Skip to content

Conversation

@prwhelan
Copy link
Member

@prwhelan prwhelan commented Dec 2, 2024

Prevent writes to the Transform system index while it is being reindexed during upgrade.

  • Add a new REST API, _transform/set_upgrade_mode, similar to _ml, which will enable via ?enabled or disable (when omitted).
  • Add a new Transport action, enabling or disabling upgrade mode. When enabled, all Transforms will abort and move into the waiting state, preventing Transforms from running and writing to the system index.
  • Hook into the System Index upgrade action, calling the new Transform action to set and unset upgrade mode.
  • Stop all API calls that modify the Transform state in the system index

Prevent writes to the Transform system index while it is being reindexed
during upgrade.

- Add a new REST API, `_transform/set_upgrade_mode`, similar to `_ml`,
  which will enable via `?enabled` or disable (when omitted).
- Add a new Transport action, enabling or disabling upgrade mode.  When
  enabled, all Transforms will abort and move into the `waiting` state,
  preventing Transforms from running and writing to the system index.
- Hook into the System Index upgrade action, calling the new Transform
  action to set and unset upgrade mode.
@prwhelan prwhelan added >enhancement :ml/Transform Transform Team:ML Meta label for the ML team auto-backport Automatically create backport pull requests when merged v9.0.0 v8.18.0 labels Dec 2, 2024
@elasticsearchmachine
Copy link
Collaborator

Hi @prwhelan, I've created a changelog YAML for you.

@prwhelan prwhelan marked this pull request as ready for review December 3, 2024 00:44
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

Copy link
Member

@davidkyle davidkyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

) {
var wasAlreadyInUpgradeMode = (boolean) preUpgradeMetadata.getOrDefault("already_in_upgrade_mode", false);
if (wasAlreadyInUpgradeMode) {
// Transform was already in upgrade mode before system indices upgrade started - we shouldn't disable it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 nice

prwhelan and others added 2 commits December 4, 2024 16:12
@prwhelan
Copy link
Member Author

@elasticmachine update branch

@prwhelan
Copy link
Member Author

@elasticmachine update branch

@prwhelan prwhelan merged commit e1a954a into elastic:main Dec 30, 2024
16 checks passed
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.x

prwhelan added a commit to prwhelan/elasticsearch that referenced this pull request Dec 30, 2024
Prevent writes to the Transform system index while it is being reindexed
during upgrade.

- Add a new REST API, `_transform/set_upgrade_mode`, similar to `_ml`,
  which will enable via `?enabled` or disable (when omitted).
- Add a new Transport action, enabling or disabling upgrade mode.  When
  enabled, all Transforms will abort and move into the `waiting` state,
  preventing Transforms from running and writing to the system index.
- Hook into the System Index upgrade action, calling the new Transform
  action to set and unset upgrade mode.

Co-authored-by: David Kyle <david.kyle@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
elasticsearchmachine pushed a commit that referenced this pull request Dec 30, 2024
Prevent writes to the Transform system index while it is being reindexed
during upgrade.

- Add a new REST API, `_transform/set_upgrade_mode`, similar to `_ml`,
  which will enable via `?enabled` or disable (when omitted).
- Add a new Transport action, enabling or disabling upgrade mode.  When
  enabled, all Transforms will abort and move into the `waiting` state,
  preventing Transforms from running and writing to the system index.
- Hook into the System Index upgrade action, calling the new Transform
  action to set and unset upgrade mode.

Co-authored-by: David Kyle <david.kyle@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged >enhancement :ml/Transform Transform Team:ML Meta label for the ML team v8.18.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants