Manage data -> Data migration to ECH or ECE updated #1426
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
eedugon
changed the title
shainaraskas
approved these changes
May 22, 2025
There was a problem hiding this comment.
beautiful and clear work as always! threw some nitpicks on there. I don't think there's anything major, but try to get that whitespace check notice to go away, and as long as @stefnestor gives this a green light, it's basically good to go
| @@ -17,13 +17,17 @@ products: | |||
|
|
|||
| # Migrate your {{es}} data | |||
|
|
|||
| You might have switched to {{ech}} or {{ece}} for any number of reasons, and you’re likely wondering how to get your existing {{es}} data into your new infrastructure. Along with easily creating as many new deployments with {{es}} clusters that you need, you have several options for moving your data over. Choose the option that works best for you: | |||
| You might have switched to {{ech}} (ECH) or {{ece}} (ECE) for any number of reasons, and you’re likely wondering how to get your existing {{es}} data into your new infrastructure. Along with easily creating as many new deployments with {{es}} clusters that you need, you have several options for moving your data over. Choose the option that works best for you: | |||
There was a problem hiding this comment.
for the whitespace thing: https://elastic.slack.com/archives/C07APH4RCDT/p1747837810907349
Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
shainaraskas
approved these changes
May 26, 2025
shainaraskas
approved these changes
May 26, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When working on the issue #1391 I realized that the migration guide was a bit outdated and had multiple important issues to address.
I have refined the content and applied major updates.
Updated docs:
Summary of fixes and changes:
Internal / system indices migration based on snapshot was incorrect -> feature states should be used.
Internal / system indices migration based on reindexing was incorrect -> the procedure was faulty and needed extra love and research. But considering that based on Elasticsearch docs we state that from 8.0 onwards,
feature statesare the only way to backup and restore system indices and system data streams, I have removed the reindex option for this.Snapshot repository creation for ECE was invalid, as users shouldn't create a repository for data migration at ECE-level, but directly at deployment-level (this is a gap in our docs as we don't have specific ECE docs to create custom repos only at deployment level) -> The ECH doc is valid for this, so ECE users can follow the ECH doc to create a custom repo at deployment level.
Added a lot of links to other relevant documents of the new IA. The intention of this document is to feel like a narrative document explaining what to do and how, but with links to the official snapshots or reindex documentation for more information.
For reindex method, I have added explanations and recommendations for mappings and index templates handling, without specific instructions (just pointers).
The previous doc presented the official doc about Backup Snapshot repositories in a context where it didn't belong. I've included a tip explaining a use case where that can be useful.
The previous doc suggested the usage of a
urltype repository in a way that felt incorrect (together with AWS / GCS / Azure keystore credentials which in my opinion cannot be related). I've added it as another possible use case (I'm still not 100% sure about this, and we have some contradictions in other documents that we have to address separately).Closes: #1391