Skip to content

[Issue]: Breaking Wrong Info on restoring system indices #1391

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
stefnestor opened this issue May 8, 2025 · 3 comments · Fixed by #1426
Closed

[Issue]: Breaking Wrong Info on restoring system indices #1391

stefnestor opened this issue May 8, 2025 · 3 comments · Fixed by #1426
Assignees
Labels
bug Something isn't working source:web Issues originating from the elastic.co docs Team:Platform Issues owned by the Platform Docs Team

Comments

@stefnestor
Copy link
Contributor

Type of issue

Inaccurate

What documentation page is affected

https://www.elastic.co/docs/manage-data/migrate/migrate-internal-indices

What happened?

You're not allowed to restore system indices via index +8.0 you MUST use feature states. This (a) literally directs you wrong path & (b) uses renames pattern which will prevent those from working in destination cluster.

Additional info

@stefnestor stefnestor added bug Something isn't working source:web Issues originating from the elastic.co docs labels May 8, 2025
@github-actions github-actions bot added the needs-team Issues pending triage by the Docs Team label May 8, 2025
@georgewallace georgewallace added the Team:Platform Issues owned by the Platform Docs Team label May 8, 2025
@github-actions github-actions bot removed the needs-team Issues pending triage by the Docs Team label May 8, 2025
@georgewallace
Copy link
Contributor

@stefnestor Thanks again for this other piece of feedback, we will take a look at this

@eedugon eedugon self-assigned this May 14, 2025
@eedugon
Copy link
Contributor

eedugon commented May 14, 2025

@stefnestor , thanks a lot for sharing this.

Quick question before preparing a fix:

Some feature states should be restored in conjunction with others. For example kibana and security can be restored by themselves but fleet has kibana dependencies and those should be restored together.

Is there any doc where that is explained? I don't see any explanation in the doc where feature states are introduced, and neither in the restore feature state doc.

If you share more details about that we can also improve those docs highlighting that fact :)

cc: @shainaraskas

@stefnestor
Copy link
Contributor Author

eedugon added a commit that referenced this issue May 27, 2025
When working on the issue
#1391 I realized that the
migration guide was a bit outdated and had multiple important issues to
address.

I have refined the content and applied major updates.

Updated docs:

- [Migrate your Elasticsearch
data](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/1426/manage-data/migrate)
- [Migrate system
indices](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/1426/manage-data/migrate/migrate-internal-indices)

Summary of fixes and changes:
- Internal / system indices migration based on snapshot was incorrect ->
feature states should be used.

- Internal / system indices migration based on reindexing was incorrect
-> the procedure was faulty and needed extra love and research. But
considering that based on Elasticsearch docs we state that from 8.0
onwards, `feature states` are the only way to backup and restore system
indices and system data streams, I have removed the reindex option for
this.

- Snapshot repository creation for ECE was invalid, as users shouldn't
create a repository for data migration at ECE-level, but directly at
deployment-level (this is a gap in our docs as we don't have specific
ECE docs to create custom repos only at deployment level) -> The ECH doc
is valid for this, so ECE users can follow the ECH doc to create a
custom repo at deployment level.

- Added a lot of links to other relevant documents of the new IA. The
intention of this document is to feel like a narrative document
explaining what to do and how, but with links to the official snapshots
or reindex documentation for more information.

- For reindex method, I have added explanations and recommendations for
mappings and index templates handling, without specific instructions
(just pointers).

- The previous doc presented the official doc about Backup Snapshot
repositories in a context where it didn't belong. I've included a tip
explaining a use case where that can be useful.

- The previous doc suggested the usage of a `url` type repository in a
way that felt incorrect (together with AWS / GCS / Azure keystore
credentials which in my opinion cannot be related). I've added it as
another possible use case (I'm still not 100% sure about this, and we
have some contradictions in other documents that we have to address
separately).

Closes: #1391

---------

Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working source:web Issues originating from the elastic.co docs Team:Platform Issues owned by the Platform Docs Team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants