Updated Is "Migration Right for you?" page and some dependencies #10158
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -9,18 +9,17 @@ | |||||
|
|
||||||
| # Is Migration Assistant right for you? | ||||||
|
|
||||||
| Whether Migration Assistant is right for you depends on your upgrade path, infrastructure complexity, and operational goals. This page will help you evaluate whether Migration Assistant fits your use case. | ||||||
|
|
||||||
| Migration Assistant addresses key limitations in traditional migration approaches. For example, if you're upgrading across multiple major versions—such as from Elasticsearch 6.8 to OpenSearch 2.19—Migration Assistant enables you to complete the process in a single step. Other methods, like rolling upgrades or snapshot restores, require upgrading through each major version, often with reindexing at every stage. | ||||||
|
|
||||||
| Migration Assistant also supports live traffic replication, enabling zero-downtime migrations. This makes it a strong fit for environments where minimizing service disruption is critical. | ||||||
|
|
||||||
| ## Supported migration paths | ||||||
|
|
||||||
| The following matrix shows which source versions can be directly migrated to which OpenSearch target versions: | ||||||
|
|
||||||
|
|
||||||
| <!-- Migration matrix rendering logic retained --> | ||||||
| {% comment %}First, collect all unique target versions{% endcomment %} | ||||||
| {% assign all_targets = "" | split: "" %} | ||||||
|
|
@@ -34,7 +33,7 @@ | |||||
| <table class="migration-matrix"> | ||||||
| <thead> | ||||||
| <tr> | ||||||
| <th>Source Version</th> | ||||||
|
Check failure on line 36 in _migration-assistant/overview/is-migration-assistant-right-for-you.md
|
||||||
| {% for target in unique_targets %} | ||||||
| <th>{{ target }}</th> | ||||||
| {% endfor %} | ||||||
|
|
@@ -59,28 +58,17 @@ | |||||
| **Source and target platforms** | ||||||
|
|
||||||
| - Self-managed (on premises or hosted by a cloud provider) | ||||||
| - Amazon OpenSearch Service (OpenSearch Serverless Collections are not supported) | ||||||
natebower marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
| **Supported AWS Regions** | ||||||
|
|
||||||
| Refer to [AWS Supported Regions](https://docs.aws.amazon.com/solutions/latest/migration-assistant-for-amazon-opensearch-service/plan-your-deployment.html#supported-aws-regions) for the for the full list of supported regions. | ||||||
|
|
||||||
| ## Supported features | ||||||
|
|
||||||
| Before starting an upgrade or migration, consider the cluster feature to be included. The following table lists what can be migrated using Migration Assistant, whether it is currently supported, and recommendations for how to handle each component. | ||||||
|
|
||||||
| | Feature | Supported | Recommendations | | ||||||
| | :--- |:--- | :--- | | ||||||
| | **Documents** | Yes | Migrate existing data with RFS and live traffic with capture and replay. | | ||||||
natebower marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| | **Index settings** | Yes | Migrate with the `Metadata-Migration-Tool`. | | ||||||
|
|
@@ -89,65 +77,91 @@ | |||||
| | **Component templates** | Yes | Migrate with the `Metadata-Migration-Tool`. | | ||||||
| | **Aliases** | Yes | Migrate with the `Metadata-Migration-Tool`. | | ||||||
| | **Index State Management (ISM) policies** | Expected in 2025 | Manually migrate using an API. For more information about ISM support, see [issue #944](https://github.com/opensearch-project/opensearch-migrations/issues/944). | | ||||||
| | **Elasticsearch Kibana dashboards** | Expected in 2025 | This tool is only needed when migrating from Elasticsearch Kibana dashboards to OpenSearch Dashboards. Start by exporting JSON files from Kibana and importing them into OpenSearch Dashboards. For Elasticsearch versions 7.10.2 to 7.17, use the [`dashboardsSanitizer`](https://github.com/opensearch-project/opensearch-migrations/tree/main/dashboardsSanitizer) tool before importing X-Pack visualizations like Canvas and Lens in Kibana dashboards, as they may require recreation for compatibility with OpenSearch.| | ||||||
natebower marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| | **Security constructs** | No | Configure roles and permissions based on cloud provider recommendations. For example, if using AWS, leverage AWS Identity and Access Management (IAM) for enhanced security management. | | ||||||
| | **Plugins** | No | Check plugin compatibility; some Elasticsearch plugins may not have direct OpenSearch equivalents. | | ||||||
|
|
||||||
|
|
||||||
| ## Checklist | ||||||
|
|
||||||
| Use this checklist to determine whether Migration Assistant is the right fit for your migration: | ||||||
|
|
||||||
| - Are you migrating across one or more major versions—for example, from Elasticsearch 5 to OpenSearch 3—in a single step? | ||||||
| - Are you upgrading but want the ability to safely back out, reducing the risk of data loss or service disruption? | ||||||
| - Do you need to maintain high service availability with minimal or zero downtime? | ||||||
| - Do you need to validate a new OpenSearch cluster before switching over, with rollback capabilities? | ||||||
natebower marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| - Is your environment self-managed or running on Amazon OpenSearch Service? | ||||||
| - Are you looking for tooling to migrate index settings and other metadata? | ||||||
| - Do you need to reconfigure your target cluster—for example, by changing the sharding strategy and reindexing? | ||||||
| - Are you migrating across regions, from on-premises, or from another cloud provider? | ||||||
natebower marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| - Do you need a high-performance backfill solution that can reliably reindex documents—with support for pause, resume, or checkpoint recovery? | ||||||
|
|
||||||
| If you answered "yes" to most of these questions, Migration Assistant is likely the right solution for your migration. | ||||||
|
|
||||||
| ## Migration Assistant assumptions and limitations | ||||||
|
|
||||||
| Before using Migration Assistant, review the following assumptions and limitations. | ||||||
|
|
||||||
| ### Networking and Environment | ||||||
|
Check failure on line 105 in _migration-assistant/overview/is-migration-assistant-right-for-you.md
|
||||||
Naarcha-AWS marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
| Migration Assistant requires network connectivity to AWS services and outbound internet access to build and deploy. Requirements differ based on whether you're deploying into a new or existing VPC. | ||||||
natebower marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
| #### Source and Target Connectivity | ||||||
|
Check failure on line 109 in _migration-assistant/overview/is-migration-assistant-right-for-you.md
|
||||||
Naarcha-AWS marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
| - You must establish connectivity between: | ||||||
| - The source cluster and/or Amazon S3 (for snapshots) and Migration Assistant | ||||||
sumobrian marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| - The target cluster and Migration Assistant | ||||||
natebower marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| - If the source or target resides in a private VPC without internet access, use one of the following to connect: | ||||||
| - VPC endpoints | ||||||
| - VPC peering | ||||||
| - AWS Transit Gateway | ||||||
|
|
||||||
| #### Deploying into a New VPC | ||||||
Naarcha-AWS marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
| - Migration Assistant provisions a new VPC with required components (e.g., NAT gateway, subnets). | ||||||
|
Check failure on line 121 in _migration-assistant/overview/is-migration-assistant-right-for-you.md
|
||||||
| - You must establish network access from this VPC to both the source (or S3) and target clusters. | ||||||
sumobrian marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
| #### Deploying into an Existing VPC | ||||||
Naarcha-AWS marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
|
||||||
sumobrian marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| - If installing Migration Assistant into an existing VPC (e.g., same VPC as the source or target), you may need to configure connectivity to any cluster external to the target VPC. | ||||||
| - For example, if installed in the source VPC, you may need VPC endpoints or peering to reach the target. | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
|
|
||||||
| - Ensure all required AWS services are reachable by Migration Assistant components. | ||||||
|
|
||||||
| - If the VPC has outbound access using: | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
| - Private subnets with a NAT gateway, or | ||||||
|
Check failure on line 132 in _migration-assistant/overview/is-migration-assistant-right-for-you.md
|
||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
| - Public subnets with an Internet Gateway | ||||||
|
Check failure on line 133 in _migration-assistant/overview/is-migration-assistant-right-for-you.md
|
||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
|
|
||||||
| then VPC interface endpoints are not required. | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
|
|
||||||
| - If using isolated subnets with no outbound access, you must configure VPC interface endpoints or routing to the following services: | ||||||
|
Check failure on line 137 in _migration-assistant/overview/is-migration-assistant-right-for-you.md
|
||||||
|
|
||||||
| - **Amazon Application Load Balancer (ALB)** – *(`Capture-and-Replay` only)* - Used to optionally reroute client traffic from the source to the target during migration. | ||||||
| - **Amazon CloudWatch** – Publishes migration metrics. | ||||||
| - **Amazon CloudWatch Logs** – Ingests ECS task logs. | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
| - **Amazon EC2** – Used to bootstrap Migration Assistant. When deploying using CloudFormation, the bootstrap EC2 instance requires outbound internet access (using NAT gateway or IGW) to download the latest release from GitHub. | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
| - **Amazon Elastic Block Store (Amazon EBS)** – Provides temporary disk storage. | ||||||
| - **Amazon Elastic Container Registry (Amazon ECR)** – Pulls container images. | ||||||
| - **Amazon Elastic Container Service (Amazon ECS)** – Orchestrates container workloads. | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
| - **Amazon Elastic File System (Amazon EFS)** – Stores persistent logs. | ||||||
| - **Amazon Managed Streaming for Apache Kafka** – *(`Capture-and-Replay` only)* - Used as a durable storage to capture and replay HTTP live traffic. | ||||||
| - **Amazon S3** – Stores and retrieves snapshots and artifacts. | ||||||
| - **AWS Elastic Load Balancing (ELB)** – *(Capture-and-Replay only)* – Used by the Migration Console to connect to the Application Load Balancer (ALB). | ||||||
|
There was a problem hiding this comment. nit: Duplicated between ELB and ALB in this list There was a problem hiding this comment. @AndreKurait @sumobrian Global editorial note: Please use "Elastic Load Balancing" (not ELB) and "Application Load Balancer" (without ALB) in all instances. Related resources include Network Load Balancer, Gateway Load Balancer, and Classic Load Balancer. Use just "load balancer" when referencing the common noun. There was a problem hiding this comment. I updated references and noted for future changes. I left the acronym in parenthesis since often users are more familiar with the acronym. |
||||||
| - **AWS Secrets Manager** – Stores credentials securely. | ||||||
sumobrian marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| - **AWS Systems Manager Parameter Store** – Holds configuration parameters. | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
| - **AWS Systems Manager Session Manager** – Enables secure EC2 shell access. | ||||||
| - **AWS X-Ray** – Supports distributed tracing. | ||||||
| - **Amazon VPC** – Ensure proper routing, DNS resolution, and endpoint configuration. | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
|
|
||||||
| ### Reindex-from-Snapshot | ||||||
|
|
||||||
| - The `_source` field must be enabled on all indexes to be migrated. See [Source documentation]({{site.url}}{{site.baseurl}}/field-types/metadata-fields/source/). | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
| - The source cluster must have the Amazon S3 plugin installed. | ||||||
| - Snapshots must include global cluster state (`include_global_state: true`). | ||||||
sumobrian marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| - Shards up to **80 GiB** are supported by default. Larger shard sizes can be configured, except in **AWS GovCloud**, where 80 GiB is the maximum. | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
|
|
||||||
| ### Capture-and-Replay | ||||||
|
|
||||||
| - The Traffic Capture Proxy must be deployed to intercept client traffic. | ||||||
| - Live capture is recommended only for workloads with **< 4 TB/day** of incoming traffic to the source cluster. | ||||||
| - Automatically generated document IDs are **not preserved** during replay. Clients must explicitly provide document IDs for `index` and `update` operations. | ||||||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.