Skip to content

Updated Is "Migration Right for you?" page and some dependencies #10158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Jul 10, 2025
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
df3f743
Restructured "Is Migration Assistant Right for you?" page.
sumobrian Jun 29, 2025
c62887a
Broke down limitation by component. Minor editorial changes.
sumobrian Jun 29, 2025
b58adc4
Removed "Chossing migration approach" since it will be revised and in…
sumobrian Jun 29, 2025
3e364fa
Updated requirements around VPC connectivity.
sumobrian Jul 2, 2025
38b6d73
Updated list of Migration Assistant products to vale github action pr…
sumobrian Jul 7, 2025
27dd744
Added further breakdown of networking requirements
sumobrian Jul 7, 2025
a365541
Moved limitations and assumptions
sumobrian Jul 7, 2025
a67a2be
Removed Github issue reference for Kibana
sumobrian Jul 7, 2025
200024d
Identify services that are only Capture-and-Replay
sumobrian Jul 7, 2025
0a801a8
Added ALB to list of services. Updated verbiage for ELB.
sumobrian Jul 8, 2025
dcffac5
Updated based on style-job suggestions.
sumobrian Jul 8, 2025
242887c
Updated S3 verbiage
sumobrian Jul 8, 2025
70e22e5
Remove duplicate compression bullet point.
sumobrian Jul 8, 2025
7178b11
Indicate that secrets manager is only used for basic auth creds.
sumobrian Jul 8, 2025
45ceabf
Updated SSM verbiage replacing EC2 w/ ECS
sumobrian Jul 8, 2025
6e475d6
Add writer edits
Naarcha-AWS Jul 10, 2025
1339b1b
Apply suggestions from code review
Naarcha-AWS Jul 10, 2025
b9c53f9
Apply suggestions from code review
Naarcha-AWS Jul 10, 2025
1b225ac
Apply suggestions from code review
natebower Jul 10, 2025
61321f1
Merge branch 'main' into ma_phases
Naarcha-AWS Jul 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -12,22 +12,7 @@ redirect_from:

This quickstart outlines how to deploy Migration Assistant for OpenSearch and execute an existing data migration using `Reindex-from-Snapshot` (RFS). It uses AWS for illustrative purposes. However, the steps can be modified for use with other cloud providers.

## Prerequisites and assumptions

Before using this quickstart, make sure you fulfill the following prerequisites:

* Verify that your migration path [is supported]({{site.url}}{{site.baseurl}}/migration-assistant/overview/is-migration-assistant-right-for-you/#supported-migration-paths). Note that we test with the exact versions specified, but you should be able to migrate data on alternative minor versions as long as the major version is supported.
* The source cluster must be deployed Amazon Simple Storage Service (Amazon S3) plugin.
* The target cluster must be deployed.
* Verify that the `CDKToolkit` stack exists and is set to `CREATE_COMPLETE`. For more information about how to bootstrap your AWS account in the required AWS Region, see [the CDKToolkit documentation](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html).

The steps in this guide assume the following:

* In this guide, a snapshot will be taken and stored in Amazon S3; the following assumptions are made about this snapshot:
* The `_source` flag is enabled on all indexes to be migrated.
* The snapshot includes the global cluster state (`include_global_state` is `true`).
* Shard sizes of up to approximately 80 GB are supported. Larger shards cannot be migrated. If this presents challenges for your migration, contact the [migration team](https://opensearch.slack.com/archives/C054JQ6UJFK).
* Migration Assistant will be installed in the same AWS Region and have access to both the source snapshot and target cluster.
Before using this quickstart, make sure you review ["Is Migration Assistant right for you?"]({{site.url}}{{site.baseurl}}/migration-assistant/overview/is-migration-assistant-right-for-you/#supported-migration-paths).

---

Expand Down
6 changes: 3 additions & 3 deletions _migration-assistant/overview/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ has_children: true
has_toc: false
permalink: /migration-assistant/overview/
items:
- heading: "Is Migration Assistant right for you?"
description: "Evaluate whether Migration Assistant is right for your use case."
link: "/migration-assistant/overview/is-migration-assistant-right-for-you/"
- heading: "Key components"
description: "Get familiar with the key components of Migration Assistant."
link: "/migration-assistant/overview/key-components/"
- heading: "Architecture"
description: "Understand how Migration Assistant integrates into your infrastructure."
link: "/migration-assistant/overview/architecture/"
- heading: "Is Migration Assistant right for you?"
description: "Evaluate whether Migration Assistant is right for your use case."
link: "/migration-assistant/overview/is-migration-assistant-right-for-you/"
---

# Migration Assistant overview
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,17 +9,56 @@

# Is Migration Assistant right for you?

Deciding whether to use Migration Assistant depends on your specific upgrade path, infrastructure complexity, and operational goals. This page will help you evaluate whether Migration Assistant is right for your use case—or whether another tool might be a better fit.
Whether Migration Assistant is right for you depends on your upgrade path, infrastructure complexity, and operational goals. This page will help you evaluate whether Migration Assistant fits your use case.

Migration Assistant was built to fill important gaps in common migration strategies. For example, if you're upgrading across multiple major versions—such as from Elasticsearch 6.8 to OpenSearch 2.19—Migration Assistant lets you do this in a single step. Other methods, like rolling upgrades or snapshot restores, require you to upgrade through each major version, often reindexing your data at every step.
Migration Assistant addresses key limitations in traditional migration approaches. For example, if you're upgrading across multiple major versions—such as from Elasticsearch 6.8 to OpenSearch 2.19—Migration Assistant enables you to complete the process in a single step. Other methods, like rolling upgrades or snapshot restores, require upgrading through each major version, often with reindexing at every stage.

Migration Assistant also supports live traffic replication, allowing for zero-downtime migrations. This makes it a strong choice for production environments, where minimizing service disruption is critical.
Migration Assistant also supports live traffic replication, enabling zero-downtime migrations. This makes it a strong fit for environments where minimizing service disruption is critical.

If your migration is limited to a static cluster configuration (like index templates and aliases), or if you're not concerned about downtime, simpler tools may be sufficient. But for complex migrations involving real-time traffic or major version jumps, Migration Assistant offers robust, flexible capabilities.
## Migration Assistant assumptions and limitations

Before using Migration Assistant, review the following assumptions and limitations. They are grouped by scope to clarify which apply generally and which are specific to components.

### General


- If deploying on AWS, the identity used to deploy Migration Assistant must have permission to install all required resources. For a full list of services, see [AWS Services in this Solution](https://docs.aws.amazon.com/solutions/latest/migration-assistant-for-amazon-opensearch-service/architecture-details.html#aws-services-in-this-solution).
- The target cluster must be deployed and reachable by Migration Assistant components, including the Migration Console, Reindex-from-Snapshot, and Traffic Replayer, depending on which features are used.

Check failure on line 26 in _migration-assistant/overview/is-migration-assistant-right-for-you.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.Spelling] Error: Replayer. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks. Raw Output: {"message": "[OpenSearch.Spelling] Error: Replayer. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_migration-assistant/overview/is-migration-assistant-right-for-you.md", "range": {"start": {"line": 26, "column": 156}}}, "severity": "ERROR"}
- The `_source` field must be enabled on all indexes to be migrated. See [Source documentation]({{site.url}}{{site.baseurl}}/field-types/metadata-fields/source/) for more information.
- If deploying to AWS, ensure the `CDKToolkit` stack exists and is in the `CREATE_COMPLETE` state. For setup instructions, see the [CDK Toolkit documentation](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html).
- If deploying Migration Assistant into an existing AWS VPC, you must configure VPC interface endpoints and IAM permissions for the following AWS services:
- **Amazon CloudWatch Logs** – for log ingestion from ECS tasks.
- **Amazon CloudWatch** – for metrics published during migration
- **Amazon Elastic Container Registry (ECR)** – for pulling container images.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this enough detail for customers? there are two ECR endpoints
com.amazonaws.region.ecr.dkr and com.amazonaws.region.ecr.api, we need the former to pull images not the later

https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html#ecr-setting-up-vpc-create

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a level of detail that should be included on another page. On this page I'd like customers to think about the access that they will need.

- **Amazon Elastic Container Service (ECS)** – for task orchestration and migration.
- **Elastic Load Balancing (ELB)** – for routing traffic to Capture Proxy.
- **AWS Secrets Manager** – for storing credentials.
- **AWS Systems Manager Parameter Store** – for storing and accessing parameter values.
- **AWS Systems Manager Session Manager** – for secure shell access to EC2 (i.e., bootstrap instance).
- **Amazon EC2** – for launching the bootstrap instance responsible for build and deployment.
- **Amazon Elastic Block Store (EBS)** – for disk storage during migration.
- **Amazon Virtual Private Cloud (VPC)** – for private networking and VPC endpoints.
- **AWS X-Ray** – for distributed tracing across components.
- **Amazon Elastic File System (EFS)** – for persistent logging.

### Reindex-from-Snapshot

- The source cluster must have the Amazon S3 plugin installed.
- Snapshots must include global cluster state (`include_global_state` is `true`).
- Shards up to 80 GiB are supported by default. Larger shards can be configured, except in AWS GovCloud, where 80 GiB is the maximum supported size.

### Capture-and-Replay

- The Traffic Capture Proxy must be deployed to intercept relevant client traffic.
- Live capture is recommended only for environments with less than 4 TB/day of incoming traffic to the source cluster.
- Automatically generated document IDs are not preserved for index requests. Clients must explicitly provide document IDs when indexing or updating documents.

---

## Supported migration paths

The following matrix shows which source versions can be directly migrated to which OpenSearch target versions:
The matrix below shows which source versions can be directly migrated to which OpenSearch target versions:

Check warning on line 60 in _migration-assistant/overview/is-migration-assistant-right-for-you.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.DirectionAboveBelow] Use 'following or later' instead of 'below' for versions or orientation within a document. Use 'above' and 'below' only for physical space or screen descriptions. Raw Output: {"message": "[OpenSearch.DirectionAboveBelow] Use 'following or later' instead of 'below' for versions or orientation within a document. Use 'above' and 'below' only for physical space or screen descriptions.", "location": {"path": "_migration-assistant/overview/is-migration-assistant-right-for-you.md", "range": {"start": {"line": 60, "column": 12}}}, "severity": "WARNING"}


<!-- Migration matrix rendering logic retained -->
{% comment %}First, collect all unique target versions{% endcomment %}
Expand All @@ -34,7 +73,7 @@
<table class="migration-matrix">
<thead>
<tr>
<th></th>
<th>Source Version</th>

Check failure on line 76 in _migration-assistant/overview/is-migration-assistant-right-for-you.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.TableHeadings] 'Source Version' is a table heading and should be in sentence case. Raw Output: {"message": "[OpenSearch.TableHeadings] 'Source Version' is a table heading and should be in sentence case.", "location": {"path": "_migration-assistant/overview/is-migration-assistant-right-for-you.md", "range": {"start": {"line": 76, "column": 11}}}, "severity": "ERROR"}
{% for target in unique_targets %}
<th>{{ target }}</th>
{% endfor %}
Expand All @@ -59,28 +98,17 @@
**Source and target platforms**

- Self-managed (on premises or hosted by a cloud provider)
- Amazon OpenSearch Service


**AWS Regions**
- Amazon OpenSearch Service (OpenSearch Serverless Collections are not supported)

Migration Assistant is supported in the following AWS Regions:
**Supported AWS Regions**

- US East (N. Virginia, Ohio)
- US West (Oregon, N. California)
- Europe (Frankfurt, Ireland, London)
- Asia Pacific (Tokyo, Singapore, Sydney)
- AWS GovCloud (US-East, US-West)[^1]
Refer to [AWS Supported Regions](https://docs.aws.amazon.com/solutions/latest/migration-assistant-for-amazon-opensearch-service/plan-your-deployment.html#supported-aws-regions) for the for the full list of supported regions.

[^1]: In AWS GovCloud (US), `reindex-from-snapshot` (RFS) is limited to shard sizes of 80 GiB or smaller.
## Supported features

Before starting an upgrade or migration, consider the cluster feature to be included. The table below lists what can be migrated using Migration Assistant, whether it is currently supported, and recommendations for how to handle each component.

Check warning on line 109 in _migration-assistant/overview/is-migration-assistant-right-for-you.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.DirectionAboveBelow] Use 'following table' instead of 'table below' for versions or orientation within a document. Use 'above' and 'below' only for physical space or screen descriptions. Raw Output: {"message": "[OpenSearch.DirectionAboveBelow] Use 'following table' instead of 'table below' for versions or orientation within a document. Use 'above' and 'below' only for physical space or screen descriptions.", "location": {"path": "_migration-assistant/overview/is-migration-assistant-right-for-you.md", "range": {"start": {"line": 109, "column": 91}}}, "severity": "WARNING"}


## Supported components

Before starting a migration, consider the scope of the components involved. The following table outlines components that should potentially be migrated, indicates whether they are supported by Migration Assistant, and provides recommendations.

| Component | Supported | Recommendations |
| Feature | Supported | Recommendations |
| :--- |:--- | :--- |
| **Documents** | Yes | Migrate existing data with RFS and live traffic with capture and replay. |
| **Index settings** | Yes | Migrate with the `Metadata-Migration-Tool`. |
Expand All @@ -95,59 +123,19 @@

[^2]: Support for Kibana 5.0 through 7.10.2 migration paths to OpenSearch Dashboards will be added in a future version. Kibana 8 and later are not supported. For more information, see [issue #944](https://github.com/opensearch-project/opensearch-migrations/issues/944).

## Choosing your migration approach

Use the following checklist to determine which Migration Assistant components best fit your use case.

### Metadata migration

Use [metadata migration]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/migrating-metadata/) if:

- You need to migrate while mitigating breaking changes between the source and target clusters, such as differences in mappings, settings, aliases, or component templates.
- You want a relatively consistent configuration between the source and target clusters.

### Backfill migration

Use [backfill migration]({{site.url}}{{site.baseurl}}/migration-assistant/migration-phases/backfill/) if:

- You need to move historical data without disrupting live traffic.
- You want to backfill indexes from a specific point in time without impacting the source cluster.
- You want to verify historical data in the target cluster before switching over.
- You want to backfill using an existing or incremental snapshot.
- You need the fastest backfill option that includes reindexing.
- You want the ability to pause and resume migration.

### RFS

Use [RFS]({{site.url}}{{site.baseurl}}/migration-assistant/deploying-migration-assistant/getting-started-data-migration/) if:

- You already use OpenSearch snapshots for backups.
- You need to migrate documents at scale in parallel, such as with Amazon Elastic Container Service (Amazon ECS).
- You require a data migration path as part of a zero-downtime migration.
- Your AWS Region supports RFS and your shard sizes are within supported limits.

### Combination of all three

Use a combination of all three migration types if:

- You're performing a complex, multi-version migration.
- You require zero downtime and full validation of the target environment.
- You want end-to-end tooling for metadata, data movement, and cluster behavior comparison.
- You're cloning an existing cluster and changing the source's configuration.
- You're setting up disaster recovery.

## Checklist

Use this checklist to decide whether Migration Assistant is right for you:

- Are you migrating across one or more major versions?

- Do you need to maintain service availability with zero downtime?

- Do you need to validate a new OpenSearch cluster before switching over?
Use this checklist to determine whether Migration Assistant is the right fit for your migration:

- Are you migrating across one or more major versions—for example, from Elasticsearch 5 to OpenSearch 3—in a single step?
- Are you upgrading but want the ability to safely back out, reducing the risk of data loss or service disruption?
- Do you need to maintain high service availability with minimal or zero downtime?
- Do you need to validate a new OpenSearch cluster before switching over, with rollback capabilities?
- Is your environment self-managed or running on Amazon OpenSearch Service?

- Are you looking for tooling that can automate metadata migration and performance comparison?
- Are you looking for tooling to migrate index settings and other metadata?
- Do you need to reconfigure your target cluster—for example, by changing the sharding strategy and reindexing?
- Are you migrating across regions, from on-premises, or from another cloud provider?
- Do you need a high-performance backfill solution that can reliably reindex documents—with support for pause, resume, or checkpoint recovery?

If you answered "yes" to most of these questions, Migration Assistant is likely the right solution for your migration.
Loading