You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In relation with elastic/cloud#134936, we have
realized that the doc
`/deploy-manage/tools/snapshot-and-restore/minio-on-premise-repository.md`
was really outdated and needed some love.
This is the main page to review:
[MinIO self-managed
repository](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/1996/deploy-manage/tools/snapshot-and-restore/minio-on-premise-repository)
Applied changes:
- Changed wording to `MinIO` across all docs. No minio or Minio anymore.
- Fixed a lot of broken links to MinIO docs.
- Removed screenshots (thanks @Kushmaro for pointing that out), and
replacing them by proper links and instructions.
- Adding a note to recommend installing MinIO in separate hosts,
although we support installing it in ECE allocator hosts.
- Removed the manual steps to restore a snapshot from another cluster as
now that's offered and documented as an ECE feature. I've linked to the
right doc for that.
- Improved the structure of certain sections as they were hard to
understand (the bucket creation step for example was incorrect).
- Renamed `offline` to `air-gapped` (for consistency with the new IA and
new docs).
- Changed title from `on-premise` to `self-managed` (because ECE hosts
or the customer minio hosts could be on-premises or on cloud VMs
technically. What they are in reality is `self-managed`).
- Added useful links to other parts of the documentation.
- Removed the statements about 6.x and 5.x versions of Elasticsearch.
These are unneeded as in ECE 4.0 it's mandatory for all deployments to
be on 8.0.0 or above. So there's no need to mention it.
Closeselastic/cloud#134936
---------
Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>
Copy file name to clipboardExpand all lines: deploy-manage/deploy/cloud-enterprise/add-custom-bundles-plugins.md
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -184,7 +184,7 @@ In this example, we assume the Identity Provider does not publish its SAML metad
184
184
185
185
## Example: Custom JVM trust store bundle [ece-add-custom-bundle-example-cacerts]
186
186
187
-
If you are using SSL certificates signed by non-public certificate authorities, {{es}} is not able to communicate with the services using those certificates unless you import a custom JVM trust store containing the certificates of your signing authority into your {{ece}} installation. You’ll need the trust store to access snapshot repositories like Minio, for your {{ece}} proxy, or to reindex from remote.
187
+
If you are using SSL certificates signed by non-public certificate authorities, {{es}} is not able to communicate with the services using those certificates unless you import a custom JVM trust store containing the certificates of your signing authority into your {{ece}} installation. You’ll need the trust store to access snapshot repositories like MinIO, for your {{ece}} proxy, or to reindex from remote.
If you are installing ECE without internet access (commonly called an offline or air-gapped installation), you will need to use an on-premise storage service. We suggest that you use [Minio](https://www.minio.io/). For our installation notes, check [Snapshotting to Minio On-Premise Storage](minio-on-premise-repository.md).
39
+
If you are installing ECE without internet access (commonly called an offline or air-gapped installation), you will need to use an on-premise storage service. We suggest that you use [MinIO](https://www.minio.io/). For our installation notes, check [Snapshotting to MinIO On-Premise Storage](minio-on-premise-repository.md).
40
40
::::
41
41
42
42
@@ -47,7 +47,7 @@ The following guides provide instructions on adding a snapshot repository in ECE
Minio is a popular, open-source distributed object storage server compatible with the Amazon AWS S3 API. You can use it with {{ece}} installations when you want to store your {{es}} snapshots locally.
13
+
[MinIO](https://min.io/docs/minio/container/index.html) is a popular, open-source object storage server compatible with the Amazon AWS S3 API. As an [S3 compatible service](/deploy-manage/tools/snapshot-and-restore/s3-repository.md#repository-s3-compatible-services), MinIO is supported for use as a snapshot repository in {{ece}} (ECE).
14
14
15
+
This guide walks you through integrating MinIO with ECE to store your {{es}} snapshots.
15
16
16
-
## Create a test environment [ece-minio-test]
17
+
::::{important}
18
+
Avoid running MinIO directly on ECE hosts. Sharing infrastructure can lead to resource contention, especially disk I/O, and may affect the performance and stability of your Elastic workloads. It also complicates upgrades, troubleshooting, and supportability.
17
19
18
-
We recommend following the [Minio Quickstart Guide Docker Container instructions](https://docs.minio.io/docs/minio-docker-quickstart-guide) to create a simple Minio standalone installation for your initial evaluation and development.
19
-
20
-
Be sure to use the `docker -v` option to map persistent storage to the container.
21
-
22
-
23
-
## Production environment prerequisites [ece-minio-requirements]
24
-
25
-
Installing Minio for production requires a high-availability configuration where Minio is running in [Distributed mode](https://docs.minio.io/docs/distributed-minio-quickstart-guide).
26
-
27
-
As mentioned in the Minio documentation, you will need to have 4-16 Minio drive mounts. There is no hard limit on the number of Minio nodes. It might be convenient to place the Minio node containers on your ECE hosts to ensure you have a suitable level of availability, but those can not be located on the same hosts as ECE proxies since they both listen on the same port.
28
-
29
-
The following illustration is a sample architecture for a [large ECE installation](../../deploy/cloud-enterprise/deploy-large-installation.md). Note that there is at least one MinIO container in *each* availability zone.
30
-
31
-
There are a number of different ways of orchestrating the Minio deployment (Docker Compose, Kubernetes, and so on). We suggest you use the method most familiar to you.
32
-
33
-
We recommend:
34
-
35
-
* Using a single Minio endpoint with the {{ece}} installation, to simplify repository management.
36
-
* Securing access to the Minio endpoint with TLS.
20
+
If you're evaluating MinIO in a test system, do not place MinIO containers on the same hosts as ECE proxies, as both services use the same port.
This section provides guidance and recommendations for deploying MinIO. It does not include installation steps. As MinIO is a third-party product, its deployment, configuration, and maintenance are outside the scope of Elastic support.
43
26
44
-
## Create an offline installation [ece-minio-offline-installation]
27
+
For installation instructions, refer to the official [MinIO documentation](https://min.io/docs/).
45
28
46
-
If you are installing MinIO offline, the process is very similar to the [offline installation of {{ece}}](../../deploy/cloud-enterprise/air-gapped-install.md). There are two options:
29
+
The performance and reliability of MinIO depend on its configuration and the underlying infrastructure. Consider the following best practices:
47
30
48
-
* Use a private Docker repository and [install the Minio images in the private repository](https://docs.docker.com/registry/deploying/).
49
-
* Download the Minio images from an internet-connected machine, then use docker save to bundle the images into tar files. Copy the TAR files to the target hosts and use `docker load` to install.
31
+
* For production use, deploy MinIO in [distributed mode](https://min.io/docs/minio/linux/operations/install-deploy-manage/deploy-minio-multi-node-multi-drive.html#minio-mnmd).
32
+
* Use a single MinIO endpoint with the ECE installation, to simplify repository configuration.
33
+
* Secure access to the MinIO endpoint with TLS.
50
34
51
-
Gather the following after your installation:
35
+
After deployment, make sure you collect the following values:
52
36
53
-
*Minio AccessKey
54
-
*Minio SecretKey
55
-
*Endpoint URL
37
+
*MinIO Access Key
38
+
*MinIO Secret Key
39
+
*MinIO endpoint URL
56
40
57
41
::::{tip}
58
-
Minio might report various Endpoint URLs, be sure to choose the one that will be routable from your {{es}} Docker containers.
42
+
MinIO may report multiple endpoint URLs. Be sure to select the one reachable from your {{es}} containers running on ECE allocator hosts.
59
43
::::
60
44
45
+
### Testing and evaluation
61
46
47
+
Use the [MinIO Quickstart Guide](https://charts.min.io/) or the [container deployment guide](https://min.io/docs/minio/container/index.html) to spin up a simple standalone MinIO container. Use `-v` to map persistent storage when using the `docker` or `podman` options.
62
48
63
-
## Create the S3 bucket [ece-minio-create-s3-bucket]
64
-
65
-
How you create the AWS S3 bucket depends on what version of {{es}} you are using:
49
+
### Production environments
66
50
67
-
* For version 7.x:
51
+
Set up MinIO in distributed mode across multiple nodes and drives. You can use Docker Compose, Kubernetes, or another orchestration tool of your choice.
68
52
69
-
1. Using the Minio browser or an S3 client application, create an S3 bucket to store your snapshots.
70
-
2.[Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md) and [add the S3 repository plugin](elasticsearch://reference/elasticsearch-plugins/plugin-management.md) to your cluster.
53
+
## Create the S3 bucket [ece-minio-create-s3-bucket]
71
54
72
-
* For versions 8.0 and later, {{es}} has built-in support for AWS S3 repositories; no repository plugin is needed. Use the Minio browser or an S3 client application to create an S3 bucket to store your snapshots.
55
+
After installing MinIO you will need to create a bucket to store your deployments' snapshots. Use the MinIO browser or an S3 client application to create an S3 bucket to store your snapshots.
73
56
74
57
::::{tip}
75
58
Don’t forget to make the bucket name DNS-friendly, for example no underscores or uppercase letters. For more details, read the [bucket restrictions](https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html).
76
59
::::
77
60
61
+
## {{ece}} configuration [ece-install-with-minio]
78
62
63
+
This section describes the configuration changes required to use MinIO storage within ECE to make periodic snapshots of your deployments. The required steps include:
79
64
80
-
## {{ece}} configuration [ece-install-with-minio]
65
+
* Configuring the repository at ECE level
66
+
* Associating it with your deployments
67
+
* Applying specific YAML settings to the deployments
81
68
82
-
You can configure existing deployments, or create new ones, with the following changes to use Minio storage.
69
+
### Prerequisites
83
70
71
+
Before integrating ECE with MinIO, ensure you have the following details from your MinIO deployment:
72
+
73
+
* MinIO Access Key
74
+
* MinIO Secret Key
75
+
* MinIO endpoint URL
76
+
* S3 bucket name
84
77
85
78
### Add the repository to {{ece}} [ece-add-repository]
86
79
87
-
You must add the new repository to {{ece}} before it can be used with your {{es}} clusters.
80
+
You must add the new repository at ECE platform level before it can be used by your {{es}} deployments.
88
81
89
82
1.[Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md).
90
83
2. From the **Platform** menu, select **Repositories**.
@@ -97,76 +90,50 @@ You must add the new repository to {{ece}} before it can be used with your {{es}
Once the MinIO repository is created at the ECE platform level, you can associate it with your {{es}} deployments in two ways:
122
107
108
+
* For new deployments, select the repository from the **Snapshot repository** drop-down list while [creating the deployment](/deploy-manage/deploy/cloud-enterprise/create-deployment.md).
123
109
124
-
### Additional settings for 6.x clusters [ece-6.x-settings]
110
+
* For existing deployments, associate the repository by following the instructions in [Manage {{es}} clusters repositories](/deploy-manage/tools/snapshot-and-restore/cloud-enterprise.md#ece-manage-repositories-clusters).
125
111
126
-
For {{es}} versions 6.0 and later, after selecting the repository, you also need to set your **User Settings** YAML to specify the endpoint and protocol. For example:
112
+
### Additional settings for {{es}} [ece-6.x-settings]
113
+
114
+
After selecting the repository, you also need to configure your [{{es}} user settings YAML](/deploy-manage/deploy/cloud-enterprise/edit-stack-settings-elasticsearch.md) to specify the endpoint and protocol. For example:
Check the [{{es}} S3 plugin details](https://www.elastic.co/guide/en/elasticsearch/plugins/6.8/repository-s3-client.html) for more information.
133
120
121
+
Refer to the [{{es}} S3 plugin details](/deploy-manage/tools/snapshot-and-restore/s3-repository.md) for more information.
134
122
135
-
## Upgrade from 5.x to 6.x {{es}} clusters [ece-upgrade-minio]
123
+
#### Add S3 repository plugin (only for {{es}} 7.x)
136
124
137
-
The configuration options for the {{es}} S3 repository plugin have changed from 5.x to 6.x versions and you must copy the endpoint and protocol values from your repository configuration to your **User Settings** YAML before you upgrade.
125
+
For {{es}} clusters in version 7.x you must add the S3 repository plugin to your cluster. Refer to [Managing plugins for ECE](elasticsearch://reference/elasticsearch-plugins/plugin-management.md#managing-plugins-for-ece) for more details.
138
126
127
+
::::{note}
128
+
For versions 8.0 and later, {{es}} has built-in support for AWS S3 repositories; no repository plugin is needed.
129
+
::::
139
130
140
131
## Verify snapshots [ece-minio-verify-snapshot]
141
132
142
-
The cluster should make a snapshot when the repository is set up. You can check that by going to the **Elasticsearch** and then the **Snapshots** page.
143
-
144
-
As an extra verification step, you can restore a cluster using the snapshots that have been taken.
145
-
146
-
1. [Log into the Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md).
147
-
2. Get the plan from your test cluster.
148
-
149
-
1. From the **Deployments** page, select your deployment.
150
-
151
-
Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
152
-
153
-
2. From your deployment menu, go to the **Edit** page then go to the bottom of the page and select **advanced {{es}} configuration**.
154
-
3. Copy the JSON format under the **Deployment configuration** heading.
155
-
156
-
3. Create a new {{es}} cluster as your target.
157
-
4. On the new cluster, open the advanced cluster configuration editor. In the transient section, add the `restore_snapshot` settings to the plan.
158
-
159
-
```json
160
-
...
161
-
"transient": {
162
-
"restore_snapshot": {
163
-
"repository_name": "<Minio repository name>",
164
-
"snapshot_name": "latest_success"
165
-
}
166
-
}
167
-
```
133
+
The cluster should make periodic snapshots when the repository is set up and associated to it. You can check this in the **Elasticsearch > Snapshots** section of the deployment page in the [Cloud UI](../../deploy/cloud-enterprise/log-into-cloud-ui.md).
168
134
169
-
5. Select **Save** to restore from the snapshot. When the plan update is complete, you can check the restored indexes in your target cluster.
135
+
As an extra verification step, you can [restore snapshots across clusters](/deploy-manage/tools/snapshot-and-restore/ece-restore-across-clusters.md).
170
136
171
-
More details are available to [work with snapshots](../snapshot-and-restore.md).
137
+
Refer to [work with snapshots](../snapshot-and-restore.md) for more information around {{es}} snapshot and restore.
172
138
139
+
For additional considerations on performance, reliability, and troubleshooting when using MinIO as a snapshot repository, refer to [Using MinIO with {{es}}](/deploy-manage/tools/snapshot-and-restore/s3-repository.md#using-minio-with-elasticsearch).
0 commit comments