Allow APIKey to be used as authentication method for Elasticsearch

[l-trotta]

APIKey is a popular authentication method for elasticsearch, available both for local installations and for cloud instances; in fact, it's the recommended way to setup the Elasticsearch Java client!

This PR adds spring.elasticsearch.apikey as one of the elasticsearch properties, so that users will be able to setup a connection simply with:

spring.elasticsearch.uris=${ES_SERVER_URL}
spring.elasticsearch.apikey=${ES_APIKEY}

If this gets merged I will also take care of the documentation of affected repos (spring-ai, spring-data-elasticsearch) to add and describe the new property.

Disclaimer: I am one of the maintainer of https://github.com/elastic/elasticsearch-java

[snicoll]

Thanks for the PR

it's the recommended way to setup the Elasticsearch Java client!

I wasn't aware of that. How does it work with Testcontainers and Docker compose then? Both of them retrieve a password from the environment. I can see you have updated ElasticsearchConnectionDetails but none of their two implementations.

[l-trotta]

Creating the API key has to be done through either an API call or in the dedicated Kibana menu, this is because you can decide to create one with certain privileges, or with an expiration date. So the default for both implementations of ElasticsearchConnectionDetails would be null, because the API key cannot be retreived from the environment, but only created after the elasticsearch instance is running.

But it works the exact same for cloud, testcontainers or docker, it's just an additional step unlike username and password credentials.

[ezimuel]

@l-trotta and @snicoll maybe we can simplify the Elasticsearch testing using start-local. This script installs Elasticsearch with a .env file containing all the information for the connection, including the API key (ES_LOCAL_API_KEY). If you only need Elasticsearch and not Kibana you can use the --esonly option.

[eddumelendez]

Hi, what about providing and env var in Elasticsearch image which allows to configure the API Key? That would benefit Docker Compose and Testcontainers implementations. Also, if this is the recommended way then the image could provide a default way to enable this.

On the Testcontainers side we can add a custom implementation to generate the API Key but I am more in favor to provide the same experience for Docker Compose and Testcontainers users by just enabling the API key with a env var.

[bclozel]

@eddumelendez Looking at what @l-trotta pointed out, it seems that one of the main driver behind API keys is to have fine-grained roles per index, since you need to provide those during API key generation. I guess that for a given user, you want to use different keys for specific usage patterns and restrict roles accordingly.
Is this something that testcontainers is already supporting in some way, or is it something that developers should write as part of their tests?

[eddumelendez]

Hi @bclozel, currently, developers must extend and override ElasticsearchContainer, implement containerIsStarted as part of the Testcontainers Lifecycle and make a similar call to start-local using a HTTP library.

[eddumelendez]

I've uploaded an example using Testcontainers and Elasticsearch using API Key.