Elasticsearch Testcontainer image version is not compatible out of the box with Spring Boot Data Elasticsearch

[kaiquevieirasoares]

Closes #2146

Description:
This PR addresses the issue where Spring Boot 4.0 requires Elasticsearch 9+, but the Initializr was hardcoded to use 7.17.10.

Changes:

• Added elasticsearch9 to SimpleDockerServiceResolver with image 9.3.3.
• Modified ElasticsearchProjectGenerationConfiguration to check the Spring Boot version. If >= 4.0.0-M1, it switches to the ES 9 image.
• Injected xpack.security.http.ssl.enabled=false and xpack.security.transport.ssl.enabled=false into the Docker Compose file for Boot 4 projects to prevent the Connection closed by peer error caused by the default TLS settings in ES 9.
• Preserved the existing ES 7.x logic for Spring Boot 3.x projects.
• Added corresponding tests.

[kaiquevieirasoares]

Hi @eddumelendez ! Thanks for the review.
You are completely right. I initially added those SSL flags thinking they were strictly required to bypass a Connection closed by peer issue on ES 9, but since xpack.security.enabled=false handles it just like in ElasticsearchContainer, they are indeed redundant. I just pushed a new commit removing them to keep it aligned.
Also, my bad on the Javadoc! I accidentally removed it while fixing Checkstyle issues. I've restored it now.

[mhalbritter]

Unfortunately, this doesn't work with Testcontainers. The PR generates this code:

return new ElasticsearchContainer(DockerImageName.parse("docker.elastic.co/elasticsearch/elasticsearch:9.3.3"));

which still has TLS enabled.

We would need that:

return new ElasticsearchContainer(DockerImageName.parse("docker.elastic.co/elasticsearch/elasticsearch:9.3.3"))
		.withEnv("xpack.security.enabled", "false");

[mhalbritter]

Not caused by this PR: And it seems that Boot lacks support for the docker.elastic.co image when using Elasticsearch. I've opened spring-projects/spring-boot#50088 for that. I wonder why we use docker.elastic.co/elasticsearch/elasticsearch instead of elasticsearch from Docker Hub.

[kaiquevieirasoares]

Hi @mhalbritter ! Thanks for the review and for taking over!

I was looking into implementing the .withEnv("xpack.security.enabled", "false") for the generated Java code, but I noticed an architectural limitation. Currently, the ServiceConnection record and the DockerService API don't seem to expose a mechanism or a modifier to append custom method invocations (like .withEnv(...)) to the generated TestcontainersConfiguration class.

Since the generator seems to build the return new (DockerImageName.parse(...)) string statically based on the ServiceConnection fields, how would you prefer to handle this? Should we introduce a new field/modifier to the ServiceConnection API to support custom method chaining, or is there an existing customizer/approach I might have missed to inject this environment variable into the Java generation?

Also, regarding your note on issue #50088: I understand it's not caused by this PR, but if you'd like me to update the image reference to use the Docker Hub one while we're at it, I can easily change that in SimpleDockerServiceResolver. Just let me know if you want that included here!

[mhalbritter]

For the image name, i've opened #2151.

Since the generator seems to build the return new (DockerImageName.parse(...)) string statically based on the ServiceConnection fields, how would you prefer to handle this?

I have to look deeper into this, likely next week. AFAIK there's no API to do that right now, so we'd have to touch io.spring.start.site.container.ServiceConnections.ServiceConnection and io.spring.start.site.extension.dependency.testcontainers.TestContainersApplicationCodeProjectContributor i guess.

[kaiquevieirasoares]

Thanks for the update, @mhalbritter! That makes perfect sense. I was looking into ServiceConnection earlier and indeed noticed there wasn't a clear way to inject custom environment variables yet.
Take your time looking into it next week! Let me know if you'd like me to help implement those API changes in ServiceConnection and the contributor once you decide on the approach, or if you prefer to handle that core API update on your end. Have a great weekend!

[eddumelendez]

Could be possible to add @Ssl to ElasticsearchContainer instead? and generate something like

@Container
@ServiceConnection
@Ssl
private static ElasticsearchContainer elasticsearch = new ElasticsearchContainer(
		"docker.elastic.co/elasticsearch/elasticsearch:9.2.3");

[kaiquevieirasoares]

Hi @eddumelendez and @mhalbritter! Just wanted to share a quick update before the weekend.

I ran a local PoC using the @Ssl annotation with the 9.3.3 image, and fired a real ElasticsearchClient.ping() over the network to enforce the connection. It works perfectly! The test passes beautifully without needing to disable xpack.security. This is definitely the right architectural path.

Since we confirmed earlier that the current ServiceConnection generation API doesn't yet support injecting custom annotations, I'll leave this on standby. Let me know once the core API enhancements are in place (or if you decide on a different approach), and I'll gladly update the PR to generate this exact configuration.

Have a great weekend, everyone!

[mhalbritter]

Since the bug in Boot is now fixed, we need to add infra for the @Ssl annotation. I've created #2162 for that.

[kaiquevieirasoares]

Just an update on this: I've submitted a Pull Request to enhance the ServiceConnection infrastructure.

Instead of a narrow fix, I've implemented support for parameterized annotations across Java, Kotlin, and Groovy. This not only unblocks the @Ssl requirement for Elasticsearch but also makes the infrastructure 'future-proof' for any other service that might need custom annotation attributes later on.

Once the foundation PR is merged, I'll follow up with the specific implementation for Elasticsearch.

[mhalbritter]

The ServiceConnection PR is now merged. Can you update this PR to add the necessary annotations?

[kaiquevieirasoares]

Awesome! I'll get to work on updating this PR to use the new annotation infrastructure right away.

[kaiquevieirasoares]

Hi, @mhalbritter ,

I’ve updated the PR with the final logic and tests.

I’ve decided to keep the @ssl annotation enabled for the entire Spring Boot 4.x branch (isBoot4). During testing, I noticed that version 4.0.6 still has some internal limitations with Elasticsearch 9, but since @ssl is the expected architectural standard for the 4.x line, I believe it’s best to provide the correct code generation. The integration works perfectly starting with version 4.0.7.

All tests passed, covering both Boot 3.x (without annotation) and Boot 4.x (with annotation) scenarios.

If you have any questions, I’m here.