[CI] FileSettingsServiceIT testSettingsAppliedOnStart failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic-platform-support #9458 / almalinux-8-aarch64_checkpart1_platform-support-arm
• elasticsearch-periodic-platform-support #9430 / almalinux-8-aarch64_checkpart1_platform-support-arm
• elasticsearch-periodic-platform-support #9367 / almalinux-8-aarch64_checkpart1_platform-support-arm

Reproduction Line:

./gradlew ":server:internalClusterTest" --tests "org.elasticsearch.reservedstate.service.FileSettingsServiceIT.testSettingsAppliedOnStart" -Dtests.seed=6F05AE11A1C290C9 -Dtests.locale=pt-PT -Dtests.timezone=Europe/Kyiv -Druntime.java=24

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

junit.framework.AssertionFailedError: Expected exception ExecutionException but no exception was thrown

Issue Reasons:

• [main] 3 failures in test testSettingsAppliedOnStart (0.3% fail rate in 1000 executions)
• [main] 3 failures in step almalinux-8-aarch64_checkpart1_platform-support-arm (21.4% fail rate in 14 executions)
• [main] 3 failures in pipeline elasticsearch-periodic-platform-support (21.4% fail rate in 14 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[ldematte]

I've investigated this briefly: the test fails when it tries to check that a cluster settings update does fail when it tries to update a file setting (read-only); so it may not be related to files settings, but to cluster settings. The test also seems to be failing very rarely, which does not help to track down which PR introduced the fault.

According to the dashboard, this started to fail on Monday (13/07). It definitely does not look like a test failure, so I'm assigning a medium priority, which we could raise if we discover this is a new bug.