Kafka-UI does not correctly support configuring clusters via environment variables (binding failures)

[Heniland]

Issue submitter TODO list

• I've looked up my issue in FAQ
• I've searched for an already existing issues here
• I've tried running main-labeled docker image and the issue still persists there
• I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

Kafka-UI does not correctly support configuring clusters purely through environment variables.
When providing cluster connection information (name, bootstrapServers, security.protocol, sasl.jaas.config, etc.) via environment variables, the application fails to start due to Spring Boot binding errors.

This severely limits the ability to use Kafka-UI in cloud-native deployments where secrets and configs are dynamically injected.

Expected behavior

Kafka-UI should fully support defining clusters via environment variables, matching the documented environment variable naming conventions.

It should allow overriding individual cluster fields (such as sasl.jaas.config) via environment variables, merging them cleanly with any static configuration provided.

Your installation details

installed with helm version 1.4.11

kafka:
      clusters:
        - name: "kafka-prod"
          bootstrapServers: "boot-...amazonaws.com:9092"
        - name: "kafka-dev"
          bootstrapServers: "b-1....amazonaws.com:9092"
        - name: "kafka-dev-hanoch-test"
          bootstrapServers: "b-1...:9096"
          properties:
            security.protocol: "SASL_SSL"
            sasl.mechanism: "SCRAM-SHA-512"
            sasl.jaas.config: >
            org.apache.kafka.common.security.scram.ScramLoginModule required username="username" password="password";
        - name: "kafka-dev-hanoch-test-unauthenticated"
          bootstrapServers: "b-1...amazonaws.com:9092"
    auth:
      type: OAUTH2
      oauth2:
        client:
          azure:
            scope: openid
            client-name: azure
            provider: azure
            redirect-uri: "https://..."
            issuer-uri: "https://login.../v2.0"
            jwk-set-uri: "https://.../discovery/v2.0/keys"
    management:
      health:
        ldap:
          enabled: false

• All the above static configration is working. the problem is only when i pass ENV values

Steps to reproduce

1. Deploy Kafka-UI with only environment variables or a mix (static + environment overrides).
2. Set environment variables according to documentation.
3. Start the application.
4. Observe binding failure and application crash.

I used [AWS Secrets Manager] -> [ExternalSecret] -> [Kafka-UI Pod] (This step works, I added it here for completeness)

NOTE: I used ENV like "KAFKA_CLUSTERS_4_" because i already have a static configuration and they all are working. the motivation for ENV is to pass the password from ExternalSecret.

Screenshots

No response

Logs

 _   _ ___    __             _                _          _  __      __ _
| | | |_ _|  / _|___ _ _    /_\  _ __ __ _ __| |_  ___  | |/ /__ _ / _| |_____
| |_| || |  |  _/ _ | '_|  / _ \| '_ / _` / _| ' \/ -_) | ' </ _` |  _| / / _`|
 \___/|___| |_| \___|_|   /_/ \_| .__\__,_\__|_||_\___| |_|\_\__,_|_| |_\_\__,|
                                 |_|
2025-04-28 12:45:47,974 INFO  [main] i.k.u.KafkaUiApplication: Starting KafkaUiApplication v371be00 using Java 21.0.6 with PID 1 (/api.jar started by kafkaui in /)
2025-04-28 12:45:48,004 DEBUG [main] i.k.u.KafkaUiApplication: Running with Spring Boot v3.4.4, Spring v6.2.5
2025-04-28 12:45:48,005 INFO  [main] i.k.u.KafkaUiApplication: No active profile set, falling back to 1 default profile: "default"
2025-04-28 12:45:53,609 WARN  [main] o.s.b.w.r.c.AnnotationConfigReactiveWebServerApplicationContext: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.boot.context.properties.ConfigurationPropertiesBindException: Error creating bean with name 'clustersProperties': Could not bind properties to 'ClustersProperties' : prefix=kafka, ignoreInvalidFields=false, ignoreUnknownFields=true
2025-04-28 12:45:53,641 INFO  [main] o.s.b.a.l.ConditionEvaluationReportLogger:
Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2025-04-28 12:45:53,725 ERROR [main] o.s.b.d.LoggingFailureAnalysisReporter:
***************************
APPLICATION FAILED TO START
***************************
Description:
Binding to target [Bindable@7433ca19 type = java.util.List<io.kafbat.ui.config.ClustersProperties$Cluster>, value = 'provided', annotations = array<Annotation>[[empty]], bindMethod = [null]] failed:
    Property: kafka.clusters[4].bootstrapservers
    Value: "b-1...amazonaws.com:9096"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_BOOTSTRAPSERVERS"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].name
    Value: "kafka-dev-hanoch-as"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_NAME"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.sasl.jaas.config
    Value: "'org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin-secret\";'"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SASL_JAAS_CONFIG"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.sasl.mechanism
    Value: "PLAIN"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SASL_MECHANISM"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.security.protocol
    Value: "SASL_PLAINTEXT"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SECURITY_PROTOCOL"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
Action:
Update your application's configuration

Additional context

Spring Boot v3.4.4
Spring v6.2.5

[github-actions]

Hi Heniland! 👋

Welcome, and thank you for opening your first issue in the repo!

Please wait for triaging by our maintainers.

As development is carried out in our spare time, you can support us by sponsoring our activities or even funding the development of specific issues.
Sponsorship link

If you plan to raise a PR for this issue, please take a look at our contributing guide.

[Haarolean]

It should allow overriding individual cluster fields (such as sasl.jaas.config) via environment variables

How do you see this possible?
Cluster properties are defined like this:,kafka.clusters.0.property. Overriding them requires to know which cluster is which element of an array is in runtime, which is impossible as the order is not guaranteed.

[Heniland]

@Haarolean Thank you. my question has two part.

1 - Only ENV
which i expected to be fully supported as i can see it in this example here

KAFKA_CLUSTERS_4_NAME: local
KAFKA_CLUSTERS_4_BOOTSTRAPSERVERS: kafka:29092
KAFKA_CLUSTERS_4_PROPERTIES_SECURITY_PROTOCOL: SASL_PLAINTEXT
KAFKA_CLUSTERS_4_PROPERTIES_SASL_MECHANISM: PLAIN
KAFKA_CLUSTERS_4_PROPERTIES_SASL_JAAS_CONFIG: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";'

Is this possible?

2 - MIXED
What if we add an id to it and use the id in env?
example.
static file

 - name: "kafka-dev-hanoch-test"
  id: 10    <-- IF WE ADD SOMETHING LIKE THIS
  bootstrapServers: "kafka:29092"

  properties:
    security.protocol: "SASL_SSL"
    sasl.mechanism: "SCRAM-SHA-512"

and use it on ENV (like ID_10)

KAFKA_CLUSTERS_ID_10_PROPERTIES_SASL_JAAS_CONFIG: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";'

[Haarolean]

1. Yes, it's possible, should work fine.
2. We use spring under the hood, which has its own mechanisms for property loading and stuff, overriding this behavior is unfathomable tbh. Why should this be an app's issue, rather than helm charts' one? You can define your configs in a config map, add placeholders, and override them via secret injection or something.

[Heniland]

Okay let's drop 2 and thank you for the explanation.

BUT i have a problem with 1 too

No mater what i put in the ENV, I always get the following error.

Binding to target [Bindable@42cc420b type = java.util.List<io.kafbat.ui.config.ClustersProperties$Cluster>, value = 'provided', annotations = array<Annotation>[[empty]], bindMethod = [null]] failed:

take a look at this full output from the above example we talked

 _   _ ___    __             _                _          _  __      __ _
| | | |_ _|  / _|___ _ _    /_\  _ __ __ _ __| |_  ___  | |/ /__ _ / _| |_____
| |_| || |  |  _/ _ | '_|  / _ \| '_ / _` / _| ' \/ -_) | ' </ _` |  _| / / _`|
 \___/|___| |_| \___|_|   /_/ \_| .__\__,_\__|_||_\___| |_|\_\__,_|_| |_\_\__,|
                                 |_|
2025-04-28 14:14:36,745 INFO  [main] i.k.u.KafkaUiApplication: Starting KafkaUiApplication v371be00 using Java 21.0.6 with PID 1 (/api.jar started by kafkaui in /)
2025-04-28 14:14:36,748 DEBUG [main] i.k.u.KafkaUiApplication: Running with Spring Boot v3.4.4, Spring v6.2.5
2025-04-28 14:14:36,750 INFO  [main] i.k.u.KafkaUiApplication: No active profile set, falling back to 1 default profile: "default"
2025-04-28 14:14:39,376 WARN  [main] o.s.b.w.r.c.AnnotationConfigReactiveWebServerApplicationContext: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.boot.context.properties.ConfigurationPropertiesBindException: Error creating bean with name 'clustersProperties': Could not bind properties to 'ClustersProperties' : prefix=kafka, ignoreInvalidFields=false, ignoreUnknownFields=true
2025-04-28 14:14:39,401 INFO  [main] o.s.b.a.l.ConditionEvaluationReportLogger:
Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2025-04-28 14:14:39,435 ERROR [main] o.s.b.d.LoggingFailureAnalysisReporter:
***************************
APPLICATION FAILED TO START
***************************
Description:
Binding to target [Bindable@42cc420b type = java.util.List<io.kafbat.ui.config.ClustersProperties$Cluster>, value = 'provided', annotations = array<Annotation>[[empty]], bindMethod = [null]] failed:
    Property: kafka.clusters[4].bootstrapservers
    Value: "b-1....amazonaws.com:9096,b-2....amazonaws.com:9096,b-3....amazonaws.com:9096"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_BOOTSTRAPSERVERS"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].name
    Value: "local"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_NAME"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.sasl.jaas.config
    Value: "org.apache.kafka.common.security.plain.PlainLoginModule required username="username" password="password"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SASL_JAAS_CONFIG"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.sasl.mechanism
    Value: "PLAIN"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SASL_MECHANISM"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.security.protocol
    Value: "SASL_PLAINTEXT"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SECURITY_PROTOCOL"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
Action:
Update your application's configuration

[Haarolean]

Can you try a config from here?
https://github.com/kafbat/kafka-ui/blob/main/documentation/compose/ui-sasl.yaml

[Heniland]

@Haarolean First of all, I really appreciate your quick response. thank you for that.

To give you the full picture: I'm not using Docker Compose; instead, I have a managed AWS cluster setup.

Cluster: AWS MSK (version 3.6.1)
Kafka UI: Helm (version 1.4.11)

Regarding the link you shared, I didn’t notice any configuration differences compared to what I already had. Could you please point me to something specific to try?

For completeness, even with DYNAMIC_CONFIG_ENABLED: true, I still encounter exactly the same error.

[Haarolean]

My guess is that since this config works with docker compose, this should work elsewhere => meaning your env vars are probably borked. Can you output env from your container instance? Please don't forget to remove secrets and stuff.

[Heniland]

$ env
AUTH_OAUTH2_CLIENT_AZURE_CLIENTSECRET=redacted_value
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT=tcp://172.20.0.1:443
LANGUAGE=en_US:en
HOSTNAME=kafbat-ui-kafka-ui-6fb75fb449-qdjmh
SHLVL=1
HOME=/home/kafkaui
AUTH_TYPE=OAUTH2
KAFKA_CLUSTERS_4_PROPERTIES_SASL_MECHANISM=PLAIN
KAFKA_CLUSTERS_4_PROPERTIES_SASL_JAAS_CONFIG='org.apache.kafka.common.security.scram.ScramLoginModule required username="username" password="password";'
SERVER_SERVLET_CONTEXT_PATH=/
SERVER_FORWARDHEADERSSTRATEGY=native
SPRING_CONFIG_ADDITIONAL-LOCATION=/kafka-ui/config.yml
KAFBAT_UI_KAFKA_UI_SERVICE_PORT_HTTP=8080
TERM=xterm
KUBERNETES_PORT_443_TCP_ADDR=172.20.0.1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KUBERNETES_PORT_443_TCP_PORT=443
JAVA_OPTS=
KUBERNETES_PORT_443_TCP_PROTO=tcp
LANG=en_US.UTF-8
KAFBAT_UI_KAFKA_UI_SERVICE_HOST=172.20.245.39
KAFBAT_UI_KAFKA_UI_PORT_8080_TCP_ADDR=172.20.245.39
KAFKA_CLUSTERS_4_PROPERTIES_SECURITY_PROTOCOL=SASL_PLAINTEXT
KAFKA_CLUSTERS_4_BOOTSTRAPSERVERS=b-1......amazonaws.com:9096,b-2......amazonaws.com:9096,b-3......amazonaws.com:9096
KAFBAT_UI_KAFKA_UI_PORT_8080_TCP_PORT=8080
KAFBAT_UI_KAFKA_UI_PORT_8080_TCP_PROTO=tcp
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP=tcp://172.20.0.1:443
KAFBAT_UI_KAFKA_UI_SERVICE_PORT=8080
KAFBAT_UI_KAFKA_UI_PORT=tcp://172.20.245.39:8080
LC_ALL=en_US.UTF-8
JAVA_HOME=/usr/lib/jvm/zulu21
DYNAMIC_CONFIG_ENABLED=true
KUBERNETES_SERVICE_HOST=172.20.0.1
PWD=/
SERVER_PORT=8080
TZ=Etc/UTC
MANAGEMENT_HEALTH_LDAP_ENABLED=FALSE
KAFKA_CLUSTERS_4_NAME=test-cluster
KAFBAT_UI_KAFKA_UI_PORT_8080_TCP=tcp://172.20.245.39:8080
AUTH_OAUTH2_CLIENT_AZURE_CLIENTID=redacted_value

[Haarolean]

@Heniland can you try KAFKA_CLUSTERS_0_xxx rather than KAFKA_CLUSTERS_4_xxx? I think it's not fond of you starting the list like this.