Kafka-UI does not correctly support configuring clusters via environment variables (binding failures) #1045

Heniland · 2025-04-28T13:11:10Z

Issue submitter TODO list

I've looked up my issue in FAQ
I've searched for an already existing issues here
I've tried running main-labeled docker image and the issue still persists there
I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

Kafka-UI does not correctly support configuring clusters purely through environment variables.
When providing cluster connection information (name, bootstrapServers, security.protocol, sasl.jaas.config, etc.) via environment variables, the application fails to start due to Spring Boot binding errors.

This severely limits the ability to use Kafka-UI in cloud-native deployments where secrets and configs are dynamically injected.

Expected behavior

Kafka-UI should fully support defining clusters via environment variables, matching the documented environment variable naming conventions.

It should allow overriding individual cluster fields (such as sasl.jaas.config) via environment variables, merging them cleanly with any static configuration provided.

Your installation details

installed with helm version 1.4.11

kafka:
      clusters:
        - name: "kafka-prod"
          bootstrapServers: "boot-...amazonaws.com:9092"
        - name: "kafka-dev"
          bootstrapServers: "b-1....amazonaws.com:9092"
        - name: "kafka-dev-hanoch-test"
          bootstrapServers: "b-1...:9096"
          properties:
            security.protocol: "SASL_SSL"
            sasl.mechanism: "SCRAM-SHA-512"
            sasl.jaas.config: >
            org.apache.kafka.common.security.scram.ScramLoginModule required username="username" password="password";
        - name: "kafka-dev-hanoch-test-unauthenticated"
          bootstrapServers: "b-1...amazonaws.com:9092"
    auth:
      type: OAUTH2
      oauth2:
        client:
          azure:
            scope: openid
            client-name: azure
            provider: azure
            redirect-uri: "https://..."
            issuer-uri: "https://login.../v2.0"
            jwk-set-uri: "https://.../discovery/v2.0/keys"
    management:
      health:
        ldap:
          enabled: false

All the above static configration is working. the problem is only when i pass ENV values

Steps to reproduce

Deploy Kafka-UI with only environment variables or a mix (static + environment overrides).
Set environment variables according to documentation.
Start the application.
Observe binding failure and application crash.

I used [AWS Secrets Manager] -> [ExternalSecret] -> [Kafka-UI Pod] (This step works, I added it here for completeness)

NOTE: I used ENV like "KAFKA_CLUSTERS_4_" because i already have a static configuration and they all are working. the motivation for ENV is to pass the password from ExternalSecret.

Screenshots

No response

Logs

 _   _ ___    __             _                _          _  __      __ _
| | | |_ _|  / _|___ _ _    /_\  _ __ __ _ __| |_  ___  | |/ /__ _ / _| |_____
| |_| || |  |  _/ _ | '_|  / _ \| '_ / _` / _| ' \/ -_) | ' </ _` |  _| / / _`|
 \___/|___| |_| \___|_|   /_/ \_| .__\__,_\__|_||_\___| |_|\_\__,_|_| |_\_\__,|
                                 |_|
2025-04-28 12:45:47,974 INFO  [main] i.k.u.KafkaUiApplication: Starting KafkaUiApplication v371be00 using Java 21.0.6 with PID 1 (/api.jar started by kafkaui in /)
2025-04-28 12:45:48,004 DEBUG [main] i.k.u.KafkaUiApplication: Running with Spring Boot v3.4.4, Spring v6.2.5
2025-04-28 12:45:48,005 INFO  [main] i.k.u.KafkaUiApplication: No active profile set, falling back to 1 default profile: "default"
2025-04-28 12:45:53,609 WARN  [main] o.s.b.w.r.c.AnnotationConfigReactiveWebServerApplicationContext: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.boot.context.properties.ConfigurationPropertiesBindException: Error creating bean with name 'clustersProperties': Could not bind properties to 'ClustersProperties' : prefix=kafka, ignoreInvalidFields=false, ignoreUnknownFields=true
2025-04-28 12:45:53,641 INFO  [main] o.s.b.a.l.ConditionEvaluationReportLogger:
Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2025-04-28 12:45:53,725 ERROR [main] o.s.b.d.LoggingFailureAnalysisReporter:
***************************
APPLICATION FAILED TO START
***************************
Description:
Binding to target [Bindable@7433ca19 type = java.util.List<io.kafbat.ui.config.ClustersProperties$Cluster>, value = 'provided', annotations = array<Annotation>[[empty]], bindMethod = [null]] failed:
    Property: kafka.clusters[4].bootstrapservers
    Value: "b-1...amazonaws.com:9096"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_BOOTSTRAPSERVERS"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].name
    Value: "kafka-dev-hanoch-as"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_NAME"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.sasl.jaas.config
    Value: "'org.apache.kafka.common.security.plain.PlainLoginModule required username=\"admin\" password=\"admin-secret\";'"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SASL_JAAS_CONFIG"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.sasl.mechanism
    Value: "PLAIN"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SASL_MECHANISM"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.security.protocol
    Value: "SASL_PLAINTEXT"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SECURITY_PROTOCOL"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
Action:
Update your application's configuration

Additional context

Spring Boot v3.4.4
Spring v6.2.5

The text was updated successfully, but these errors were encountered:

github-actions · 2025-04-28T13:11:36Z

Hi Heniland! 👋

Welcome, and thank you for opening your first issue in the repo!

Please wait for triaging by our maintainers.

As development is carried out in our spare time, you can support us by sponsoring our activities or even funding the development of specific issues.
Sponsorship link

If you plan to raise a PR for this issue, please take a look at our contributing guide.

Haarolean · 2025-04-28T13:28:07Z

It should allow overriding individual cluster fields (such as sasl.jaas.config) via environment variables

How do you see this possible?
Cluster properties are defined like this:,kafka.clusters.0.property. Overriding them requires to know which cluster is which element of an array is in runtime, which is impossible as the order is not guaranteed.

Heniland · 2025-04-28T13:48:43Z

@Haarolean Thank you. my question has two part.

1 - Only ENV
which i expected to be fully supported as i can see it in this example here

KAFKA_CLUSTERS_4_NAME: local
KAFKA_CLUSTERS_4_BOOTSTRAPSERVERS: kafka:29092
KAFKA_CLUSTERS_4_PROPERTIES_SECURITY_PROTOCOL: SASL_PLAINTEXT
KAFKA_CLUSTERS_4_PROPERTIES_SASL_MECHANISM: PLAIN
KAFKA_CLUSTERS_4_PROPERTIES_SASL_JAAS_CONFIG: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";'

Is this possible?

2 - MIXED
What if we add an id to it and use the id in env?
example.
static file

 - name: "kafka-dev-hanoch-test"
  id: 10    <-- IF WE ADD SOMETHING LIKE THIS
  bootstrapServers: "kafka:29092"

  properties:
    security.protocol: "SASL_SSL"
    sasl.mechanism: "SCRAM-SHA-512"

and use it on ENV (like ID_10)

KAFKA_CLUSTERS_ID_10_PROPERTIES_SASL_JAAS_CONFIG: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";'

Haarolean · 2025-04-28T13:57:26Z

Yes, it's possible, should work fine.
We use spring under the hood, which has its own mechanisms for property loading and stuff, overriding this behavior is unfathomable tbh. Why should this be an app's issue, rather than helm charts' one? You can define your configs in a config map, add placeholders, and override them via secret injection or something.

Heniland · 2025-04-28T14:21:58Z

Okay let's drop 2 and thank you for the explanation.

BUT i have a problem with 1 too

No mater what i put in the ENV, I always get the following error.

Binding to target [Bindable@42cc420b type = java.util.List<io.kafbat.ui.config.ClustersProperties$Cluster>, value = 'provided', annotations = array<Annotation>[[empty]], bindMethod = [null]] failed:

take a look at this full output from the above example we talked

 _   _ ___    __             _                _          _  __      __ _
| | | |_ _|  / _|___ _ _    /_\  _ __ __ _ __| |_  ___  | |/ /__ _ / _| |_____
| |_| || |  |  _/ _ | '_|  / _ \| '_ / _` / _| ' \/ -_) | ' </ _` |  _| / / _`|
 \___/|___| |_| \___|_|   /_/ \_| .__\__,_\__|_||_\___| |_|\_\__,_|_| |_\_\__,|
                                 |_|
2025-04-28 14:14:36,745 INFO  [main] i.k.u.KafkaUiApplication: Starting KafkaUiApplication v371be00 using Java 21.0.6 with PID 1 (/api.jar started by kafkaui in /)
2025-04-28 14:14:36,748 DEBUG [main] i.k.u.KafkaUiApplication: Running with Spring Boot v3.4.4, Spring v6.2.5
2025-04-28 14:14:36,750 INFO  [main] i.k.u.KafkaUiApplication: No active profile set, falling back to 1 default profile: "default"
2025-04-28 14:14:39,376 WARN  [main] o.s.b.w.r.c.AnnotationConfigReactiveWebServerApplicationContext: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.boot.context.properties.ConfigurationPropertiesBindException: Error creating bean with name 'clustersProperties': Could not bind properties to 'ClustersProperties' : prefix=kafka, ignoreInvalidFields=false, ignoreUnknownFields=true
2025-04-28 14:14:39,401 INFO  [main] o.s.b.a.l.ConditionEvaluationReportLogger:
Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2025-04-28 14:14:39,435 ERROR [main] o.s.b.d.LoggingFailureAnalysisReporter:
***************************
APPLICATION FAILED TO START
***************************
Description:
Binding to target [Bindable@42cc420b type = java.util.List<io.kafbat.ui.config.ClustersProperties$Cluster>, value = 'provided', annotations = array<Annotation>[[empty]], bindMethod = [null]] failed:
    Property: kafka.clusters[4].bootstrapservers
    Value: "b-1....amazonaws.com:9096,b-2....amazonaws.com:9096,b-3....amazonaws.com:9096"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_BOOTSTRAPSERVERS"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].name
    Value: "local"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_NAME"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.sasl.jaas.config
    Value: "org.apache.kafka.common.security.plain.PlainLoginModule required username="username" password="password"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SASL_JAAS_CONFIG"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.sasl.mechanism
    Value: "PLAIN"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SASL_MECHANISM"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
    Property: kafka.clusters[4].properties.security.protocol
    Value: "SASL_PLAINTEXT"
    Origin: System Environment Property "KAFKA_CLUSTERS_4_PROPERTIES_SECURITY_PROTOCOL"
    Reason: The elements [kafka.clusters[4].bootstrapservers,kafka.clusters[4].name,kafka.clusters[4].properties.sasl.jaas.config,kafka.clusters[4].properties.sasl.mechanism,kafka.clusters[4].properties.security.protocol] were left unbound.
Action:
Update your application's configuration

Haarolean · 2025-04-28T15:16:30Z

Can you try a config from here?
https://github.com/kafbat/kafka-ui/blob/main/documentation/compose/ui-sasl.yaml

Heniland · 2025-04-28T15:56:53Z

@Haarolean First of all, I really appreciate your quick response. thank you for that.

To give you the full picture: I'm not using Docker Compose; instead, I have a managed AWS cluster setup.

Cluster: AWS MSK (version 3.6.1)
Kafka UI: Helm (version 1.4.11)

Regarding the link you shared, I didn’t notice any configuration differences compared to what I already had. Could you please point me to something specific to try?

For completeness, even with DYNAMIC_CONFIG_ENABLED: true, I still encounter exactly the same error.

Haarolean · 2025-04-28T15:59:26Z

My guess is that since this config works with docker compose, this should work elsewhere => meaning your env vars are probably borked. Can you output env from your container instance? Please don't forget to remove secrets and stuff.

Heniland · 2025-04-28T16:24:10Z

$ env
AUTH_OAUTH2_CLIENT_AZURE_CLIENTSECRET=redacted_value
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT=tcp://172.20.0.1:443
LANGUAGE=en_US:en
HOSTNAME=kafbat-ui-kafka-ui-6fb75fb449-qdjmh
SHLVL=1
HOME=/home/kafkaui
AUTH_TYPE=OAUTH2
KAFKA_CLUSTERS_4_PROPERTIES_SASL_MECHANISM=PLAIN
KAFKA_CLUSTERS_4_PROPERTIES_SASL_JAAS_CONFIG='org.apache.kafka.common.security.scram.ScramLoginModule required username="username" password="password";'
SERVER_SERVLET_CONTEXT_PATH=/
SERVER_FORWARDHEADERSSTRATEGY=native
SPRING_CONFIG_ADDITIONAL-LOCATION=/kafka-ui/config.yml
KAFBAT_UI_KAFKA_UI_SERVICE_PORT_HTTP=8080
TERM=xterm
KUBERNETES_PORT_443_TCP_ADDR=172.20.0.1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KUBERNETES_PORT_443_TCP_PORT=443
JAVA_OPTS=
KUBERNETES_PORT_443_TCP_PROTO=tcp
LANG=en_US.UTF-8
KAFBAT_UI_KAFKA_UI_SERVICE_HOST=172.20.245.39
KAFBAT_UI_KAFKA_UI_PORT_8080_TCP_ADDR=172.20.245.39
KAFKA_CLUSTERS_4_PROPERTIES_SECURITY_PROTOCOL=SASL_PLAINTEXT
KAFKA_CLUSTERS_4_BOOTSTRAPSERVERS=b-1......amazonaws.com:9096,b-2......amazonaws.com:9096,b-3......amazonaws.com:9096
KAFBAT_UI_KAFKA_UI_PORT_8080_TCP_PORT=8080
KAFBAT_UI_KAFKA_UI_PORT_8080_TCP_PROTO=tcp
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP=tcp://172.20.0.1:443
KAFBAT_UI_KAFKA_UI_SERVICE_PORT=8080
KAFBAT_UI_KAFKA_UI_PORT=tcp://172.20.245.39:8080
LC_ALL=en_US.UTF-8
JAVA_HOME=/usr/lib/jvm/zulu21
DYNAMIC_CONFIG_ENABLED=true
KUBERNETES_SERVICE_HOST=172.20.0.1
PWD=/
SERVER_PORT=8080
TZ=Etc/UTC
MANAGEMENT_HEALTH_LDAP_ENABLED=FALSE
KAFKA_CLUSTERS_4_NAME=test-cluster
KAFBAT_UI_KAFKA_UI_PORT_8080_TCP=tcp://172.20.245.39:8080
AUTH_OAUTH2_CLIENT_AZURE_CLIENTID=redacted_value

Haarolean · 2025-05-13T20:14:35Z

@Heniland can you try KAFKA_CLUSTERS_0_xxx rather than KAFKA_CLUSTERS_4_xxx? I think it's not fond of you starting the list like this.

kapybro · 2025-05-13T20:34:41Z

Further user feedback is requested. Please reply within 7 days or we might close the issue.

kapybro · 2025-05-20T21:28:18Z

No feedback received within 7 days. Auto closing.

Heniland · 2025-05-21T10:46:46Z

@Haarolean It's working now, Thank you

Heniland added status/triage Issues pending maintainers triage type/bug Something isn't working labels Apr 28, 2025

kapybro bot added status/triage/manual Manual triage in progress status/triage/completed Automatic triage completed and removed status/triage Issues pending maintainers triage labels Apr 28, 2025

Haarolean added the status/feedback-requested label May 13, 2025

kapybro bot assigned Heniland May 13, 2025

kapybro bot closed this as completed May 20, 2025

kapybro bot removed the status/feedback-requested label May 20, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Kafka-UI does not correctly support configuring clusters via environment variables (binding failures) #1045

Kafka-UI does not correctly support configuring clusters via environment variables (binding failures) #1045

Heniland commented Apr 28, 2025 •

edited

Loading

github-actions bot commented Apr 28, 2025

Uh oh!

Haarolean commented Apr 28, 2025

Uh oh!

Heniland commented Apr 28, 2025 •

edited

Loading

Uh oh!

Haarolean commented Apr 28, 2025

Uh oh!

Heniland commented Apr 28, 2025 •

edited

Loading

Uh oh!

Haarolean commented Apr 28, 2025

Uh oh!

Heniland commented Apr 28, 2025

Uh oh!

Haarolean commented Apr 28, 2025

Uh oh!

Heniland commented Apr 28, 2025 •

edited

Loading

Uh oh!

Haarolean commented May 13, 2025

Uh oh!

kapybro bot commented May 13, 2025

Uh oh!

kapybro bot commented May 20, 2025

Uh oh!

Heniland commented May 21, 2025

Uh oh!

Uh oh!

Kafka-UI does not correctly support configuring clusters via environment variables (binding failures) #1045

Kafka-UI does not correctly support configuring clusters via environment variables (binding failures) #1045

Comments

Heniland commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue submitter TODO list

Describe the bug (actual behavior)

Expected behavior

Your installation details

Steps to reproduce

Screenshots

Logs

Additional context

github-actions bot commented Apr 28, 2025

Uh oh!

Haarolean commented Apr 28, 2025

Uh oh!

Heniland commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Haarolean commented Apr 28, 2025

Uh oh!

Heniland commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Haarolean commented Apr 28, 2025

Uh oh!

Heniland commented Apr 28, 2025

Uh oh!

Haarolean commented Apr 28, 2025

Uh oh!

Heniland commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Haarolean commented May 13, 2025

Uh oh!

kapybro bot commented May 13, 2025

Uh oh!

kapybro bot commented May 20, 2025

Uh oh!

Heniland commented May 21, 2025

Uh oh!

Heniland commented Apr 28, 2025 •

edited

Loading

Heniland commented Apr 28, 2025 •

edited

Loading

Heniland commented Apr 28, 2025 •

edited

Loading

Heniland commented Apr 28, 2025 •

edited

Loading