|
| 1 | +== Camel Kafka with OAuth authentication example |
| 2 | + |
| 3 | +=== Introduction |
| 4 | + |
| 5 | +An example which shows how to integrate https://camel.apache.org/components/next/kafka-component.html[Camel with Kafka] with OAuth authentication using a client secret. The authentication is handled by Keycloak. |
| 6 | + |
| 7 | +This example requires docker-compose as it will build and run a keycloak and kafka broker (setup with kraft mode). |
| 8 | + |
| 9 | +On the Kafka side it uses https://github.com/strimzi/strimzi-kafka-oauth[Strimzi Oauth for Apache Kafka], this library must also be set on the client side. |
| 10 | + |
| 11 | +The Kafka Oauth client side configuration is set in the `src/main/resources/application.properties`. You may want to learn from the Strimzi OAuth project the numerous configurations to have it working with your Kafka Broker, for example you may want to use OAuth Refresh tokens or use JWT tokens. |
| 12 | + |
| 13 | +=== The Kafka Oauth configuration |
| 14 | + |
| 15 | +The configuration is in `src/main/resources/application.properties`, you are welcome to learn more from the https://kafka.apache.org/documentation/#security[Kafka Security] and https://github.com/strimzi/strimzi-kafka-oauth[Strimzi OAuth] documentations. |
| 16 | +---- |
| 17 | +camel.component.kafka.security-protocol = SASL_PLAINTEXT |
| 18 | +camel.component.kafka.sasl-mechanism = OAUTHBEARER |
| 19 | +camel.component.kafka.sasl-jaas-config = org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required \ |
| 20 | + oauth.client.id="kafka-producer-client" \ |
| 21 | + oauth.client.secret="kafka-producer-client-secret" \ |
| 22 | + oauth.username.claim="preferred_username" \ |
| 23 | + oauth.ssl.truststore.location="docker/certificates/ca-truststore.p12" \ |
| 24 | + oauth.ssl.truststore.type="pkcs12" \ |
| 25 | + oauth.ssl.truststore.password="changeit" \ |
| 26 | + oauth.token.endpoint.uri="https://keycloak:8443/realms/demo/protocol/openid-connect/token" ; |
| 27 | +camel.component.kafka.additional-properties[sasl.login.callback.handler.class]=io.strimzi.kafka.oauth.client.JaasClientOauthLoginCallbackHandler |
| 28 | +---- |
| 29 | + |
| 30 | +=== Building and running Keycloak and Kafka |
| 31 | + |
| 32 | +* Set the hosts in /etc/hosts |
| 33 | + |
| 34 | +We have to set the IP addresses in /etc/hosts (check your OS how to do it), verify the current IP address and correctly set it as the example shows. |
| 35 | +---- |
| 36 | +192.168.0.104 keycloak |
| 37 | +192.168.0.104 kafka |
| 38 | +---- |
| 39 | + |
| 40 | +* Build the project |
| 41 | + |
| 42 | +This command will download the required Strimzi OAuth libraries for Kafka and add it to the Kafka Broker directory, it will also build the camel spring boot example. |
| 43 | +---- |
| 44 | +mvn package |
| 45 | +---- |
| 46 | + |
| 47 | +* Launch the Keycloak server |
| 48 | + |
| 49 | +---- |
| 50 | +cd docker |
| 51 | +docker-compose -f keycloak/compose.yml up |
| 52 | +---- |
| 53 | + |
| 54 | +It must show the `demo` realm was imported successfully. |
| 55 | +---- |
| 56 | +[org.keycloak.exportimport.dir.DirImportProvider] (main) Importing from directory /opt/keycloak/bin/../data/import |
| 57 | +[org.keycloak.exportimport.util.ImportUtils] (main) Realm 'demo' imported |
| 58 | +[org.keycloak.services] (main) KC-SERVICES0032: Import finished successfully |
| 59 | +---- |
| 60 | + |
| 61 | +It also shows the server started. |
| 62 | +---- |
| 63 | +[io.quarkus] (main) Keycloak 26.0.8 on JVM (powered by Quarkus 3.15.1) started in 9.169s. Listening on: http://0.0.0.0:8080 and https://0.0.0.0:8443 |
| 64 | +[io.quarkus] (main) Profile prod activated. |
| 65 | +[io.quarkus] (main) Installed features: [agroal, cdi, hibernate-orm, jdbc-h2, keycloak, narayana-jta, opentelemetry, reactive-routes, rest, rest-jackson, smallrye-context-propagation, vertx] |
| 66 | +---- |
| 67 | + |
| 68 | +* Launch the Kafka broker |
| 69 | + |
| 70 | +Open another terminal console and go to the docker directory: |
| 71 | +---- |
| 72 | +cd docker |
| 73 | +docker-compose -f kafka-oauth-strimzi/compose.yml up --build |
| 74 | +---- |
| 75 | + |
| 76 | +It should show the kafka broker authenticated to the keycloak server using the kafka-broker client id. |
| 77 | +---- |
| 78 | +loginWithClientSecret() - tokenEndpointUrl: http://keycloak:8080/realms/demo/protocol/openid-connect/token, clientId: kafka-broker, clientSecret: k*********, scope: null, audience: null, connectTimeout: 20, readTimeout |
| 79 | +: 60, retries: 0, retryPauseMillis: 0 (io.strimzi.kafka.oauth.common.OAuthAuthenticator) |
| 80 | +---- |
| 81 | + |
| 82 | +It should show the kafka broker started |
| 83 | +---- |
| 84 | +Kafka version: 3.9.0 (org.apache.kafka.common.utils.AppInfoParser) |
| 85 | +
|
| 86 | +[KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer) |
| 87 | +---- |
| 88 | + |
| 89 | +=== Run the camel example |
| 90 | + |
| 91 | +As the project was already built, it's ready to run: |
| 92 | + |
| 93 | +---- |
| 94 | +mvn spring-boot:run |
| 95 | +---- |
| 96 | + |
| 97 | +It should display the kafka OAuth settings, example: |
| 98 | +---- |
| 99 | +sasl.login.callback.handler.class = class io.strimzi.kafka.oauth.client.JaasClientOauthLoginCallbackHandler |
| 100 | +sasl.mechanism = OAUTHBEARER |
| 101 | +security.protocol = SASL_PLAINTEXT |
| 102 | +---- |
| 103 | + |
| 104 | +It should show the producer message and the consumer message. |
| 105 | +---- |
| 106 | +[a_Topic1]] route1 : >> Message sent: Hi from Camel - Wed Jan 15 12:11:42 WET 2025 |
| 107 | +[a_Topic1]] route2 : << Message received: Hi from Camel - Wed Jan 15 12:11:42 WET 2025 |
| 108 | +---- |
| 109 | + |
| 110 | +The kafka broker log should display the OAuth logging. |
| 111 | +---- |
| 112 | +DEBUG Set validated token on callback: BearerTokenWithPayloadImpl (principalName: service-account-kafka-producer-client, groups: null, lifetimeMs: 1736978965000 [2025-01-15T22:09:25 UTC], startTimeMs: 1736942965000 [2025-01-15T12:09:25 UTC], scope: [profile, email], payload: {"exp":1736978965,"iat":1736942965,"jti":"43781656-a432-47f5-b0ae-c44e3224bb2b","iss":"https://keycloak:8443/realms/demo","sub":"f288b7db-a3e4-4cf4-80d3-2e5118bb2c9c","typ":"Bearer","azp":"kafka-producer-client","acr":"1","scope":"email profile","email_verified":false,"clientHost":"192.168.0.104","preferred_username":"service-account-kafka-producer-client","clientAddress":"192.168.0.104","email":"service-account-kafka-producer-client@placeholder.org","client_id":"kafka-producer-client"}, sessionId: 1893424185) (io.strimzi.kafka.oauth.server.JaasServerOauthValidatorCallbackHandler) |
| 113 | +---- |
| 114 | + |
| 115 | +Press `Ctrl-C` to exit. |
| 116 | + |
| 117 | +=== Help and contributions |
| 118 | + |
| 119 | +If you hit any problem using Camel or have some feedback, |
| 120 | +then please https://camel.apache.org/community/support/[let us know]. |
| 121 | + |
| 122 | +We also love contributors, |
| 123 | +so https://camel.apache.org/community/contributing/[get involved] :-) |
| 124 | + |
| 125 | +The Camel riders! |
| 126 | + |
0 commit comments