From 0e63c95730d768575ea7e0f8a2c11bf9215b201b Mon Sep 17 00:00:00 2001 From: Matthias Wessendorf Date: Thu, 23 May 2024 16:12:58 +0200 Subject: [PATCH 1/9] :camel: Adding apache camel k blog post Signed-off-by: Matthias Wessendorf --- blog/config/nav.yml | 1 + .../articles/knative-meets-apache-camel.md | 96 +++++++++++++++++++ 2 files changed, 97 insertions(+) create mode 100644 blog/docs/articles/knative-meets-apache-camel.md diff --git a/blog/config/nav.yml b/blog/config/nav.yml index 3c8acfc6e76..91b1de7b1a8 100644 --- a/blog/config/nav.yml +++ b/blog/config/nav.yml @@ -47,6 +47,7 @@ nav: - releases/announcing-knative-v0-3-release.md - releases/announcing-knative-v0-2-release.md - Articles: + - articles/knative-meets-apache-camel.md - articles/knative-backstage-plugins.md - articles/demystifying-activator-on-path.md - articles/knative-eventing-vision.md diff --git a/blog/docs/articles/knative-meets-apache-camel.md b/blog/docs/articles/knative-meets-apache-camel.md new file mode 100644 index 00000000000..6e22e24acf0 --- /dev/null +++ b/blog/docs/articles/knative-meets-apache-camel.md @@ -0,0 +1,96 @@ +# Event Sourcing with Apache Camel K and Knative Eventing + +**Author: Matthias Weßendorf, Senior Principal Software Engineer @ Red Hat** + +## Why Apache Camel K? + +The [Apache Camel](https://camel.apache.org/){:target="_blank"} is a popular Open Source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data. With [Apache Camel K](https://camel.apache.org/camel-k/latest){:target="_blank"} the project provides a lightweight integration framework built from Apache Camel that runs natively on Kubernetes and is specifically designed for serverless and microservice architectures. + +The Camel K framework does also support Knative, allowing developers to [bind](https://camel.apache.org/camel-k/latest/kamelets/kamelets-user.html#kamelets-usage-binding){:target="_blank"} any Kamelet to a Knative component. A Kamelet can act as "source" of data or alternatively as "sink". There are several Kamelets available for integrating and connecting to 3rd party services or products, such as Amazon Web Services (AWS), Google Cloud or even tradition message systems like AMQP 1.0 or JMS brokers like Apache Artemis. The full list of Kamelets can be found [here](https://camel.apache.org/camel-kamelets/latest/index.html){:target="_blank"}. + +## Installation + +The [Installation](https://camel.apache.org/camel-k/next/installation/installation.html) from Apache Camel K offers a few choices, such as CLI, Kustomize, OLM or Helm, like: + +``` +$ helm repo add camel-k https://apache.github.io/camel-k/charts/ +$ helm install my-camel-k camel-k/camel-k +``` + +Beside Camel K we also need to have Knative Eventing installed, as discussed [here](https://knative.dev/docs/install/yaml-install/eventing/install-eventing-with-yaml/){:target="_blank"}. + +## Creating a Knative Broker instance + +We are using a Knative Broker as the heart of our system, acting as an [Event Mesh](https://knative.dev/docs/eventing/event-mesh/){:target="_blank"} for both event producers and event consumers: + +```yaml +apiVersion: eventing.knative.dev/v1 +kind: Broker +metadata: + namespace: default + name: demo-broker +``` + +Now event producers can send events to it and event consumers can receive events. + +## Using Kamelets as Event Sources + +In order to bind a Kamelet to a Knative component, like the above broker, we are using the `Pipe` API. A Pipe allows to declaratively move data from a system described by a Kamelet _towards_ a Knative destination **or** _from_ a Knative destination to another (external) system described by a Kamelet. + +Below is a `Pipe` that uses a ready-to-use `Kamelet`, the `timer-source` + +```yaml +apiVersion: camel.apache.org/v1 +kind: Pipe +metadata: + name: beer-source-pipe +spec: + source: + ref: + kind: Kamelet + apiVersion: camel.apache.org/v1 + name: timer-source + properties: + message: Hello Knative Eventing! + sink: + properties: + cloudEventsType: com.corp.my.timer.source + ref: + kind: Broker + apiVersion: eventing.knative.dev/v1 + name: demo-broker +``` + +The `timer-source` Kamelet is referenced as the `source` of the `Pipe` and sends periodically (default is `1000ms`) the value of its `message` property to the outbound `sink`. Here we use the Knative Broker, which accepts CloudEvents. The conversion of the message payload to CloudEvents format is done by Apache Camel for us. On the `sink` we can also define the `type` of the CloudEvent to be send. + +## Using Kamelets as Event Consumers + +In order to consume message from the Knative broker, using Apache Camel K we need a different `Pipe` where the above broker acts as the source of events and a Kamelet is used as sink to receive the CloudEvents: + +```yaml +apiVersion: camel.apache.org/v1 +kind: Pipe +metadata: + name: log-sink-pipe +spec: + source: + ref: + kind: Broker + apiVersion: eventing.knative.dev/v1 + name: demo-broker + properties: + type: com.corp.my.timer.source + sink: + ref: + kind: Kamelet + apiVersion: camel.apache.org/v1 + name: log-sink +``` + +The `demo-broker` is referenced as the `source` of the `Pipe` and within the `properties` we define which CloudEvent `type` we are interested in. On a matching CloudEvent, the event is routed to the referenced `sink`. In this example we are using a simple `log-sink` Kamelet, which will just print the received data on its standard out log. + +> NOTE: In order for the above to work, the Apache Camel K operator will indeed create a Knative `Trigger` from the `Pipe` data, where the `spec.broker` will match our `demo-broker` and the `spec.filter.attributes.types` field will be set to `com.corp.my.timer.source` to ensure only matching CloudEvent types are being forwarded. + +## Conclusion + +With Apache Camel K the Knative Eventing ecosystem benefits from a huge number of predefined `Kamelet`s for integration with a lot of services and products. Sending events from Google Cloud to AWS is possible. Knative Eventing acts as the heart of the routing, with the Knative `Broker` and `Trigger` APIs as the Event Mesh for your Kubernetes cluster! From 9880ac01c2ed2e6ec2b88130b0f975f1859af170 Mon Sep 17 00:00:00 2001 From: Matthias Wessendorf Date: Mon, 27 May 2024 13:23:28 +0200 Subject: [PATCH 2/9] Update blog/docs/articles/knative-meets-apache-camel.md Co-authored-by: David Simansky --- blog/docs/articles/knative-meets-apache-camel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/docs/articles/knative-meets-apache-camel.md b/blog/docs/articles/knative-meets-apache-camel.md index 6e22e24acf0..aef69ea3851 100644 --- a/blog/docs/articles/knative-meets-apache-camel.md +++ b/blog/docs/articles/knative-meets-apache-camel.md @@ -6,7 +6,7 @@ The [Apache Camel](https://camel.apache.org/){:target="_blank"} is a popular Open Source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data. With [Apache Camel K](https://camel.apache.org/camel-k/latest){:target="_blank"} the project provides a lightweight integration framework built from Apache Camel that runs natively on Kubernetes and is specifically designed for serverless and microservice architectures. -The Camel K framework does also support Knative, allowing developers to [bind](https://camel.apache.org/camel-k/latest/kamelets/kamelets-user.html#kamelets-usage-binding){:target="_blank"} any Kamelet to a Knative component. A Kamelet can act as "source" of data or alternatively as "sink". There are several Kamelets available for integrating and connecting to 3rd party services or products, such as Amazon Web Services (AWS), Google Cloud or even tradition message systems like AMQP 1.0 or JMS brokers like Apache Artemis. The full list of Kamelets can be found [here](https://camel.apache.org/camel-kamelets/latest/index.html){:target="_blank"}. +The Camel K framework does also support Knative, allowing developers to [bind](https://camel.apache.org/camel-k/latest/kamelets/kamelets-user.html#kamelets-usage-binding){:target="_blank"} any Kamelet to a Knative component. A Kamelet can act as "source" of data or alternatively as "sink". There are several Kamelets available for integrating and connecting to 3rd party services or products, such as Amazon Web Services (AWS), Google Cloud or even tradition message systems like AMQP 1.0 or JMS brokers like Apache Artemis. The full list of Kamelets can be found in the [documentation](https://camel.apache.org/camel-kamelets/latest/index.html){:target="_blank"}. ## Installation From 0f9e937d053156a608db63b15b188667e3764d30 Mon Sep 17 00:00:00 2001 From: Matthias Wessendorf Date: Mon, 27 May 2024 13:23:35 +0200 Subject: [PATCH 3/9] Update blog/docs/articles/knative-meets-apache-camel.md Co-authored-by: David Simansky --- blog/docs/articles/knative-meets-apache-camel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/docs/articles/knative-meets-apache-camel.md b/blog/docs/articles/knative-meets-apache-camel.md index aef69ea3851..1713e9ad2e7 100644 --- a/blog/docs/articles/knative-meets-apache-camel.md +++ b/blog/docs/articles/knative-meets-apache-camel.md @@ -37,7 +37,7 @@ Now event producers can send events to it and event consumers can receive events In order to bind a Kamelet to a Knative component, like the above broker, we are using the `Pipe` API. A Pipe allows to declaratively move data from a system described by a Kamelet _towards_ a Knative destination **or** _from_ a Knative destination to another (external) system described by a Kamelet. -Below is a `Pipe` that uses a ready-to-use `Kamelet`, the `timer-source` +Below is a `Pipe` that uses a ready-to-use `Kamelet`, a `timer-source` ```yaml apiVersion: camel.apache.org/v1 From 65eb4fc8505ad737e795f9e05f80d2532347915a Mon Sep 17 00:00:00 2001 From: Matthias Wessendorf Date: Mon, 27 May 2024 13:23:48 +0200 Subject: [PATCH 4/9] Update blog/docs/articles/knative-meets-apache-camel.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Christoph Stäbler --- blog/docs/articles/knative-meets-apache-camel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/docs/articles/knative-meets-apache-camel.md b/blog/docs/articles/knative-meets-apache-camel.md index 1713e9ad2e7..e48d7f413a7 100644 --- a/blog/docs/articles/knative-meets-apache-camel.md +++ b/blog/docs/articles/knative-meets-apache-camel.md @@ -65,7 +65,7 @@ The `timer-source` Kamelet is referenced as the `source` of the `Pipe` and sends ## Using Kamelets as Event Consumers -In order to consume message from the Knative broker, using Apache Camel K we need a different `Pipe` where the above broker acts as the source of events and a Kamelet is used as sink to receive the CloudEvents: +In order to consume messages from the Knative broker, using Apache Camel K, we need a different `Pipe` where the above Broker acts as the source of events and a Kamelet is used as sink to receive the CloudEvents: ```yaml apiVersion: camel.apache.org/v1 From bf0863c16e2a13688c5820f6c3ff92f6ffe6ed48 Mon Sep 17 00:00:00 2001 From: Matthias Wessendorf Date: Mon, 27 May 2024 13:23:55 +0200 Subject: [PATCH 5/9] Update blog/docs/articles/knative-meets-apache-camel.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Christoph Stäbler --- blog/docs/articles/knative-meets-apache-camel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/docs/articles/knative-meets-apache-camel.md b/blog/docs/articles/knative-meets-apache-camel.md index e48d7f413a7..d33c3fd5674 100644 --- a/blog/docs/articles/knative-meets-apache-camel.md +++ b/blog/docs/articles/knative-meets-apache-camel.md @@ -43,7 +43,7 @@ Below is a `Pipe` that uses a ready-to-use `Kamelet`, a `timer-source` apiVersion: camel.apache.org/v1 kind: Pipe metadata: - name: beer-source-pipe + name: timer-source-pipe spec: source: ref: From 64b4b7ecfcbab85eab3c8854bb5d90e4f1ca8e36 Mon Sep 17 00:00:00 2001 From: Matthias Wessendorf Date: Mon, 27 May 2024 13:24:03 +0200 Subject: [PATCH 6/9] Update blog/docs/articles/knative-meets-apache-camel.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Christoph Stäbler --- blog/docs/articles/knative-meets-apache-camel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/docs/articles/knative-meets-apache-camel.md b/blog/docs/articles/knative-meets-apache-camel.md index d33c3fd5674..cb5467eee5d 100644 --- a/blog/docs/articles/knative-meets-apache-camel.md +++ b/blog/docs/articles/knative-meets-apache-camel.md @@ -17,7 +17,7 @@ $ helm repo add camel-k https://apache.github.io/camel-k/charts/ $ helm install my-camel-k camel-k/camel-k ``` -Beside Camel K we also need to have Knative Eventing installed, as discussed [here](https://knative.dev/docs/install/yaml-install/eventing/install-eventing-with-yaml/){:target="_blank"}. +Besides Camel K we also need to have Knative Eventing installed, as described in the [documentation](https://knative.dev/docs/install/yaml-install/eventing/install-eventing-with-yaml/){:target="_blank"}. ## Creating a Knative Broker instance From f26aabdc0dda88e14c57f4d1e0599fa4111bd5f5 Mon Sep 17 00:00:00 2001 From: Matthias Wessendorf Date: Mon, 27 May 2024 14:31:58 +0200 Subject: [PATCH 7/9] Addressing feedback Signed-off-by: Matthias Wessendorf --- blog/docs/articles/knative-meets-apache-camel.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/blog/docs/articles/knative-meets-apache-camel.md b/blog/docs/articles/knative-meets-apache-camel.md index cb5467eee5d..eebb5f24bfc 100644 --- a/blog/docs/articles/knative-meets-apache-camel.md +++ b/blog/docs/articles/knative-meets-apache-camel.md @@ -6,11 +6,11 @@ The [Apache Camel](https://camel.apache.org/){:target="_blank"} is a popular Open Source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data. With [Apache Camel K](https://camel.apache.org/camel-k/latest){:target="_blank"} the project provides a lightweight integration framework built from Apache Camel that runs natively on Kubernetes and is specifically designed for serverless and microservice architectures. -The Camel K framework does also support Knative, allowing developers to [bind](https://camel.apache.org/camel-k/latest/kamelets/kamelets-user.html#kamelets-usage-binding){:target="_blank"} any Kamelet to a Knative component. A Kamelet can act as "source" of data or alternatively as "sink". There are several Kamelets available for integrating and connecting to 3rd party services or products, such as Amazon Web Services (AWS), Google Cloud or even tradition message systems like AMQP 1.0 or JMS brokers like Apache Artemis. The full list of Kamelets can be found in the [documentation](https://camel.apache.org/camel-kamelets/latest/index.html){:target="_blank"}. +The Camel K framework also supports Knative, allowing developers to [bind](https://camel.apache.org/camel-k/latest/kamelets/kamelets-user.html#kamelets-usage-binding){:target="_blank"} any Kamelet to a Knative component. A Kamelet can act as "source" of data or alternatively as "sink". There are several Kamelets available for integrating and connecting to 3rd party services or products, such as Amazon Web Services (AWS), Google Cloud or even tradition message systems like AMQP 1.0 or JMS brokers like Apache Artemis. The full list of Kamelets can be found in the [documentation](https://camel.apache.org/camel-kamelets/latest/index.html){:target="_blank"}. ## Installation -The [Installation](https://camel.apache.org/camel-k/next/installation/installation.html) from Apache Camel K offers a few choices, such as CLI, Kustomize, OLM or Helm, like: +The [Installation](https://camel.apache.org/camel-k/next/installation/installation.html) from Apache Camel K offers a few choices, such as CLI, Kustomize, OLM or Helm. Example of Helm installation: ``` $ helm repo add camel-k https://apache.github.io/camel-k/charts/ @@ -89,8 +89,11 @@ spec: The `demo-broker` is referenced as the `source` of the `Pipe` and within the `properties` we define which CloudEvent `type` we are interested in. On a matching CloudEvent, the event is routed to the referenced `sink`. In this example we are using a simple `log-sink` Kamelet, which will just print the received data on its standard out log. -> NOTE: In order for the above to work, the Apache Camel K operator will indeed create a Knative `Trigger` from the `Pipe` data, where the `spec.broker` will match our `demo-broker` and the `spec.filter.attributes.types` field will be set to `com.corp.my.timer.source` to ensure only matching CloudEvent types are being forwarded. + +!!! note + + In order for the above to work, the Apache Camel K operator will indeed create a Knative `Trigger` from the `Pipe` data, where the `spec.broker` will match our `demo-broker` and the `spec.filter.attributes.types` field will be set to `com.corp.my.timer.source` to ensure only matching CloudEvent types are being forwarded. ## Conclusion -With Apache Camel K the Knative Eventing ecosystem benefits from a huge number of predefined `Kamelet`s for integration with a lot of services and products. Sending events from Google Cloud to AWS is possible. Knative Eventing acts as the heart of the routing, with the Knative `Broker` and `Trigger` APIs as the Event Mesh for your Kubernetes cluster! +With Apache Camel K the Knative Eventing ecosystem benefits from a huge number of predefined `Kamelet`s for integration with a lot of services and products. Sending events from Google Cloud to AWS is possible. Knative Eventing acts as the heart of the routing, with the Knative Broker and Trigger APIs as the Event Mesh for your Kubernetes cluster! From bca817fe349dfc988e7a9c2ca1ff515c92ed1f63 Mon Sep 17 00:00:00 2001 From: Matthias Wessendorf Date: Mon, 27 May 2024 15:28:15 +0200 Subject: [PATCH 8/9] Update blog/docs/articles/knative-meets-apache-camel.md Co-authored-by: David Simansky --- blog/docs/articles/knative-meets-apache-camel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blog/docs/articles/knative-meets-apache-camel.md b/blog/docs/articles/knative-meets-apache-camel.md index eebb5f24bfc..972d22510dd 100644 --- a/blog/docs/articles/knative-meets-apache-camel.md +++ b/blog/docs/articles/knative-meets-apache-camel.md @@ -96,4 +96,4 @@ The `demo-broker` is referenced as the `source` of the `Pipe` and within the `pr ## Conclusion -With Apache Camel K the Knative Eventing ecosystem benefits from a huge number of predefined `Kamelet`s for integration with a lot of services and products. Sending events from Google Cloud to AWS is possible. Knative Eventing acts as the heart of the routing, with the Knative Broker and Trigger APIs as the Event Mesh for your Kubernetes cluster! +With Apache Camel K the Knative Eventing ecosystem benefits from a huge number of predefined Kamelets for integration with a lot of services and products. Sending events from Google Cloud to AWS is possible. Knative Eventing acts as the heart of the routing, with the Knative Broker and Trigger APIs as the Event Mesh for your Kubernetes cluster! From 9d94e0cfe95f42fd4b6913d2e12d0b324ac173d0 Mon Sep 17 00:00:00 2001 From: Matthias Wessendorf Date: Mon, 27 May 2024 15:44:39 +0200 Subject: [PATCH 9/9] :tada: adding the post to be a featured post Signed-off-by: Matthias Wessendorf --- blog/docs/index.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/blog/docs/index.md b/blog/docs/index.md index ce254caf0a4..d784132826b 100644 --- a/blog/docs/index.md +++ b/blog/docs/index.md @@ -27,6 +27,11 @@ Details on the 1.14 release of the Knative project. [Read more :octicons-arrow-right-24:](releases/announcing-knative-v1-14-release.md){ .md-button } +### Knative meets Apache Camel K! +Event Sourcing with Apache Camel K and Knative Eventing. + +[Read more :octicons-arrow-right-24:](articles/knative-meets-apache-camel.md){ .md-button } + ### Knative Completes Third-Party Security Audit A third-party audit by Ada Logics found a small number of issues, including [one CVE](https://github.com/knative/serving/security/advisories/GHSA-qmvj-4qr9-v547).