Helm Charts for Data Provision and Data Consumption infrastructure, with DTR, 2x EDCs, Data Storage Service

deployment/data-consumer/.helmignore

@@ -0,0 +1,47 @@
+# #############################################################################
+# Copyright (c) 2022,2024 Bayerische Motoren Werke Aktiengesellschaft (BMW AG)
+# Copyright (c) 2021,2024 Contributors to the Eclipse Foundation
+#
+# See the NOTICE file(s) distributed with this work for additional
+# information regarding copyright ownership.
+#
+# This program and the accompanying materials are made available under the
+# terms of the Apache License, Version 2.0 which is available at
+# https://www.apache.org/licenses/LICENSE-2.0.
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# #############################################################################
+
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/
+
+# Ignore ENV specific values at release
+values?*.yaml
+values?*.yml

deployment/data-consumer/Chart.yaml

@@ -0,0 +1,37 @@
+# #############################################################################
+# Copyright (c) 2022,2024 Bayerische Motoren Werke Aktiengesellschaft (BMW AG)
+# Copyright (c) 2021,2024 Contributors to the Eclipse Foundation
+#
+# See the NOTICE file(s) distributed with this work for additional
+# information regarding copyright ownership.
+#
+# This program and the accompanying materials are made available under the
+# terms of the Apache License, Version 2.0 which is available at
+# https://www.apache.org/licenses/LICENSE-2.0.
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# #############################################################################
+---
+apiVersion: v2
+name: tx-data-provider
+description: A Helm chart for Kubernetes
+
+type: application
+version: 0.1.3
+appVersion: 0.0.1
+
+dependencies:
+  - name: tractusx-connector
+    repository: https://eclipse-tractusx.github.io/charts/dev
+    version: 0.8.0
+    condition: tractusx-connector.enabled
+  - name: vault
+    version: 0.20.0
+    repository: https://helm.releases.hashicorp.com
+    condition: vault.enabled

deployment/data-consumer/README.md

@@ -0,0 +1,49 @@
+# Dataprovider Helm Chart
+
+This chart includes EDC, Digital Twin Registry and a Submodel Server.  
+The Submodel Server images is based on an older Catena-X demo (catenax at home) since this is what we were using in our
+testing environments.
+
+
+-> values-int.yaml -> in 
+
+## Prerequisites
+
+- Running Kubernetes cluster
+- Helm is installed
+
+Example for local usage:
+
+- [Minikube](https://minikube.sigs.k8s.io/docs/start/)
+- [Minikube ingress addon](https://minikube.sigs.k8s.io/docs/handbook/addons/ingress-dns/)
+
+## Installing
+
+Run the Chart with
+
+```shell
+helm dependency update
+helm install dataprovider . --timeout 10m0s
+```
+
+Remove the chart by running
+
+```shell
+helm uninstall dataprovider
+```
+## Configuration
+
+For Configuration of digital twin registry see [sldt-digital-twin-registry/tree/main/charts/registry](https://github.com/eclipse-tractusx/sldt-digital-twin-registry/tree/main/charts/registry)
+
+Configuration of EDC see [tractusx-edc/tree/main/charts/tractusx-connecto](https://github.com/eclipse-tractusx/tractusx-edc/tree/main/charts/tractusx-connector)
+
+EDC requires connection to HashiCorp Vault and Manage Identity Wallet. These have to be configured for the dataprovider to be operative. 
+
+The Chart can be used as dataprovider by setting `simple-data-backend.enabled`, `digital-twin-registry.enabled` and `seedTestdata` to `false`.
+
+## Testdata seeding
+
+After the installation, a Post-Install Helm Hook will be started which initiates the seeding of testdata. The Hook executes a python script which uploads a provided test dataset to the dataprovider. Documentation to the python script can be found here [README](https://github.com/eclipse-tractusx/item-relationship-service/blob/main/local/testing/testdata/README.md)
+
+Test data set and upload script are stored in [resources/](resources) and provided to the hook as config map.  
+A custom config map can be used to provided e.g. `testdataConfigMap: my-custom-testdata-configmap`.

deployment/data-consumer/templates/_helpers.tpl

@@ -0,0 +1,133 @@
+{{/*
+*******************************************************************************
+ * Copyright (c) 2022,2024 Bayerische Motoren Werke Aktiengesellschaft (BMW AG)
+ * Copyright (c) 2021,2024 Contributors to the Eclipse Foundation
+ *
+ * See the NOTICE file(s) distributed with this work for additional
+ * information regarding copyright ownership.
+ *
+ * This program and the accompanying materials are made available under the
+ * terms of the Apache License, Version 2.0 which is available at
+ * https://www.apache.org/licenses/LICENSE-2.0.
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ *******************************************************************************
+ */}}
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "dataprovider.fullname" -}}
+    {{- if .Values.fullnameOverride }}
+        {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
+    {{- else }}
+        {{- $name := default .Chart.Name .Values.nameOverride }}
+        {{- if contains $name .Release.Name }}
+            {{- .Release.Name | trunc 63 | trimSuffix "-" }}
+        {{- else }}
+            {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+        {{- end }}
+    {{- end }}
+{{- end }}
+
+{{/*
+Submodel URL helpers
+*/}}
+{{- define "simple-data-backend.host" -}}
+    {{- if .Values.backendUrl }}
+        {{- tpl .Values.backendUrl . }}
+    {{- else if (index .Values "simple-data-backend" "ingress" "enabled") }}
+        {{- with (first (index .Values "simple-data-backend" "ingress" "hosts")) }}
+            {{- printf "https://%s" .host }}
+        {{- end }}
+    {{- else }}
+        {{- printf "http://%s%s:8080" .Release.Name "-simple-data-backend" }}
+    {{- end }}
+{{- end }}
+
+{{/*
+Registry URL helpers
+*/}}
+{{- define "registry.host" -}}
+    {{- if index .Values "digital-twin-registry" "registry" "ingress" "enabled" }}
+        {{- printf "https://%s" (index .Values "digital-twin-registry" "registry" "host") }}
+    {{- else }}
+        {{- printf "http://%s-%s:8080" .Release.Name "digital-twin-registry" }}
+    {{- end }}
+{{- end }}
+{{- define "registry.path" -}}
+    {{- if index .Values "digital-twin-registry" "registry" "ingress" "enabled" }}
+        {{- index .Values "digital-twin-registry" "registry" "ingress" "urlPrefix" }}
+    {{- else }}
+    {{- print "" }}
+    {{- end }}
+{{- end }}
+{{- define "registry.url" -}}
+    {{- if .Values.registryUrl }}
+        {{- tpl .Values.registryUrl . }}
+    {{ else }}
+        {{- printf "%s%s%s" (include "registry.host" .) (include "registry.path" .) "/api/v3" }}
+    {{- end }}
+{{- end }}
+
+{{/*
+EDC URL helpers
+*/}}
+
+{{- define "edc.controlplane.host" -}}
+    {{- if .Values.controlplanePublicUrl }}
+        {{- tpl .Values.controlplanePublicUrl . }}
+    {{ else }}
+        {{- with (first (index .Values "tractusx-connector" "controlplane" "ingresses")) }}
+            {{- if .enabled }}
+                {{- printf "https://%s" .hostname }}
+            {{- else }}
+                {{- printf "http://%s-%s:8084" $.Release.Name "tractusx-connector-controlplane" }}
+            {{- end }}
+        {{- end }}
+    {{ end  }}
+{{- end }}
+
+{{- define "edc.controlplane.management.host" -}}
+    {{- if .Values.controlplaneManagementUrl }}
+        {{- tpl .Values.controlplaneManagementUrl . }}
+    {{ else }}
+        {{- with (first (index .Values "tractusx-connector" "controlplane" "ingresses")) }}
+            {{- if .enabled }}
+                {{- printf "https://%s" .hostname }}
+            {{- else }}
+                {{- printf "http://%s-%s:8081" $.Release.Name "tractusx-connector-controlplane" }}
+            {{- end }}
+        {{- end }}
+    {{ end  }}
+{{- end }}
+
+{{- define "edc.dataplane.host" -}}
+    {{- if .Values.dataplaneUrl }}
+        {{- tpl .Values.dataplaneUrl . }}
+    {{ else }}
+        {{- with (first (index .Values "tractusx-connector" "dataplane" "ingresses")) }}
+            {{- if .enabled }}
+                {{- printf "https://%s" .hostname }}
+            {{- else }}
+                {{- printf "http://%s-%s:8081" $.Release.Name "tractusx-connector-dataplane" }}
+            {{- end }}
+        {{- end }}
+    {{ end  }}
+{{- end }}
+
+{{- define "edc.key" -}}
+    {{- index .Values "tractusx-connector" "controlplane" "endpoints" "management" "authKey" }}
+{{- end }}
+
+{{- define "edc.bpn" -}}
+    {{- index .Values "tractusx-connector" "participant" "id" }}
+{{- end }}
+

deployment/data-consumer/templates/post-install-vault-setup.yaml

@@ -0,0 +1,79 @@
+# #############################################################################
+# Copyright (c) 2021,2024 Contributors to the Eclipse Foundation
+#
+# See the NOTICE file(s) distributed with this work for additional
+# information regarding copyright ownership.
+#
+# This program and the accompanying materials are made available under the
+# terms of the Apache License, Version 2.0 which is available at
+# https://www.apache.org/licenses/LICENSE-2.0.
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# #############################################################################
+---
+{{ if .Values.vault.enabled }}
+{{- $vaultToken := index .Values "tractusx-connector" "vault" "hashicorp" "token" -}}
+{{- $vaultUrl := tpl (index .Values "tractusx-connector" "vault" "hashicorp" "url") . -}}
+{{- $fullName := include "dataprovider.fullname" . -}}
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ $fullName }}-post-install-vault-setup
+  labels:
+    app.kubernetes.io/managed-by: {{ .Release.Service | quote }}
+    app.kubernetes.io/instance: {{ .Release.Name | quote }}
+    app.kubernetes.io/version: {{ .Chart.AppVersion }}
+    helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+  annotations:
+    # This is what defines this resource as a hook. Without this line, the
+    # job is considered part of the release.
+    "helm.sh/hook": post-install,post-upgrade
+    "helm.sh/hook-weight": "-5"
+    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
+spec:
+  template:
+    metadata:
+      name: "{{ .Release.Name }}"
+      labels:
+        app.kubernetes.io/managed-by: {{ .Release.Service | quote }}
+        app.kubernetes.io/instance: {{ .Release.Name | quote }}
+        helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+    spec:
+      restartPolicy: Never
+      containers:
+        - name: post-install-job
+          image: "alpine:3.19"
+          imagePullPolicy: "IfNotPresent"
+          command:
+            - "/bin/sh"
+            - "-c"
+            - |
+              sleep 10
+              {{- range $key, $value := .Values.secrets }}
+              wget --header 'Content-Type: application/json' --header 'X-Vault-Token: {{ $vaultToken }}' \
+              --post-data '{"data": {"content": "{{ $value }}"}}' "{{ $vaultUrl }}/v1/secret/data/{{ $key }}"
+              {{- end }}
+
+              wget --header 'Content-Type: application/json' --header 'X-Vault-Token: {{ $vaultToken }}' \
+              --post-file=/opt/config/cert.json "{{ $vaultUrl }}/v1/secret/data/tokenSignerPublicKey"
+
+              wget --header 'Content-Type: application/json' --header 'X-Vault-Token: {{ $vaultToken }}' \
+              --post-file=/opt/config/key.json "{{ $vaultUrl }}/v1/secret/data/tokenSignerPrivateKey"
+
+              wget --header 'Content-Type: application/json' --header 'X-Vault-Token: {{ $vaultToken }}' \
+              --post-file=/opt/config/aes-secret.json "{{ $vaultUrl }}/v1/secret/data/tokenEncryptionAesKey"
+          volumeMounts:
+            - name: config-volume
+              mountPath: /opt/config
+      volumes:
+        - name: config-volume
+          configMap:
+            name: {{ $fullName }}-vault-edc-configmap
+            defaultMode: 0777
+{{ end }}

deployment/data-consumer/templates/vault-edc-configmap.yaml

@@ -0,0 +1,44 @@
+# #############################################################################
+# Copyright (c) 2021,2024 Contributors to the Eclipse Foundation
+#
+# See the NOTICE file(s) distributed with this work for additional
+# information regarding copyright ownership.
+#
+# This program and the accompanying materials are made available under the
+# terms of the Apache License, Version 2.0 which is available at
+# https://www.apache.org/licenses/LICENSE-2.0.
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# #############################################################################
+---
+{{ $fullName := include "dataprovider.fullname" . -}}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ $fullName }}-vault-edc-configmap
+data:
+  cert.json: |-
+      {
+        "data": {
+          "content": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsP1Wl50viKzVqw8HGFWP\nis+M8Im4daTFMned5Qr2z90FNgkj1EVhip0mOdD6kDg3bW4RxyL6z3jWi19JKBHZ\n68UTgZNdPbhhPNLEGcQpu8uwgFcWKL4P/IOykEeE8ResGOVg/HzNE7HkTgiBdr2C\nMTEXL3zTmdr0vbFGMbOTPyOvKMoy/2FaJaJAPXo3poGqfRvr6Gu6top2ktRd/z8N\nhBpuzx9QypIsE62ooLNPpqzjezfvzJbc3tko/cXNOyGoZWuEKMTfKwYq3ZjsTZKk\nbjVNgdoEFSWpd6Tqk76B1Cboxv1CD8xw0cQ149eW0IuLkAk3eF5eZR4iQ5YAOZqA\nUQIDAQAB\n-----END PUBLIC KEY-----\n"
+        }
+      }
+
+  key.json: |-
+    {
+      "data": {
+        "content": "-----BEGIN PRIVATE KEY-----\nMIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQCw/VaXnS+IrNWr\nDwcYVY+Kz4zwibh1pMUyd53lCvbP3QU2CSPURWGKnSY50PqQODdtbhHHIvrPeNaL\nX0koEdnrxROBk109uGE80sQZxCm7y7CAVxYovg/8g7KQR4TxF6wY5WD8fM0TseRO\nCIF2vYIxMRcvfNOZ2vS9sUYxs5M/I68oyjL/YVolokA9ejemgap9G+voa7q2inaS\n1F3/Pw2EGm7PH1DKkiwTraigs0+mrON7N+/Mltze2Sj9xc07Iahla4QoxN8rBird\nmOxNkqRuNU2B2gQVJal3pOqTvoHUJujG/UIPzHDRxDXj15bQi4uQCTd4Xl5lHiJD\nlgA5moBRAgMBAAECggEAKD8XjYb8G+WHeexDJgSwzTUonLsIg9H52KHMORz+5mIh\nUPoPmHHFfj6BhoSvsZNjAUKWDtU0uPCGwu8iRNcYWa15I841lfcjP3BDEQPjJJXr\nNyf2fUHJA1gURwxIXgWOyCOC5C9h9/BMFPWIsQ5jeFmsJsuJF5OrcyZIar1lxqWu\nQ+HC7f/7JNkpR26uIyGjs1OXwfp+mHqze2Qf8hLWIXcN9tBCQZ75Cg7rarNVimMC\n59QD80JZCHTaCX1ZtE1T8HM+53Ob78lnFCuBfiBT/S3O/NXVsEN9q6rMWKhETVWR\nUX56EqZ7XGSMOzuZyK7kj1QsHzEMrrHjwDSNSjAqFQKBgQC6hEeAWPCEM+WVoF9n\nmhvwZVZv/PPyLAarykBTGoeHR2hqNyih9JmcXL+XQHMlhy1Ka8NtJHvfyB1xhXgF\n/d91i/Yq02+nZoJPNnVWo8zoXIAIq+xg9CBiu0agBxv45PjJkEkQmmEG4Iej1+Kf\n5/+dI7sFjE2T4q/lLK0Aw3x+zQKBgQDy7Ho7eRi5CV1Ks+r5lpGGdM15hbE5tviE\nfmJvaEUh3oWuwdkyFjD/QEPITG1bFuvhATdEEWxTbuswNEbELLhKCPcixcI0sLUO\n6BeNi1YD6ouuqsWBLvmE4hvoDR7RlkpUduxWpZ2tNIDJYYTwCERhcYK9OtHU15kc\nlS0pEjF/lQKBgEh+28/OQgYQqd7ji9GX+94PdW5n0mXBqQIixafHewAgyDvonpl8\nmixFfI6MlXTzuq3ffwEwGhncDV2vc/xYNf/ZW+A/eHmHhYTGdQss9ZsnQPid4m24\n1dGqWwQeX0f5r52gwFV8u9PRd8c+RS7EHP12At5gL1MY3CdmmwPd98jNAoGASvPV\n/xWtICKYi10aCip/+kl9wJoUhadD5LWOL6uvcPTUsIgVONQKCCfPAjU6pJlc7E7Q\nu4rYrqGRpYzrrMnTjtxXTH5SHqnLI69O3Rh50LmEob8FM4fH601MqPurX6WMh6Ut\n5Moy7Wc+uWQCfYE/gAVi/nnwlkhzcJNCnOKFLUUCgYA4nbhLrdEeCBJas2+Z9aGw\n/SurtHF2z2meAPO3VaYslf0D0xKzUIv5hkkzaNgs0pFGt0BcKfTWkYy/Hh4QXZhm\n//ZrqbkVsqzDOcqGwQcDmUEN/T+vprUJVKqpNHXT6fz3OTydxLQ7K7SXzwXWYfRW\ncpjdr0c1tVVLMyNG+Wahrg==\n-----END PRIVATE KEY-----\n"
+      }
+    }
+  aes-secret.json: |-
+    {
+      "data": {
+        "content": {{ randAlphaNum 32 | b64enc | quote }}
+      }
+    }