diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b8b5427..f0db023 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,76 +2,126 @@ ## Build and generate template yml locally -``` +```bash helm package ./charts/cadence -helm template cadence-release cadence-0.1.8.tgz > template_out.yaml +# Replace with current chart version +helm template cadence-release cadence-0.2.0.tgz > template_out.yaml ``` ## Build and deploy to a k8s cluster -1. Build helm package and deploy to a k8s cluster +### 1. Update dependencies (if needed) +```bash +cd charts/cadence +helm dependency update +cd ../.. ``` + +### 2. Build helm package and deploy to a k8s cluster +```bash helm package ./charts/cadence -helm upgrade --install cadence-release cadence-0.1.8.tgz \ +# Replace with current chart version +helm upgrade --install cadence-release cadence-0.2.0.tgz \ -n cadencetest \ --create-namespace ``` -2. Port forward to check the UI -``` +### 3. Port forward to check the UI +```bash kubectl port-forward svc/cadence-release-web 8088:8088 -n cadencetest ``` -Visit localhost:8088 and validate it is accessible. +Visit http://localhost:8088 and validate it is accessible. -3. Port forward frontend service to run CLI commands -``` +### 4. Port forward frontend service to run CLI commands +```bash kubectl port-forward svc/cadence-release-frontend 7833:7833 -n cadencetest ``` -4. (optional) Register a domain: -``` +### 5. (Optional) Register a domain: +```bash cadence \ --address localhost:7833 \ --transport grpc \ - --domain samples-domain \ + --domain default \ domain register \ --retention 1 ``` -5. Run samples: -- Clone https://github.com/uber-common/cadence-samples +### 6. Run samples: +- Clone https://github.com/cadence-workflow/cadence-samples -- Run sample worker (run at samples repo root) -``` +- Run sample worker (execute at samples repo root): +```bash ./bin/helloworld -m worker ``` -- Trigger a workflow (run at samples repo root) -``` +- Trigger a workflow (execute at samples repo root): +```bash ./bin/helloworld -m trigger ``` -6. Visit localhost:8088 and validate the new workflow exists! +### 7. Validate deployment +Visit http://localhost:8088 and validate the new workflow exists! -## Generate helmdocs +## Generate helm documentation Install [helm-docs](https://github.com/norwoodj/helm-docs): -``` +```bash go install github.com/norwoodj/helm-docs/cmd/helm-docs@latest ``` -Run it -``` +Generate documentation: +```bash helm-docs ``` -cadencechart/README.md file should be updated. +The `charts/cadence/README.md` file should be updated automatically. + +## Release Process + +### Before making a release: + +1. **Update appVersion and dependencies**: + - Check if Cadence has a new release and update `appVersion` in `Chart.yaml` + - Verify dependencies are using latest-1 stable versions: + - Cassandra: Currently using `11.x.x` from Bitnami + - PostgreSQL: Currently using `16.x.x` from Bitnami + - MySQL: Currently using `12.x.x` from Bitnami + - Update `global.image.tag` in `values.yaml` to match `appVersion` + +2. **Update dependencies**: + ```bash + cd charts/cadence + helm dependency update + cd ../.. + ``` + +3. **Test the changes**: + - Build and deploy locally following the steps above + - Validate all functionality works as expected + +4. **Increment chart version**: Update the `version` field in `charts/cadence/Chart.yaml`: + - Patch version (0.2.1): Bug fixes, dependency updates + - Minor version (0.3.0): New features, breaking changes + - Major version (1.0.0): Major breaking changes + +5. **Update documentation**: Run `helm-docs` to regenerate the README.md + +### Publishing: + +After making changes to templates and incrementing the chart version in `charts/cadence/Chart.yaml`, merge your changes to the main branch. + +The automation will handle publishing the new version using [Chart Releaser Action](https://helm.sh/docs/howto/chart_releaser_action/). The Cadence chart is hosted on GitHub Pages. +After the new version is available in the helm repository, deploy it to a Kubernetes cluster to validate the release. -## Publish chart +## Version Management Checklist -After making changes to templates, increment the chart version in charts/cadence/Chart.yaml. -Then merge your changes and automation will take care of publishing the new version. -Cadence chart is hosted on github pages and automation is done using [Chart Releaser Action](https://helm.sh/docs/howto/chart_releaser_action/). -After new version is available in helm repo, deploy it to a K8s cluster to validate. +When updating versions, ensure consistency across: +- [ ] `Chart.yaml`: `version` (chart version) and `appVersion` (Cadence version) +- [ ] `values.yaml`: `global.image.tag` (should match appVersion) +- [ ] Dependencies in `Chart.yaml` (check for updates, aim for latest-1 stable) +- [ ] Update dependencies with `helm dependency update` +- [ ] Test deployment locally +- [ ] Run `helm-docs` to update documentation \ No newline at end of file diff --git a/charts/cadence/templates/.helmignore b/charts/cadence/.helmignore similarity index 95% rename from charts/cadence/templates/.helmignore rename to charts/cadence/.helmignore index 0e8a0eb..b1eeb65 100644 --- a/charts/cadence/templates/.helmignore +++ b/charts/cadence/.helmignore @@ -14,10 +14,11 @@ *.swp *.bak *.tmp -*.orig *~ # Various IDEs .project .idea/ *.tmproj .vscode/ +# Cadence +docs/ \ No newline at end of file diff --git a/charts/cadence/Chart.lock b/charts/cadence/Chart.lock new file mode 100644 index 0000000..66a616e --- /dev/null +++ b/charts/cadence/Chart.lock @@ -0,0 +1,12 @@ +dependencies: +- name: cassandra + repository: oci://registry-1.docker.io/bitnamicharts + version: 11.4.2 +- name: postgresql + repository: oci://registry-1.docker.io/bitnamicharts + version: 16.7.13 +- name: mysql + repository: oci://registry-1.docker.io/bitnamicharts + version: 12.3.5 +digest: sha256:d355eb032d520a4005678225960f11fbce541e4aaea7fce4a7e90ecebcbe1807 +generated: "2025-06-25T16:33:38.605196+02:00" diff --git a/charts/cadence/Chart.yaml b/charts/cadence/Chart.yaml index 28111cf..288146a 100644 --- a/charts/cadence/Chart.yaml +++ b/charts/cadence/Chart.yaml @@ -1,6 +1,6 @@ apiVersion: v2 name: cadence -version: 0.1.9 +version: 0.2.0 appVersion: "1.3.1" description: | @@ -26,3 +26,17 @@ sources: maintainers: - name: Cadence Community url: https://github.com/cadence-workflow/cadence-charts + +dependencies: + - name: cassandra + version: 11.x.x + repository: oci://registry-1.docker.io/bitnamicharts + condition: cassandra.enabled + - name: postgresql + version: 16.x.x + repository: oci://registry-1.docker.io/bitnamicharts + condition: postgresql.enabled + - name: mysql + version: 12.x.x + repository: oci://registry-1.docker.io/bitnamicharts + condition: mysql.enabled \ No newline at end of file diff --git a/charts/cadence/README.md b/charts/cadence/README.md index 2c944da..562be51 100644 --- a/charts/cadence/README.md +++ b/charts/cadence/README.md @@ -1,6 +1,6 @@ # cadence -![Version: 0.1.9](https://img.shields.io/badge/Version-0.1.9-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.3.1](https://img.shields.io/badge/AppVersion-1.3.1-informational?style=flat-square) +![Version: 0.2.0](https://img.shields.io/badge/Version-0.2.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.3.1](https://img.shields.io/badge/AppVersion-1.3.1-informational?style=flat-square) Cadence is a distributed, scalable, durable, and highly available orchestration engine to execute asynchronous long-running business logic in a scalable and resilient way. @@ -33,9 +33,163 @@ This chart deploys Uber Cadence server components and web UI. | cassandra.deployment.image.tag | string | `"4.1.1"` | | | cassandra.deployment.resources | object | `{"limits":{"cpu":1,"memory":"2Gi"},"requests":{"cpu":"500m","memory":"1Gi"}}` | Resource limits and requests for Cassandra | | cassandra.endpoint | string | `""` | External Cassandra endpoint to connect to. Required when cassandra.deployment.enabled is set to false | -| cassandra.schema.version | string | `"0.40"` | Cassandra schema version of the Cadence keyspace to use | -| cassandra.schema.visibility_version | string | `"0.9"` | Cassandra schema version of the Cadence visibility keyspace to use | -| dynamicConfig.values | object | `{"history.workflowIDExternalRateLimitEnabled":[{"value":true}]}` | Dynamic config values to be set in the Cadence server List of keys can be found at https://pkg.go.dev/github.com/uber/cadence@v1.3.0/common/dynamicconfig/dynamicproperties | +| config.archival.history.enableRead | bool | `false` | Enable reading from archives | +| config.archival.history.provider | object | `{"filestore":{"dirMode":"0755","fileMode":"0644"},"gstorage":{"credentialsPath":""},"s3store":{"endpoint":"","region":"","s3ForcePathStyle":false},"type":"filestore"}` | Archive providers configuration | +| config.archival.history.provider.filestore.dirMode | string | `"0755"` | Directory mode for archive directories | +| config.archival.history.provider.filestore.fileMode | string | `"0644"` | File mode for archived files | +| config.archival.history.provider.gstorage.credentialsPath | string | `""` | Path to service account key file | +| config.archival.history.provider.s3store.endpoint | string | `""` | S3 endpoint (for S3-compatible storage) | +| config.archival.history.provider.s3store.region | string | `""` | AWS region | +| config.archival.history.provider.s3store.s3ForcePathStyle | bool | `false` | Force path style URLs | +| config.archival.history.provider.type | string | `"filestore"` | Storage type: filestore, s3, gcs | +| config.archival.history.status | string | `"disabled"` | Archival status: enabled, disabled, paused | +| config.archival.visibility.enableRead | bool | `false` | Enable reading from archives | +| config.archival.visibility.provider | object | `{"filestore":{"dirMode":"0755","fileMode":"0644"},"gstorage":{"credentialsPath":""},"s3store":{"endpoint":"","region":"","s3ForcePathStyle":false},"type":"filestore"}` | Archive providers configuration | +| config.archival.visibility.provider.filestore.dirMode | string | `"0755"` | Directory mode for archive directories | +| config.archival.visibility.provider.filestore.fileMode | string | `"0644"` | File mode for archived files | +| config.archival.visibility.provider.gstorage.credentialsPath | string | `""` | Path to service account key file | +| config.archival.visibility.provider.s3store.endpoint | string | `""` | S3 endpoint (for S3-compatible storage) | +| config.archival.visibility.provider.s3store.region | string | `""` | AWS region | +| config.archival.visibility.provider.s3store.s3ForcePathStyle | bool | `false` | Force path style URLs | +| config.archival.visibility.provider.type | string | `"filestore"` | Storage type: filestore, s3, gcs | +| config.archival.visibility.status | string | `"disabled"` | Archival status: enabled, disabled, paused | +| config.asyncWorkflowQueues.default-queue | object | `{"config":{"cluster":"default","topic":"cadence-async-wf"},"type":"kafka"}` | Async workflow queue providers | +| config.asyncWorkflowQueues.default-queue.config | object | `{"cluster":"default","topic":"cadence-async-wf"}` | Queue configuration | +| config.asyncWorkflowQueues.default-queue.config.cluster | string | `"default"` | Kafka cluster reference | +| config.asyncWorkflowQueues.default-queue.config.topic | string | `"cadence-async-wf"` | Kafka topic for async workflows | +| config.asyncWorkflowQueues.default-queue.type | string | `"kafka"` | Queue type: kafka | +| config.asyncWorkflowQueues.enabled | bool | `false` | Enable async workflow queues | +| config.blobstore.filestore.outputDirectory | string | `"/etc/cadence/blobstore"` | Output directory for blob storage | +| config.cluster.clusterGroup | string | `nil` | Cluster group configuration with additional clusters | +| config.cluster.clusterRedirectionPolicy | object | `{"policy":"noop"}` | Cluster redirection policy for cross-cluster operations | +| config.cluster.clusterRedirectionPolicy.policy | string | `"noop"` | Policy for handling cross-cluster requests (noop, selected-apis-forwarding, all-domain-apis-forwarding, selected-apis-forwarding-v2) | +| config.cluster.currentClusterName | string | `"cluster0"` | Name of the current cluster | +| config.cluster.failoverVersionIncrement | int | `10` | Version increment used during cluster failover operations | +| config.cluster.initialFailoverVersion | int | `0` | Initial failover version for this cluster | +| config.cluster.isNotPrimary | bool | `false` | Whether this cluster is not the primary cluster | +| config.cluster.primaryClusterName | string | `"cluster0"` | Name of the primary cluster in a multi-cluster setup | +| config.cluster.rpcTransport | string | `"grpc"` | RPC transport protocol (grpc or tchannel) | +| config.domainDefaults.archival | object | `{"history":{"URI":"","status":"disabled"},"visibility":{"URI":"","status":"disabled"}}` | Default archival settings for new domains - Documentation for S3 here: https://github.com/cadence-workflow/cadence/blob/v1.3.0/common/archiver/s3store/README.md - Documentation for GStorage here: https://github.com/cadence-workflow/cadence/blob/v1.3.0/common/archiver/gcloud/README.md | +| config.domainDefaults.archival.history.URI | string | `""` | Default history archival URI | +| config.domainDefaults.archival.history.status | string | `"disabled"` | Default history archival status: enabled, disabled | +| config.domainDefaults.archival.visibility.URI | string | `""` | Default visibility archival URI | +| config.domainDefaults.archival.visibility.status | string | `"disabled"` | Default visibility archival status: enabled, disabled | +| config.dynamicConfig.client | string | `"filebased"` | Dynamic config client type: noop, filebased, configstore | +| config.dynamicConfig.filebased.filepath | string | `"/etc/cadence/config/dynamicconfig/config.yaml"` | Path to dynamic config file | +| config.dynamicConfig.filebased.pollInterval | string | `"60s"` | Poll interval for config changes | +| config.kafka.brokers | string | `"kafka-service.kafka-namespace.svc.cluster.local"` | Kafka broker service. Can reference Kubernetes services | +| config.kafka.enabled | bool | `false` | Enable Kafka for async workflows | +| config.kafka.port | int | `9092` | Kafka port | +| config.kafka.sasl.enabled | bool | `false` | Enable SASL authentication | +| config.kafka.sasl.mechanism | string | `"PLAIN"` | SASL mechanism: plain, sha512 or sha256 | +| config.kafka.sasl.password | string | `""` | SASL password | +| config.kafka.sasl.username | string | `""` | SASL username | +| config.kafka.tls.caFile | string | `""` | CA certificate file to verify server certificates | +| config.kafka.tls.caFiles | list | `[]` | Multiple CA certificate files (alternative to caFile) | +| config.kafka.tls.certFile | string | `""` | Client certificate file for mutual TLS | +| config.kafka.tls.enableHostVerification | bool | `true` | Verify server hostname matches certificate | +| config.kafka.tls.enabled | bool | `false` | Enable TLS | +| config.kafka.tls.keyFile | string | `""` | Client private key file for mutual TLS | +| config.kafka.tls.requireClientAuth | bool | `false` | Require client certificate authentication | +| config.kafka.tls.serverName | string | `""` | Override server name for certificate verification | +| config.kafka.topicProperties | object | `{}` | Topic properties (optional) | +| config.kafka.visibilityDLQTopic | string | `"cadence-visibility-dlq"` | Kafka visibility DLQ topic name | +| config.kafka.visibilityTopic | string | `"cadence-visibility"` | Kafka visibility topic name | +| config.log.encoding | string | `"json"` | Log encoding format: json, console (defaults to "json") | +| config.log.level | string | `nil` | Logging level: debug, info, warn, error (inherits from global.log if not specified) | +| config.log.levelKey | string | `"level"` | Log level key name (defaults to "level") | +| config.log.outputFile | string | `""` | Output file path for logging (if stdout is false) | +| config.log.stdout | string | `nil` | Enable stdout logging (inherits from global.log if not specified) | +| config.log.useEnvVars | bool | `false` | Allow using environment variables for log configuration. If enabled, it will use ENV variable of each server component. | +| config.persistence.advancedVisibilityStore | string | `"es-visibility"` | Name of the advanced visibility datastore | +| config.persistence.database.cassandra.allowedAuthenticators | list | `[]` | Allowed authenticators for custom authentication | +| config.persistence.database.cassandra.connectAttributes | object | `{}` | Additional connection attributes | +| config.persistence.database.cassandra.connectTimeout | string | `"10s"` | Connection timeout | +| config.persistence.database.cassandra.consistency | string | `"LOCAL_QUORUM"` | Default consistency level | +| config.persistence.database.cassandra.datacenter | string | `""` | Datacenter filter for Cassandra | +| config.persistence.database.cassandra.hostSelectionPolicy | string | `"tokenaware,roundrobin"` | Host selection policy | +| config.persistence.database.cassandra.hosts | string | `"cassandra-service.cadence.svc.cluster.local"` | Cassandra hosts. Can reference Kubernetes services | +| config.persistence.database.cassandra.keyspace | string | `"cadence"` | Cassandra keyspace for main data | +| config.persistence.database.cassandra.maxConns | int | `10` | Maximum number of connections | +| config.persistence.database.cassandra.password | string | `"cassandra"` | Cassandra password | +| config.persistence.database.cassandra.port | int | `9042` | Cassandra port | +| config.persistence.database.cassandra.protoVersion | int | `4` | Cassandra protocol version | +| config.persistence.database.cassandra.region | string | `""` | AWS region filter for Cassandra (if using AWS Keyspaces) | +| config.persistence.database.cassandra.serialConsistency | string | `"LOCAL_SERIAL"` | Serial consistency level | +| config.persistence.database.cassandra.timeout | string | `"1s"` | Query timeout | +| config.persistence.database.cassandra.tls.caFile | string | `""` | CA certificate file to verify server certificates | +| config.persistence.database.cassandra.tls.caFiles | list | `[]` | Multiple CA certificate files (alternative to caFile) | +| config.persistence.database.cassandra.tls.certFile | string | `""` | Client certificate file for mutual TLS | +| config.persistence.database.cassandra.tls.enableHostVerification | bool | `true` | Verify server hostname matches certificate | +| config.persistence.database.cassandra.tls.enabled | bool | `false` | Enable TLS | +| config.persistence.database.cassandra.tls.keyFile | string | `""` | Client private key file for mutual TLS | +| config.persistence.database.cassandra.tls.requireClientAuth | bool | `false` | Require client certificate authentication | +| config.persistence.database.cassandra.tls.serverName | string | `"cassandra"` | Override server name for certificate verification | +| config.persistence.database.cassandra.user | string | `"cassandra"` | Cassandra username | +| config.persistence.database.cassandra.visibilityKeyspace | string | `"cadence_visibility"` | Cassandra keyspace for visibility data | +| config.persistence.database.driver | string | `"cassandra"` | Database driver: cassandra, mysql, postgres | +| config.persistence.database.mysql.port | int | `3306` | Default port for MySQL (overrides sql.port if specified) | +| config.persistence.database.mysql.txIsolationCompat | bool | `false` | Enable transaction isolation compatibility mode | +| config.persistence.database.postgres.port | int | `5432` | Default port for PostgreSQL (overrides sql.port if specified) | +| config.persistence.database.sql.connectAttributes | object | `{}` | Connection attributes (key-value pairs for connection string) | +| config.persistence.database.sql.dbname | string | `"cadence"` | Database name for main data | +| config.persistence.database.sql.decodingTypes | list | `["thriftrw"]` | Decoding types for SQL blobs | +| config.persistence.database.sql.encodingType | string | `"thriftrw"` | Encoding type for SQL blobs | +| config.persistence.database.sql.hosts | string | `"mysql-service.mysql-namespace.svc.cluster.local"` | Database host. Can reference Kubernetes services | +| config.persistence.database.sql.maxConnLifetime | string | `"1h"` | Maximum connection lifetime | +| config.persistence.database.sql.maxConns | int | `20` | Maximum number of connections | +| config.persistence.database.sql.maxIdleConns | int | `20` | Maximum number of idle connections | +| config.persistence.database.sql.multipleDatabasesConfig | list | `[]` | Multiple databases configuration (when useMultipleDatabases is true) | +| config.persistence.database.sql.numShards | int | `1` | Number of database shards (default: 1) | +| config.persistence.database.sql.password | string | `""` | Database password | +| config.persistence.database.sql.port | string | `nil` | Database port (will use driver default if not specified) | +| config.persistence.database.sql.tls.caFile | string | `""` | Path to CA certificate file | +| config.persistence.database.sql.tls.caFiles | list | `[]` | Multiple CA certificate files | +| config.persistence.database.sql.tls.certFile | string | `""` | Path to client certificate file | +| config.persistence.database.sql.tls.enableHostVerification | bool | `true` | Enable hostname verification (inverse of skipHostVerification) | +| config.persistence.database.sql.tls.enabled | bool | `false` | Enable TLS | +| config.persistence.database.sql.tls.keyFile | string | `""` | Path to client private key file | +| config.persistence.database.sql.tls.requireClientAuth | bool | `false` | Require client authentication for mutual TLS | +| config.persistence.database.sql.tls.serverName | string | `""` | Server name for certificate verification | +| config.persistence.database.sql.tls.sslMode | string | `""` | For MySQL: false, true, skip-verify, preferred. (Additional this should work: required, verify-ca, verify-identity) | +| config.persistence.database.sql.useMultipleDatabases | bool | `false` | Use multiple databases for sharding | +| config.persistence.database.sql.user | string | `"cadence"` | Database username | +| config.persistence.database.sql.visibilityDbname | string | `"cadence_visibility"` | Database name for visibility data | +| config.persistence.defaultStore | string | `"default"` | Name of the default datastore | +| config.persistence.elasticsearch.awsSigning | object | `{"enabled":false,"region":"","service":"es"}` | Enable AWS signing (for AWS Elasticsearch) | +| config.persistence.elasticsearch.enabled | bool | `false` | Enable Elasticsearch for advanced visibility | +| config.persistence.elasticsearch.hosts | string | `"elasticsearch-service.elastic-namespace.svc.cluster.local"` | Elasticsearch host. | +| config.persistence.elasticsearch.password | string | `""` | Elasticsearch password | +| config.persistence.elasticsearch.port | int | `9200` | Elasticsearch port | +| config.persistence.elasticsearch.protocol | string | `""` | Protocol to use (http/https). If not specified, auto-detected based on TLS settings | +| config.persistence.elasticsearch.tls.caFile | string | `""` | CA certificate file to verify server certificates | +| config.persistence.elasticsearch.tls.caFiles | list | `[]` | Multiple CA certificate files (alternative to caFile) | +| config.persistence.elasticsearch.tls.certFile | string | `""` | Client certificate file for mutual TLS | +| config.persistence.elasticsearch.tls.enableHostVerification | bool | `true` | Verify server hostname matches certificate | +| config.persistence.elasticsearch.tls.enabled | bool | `false` | Enable TLS | +| config.persistence.elasticsearch.tls.keyFile | string | `""` | Client private key file for mutual TLS | +| config.persistence.elasticsearch.tls.requireClientAuth | bool | `false` | Require client certificate authentication | +| config.persistence.elasticsearch.tls.serverName | string | `""` | Override server name for certificate verification | +| config.persistence.elasticsearch.user | string | `""` | Elasticsearch username | +| config.persistence.elasticsearch.version | string | `"v7"` | Elasticsearch version (v6, use v7 for v7 or higher) | +| config.persistence.elasticsearch.visibilityIndex | string | `"cadence-visibility"` | Elasticsearch visibility index name | +| config.persistence.enablePersistenceLatencyHistogramMetrics | bool | `false` | Enable persistence latency histogram metrics | +| config.persistence.numHistoryShards | int | `4` | Number of history shards for partitioning (CANNOT BE CHANGED ONCE SET) | +| config.persistence.visibilityStore | string | `"visibility"` | Name of the visibility datastore (basic visibility) | +| config.publicClient.hostPort | string | `""` | Frontend service address (defaults to current cluster's RPC address) | +| config.publicClient.refreshInterval | string | `"10s"` | DNS refresh interval | +| config.publicClient.transport | string | `"grpc"` | Transport protocol: grpc, tchannel | +| config.ringpop.bootstrapMode | string | `"dns"` | Bootstrap mode: dns, hosts, file | +| config.ringpop.maxJoinDuration | string | `"30s"` | Maximum duration to wait for joining the ring | +| config.ringpop.name | string | `"cadence"` | Ringpop cluster name | +| config.services.grpcMaxMsgSize | int | `4194304` | gRPC max message size | +| config.services.metrics.prometheus.timerType | string | `"histogram"` | Timer type: histogram, summary | +| config.services.metrics.statsd.endpoint | string | `""` | StatsD endpoint. Can reference Kubernetes services | +| config.services.metrics.statsd.prefixes | object | `{"frontend":"cadence-frontend","history":"cadence-history","matching":"cadence-matching","worker":"cadence-worker"}` | Metric prefixes for each service | +| config.services.metrics.type | string | `"prometheus"` | Metrics type: statsd, prometheus | +| config.services.pprof.enabled | bool | `false` | Enable pprof endpoints | +| config.services.pprof.ports | object | `{"frontend":6060,"history":6062,"matching":6061,"worker":6063}` | Pprof ports for each service | +| dynamicConfig.values | object | `{"system.minRetentionDays":[{"constraints":{},"value":0}],"system.readVisibilityStoreName":[{"value":"db"}],"system.writeVisibilityStoreName":[{"value":"db"}]}` | Dynamic config values to be set in the Cadence server List of keys can be found at https://pkg.go.dev/github.com/uber/cadence/common/dynamicconfig/dynamicproperties | | frontend.affinity | object | `{}` | Affinity rules (inherits from global if not specified) | | frontend.containerSecurityContext | object | `{}` | Container security context (inherits from global if not specified) | | frontend.env | list | `[]` | Environment variables for frontend service | @@ -58,8 +212,8 @@ This chart deploys Uber Cadence server components and web UI. | fullnameOverride | string | `nil` | Provide a name to override the full names of resources | | global.affinity | object | `{}` | Global affinity rules | | global.containerSecurityContext | object | `{}` | Global container security context | -| global.env | list | `[{"name":"ENABLE_ES","value":"false"},{"name":"SKIP_SCHEMA_SETUP","value":"true"},{"name":"RINGPOP_BOOTSTRAP_MODE","value":"dns"},{"name":"BIND_ON_IP","value":"0.0.0.0"}]` | Global environment variables (shared only by Cadence Server services [frontend, worker, matching and history]) | -| global.image | object | `{"pullPolicy":"IfNotPresent","repository":"docker.io/ubercadence/server","tag":"v1.3.1-auto-setup"}` | Global image configuration (shared only by Cadence Server services [frontend, worker, matching and history]) | +| global.env | list | `[]` | Global environment variables (shared only by Cadence Server services [frontend, worker, matching and history]) | +| global.image | object | `{"pullPolicy":"IfNotPresent","repository":"docker.io/ubercadence/server","tag":"v1.3.1"}` | Global image configuration (shared only by Cadence Server services [frontend, worker, matching and history]) | | global.imagePullSecrets | list | `[]` | Image pull secrets for private registries | | global.log | object | `{"level":"info","stdout":true}` | Global logging configuration (shared only by Cadence Server services [frontend, worker, matching and history]) | | global.log.level | string | `"info"` | Logging level (debug, info, warn, error) | @@ -68,6 +222,9 @@ This chart deploys Uber Cadence server components and web UI. | global.podSecurityContext | object | `{}` | Global pod security context | | global.priorityClassName | string | `""` | Global priority class name for pod scheduling | | global.secretEnv | list | `[]` | Global secret environment variables (shared only by Cadence Server services [frontend, worker, matching and history]) | +| global.tls | object | `{"volumeMounts":[],"volumes":[]}` | Global TLS volumes configuration | +| global.tls.volumeMounts | list | `[]` | Volume mounts for TLS certificates | +| global.tls.volumes | list | `[]` | Additional volumes for TLS certificates (The mode is important to have the minimum permissions) | | global.tolerations | list | `[]` | Global tolerations | | global.topologySpreadConstraints | list | `[]` | Global topology spread constraints | | history.affinity | object | `{}` | Affinity rules (inherits from global if not specified) | @@ -126,6 +283,21 @@ This chart deploys Uber Cadence server components and web UI. | networkPolicy.enabled | bool | `false` | Enable network policies | | networkPolicy.ingress | list | `[]` | Ingress rules | | rbac.create | bool | `false` | Enable RBAC creation | +| schema.checkSchema.cassandra.image.pullPolicy | string | `"IfNotPresent"` | | +| schema.checkSchema.cassandra.image.repository | string | `"cassandra"` | | +| schema.checkSchema.cassandra.image.tag | string | `"4.0"` | | +| schema.checkSchema.elasticsearch.image.pullPolicy | string | `"IfNotPresent"` | | +| schema.checkSchema.elasticsearch.image.repository | string | `"alpine/curl"` | | +| schema.checkSchema.elasticsearch.image.tag | string | `"latest"` | | +| schema.checkSchema.mysql.image.pullPolicy | string | `"IfNotPresent"` | | +| schema.checkSchema.mysql.image.repository | string | `"alpine/mysql"` | | +| schema.checkSchema.mysql.image.tag | string | `"latest"` | | +| schema.checkSchema.postgres.image.pullPolicy | string | `"IfNotPresent"` | | +| schema.checkSchema.postgres.image.repository | string | `"alpine/psql"` | | +| schema.checkSchema.postgres.image.tag | string | `"latest"` | | +| schema.setupJob.enabled | bool | `true` | | +| schema.version.default | string | `"0.42"` | | +| schema.version.visibility | string | `"0.9"` | | | serviceAccount.annotations | object | `{}` | Annotations for service account | | serviceAccount.automountServiceAccountToken | bool | `true` | Automatically mount service account token | | serviceAccount.create | bool | `true` | Enable service account creation | @@ -133,7 +305,7 @@ This chart deploys Uber Cadence server components and web UI. | web.affinity | object | `{}` | Affinity rules (inherits from global if not specified) | | web.containerSecurityContext | object | `{}` | Container security context (inherits from global if not specified) | | web.env | list | `[{"name":"CADENCE_WEB_PORT","value":"8088"}]` | Environment variables for web UI | -| web.image | object | `{"imagePullSecrets":[],"pullPolicy":"IfNotPresent","repository":"docker.io/ubercadence/web","tag":"v4.0.3"}` | Image configuration for Web UI | +| web.image | object | `{"imagePullSecrets":[],"pullPolicy":"IfNotPresent","repository":"docker.io/ubercadence/web","tag":"v4.0.5"}` | Image configuration for Web UI | | web.ingress.annotations | object | `{}` | Ingress annotations | | web.ingress.className | string | `""` | Ingress class name | | web.ingress.enabled | bool | `false` | Enable ingress | diff --git a/charts/cadence/charts/cassandra-11.4.2.tgz b/charts/cadence/charts/cassandra-11.4.2.tgz new file mode 100644 index 0000000..d90555c Binary files /dev/null and b/charts/cadence/charts/cassandra-11.4.2.tgz differ diff --git a/charts/cadence/charts/mysql-12.3.5.tgz b/charts/cadence/charts/mysql-12.3.5.tgz new file mode 100644 index 0000000..1d232f7 Binary files /dev/null and b/charts/cadence/charts/mysql-12.3.5.tgz differ diff --git a/charts/cadence/charts/postgresql-16.7.13.tgz b/charts/cadence/charts/postgresql-16.7.13.tgz new file mode 100644 index 0000000..d5bb101 Binary files /dev/null and b/charts/cadence/charts/postgresql-16.7.13.tgz differ diff --git a/charts/cadence/docs/TLS.md b/charts/cadence/docs/TLS.md new file mode 100644 index 0000000..55dec0c --- /dev/null +++ b/charts/cadence/docs/TLS.md @@ -0,0 +1,708 @@ +# TLS Configuration for Database and Service Connections + +This document describes how to configure TLS (Transport Layer Security) for secure connections to various databases and services including Cassandra, MySQL, PostgreSQL, Elasticsearch, and Kafka. + +## Configuration Overview + +The TLS configuration supports multiple security scenarios from basic server authentication to full mutual TLS (mTLS). All services use the same TLS configuration structure and validation logic. Certificates can be provided either through direct file paths or by mounting them as Kubernetes volumes. + +### Basic Configuration + +```yaml +tls: + enabled: true + caFile: "/path/to/ca-cert.pem" + enableHostVerification: true +``` + +## Certificate Management + +### Kubernetes Volume Mounting + +For Kubernetes deployments, TLS certificates can be mounted as volumes from Secrets or ConfigMaps. This is the recommended approach for production environments as it provides better security and management. + +#### Global TLS Volume Configuration + +Configure TLS volumes at the global level to share certificates across all Cadence services: + +```yaml +global: + tls: + volumes: + # Single CA certificate from Secret + - name: database-ca-cert + secret: + secretName: database-tls-secret + items: + - key: ca.crt + path: ca.pem + mode: 0644 + + # Client certificate and key from Secret + - name: database-client-cert + secret: + secretName: database-client-secret + items: + - key: tls.crt + path: client.pem + mode: 0644 + - key: tls.key + path: client-key.pem + mode: 0600 # Restricted permissions for private key + + volumeMounts: + # Mount CA certificate + - name: database-ca-cert + mountPath: /etc/cadence/ssl/ca + readOnly: true + + # Mount client certificates + - name: database-client-cert + mountPath: /etc/cadence/ssl/client + readOnly: true +``` + +#### File Permissions and Security + +The `mode` field in Kubernetes volume items controls the file permissions of mounted certificates: + +- **`mode: 0644`**: Read-write for owner, read-only for group and others + - Use for: CA certificates, client certificates (public parts) + - Security level: Standard - suitable for non-sensitive certificate files + +- **`mode: 0600`**: Read-write for owner only, no access for group or others + - Use for: Private keys, sensitive certificate files + - Security level: Restrictive - required for private key security + +- **`mode: 0400`**: Read-only for owner, no access for group or others + - Use for: Extra security on private keys in read-only scenarios + - Security level: Maximum restriction + +**⚠️ Security Best Practice**: Always use `mode: 0600` or `mode: 0400` for private key files to prevent unauthorized access. + +#### Database-specific Examples + +```yaml +global: + tls: + volumes: + # PostgreSQL TLS certificates + - name: postgres-tls-certs + secret: + secretName: postgres-ssl-secret + items: + - key: root.crt + path: postgresql-ca.pem + mode: 0644 + - key: postgresql.crt + path: postgresql-client.pem + mode: 0644 + - key: postgresql.key + path: postgresql-client-key.pem + mode: 0600 # Private key - restricted access + + # MySQL TLS certificates + - name: mysql-tls-certs + secret: + secretName: mysql-ssl-secret + items: + - key: ca.pem + path: mysql-ca.pem + mode: 0644 + - key: client-cert.pem + path: mysql-client-cert.pem + mode: 0644 + - key: client-key.pem + path: mysql-client-key.pem + mode: 0600 # Private key - restricted access + + # Elasticsearch TLS certificates + - name: elasticsearch-tls-certs + secret: + secretName: elasticsearch-ssl-secret + items: + - key: ca.crt + path: elasticsearch-ca.pem + mode: 0644 + - key: client.crt + path: elasticsearch-client.pem + mode: 0644 + - key: client.key + path: elasticsearch-client-key.pem + mode: 0600 # Private key - restricted access + + # Kafka TLS certificates + - name: kafka-tls-certs + secret: + secretName: kafka-ssl-secret + items: + - key: ca.crt + path: kafka-ca.pem + mode: 0644 + - key: client.crt + path: kafka-client.pem + mode: 0644 + - key: client.key + path: kafka-client-key.pem + mode: 0600 # Private key - restricted access + + volumeMounts: + - name: postgres-tls-certs + mountPath: /etc/cadence/ssl/postgres + readOnly: true + - name: mysql-tls-certs + mountPath: /etc/cadence/ssl/mysql + readOnly: true + - name: elasticsearch-tls-certs + mountPath: /etc/cadence/ssl/elasticsearch + readOnly: true + - name: kafka-tls-certs + mountPath: /etc/cadence/ssl/kafka + readOnly: true +``` + +#### Multiple CA Certificates Example + +```yaml +global: + tls: + volumes: + - name: multiple-ca-certs + configMap: + name: database-ca-bundle + items: + - key: root-ca.crt + path: root-ca.pem + mode: 0644 + - key: intermediate-ca.crt + path: intermediate-ca.pem + mode: 0644 + volumeMounts: + - name: multiple-ca-certs + mountPath: /etc/cadence/ssl/ca-bundle + readOnly: true +``` + +## Configuration Parameters + +### `enabled` +- **Type**: `boolean` +- **Default**: `false` +- **Description**: Enables or disables TLS connections +- **Usage**: When `false`, all other TLS settings are ignored +- **Applies to**: All database and service connections + +### `sslMode` +- **Type**: `string` +- **Default**: `""` +- **Description**: SSL mode configuration (database-specific) +- **PostgreSQL values**: `disable`, `allow`, `prefer`, `require`, `verify-ca`, `verify-full` +- **MySQL values**: `false`, `true`, `skip-verify`, `preferred` +- **Usage**: Controls the level of SSL verification required + +### `caFile` +- **Type**: `string` +- **Default**: `""` +- **Description**: Path to Certificate Authority (CA) certificate file +- **Format**: PEM format +- **Usage**: Required to verify server certificates and establish trust +- **Example with volumes**: `"/etc/cadence/ssl/ca/ca.pem"` +- **Direct path**: `"/etc/ssl/certs/ca-certificates.crt"` +- **File permissions**: Recommended `mode: 0644` + +### `caFiles` +- **Type**: `array` +- **Default**: `[]` +- **Description**: Array of CA certificate file paths +- **Format**: PEM format +- **Usage**: Alternative to `caFile` when multiple CA certificates are needed +- **Note**: Can be used together with `caFile` - all certificates are combined +- **Example with volumes**: `["/etc/cadence/ssl/ca-bundle/root-ca.pem", "/etc/cadence/ssl/ca-bundle/intermediate-ca.pem"]` +- **File permissions**: Recommended `mode: 0644` + +### `certFile` +- **Type**: `string` +- **Default**: `""` +- **Description**: Path to client certificate file +- **Format**: PEM format +- **Usage**: Required for mutual TLS (mTLS) authentication +- **Dependencies**: Must be used together with `keyFile` +- **Example with volumes**: `"/etc/cadence/ssl/client/client.pem"` +- **File permissions**: Recommended `mode: 0644` + +### `keyFile` +- **Type**: `string` +- **Default**: `""` +- **Description**: Path to client private key file +- **Format**: PEM format (RSA or ECDSA) +- **Usage**: Required for mutual TLS (mTLS) authentication +- **Dependencies**: Must be used together with `certFile` +- **Security**: Should have restricted file permissions +- **Example with volumes**: `"/etc/cadence/ssl/client/client-key.pem"` +- **File permissions**: **Required** `mode: 0600` or `mode: 0400` + +### `enableHostVerification` +- **Type**: `boolean` +- **Default**: `true` +- **Description**: Enables hostname verification against server certificate +- **Security**: + - `true`: Verifies server certificate matches hostname (secure) + - `false`: Skips hostname verification (insecure - testing only) +- **Recommendation**: Always `true` in production environments + +### `requireClientAuth` +- **Type**: `boolean` +- **Default**: `false` +- **Description**: Requires client certificate authentication (mutual TLS) +- **Usage**: When `true`, server will request and verify client certificates +- **Dependencies**: Clients must provide `certFile` and `keyFile` +- **Server requirement**: Database/service must be configured for mTLS + +### `serverName` +- **Type**: `string` +- **Default**: `""` +- **Description**: Override server name for certificate verification +- **Usage**: Use when connecting via IP address or when certificate Common Name differs from hostname +- **Example**: `"database.example.com"` when connecting to `192.168.1.100` + +## Certificate Formats + +### PEM Format +- **Description**: Privacy-Enhanced Mail format (Base64 encoded, human-readable) +- **Extensions**: `.pem`, `.crt`, `.cer`, `.key` +- **Structure**: Contains `-----BEGIN CERTIFICATE-----` and `-----END CERTIFICATE-----` headers +- **Support**: Only format supported by this implementation + +## Configuration Scenarios by Database/Service Type + +### 1. PostgreSQL with TLS + +```yaml +# Using Kubernetes volumes (recommended) +global: + tls: + volumes: + - name: postgres-tls-certs + secret: + secretName: postgres-ssl-secret + items: + - key: root.crt + path: postgresql-ca.pem + mode: 0644 + - key: postgresql.crt + path: postgresql-client.pem + mode: 0644 + - key: postgresql.key + path: postgresql-client-key.pem + mode: 0600 # Critical: Private key security + volumeMounts: + - name: postgres-tls-certs + mountPath: /etc/cadence/ssl/postgres + readOnly: true + +database: + sql: + driver: "postgresql" + tls: + enabled: true + sslMode: "require" + caFile: "/etc/cadence/ssl/postgres/postgresql-ca.pem" + certFile: "/etc/cadence/ssl/postgres/postgresql-client.pem" + keyFile: "/etc/cadence/ssl/postgres/postgresql-client-key.pem" + enableHostVerification: true +``` + +### 2. MySQL with TLS + +```yaml +# Using Kubernetes volumes (recommended) +global: + tls: + volumes: + - name: mysql-tls-certs + secret: + secretName: mysql-ssl-secret + items: + - key: ca.pem + path: mysql-ca.pem + mode: 0644 + - key: client-cert.pem + path: mysql-client-cert.pem + mode: 0644 + - key: client-key.pem + path: mysql-client-key.pem + mode: 0600 # Critical: Private key security + volumeMounts: + - name: mysql-tls-certs + mountPath: /etc/cadence/ssl/mysql + readOnly: true + +database: + sql: + driver: "mysql" + tls: + enabled: true + sslMode: "true" + caFile: "/etc/cadence/ssl/mysql/mysql-ca.pem" + certFile: "/etc/cadence/ssl/mysql/mysql-client-cert.pem" + keyFile: "/etc/cadence/ssl/mysql/mysql-client-key.pem" + enableHostVerification: true +``` + +### 3. Cassandra with TLS + +```yaml +# Using Kubernetes volumes (recommended) +global: + tls: + volumes: + - name: cassandra-tls-certs + secret: + secretName: cassandra-tls-secret + items: + - key: ca.crt + path: cassandra-ca.pem + mode: 0644 + - key: client.crt + path: cassandra-client.pem + mode: 0644 + - key: client.key + path: cassandra-client-key.pem + mode: 0600 # Critical: Private key security + volumeMounts: + - name: cassandra-tls-certs + mountPath: /etc/cadence/ssl/cassandra + readOnly: true + +database: + cassandra: + tls: + enabled: true + caFile: "/etc/cadence/ssl/cassandra/cassandra-ca.pem" + certFile: "/etc/cadence/ssl/cassandra/cassandra-client.pem" + keyFile: "/etc/cadence/ssl/cassandra/cassandra-client-key.pem" + enableHostVerification: true + requireClientAuth: true +``` + +### 4. Elasticsearch with TLS + +```yaml +# Using Kubernetes volumes (recommended) +global: + tls: + volumes: + - name: elasticsearch-tls-certs + secret: + secretName: elasticsearch-ssl-secret + items: + - key: ca.crt + path: elasticsearch-ca.pem + mode: 0644 + - key: client.crt + path: elasticsearch-client.pem + mode: 0644 + - key: client.key + path: elasticsearch-client-key.pem + mode: 0600 # Critical: Private key security + volumeMounts: + - name: elasticsearch-tls-certs + mountPath: /etc/cadence/ssl/elasticsearch + readOnly: true + +elasticsearch: + tls: + enabled: true + caFile: "/etc/cadence/ssl/elasticsearch/elasticsearch-ca.pem" + certFile: "/etc/cadence/ssl/elasticsearch/elasticsearch-client.pem" + keyFile: "/etc/cadence/ssl/elasticsearch/elasticsearch-client-key.pem" + enableHostVerification: true +``` + +### 5. Kafka with TLS + +```yaml +# Using Kubernetes volumes (recommended) +global: + tls: + volumes: + - name: kafka-tls-certs + secret: + secretName: kafka-ssl-secret + items: + - key: ca.crt + path: kafka-ca.pem + mode: 0644 + - key: client.crt + path: kafka-client.pem + mode: 0644 + - key: client.key + path: kafka-client-key.pem + mode: 0600 # Critical: Private key security + volumeMounts: + - name: kafka-tls-certs + mountPath: /etc/cadence/ssl/kafka + readOnly: true + +kafka: + tls: + enabled: true + caFile: "/etc/cadence/ssl/kafka/kafka-ca.pem" + certFile: "/etc/cadence/ssl/kafka/kafka-client.pem" + keyFile: "/etc/cadence/ssl/kafka/kafka-client-key.pem" + enableHostVerification: true +``` + +### 6. Multiple Services with Shared Certificates + +```yaml +# Single certificate bundle for multiple services +global: + tls: + volumes: + - name: shared-tls-bundle + secret: + secretName: shared-ssl-secret + items: + - key: ca-bundle.crt + path: ca-bundle.pem + mode: 0644 + - key: client.crt + path: client.pem + mode: 0644 + - key: client.key + path: client-key.pem + mode: 0600 # Critical: Private key security + volumeMounts: + - name: shared-tls-bundle + mountPath: /etc/cadence/ssl/shared + readOnly: true + +database: + cassandra: + tls: + enabled: true + caFile: "/etc/cadence/ssl/shared/ca-bundle.pem" + certFile: "/etc/cadence/ssl/shared/client.pem" + keyFile: "/etc/cadence/ssl/shared/client-key.pem" + enableHostVerification: true + +elasticsearch: + tls: + enabled: true + caFile: "/etc/cadence/ssl/shared/ca-bundle.pem" + certFile: "/etc/cadence/ssl/shared/client.pem" + keyFile: "/etc/cadence/ssl/shared/client-key.pem" + enableHostVerification: true + +kafka: + tls: + enabled: true + caFile: "/etc/cadence/ssl/shared/ca-bundle.pem" + certFile: "/etc/cadence/ssl/shared/client.pem" + keyFile: "/etc/cadence/ssl/shared/client-key.pem" + enableHostVerification: true +``` + +### 7. Development/Testing with Self-signed Certificates + +```yaml +# For any database/service with self-signed certificates +global: + tls: + volumes: + - name: dev-tls-certs + secret: + secretName: dev-ssl-secret + items: + - key: self-signed-ca.crt + path: self-signed-ca.pem + mode: 0644 + - key: client.crt + path: client.pem + mode: 0644 + - key: client.key + path: client-key.pem + mode: 0600 # Even in dev, protect private keys + volumeMounts: + - name: dev-tls-certs + mountPath: /etc/cadence/ssl/dev + readOnly: true + +tls: + enabled: true + caFile: "/etc/cadence/ssl/dev/self-signed-ca.pem" + enableHostVerification: false # Only if hostname doesn't match + serverName: "service.local" # If needed for verification +``` + +**⚠️ Warning**: Only use `enableHostVerification: false` in development environments + +## Security Best Practices + +### Production Environments +- **Use Kubernetes Secrets** for certificate storage instead of direct file paths +- Always use `enableHostVerification: true` +- Use certificates from trusted Certificate Authorities +- Regularly rotate certificates before expiration +- Use mutual TLS for high-security requirements +- Mount certificates as read-only volumes +- Use appropriate `sslMode` for SQL databases +- **Always set `mode: 0600` for private key files** + +### File Permission Guidelines +- **CA Certificates**: `mode: 0644` (readable by all) +- **Client Certificates**: `mode: 0644` (readable by all) +- **Private Keys**: `mode: 0600` (owner access only) or `mode: 0400` (read-only) +- **Never use**: `mode: 0777` or overly permissive settings for any certificate files + +### Database-specific Security +- **PostgreSQL**: Use `sslMode: "verify-full"` for maximum security +- **MySQL**: Use `sslMode: "true"` and verify certificates +- **Cassandra**: Enable `requireClientAuth: true` for mutual TLS +- **Elasticsearch**: Configure cluster security with proper certificates +- **Kafka**: Use SASL_SSL for authentication combined with TLS + +### Kubernetes Security +- Store certificates in Kubernetes Secrets, not ConfigMaps +- Use RBAC to restrict access to certificate secrets +- Consider using cert-manager for automated certificate lifecycle management +- Implement certificate rotation strategies +- Use different certificates for different environments +- **Ensure proper file permissions on mounted volumes** + +### Certificate Management +- Monitor certificate expiration dates +- Implement automated certificate renewal +- Keep CA certificates up to date +- Use separate certificates for different services when possible +- Consider using Kubernetes cert-manager for automation +- **Audit file permissions regularly** + +## Troubleshooting + +### Common Issues + +1. **Certificate verification failed** + - Check if CA certificate is correct for the specific service + - Verify certificate chain is complete + - Ensure certificate hasn't expired + - Verify volume mounts are correct when using Kubernetes + +2. **SSL mode issues (SQL databases)** + - Verify `sslMode` is appropriate for your database type + - Check database server SSL configuration + - Ensure SSL is enabled on the database server + +3. **Volume mount issues** + - Verify Secret/ConfigMap exists in the correct namespace + - Check that the secret keys match the volume configuration + - Ensure proper RBAC permissions for accessing secrets + - **Verify file permissions with `mode` settings** + +4. **Permission denied errors** + - Check if `mode` is set correctly for private key files + - Ensure private keys have `mode: 0600` or more restrictive + - Verify the pod's security context allows access to the files + +5. **Service-specific connection issues** + - **Cassandra**: Verify TLS port (usually 9142) is used + - **PostgreSQL**: Check `sslmode` parameter in connection string + - **MySQL**: Verify SSL parameters are correctly set + - **Elasticsearch**: Check HTTPS port and cluster security + - **Kafka**: Verify SASL_SSL configuration + +### Validation Commands + +Test certificate validity: +```bash +# Check certificate details +openssl x509 -in /path/to/cert.pem -text -noout + +# Verify certificate chain +openssl verify -CAfile /path/to/ca.pem /path/to/cert.pem + +# Test TLS connection to different services +openssl s_client -connect cassandra-host:9142 -CAfile /path/to/ca.pem +openssl s_client -connect postgres-host:5432 -CAfile /path/to/ca.pem +openssl s_client -connect mysql-host:3306 -CAfile /path/to/ca.pem +openssl s_client -connect elasticsearch-host:9200 -CAfile /path/to/ca.pem +openssl s_client -connect kafka-host:9093 -CAfile /path/to/ca.pem +``` + +Test Kubernetes volume mounts and permissions: +```bash +# Check if certificates are mounted correctly +kubectl exec -it -- ls -la /etc/cadence/ssl/ +kubectl exec -it -- cat /etc/cadence/ssl/ca.pem + +# Verify file permissions specifically +kubectl exec -it -- ls -la /etc/cadence/ssl/client/ +kubectl exec -it -- stat /etc/cadence/ssl/client/client-key.pem + +# Verify Secret contents +kubectl get secret -o yaml +kubectl describe secret + +# Test file access +kubectl exec -it -- head -n 5 /etc/cadence/ssl/client/client-key.pem +``` + +Expected file permissions output: +```bash +# CA and client certificates (should be 644) +-rw-r--r-- 1 root root 1234 Jan 01 12:00 ca.pem +-rw-r--r-- 1 root root 1234 Jan 01 12:00 client.pem + +# Private key (should be 600) +-rw------- 1 root root 1234 Jan 01 12:00 client-key.pem +``` + +## Service-specific Integration Notes + +### PostgreSQL +```yaml +# postgresql.conf +ssl = on +ssl_cert_file = 'server.crt' +ssl_key_file = 'server.key' +ssl_ca_file = 'ca.crt' +``` + +### MySQL +```yaml +# my.cnf +[mysqld] +ssl-ca=/path/to/ca.pem +ssl-cert=/path/to/server-cert.pem +ssl-key=/path/to/server-key.pem +require_secure_transport=ON +``` + +### Cassandra +```yaml +# cassandra.yaml +client_encryption_options: + enabled: true + optional: false + keystore: /path/to/server-keystore.jks + truststore: /path/to/server-truststore.jks +``` + +### Elasticsearch +```yaml +# elasticsearch.yml +xpack.security.http.ssl.enabled: true +xpack.security.http.ssl.keystore.path: /path/to/keystore.p12 +xpack.security.http.ssl.truststore.path: /path/to/truststore.p12 +``` + +### Kafka +```yaml +# server.properties +listeners=SASL_SSL://kafka:9093 +ssl.keystore.location=/path/to/kafka.server.keystore.jks +ssl.truststore.location=/path/to/kafka.server.truststore.jks +``` + +This configuration provides a secure, production-ready TLS setup for all supported databases and services with proper certificate management through Kubernetes, including secure file permission handling. \ No newline at end of file diff --git a/charts/cadence/templates/_helpers.tpl b/charts/cadence/templates/_helpers.tpl index df7175a..9cfd00d 100644 --- a/charts/cadence/templates/_helpers.tpl +++ b/charts/cadence/templates/_helpers.tpl @@ -92,30 +92,46 @@ Check if HPA is enabled for a specific service {{- end }} {{- end }} +{{/* +Helper to generate database and service secrets based on configuration +Receives context as parameter +*/}} +{{- define "cadence.databaseSecrets" -}} +{{- $context := . -}} +{{- $secrets := list -}} + +{{- /* Cassandra password */ -}} +{{- if and (eq $context.Values.config.persistence.database.driver "cassandra") $context.Values.config.persistence.database.cassandra.password -}} +{{- $secrets = append $secrets (dict "name" "CASSANDRA_PASSWORD" "value" $context.Values.config.persistence.database.cassandra.password) -}} +{{- end -}} + +{{- /* MySQL password */ -}} +{{- if and (eq $context.Values.config.persistence.database.driver "mysql") $context.Values.config.persistence.database.sql.password -}} +{{- $secrets = append $secrets (dict "name" "MYSQL_PWD" "value" $context.Values.config.persistence.database.sql.password) -}} +{{- end -}} + +{{- /* PostgreSQL password */ -}} +{{- if and (eq $context.Values.config.persistence.database.driver "postgres") $context.Values.config.persistence.database.sql.password -}} +{{- $secrets = append $secrets (dict "name" "POSTGRES_PWD" "value" $context.Values.config.persistence.database.sql.password) -}} +{{- end -}} + +{{- /* Elasticsearch password */ -}} +{{- if and $context.Values.config.persistence.elasticsearch.enabled $context.Values.config.persistence.elasticsearch.password -}} +{{- $secrets = append $secrets (dict "name" "ES_PWD" "value" $context.Values.config.persistence.elasticsearch.password) -}} +{{- end -}} + +{{- /* Kafka SASL password */ -}} +{{- if and $context.Values.config.kafka.enabled $context.Values.config.kafka.sasl.enabled $context.Values.config.kafka.sasl.password -}} +{{- $secrets = append $secrets (dict "name" "SASL_PASSWORD" "value" $context.Values.config.kafka.sasl.password) -}} +{{- end -}} + +{{- /* Store secrets in a shared variable using a unique key */ -}} +{{- $_ := set $context "databaseSecrets" $secrets -}} +{{- end -}} + {/* Cadence GRPC Peers endpoint */}} {{- define "cadence.grpcPeers" -}} {{ include "cadence.fullname" . }}-frontend.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.frontend.grpcPort | default 7833 }} -{{- end }} - -{{/* -Generate Ringpop seeds for service discovery -*/}} -{{- define "cadence.ringpopSeeds" -}} -{{- $seeds := list }} -{{- $namespace := .Release.Namespace }} -{{- $fullname := include "cadence.fullname" . }} -{{- $seeds = append $seeds (printf "%s-frontend-headless.%s.svc.cluster.local:%d" $fullname $namespace (.Values.frontend.port | int)) }} -{{- $seeds = append $seeds (printf "%s-history-headless.%s.svc.cluster.local:%d" $fullname $namespace (.Values.history.port | int)) }} -{{- $seeds = append $seeds (printf "%s-matching-headless.%s.svc.cluster.local:%d" $fullname $namespace (.Values.matching.port | int)) }} -{{- $seeds = append $seeds (printf "%s-worker-headless.%s.svc.cluster.local:%d" $fullname $namespace (.Values.worker.port | int)) }} -{{- join "," $seeds }} -{{- end }} - -{{/* -Get the Cassandra endpoint -*/}} -{{- define "cassandra.endpoint" -}} -{{ .Values.cassandra.endpoint | default (printf "cassandra-service.%s.svc.cluster.local" $.Release.Namespace) }} -{{- end -}} \ No newline at end of file +{{- end }} \ No newline at end of file diff --git a/charts/cadence/templates/cassandra-deployment.yaml b/charts/cadence/templates/cassandra-deployment.yaml deleted file mode 100644 index c4d79b4..0000000 --- a/charts/cadence/templates/cassandra-deployment.yaml +++ /dev/null @@ -1,61 +0,0 @@ -{{- if .Values.cassandra.deployment.enabled }} -apiVersion: apps/v1 -kind: Deployment -metadata: - name: cassandra-deployment - labels: - {{- include "cadence.labels" . | nindent 4 }} -spec: - replicas: 1 - selector: - matchLabels: - app: cassandra - template: - metadata: - labels: - app: cassandra - spec: - containers: - - name: cassandra - image: {{ .Values.cassandra.deployment.image.repository }}:{{ .Values.cassandra.deployment.image.tag }} - ports: - - containerPort: 9042 - env: - - name: MAX_HEAP_SIZE - value: "512M" - - name: HEAP_NEWSIZE - value: "256M" - resources: - limits: - cpu: {{ .Values.cassandra.deployment.resources.limits.cpu }} - memory: {{ .Values.cassandra.deployment.resources.limits.memory }} - requests: - cpu: {{ .Values.cassandra.deployment.resources.requests.cpu }} - memory: {{ .Values.cassandra.deployment.resources.requests.memory }} - readinessProbe: - exec: - command: ["sh", "-c", "cqlsh $(hostname -i) -u cassandra -p cassandra -e 'describe keyspaces'"] - initialDelaySeconds: 10 - periodSeconds: 15 - timeoutSeconds: 30 - failureThreshold: 10 - livenessProbe: - exec: - command: ["sh", "-c", "cqlsh $(hostname -i) -u cassandra -p cassandra -e 'describe keyspaces'"] - initialDelaySeconds: 10 - periodSeconds: 15 - timeoutSeconds: 30 - failureThreshold: 10 ---- -apiVersion: v1 -kind: Service -metadata: - name: cassandra-service -spec: - selector: - app: cassandra - ports: - - protocol: TCP - port: 9042 - targetPort: 9042 -{{- end }} diff --git a/charts/cadence/templates/schema-elasticsearch-job.yaml b/charts/cadence/templates/schema-elasticsearch-job.yaml new file mode 100644 index 0000000..02a4d98 --- /dev/null +++ b/charts/cadence/templates/schema-elasticsearch-job.yaml @@ -0,0 +1,463 @@ +{{- if and .Values.schema.elasticSearchJob.enabled .Values.config.persistence.elasticsearch.enabled -}} +--- +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ include "cadence.fullname" . }}-schema-elasticsearch + labels: + {{- include "cadence.labels" . | nindent 4 }} + app.kubernetes.io/component: schema-elasticsearch +spec: + ttlSecondsAfterFinished: 60 + template: + metadata: + labels: + {{- include "cadence.labels" . | nindent 8 }} + app.kubernetes.io/component: schema-elasticsearch + spec: + {{- if $.Values.serviceAccount.create }} + serviceAccountName: {{ include "cadence.serviceAccountName" $ }} + {{- end }} + {{- $globalImagePullSecrets := $.Values.global.imagePullSecrets | default list }} + {{- $schemaImagePullSecrets := $.Values.frontend.imagePullSecrets | default list }} + {{- $mergedImagePullSecrets := concat $globalImagePullSecrets $schemaImagePullSecrets }} + {{- if $mergedImagePullSecrets }} + imagePullSecrets: + {{- toYaml $mergedImagePullSecrets | nindent 8 }} + {{- end }} + {{- $globalNodeSelector := $.Values.global.nodeSelector | default dict }} + {{- $schemaNodeSelector := $.Values.schema.elasticSearchJob.nodeSelector | default $globalNodeSelector }} + {{- if $schemaNodeSelector }} + nodeSelector: + {{- toYaml $schemaNodeSelector | nindent 8 }} + {{- end }} + {{- $globalAffinity := $.Values.global.affinity | default dict }} + {{- $schemaAffinity := $.Values.schema.elasticSearchJob.affinity | default $globalAffinity }} + {{- if $schemaAffinity }} + affinity: + {{- toYaml $schemaAffinity | nindent 8 }} + {{- end }} + {{- $globalTolerations := $.Values.global.tolerations | default list }} + {{- $schemaTolerations := $.Values.schema.elasticSearchJob.tolerations | default $globalTolerations }} + {{- if $schemaTolerations }} + tolerations: + {{- toYaml $schemaTolerations | nindent 8 }} + {{- end }} + restartPolicy: OnFailure + initContainers: + - name: wait-for-elasticsearch + image: {{ $.Values.schema.checkSchema.elasticsearch.image.repository | default "alpine/curl" }}:{{ $.Values.schema.checkSchema.elasticsearch.image.tag | default "latest" }} + imagePullPolicy: {{ $.Values.schema.checkSchema.elasticsearch.image.pullPolicy | default "IfNotPresent" }} + command: + - sh + - -c + - | + echo "Starting Elasticsearch readiness check..." + + # Build Elasticsearch connection parameters + build_es_connection() { + # Determine protocol - allow override from values or default based on TLS + if [ -n "$ES_PROTOCOL" ]; then + PROTOCOL="$ES_PROTOCOL" + elif [ "$TLS_ENABLED" = "true" ]; then + PROTOCOL="https" + else + PROTOCOL="http" + fi + + # Build curl options for TLS + CURL_OPTS="" + if [ "$TLS_ENABLED" = "true" ]; then + # Configure SSL verification based on host verification setting + if [ "$ENABLE_HOST_VERIFICATION" = "false" ]; then + CURL_OPTS="$CURL_OPTS -k" + fi + + # Add CA certificate if provided + if [ -n "$SSL_CA_FILE" ]; then + CURL_OPTS="$CURL_OPTS --cacert $SSL_CA_FILE" + fi + + # Add client certificate for mutual TLS if provided + if [ -n "$SSL_CLIENT_CERT" ] && [ -n "$SSL_CLIENT_KEY" ]; then + CURL_OPTS="$CURL_OPTS --cert $SSL_CLIENT_CERT --key $SSL_CLIENT_KEY" + fi + + # Override server name if specified + if [ -n "$SSL_SERVER_NAME" ]; then + CURL_OPTS="$CURL_OPTS --resolve $SSL_SERVER_NAME:$ES_PORT:$ES_HOST" + fi + + # Additional TLS options + if [ "$REQUIRE_CLIENT_AUTH" = "true" ]; then + # Client auth is required, ensure we have client cert + if [ -z "$SSL_CLIENT_CERT" ] || [ -z "$SSL_CLIENT_KEY" ]; then + echo "Error: Client authentication required but client certificate/key not provided" + exit 1 + fi + fi + fi + + # Add authentication if user/password provided + if [ -n "$ES_USER" ] && [ -n "$ES_PWD" ]; then + CURL_OPTS="$CURL_OPTS -u $ES_USER:$ES_PWD" + fi + + # Set global variables + BASE_URL="$PROTOCOL://$ES_HOST:$ES_PORT" + + echo "Connecting to Elasticsearch at: $BASE_URL" + echo "TLS Enabled: $TLS_ENABLED" + if [ "$TLS_ENABLED" = "true" ]; then + echo "Host Verification: $ENABLE_HOST_VERIFICATION" + echo "Client Auth Required: $REQUIRE_CLIENT_AUTH" + fi + } + + # Wait for Elasticsearch to be ready + echo "Waiting for Elasticsearch to be ready..." + build_es_connection + + # Check Elasticsearch health + until curl $CURL_OPTS -s -f "$BASE_URL/_cluster/health?wait_for_status=yellow&timeout=5s" > /dev/null; do + echo "Elasticsearch is not ready yet..." + sleep 10 + done + + echo "Elasticsearch is ready!" + env: + # Basic Elasticsearch connection parameters + - name: ES_HOST + value: {{ $.Values.config.persistence.elasticsearch.hosts | quote }} + - name: ES_PORT + value: {{ $.Values.config.persistence.elasticsearch.port | default 9200 | quote }} + - name: ES_PROTOCOL + value: {{ $.Values.config.persistence.elasticsearch.protocol | default "" | quote }} + # Authentication parameters + {{- if $.Values.config.persistence.elasticsearch.user }} + - name: ES_USER + value: {{ $.Values.config.persistence.elasticsearch.user | quote }} + {{- end }} + {{- if $.Values.config.persistence.elasticsearch.password }} + - name: ES_PWD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-frontend-secrets + key: ES_PWD + {{- end }} + # TLS Configuration + - name: TLS_ENABLED + value: {{ $.Values.config.persistence.elasticsearch.tls.enabled | quote }} + {{- if $.Values.config.persistence.elasticsearch.tls.enabled }} + - name: ENABLE_HOST_VERIFICATION + value: {{ $.Values.config.persistence.elasticsearch.tls.enableHostVerification | quote }} + - name: REQUIRE_CLIENT_AUTH + value: {{ $.Values.config.persistence.elasticsearch.tls.requireClientAuth | quote }} + # CA certificate file + {{- if $.Values.config.persistence.elasticsearch.tls.caFile }} + - name: SSL_CA_FILE + value: {{ $.Values.config.persistence.elasticsearch.tls.caFile | quote }} + {{- else if $.Values.config.persistence.elasticsearch.tls.caFiles }} + - name: SSL_CA_FILE + value: {{ index $.Values.config.persistence.elasticsearch.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if $.Values.config.persistence.elasticsearch.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ $.Values.config.persistence.elasticsearch.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if $.Values.config.persistence.elasticsearch.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ $.Values.config.persistence.elasticsearch.tls.keyFile | quote }} + {{- end }} + # Server name override + {{- if $.Values.config.persistence.elasticsearch.tls.serverName }} + - name: SSL_SERVER_NAME + value: {{ $.Values.config.persistence.elasticsearch.tls.serverName | quote }} + {{- end }} + {{- end }} + volumeMounts: + {{- with .Values.global.tls.volumeMounts }} + {{- toYaml . | nindent 8 }} + {{- end }} + containers: + - name: cadence-schema-elasticsearch + {{- $globalImage := .Values.global.image | default dict }} + {{- $schemaImage := $.Values.frontend.image | default dict }} + {{- $repository := $schemaImage.repository | default $globalImage.repository }} + {{- $tag := $schemaImage.tag | default $globalImage.tag }} + image: {{ $repository }}:{{ $tag }} + {{- $pullPolicy := $schemaImage.pullPolicy | default $globalImage.pullPolicy | default "IfNotPresent" }} + imagePullPolicy: {{ $pullPolicy }} + command: + - sh + - -c + - | + set -e + echo "Starting ElasticSearch schema setup..." + echo "=== Installing ElasticSearch Schema ===" + + # Build Elasticsearch connection parameters + build_es_connection() { + # Determine protocol - allow override from values or default based on TLS + if [ -n "$ES_PROTOCOL" ]; then + PROTOCOL="$ES_PROTOCOL" + elif [ "$TLS_ENABLED" = "true" ]; then + PROTOCOL="https" + else + PROTOCOL="http" + fi + + # Build curl options for TLS + CURL_OPTS="" + if [ "$TLS_ENABLED" = "true" ]; then + # Configure SSL verification based on host verification setting + if [ "$ENABLE_HOST_VERIFICATION" = "false" ]; then + CURL_OPTS="$CURL_OPTS -k" + fi + + # Add CA certificate if provided + if [ -n "$SSL_CA_FILE" ]; then + CURL_OPTS="$CURL_OPTS --cacert $SSL_CA_FILE" + fi + + # Add client certificate for mutual TLS if provided + if [ -n "$SSL_CLIENT_CERT" ] && [ -n "$SSL_CLIENT_KEY" ]; then + CURL_OPTS="$CURL_OPTS --cert $SSL_CLIENT_CERT --key $SSL_CLIENT_KEY" + fi + + # Override server name if specified + if [ -n "$SSL_SERVER_NAME" ]; then + CURL_OPTS="$CURL_OPTS --resolve $SSL_SERVER_NAME:$ES_PORT:$ES_HOST" + fi + fi + + # Add authentication if user/password provided + if [ -n "$ES_USER" ] && [ -n "$ES_PWD" ]; then + CURL_OPTS="$CURL_OPTS -u $ES_USER:$ES_PWD" + fi + + # Set global variables + BASE_URL="$PROTOCOL://$ES_HOST:$ES_PORT" + + echo "Connecting to Elasticsearch at: $BASE_URL" + echo "TLS Enabled: $TLS_ENABLED" + if [ "$TLS_ENABLED" = "true" ]; then + echo "Host Verification: $ENABLE_HOST_VERIFICATION" + echo "Client Auth Required: $REQUIRE_CLIENT_AUTH" + fi + } + + # Initialize connection parameters + build_es_connection + + # Determine schema file path based on ES version + case "$ES_VERSION" in + "v6") + SCHEMA_FILE="$CADENCE_HOME/schema/elasticsearch/v6/visibility/index_template.json" + ;; + "v7") + SCHEMA_FILE="$CADENCE_HOME/schema/elasticsearch/v7/visibility/index_template.json" + ;; + *) + echo "Error: Unsupported Elasticsearch version: $ES_VERSION" + echo "Supported versions: v6, v7, v8 (in values should be v7)" + exit 1 + ;; + esac + + echo "Using schema file: $SCHEMA_FILE" + echo "Elasticsearch version: $ES_VERSION" + + # Check if schema file exists + if [ ! -f "$SCHEMA_FILE" ]; then + echo "Error: Schema file not found: $SCHEMA_FILE" + echo "Available schema files:" + find $CADENCE_HOME/schema/elasticsearch -name "*.json" -type f || echo "No schema files found" + exit 1 + fi + + # Step 1: Install template + echo "Step 1: Installing Cadence visibility template..." + TEMPLATE_URL="$BASE_URL/_template/cadence-visibility-template" + + echo "Uploading template to: $TEMPLATE_URL" + TEMPLATE_RESPONSE=$(curl $CURL_OPTS -s -w "%{http_code}" -X PUT "$TEMPLATE_URL" -H 'Content-Type: application/json' --data-binary "@$SCHEMA_FILE") + TEMPLATE_HTTP_CODE=$(echo "$TEMPLATE_RESPONSE" | tail -c 4) + TEMPLATE_BODY=$(echo "$TEMPLATE_RESPONSE" | head -c -4) + + if [ "$TEMPLATE_HTTP_CODE" -eq 200 ] || [ "$TEMPLATE_HTTP_CODE" -eq 201 ]; then + echo "✓ Template installed successfully" + echo "Response: $TEMPLATE_BODY" + else + echo "✗ Failed to install template. HTTP Code: $TEMPLATE_HTTP_CODE" + echo "Response: $TEMPLATE_BODY" + exit 1 + fi + + # Step 2: Create visibility index + echo "Step 2: Creating visibility index..." + INDEX_URL="$BASE_URL/$VISIBILITY_INDEX" + + echo "Creating index: $INDEX_URL" + INDEX_RESPONSE=$(curl $CURL_OPTS -s -w "%{http_code}" -X PUT "$INDEX_URL") + INDEX_HTTP_CODE=$(echo "$INDEX_RESPONSE" | tail -c 4) + INDEX_BODY=$(echo "$INDEX_RESPONSE" | head -c -4) + + if [ "$INDEX_HTTP_CODE" -eq 200 ] || [ "$INDEX_HTTP_CODE" -eq 201 ]; then + echo "✓ Index created successfully" + echo "Response: $INDEX_BODY" + elif [ "$INDEX_HTTP_CODE" -eq 400 ] && echo "$INDEX_BODY" | grep -q "resource_already_exists_exception"; then + echo "✓ Index already exists" + echo "Response: $INDEX_BODY" + else + echo "✗ Failed to create index. HTTP Code: $INDEX_HTTP_CODE" + echo "Response: $INDEX_BODY" + exit 1 + fi + + # Step 3: Verify installation + echo "Step 3: Verifying installation..." + + # Check template exists + echo "Checking template..." + TEMPLATE_CHECK=$(curl $CURL_OPTS -s -f "$TEMPLATE_URL") + if [ $? -eq 0 ]; then + echo "✓ Template verification successful" + if echo "$TEMPLATE_CHECK" | grep -q "cadence-visibility-template"; then + echo "✓ Template structure is valid" + fi + else + echo "✗ Template verification failed" + exit 1 + fi + + # Check index exists and is healthy + echo "Checking index..." + INDEX_CHECK=$(curl $CURL_OPTS -s -f "$INDEX_URL") + if [ $? -eq 0 ]; then + echo "✓ Index verification successful" + + # Get index stats + INDEX_STATS=$(curl $CURL_OPTS -s "$INDEX_URL/_stats") + if [ $? -eq 0 ]; then + echo "✓ Index is healthy and accessible" + # Try to extract document count + DOC_COUNT=$(echo "$INDEX_STATS" | grep -o '"count":[0-9]*' | head -1 | cut -d':' -f2) + if [ -n "$DOC_COUNT" ]; then + echo " - Document count: $DOC_COUNT" + fi + fi + else + echo "✗ Index verification failed" + exit 1 + fi + + # Step 4: Version-specific validation + echo "Step 4: Performing version-specific validation..." + case "$ES_VERSION" in + "v6") + # Check ES6 document type mapping + TYPE_URL="$BASE_URL/$VISIBILITY_INDEX/_mapping/_doc" + TYPE_CHECK=$(curl $CURL_OPTS -s -f "$TYPE_URL") + if [ $? -eq 0 ]; then + echo "✓ ES6 document type mapping exists" + else + echo "✗ ES6 document type mapping check failed" + exit 1 + fi + ;; + "v7") + # Check ES7/8 index mapping + MAPPING_URL="$BASE_URL/$VISIBILITY_INDEX/_mapping" + MAPPING_CHECK=$(curl $CURL_OPTS -s -f "$MAPPING_URL") + if [ $? -eq 0 ]; then + echo "✓ ES7 index mapping exists" + else + echo "✗ ES7 index mapping check failed" + exit 1 + fi + ;; + esac + + # Final summary + echo "" + echo "=== Elasticsearch Schema Installation Summary ===" + echo "Template: Installed ✓" + echo "Index: Created ✓" + echo "Mapping: Verified ✓" + echo "Version: $ES_VERSION" + echo "Schema File: $SCHEMA_FILE" + echo "===============================================" + + echo "Elasticsearch schema installation completed successfully!" + exit 0 + env: + # Basic Elasticsearch connection parameters + - name: ES_HOST + value: {{ $.Values.config.persistence.elasticsearch.hosts | quote }} + - name: ES_PORT + value: {{ $.Values.config.persistence.elasticsearch.port | default 9200 | quote }} + - name: ES_PROTOCOL + value: {{ $.Values.config.persistence.elasticsearch.protocol | default "" | quote }} + - name: VISIBILITY_INDEX + value: {{ $.Values.config.persistence.elasticsearch.visibilityIndex | quote }} + - name: ES_VERSION + value: {{ $.Values.config.persistence.elasticsearch.version | quote }} + # Cadence home directory + - name: CADENCE_HOME + value: "/etc/cadence" + # Authentication parameters + {{- if $.Values.config.persistence.elasticsearch.user }} + - name: ES_USER + value: {{ $.Values.config.persistence.elasticsearch.user | quote }} + {{- end }} + {{- if $.Values.config.persistence.elasticsearch.password }} + - name: ES_PWD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-frontend-secrets + key: ES_PWD + {{- end }} + # TLS Configuration + - name: TLS_ENABLED + value: {{ $.Values.config.persistence.elasticsearch.tls.enabled | quote }} + {{- if $.Values.config.persistence.elasticsearch.tls.enabled }} + - name: ENABLE_HOST_VERIFICATION + value: {{ $.Values.config.persistence.elasticsearch.tls.enableHostVerification | quote }} + - name: REQUIRE_CLIENT_AUTH + value: {{ $.Values.config.persistence.elasticsearch.tls.requireClientAuth | quote }} + # CA certificate file + {{- if $.Values.config.persistence.elasticsearch.tls.caFile }} + - name: SSL_CA_FILE + value: {{ $.Values.config.persistence.elasticsearch.tls.caFile | quote }} + {{- else if $.Values.config.persistence.elasticsearch.tls.caFiles }} + - name: SSL_CA_FILE + value: {{ index $.Values.config.persistence.elasticsearch.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if $.Values.config.persistence.elasticsearch.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ $.Values.config.persistence.elasticsearch.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if $.Values.config.persistence.elasticsearch.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ $.Values.config.persistence.elasticsearch.tls.keyFile | quote }} + {{- end }} + # Server name override + {{- if $.Values.config.persistence.elasticsearch.tls.serverName }} + - name: SSL_SERVER_NAME + value: {{ $.Values.config.persistence.elasticsearch.tls.serverName | quote }} + {{- end }} + {{- end }} + volumeMounts: + {{- with .Values.global.tls.volumeMounts }} + {{- toYaml . | nindent 8 }} + {{- end }} + volumes: + {{- with .Values.global.tls.volumes }} + {{- toYaml . | nindent 6 }} + {{- end }} +{{- end }} \ No newline at end of file diff --git a/charts/cadence/templates/schema-job.yaml b/charts/cadence/templates/schema-job.yaml deleted file mode 100644 index bbf362b..0000000 --- a/charts/cadence/templates/schema-job.yaml +++ /dev/null @@ -1,69 +0,0 @@ -apiVersion: batch/v1 -kind: Job -metadata: - name: cadence-schema-setup-v{{ .Values.cassandra.schema.version | replace "." "-" }} - labels: - {{- include "cadence.labels" . | nindent 4 }} -spec: - backoffLimit: 4 - template: - metadata: - name: cadence-schema-setup - spec: - restartPolicy: "OnFailure" - initContainers: - # Check Cassandra DNS resolution using busybox - - name: check-cassandra-dns - image: busybox - command: ['sh', '-c', ' - until nslookup {{ include "cassandra.endpoint" . }}; do - echo "Waiting for Cassandra service DNS resolution..."; - sleep 5; - done; - echo "Cassandra service DNS resolved."; - '] - # Check Cassandra readiness using cqlsh - - name: check-cassandra-ready - {{- $globalImage := $.Values.global.image | default dict }} - {{- $serviceImage := $.Values.history.image | default dict }} - {{- $repository := $serviceImage.repository | default $globalImage.repository }} - {{- $tag := $serviceImage.tag | default $globalImage.tag }} - image: {{ $repository }}:{{ $tag }} - imagePullPolicy: Always - command: ['sh', '-c', ' - until cqlsh {{ include "cassandra.endpoint" . }} 9042 -e "SHOW VERSION"; do - echo "Waiting for Cassandra to start..."; - sleep 30; - done; - echo "Cassandra is up and running."; - '] - containers: - - name: schema-setup - {{- $globalImage := $.Values.global.image | default dict }} - {{- $serviceImage := $.Values.history.image | default dict }} - {{- $repository := $serviceImage.repository | default $globalImage.repository }} - {{- $tag := $serviceImage.tag | default $globalImage.tag }} - image: {{ $repository }}:{{ $tag }} - imagePullPolicy: Always - env: - - name: CASSANDRA_SEEDS - value: {{ include "cassandra.endpoint" . }} - - name: KEYSPACE - value: cadence - - name: VISIBILITY_KEYSPACE - value: cadence_visibility - - name: RF - value: "1" - - name: CADENCE_HOME - value: /etc/cadence - args: ['sh', '-c', ' - SCHEMA_DIR=$CADENCE_HOME/schema/cassandra/cadence/versioned; - cadence-cassandra-tool --ep $CASSANDRA_SEEDS create -k $KEYSPACE --rf $RF; - cadence-cassandra-tool --ep $CASSANDRA_SEEDS -k $KEYSPACE setup-schema -v 0.0; - cadence-cassandra-tool --ep $CASSANDRA_SEEDS -k $KEYSPACE update-schema -d $SCHEMA_DIR; - - VISIBILITY_SCHEMA_DIR=$CADENCE_HOME/schema/cassandra/visibility/versioned; - cadence-cassandra-tool --ep $CASSANDRA_SEEDS create -k $VISIBILITY_KEYSPACE --rf $RF; - cadence-cassandra-tool --ep $CASSANDRA_SEEDS -k $VISIBILITY_KEYSPACE setup-schema -v 0.0; - cadence-cassandra-tool --ep $CASSANDRA_SEEDS -k $VISIBILITY_KEYSPACE update-schema -d $VISIBILITY_SCHEMA_DIR; - '] diff --git a/charts/cadence/templates/schema-server-job.yaml b/charts/cadence/templates/schema-server-job.yaml new file mode 100644 index 0000000..64d4dcc --- /dev/null +++ b/charts/cadence/templates/schema-server-job.yaml @@ -0,0 +1,764 @@ +{{- if .Values.schema.serverJob.enabled -}} +{{- $dbDriver := .Values.config.persistence.database.driver }} +--- +apiVersion: batch/v1 +kind: Job +metadata: + name: {{ include "cadence.fullname" . }}-schema-server + labels: + {{- include "cadence.labels" . | nindent 4 }} + app.kubernetes.io/component: schema-server +spec: + ttlSecondsAfterFinished: 60 + template: + metadata: + labels: + {{- include "cadence.labels" . | nindent 8 }} + app.kubernetes.io/component: schema-server + spec: + {{- if $.Values.serviceAccount.create }} + serviceAccountName: {{ include "cadence.serviceAccountName" $ }} + {{- end }} + {{- $globalImagePullSecrets := $.Values.global.imagePullSecrets | default list }} + {{- $schemaImagePullSecrets := $.Values.frontend.imagePullSecrets | default list }} + {{- $mergedImagePullSecrets := concat $globalImagePullSecrets $schemaImagePullSecrets }} + {{- if $mergedImagePullSecrets }} + imagePullSecrets: + {{- toYaml $mergedImagePullSecrets | nindent 8 }} + {{- end }} + {{- $globalNodeSelector := $.Values.global.nodeSelector | default dict }} + {{- $schemaNodeSelector := $.Values.schema.serverJob.nodeSelector | default $globalNodeSelector }} + {{- if $schemaNodeSelector }} + nodeSelector: + {{- toYaml $schemaNodeSelector | nindent 8 }} + {{- end }} + {{- $globalAffinity := $.Values.global.affinity | default dict }} + {{- $schemaAffinity := $.Values.schema.serverJob.affinity | default $globalAffinity }} + {{- if $schemaAffinity }} + affinity: + {{- toYaml $schemaAffinity | nindent 8 }} + {{- end }} + {{- $globalTolerations := $.Values.global.tolerations | default list }} + {{- $schemaTolerations := $.Values.schema.serverJob.tolerations | default $globalTolerations }} + {{- if $schemaTolerations }} + tolerations: + {{- toYaml $schemaTolerations | nindent 8 }} + {{- end }} + restartPolicy: OnFailure + initContainers: + - name: wait-for-database + {{- if eq $dbDriver "cassandra" }} + image: {{ $.Values.schema.checkSchema.cassandra.image.repository }}:{{ $.Values.schema.checkSchema.cassandra.image.tag }} + imagePullPolicy: {{ $.Values.schema.checkSchema.cassandra.image.pullPolicy }} + command: + - sh + - -c + - | + # Create .cassandra directory for cqlshrc if it doesn't exist + mkdir -p ~/.cassandra + + # Build cqlshrc configuration file + cat > ~/.cassandra/cqlshrc << EOF + [connection] + hostname = $DB_HOST + port = $DB_PORT + + EOF + + # Add authentication section if user is provided + if [ -n "$DB_USER" ] && [ "$DB_USER" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + [authentication] + username = $DB_USER + EOF + # Add password if provided + if [ -n "$CASSANDRA_PASSWORD" ] && [ "$CASSANDRA_PASSWORD" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + password = $CASSANDRA_PASSWORD + EOF + fi + fi + + # Add SSL configuration if enabled + if [ "$TLS_ENABLED" = "true" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + + [ssl] + EOF + # Add certificate file if specified (CA certificate) + if [ -n "$SSL_CERTFILE" ] && [ "$SSL_CERTFILE" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + certfile = $SSL_CERTFILE + EOF + fi + + # Add client certificate for mutual TLS + if [ -n "$SSL_CLIENT_CERT" ] && [ "$SSL_CLIENT_CERT" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + usercert = $SSL_CLIENT_CERT + EOF + fi + + # Add client private key for mutual TLS + if [ -n "$SSL_CLIENT_KEY" ] && [ "$SSL_CLIENT_KEY" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + userkey = $SSL_CLIENT_KEY + EOF + fi + + # Add validate setting + if [ -n "$SSL_VALIDATE" ] && [ "$SSL_VALIDATE" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + validate = $SSL_VALIDATE + EOF + else + cat >> ~/.cassandra/cqlshrc << EOF + validate = true + EOF + fi + fi + + # Debug: Show generated cqlshrc (remove in production) + echo "Generated cqlshrc:" + cat ~/.cassandra/cqlshrc + echo "---" + + # Build cqlsh command + build_cqlsh_cmd() { + local cmd="cqlsh" + + # Add SSL option if enabled + if [ "$TLS_ENABLED" = "true" ]; then + cmd="$cmd --ssl" + fi + + echo "$cmd" + } + + # Compare database version with cadence schema version + until $(build_cqlsh_cmd) -e " + SELECT now() FROM system.local;" > /dev/null 2>&1 + do + echo 'Waiting for Cassandra to be ready...' + sleep 5 + done + echo "Cassandra is ready!" + env: + - name: ES_ENABLED + value: {{ $.Values.config.persistence.elasticsearch.enabled | quote }} + # Basic connection parameters + - name: DB_HOST + value: {{ $.Values.config.persistence.database.cassandra.hosts | quote }} + - name: DB_PORT + value: {{ $.Values.config.persistence.database.cassandra.port | quote }} + - name: DB_NAME + value: {{ $.Values.config.persistence.database.cassandra.keyspace | quote }} + - name: DB_VISIBILITY_NAME + value: {{ $.Values.config.persistence.database.cassandra.visibilityKeyspace | quote }} + # Authentication parameters (conditional) + {{- if $.Values.config.persistence.database.cassandra.user }} + - name: DB_USER + value: {{ $.Values.config.persistence.database.cassandra.user | quote }} + {{- end }} + {{- if $.Values.config.persistence.database.cassandra.password }} + - name: CASSANDRA_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-frontend-secrets + key: CASSANDRA_PASSWORD + {{- end }} + # TLS Configuration + - name: TLS_ENABLED + value: {{ $.Values.config.persistence.database.cassandra.tls.enabled | quote }} + {{- if $.Values.config.persistence.database.cassandra.tls.enabled }} + # SSL_CERTFILE environment variable (CA certificate) + {{- if $.Values.config.persistence.database.cassandra.tls.caFile }} + - name: SSL_CERTFILE + value: {{ $.Values.config.persistence.database.cassandra.tls.caFile | quote }} + {{- else if $.Values.config.persistence.database.cassandra.tls.caFiles }} + - name: SSL_CERTFILE + value: {{ index $.Values.config.persistence.database.cassandra.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if $.Values.config.persistence.database.cassandra.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ $.Values.config.persistence.database.cassandra.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if $.Values.config.persistence.database.cassandra.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ $.Values.config.persistence.database.cassandra.tls.keyFile | quote }} + {{- end }} + # SSL_VALIDATE environment variable + - name: SSL_VALIDATE + value: {{ $.Values.config.persistence.database.cassandra.tls.enableHostVerification | quote }} + {{- end }} + {{- else if eq $dbDriver "postgres" }} + image: {{ $.Values.schema.checkSchema.postgres.image.repository }}:{{ $.Values.schema.checkSchema.postgres.image.tag }} + imagePullPolicy: {{ $.Values.schema.checkSchema.postgres.image.pullPolicy }} + command: + - sh + - -c + - | + # Set up PostgreSQL environment variables + export PGPASSWORD="$POSTGRES_PWD" + + # Add SSL mode if TLS is enabled + if [ "$TLS_ENABLED" = "true" ] && [ -n "$SSL_MODE" ]; then + # Set SSL mode as environment variable (psql reads PGSSLMODE) + export PGSSLMODE="$SSL_MODE" + + # Add SSL certificate parameters as environment variables if provided + if [ -n "$SSL_CERTFILE" ]; then + export PGSSLROOTCERT="$SSL_CERTFILE" + fi + + if [ -n "$SSL_CLIENT_CERT" ]; then + export PGSSLCERT="$SSL_CLIENT_CERT" + fi + + if [ -n "$SSL_CLIENT_KEY" ]; then + export PGSSLKEY="$SSL_CLIENT_KEY" + fi + else + # Disable SSL if TLS is not enabled + export PGSSLMODE="disable" + fi + + # Wait for PostgreSQL to be ready with authentication and TLS + echo "Waiting for PostgreSQL to be ready..." + until psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d postgres -c "SELECT 1" >/dev/null 2>&1; do + echo 'PostgreSQL is not ready yet...' + sleep 5 + done + echo "PostgreSQL is ready!" + env: + - name: ES_ENABLED + value: {{ $.Values.config.persistence.elasticsearch.enabled | quote }} + # Basic connection parameters + - name: DB_HOST + value: {{ $.Values.config.persistence.database.sql.hosts | quote }} + - name: DB_PORT + value: {{ $.Values.config.persistence.database.postgres.port | default $.Values.config.persistence.database.sql.port | default 5432 | quote }} + - name: DB_NAME + value: {{ $.Values.config.persistence.database.sql.dbname | quote }} + - name: DB_VISIBILITY_NAME + value: {{ $.Values.config.persistence.database.sql.visibilityDbname | quote }} + - name: DB_USER + value: {{ $.Values.config.persistence.database.sql.user | quote }} + # Authentication parameters + - name: POSTGRES_PWD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-frontend-secrets + key: POSTGRES_PWD + # TLS Configuration + - name: TLS_ENABLED + value: {{ $.Values.config.persistence.database.sql.tls.enabled | quote }} + {{- if $.Values.config.persistence.database.sql.tls.enabled }} + # SSL Mode + - name: SSL_MODE + value: {{ $.Values.config.persistence.database.sql.tls.sslMode | default "require" | quote }} + # SSL_CERTFILE environment variable (CA certificate) + {{- if $.Values.config.persistence.database.sql.tls.caFile }} + - name: SSL_CERTFILE + value: {{ $.Values.config.persistence.database.sql.tls.caFile | quote }} + {{- else if $.Values.config.persistence.database.sql.tls.caFiles }} + - name: SSL_CERTFILE + value: {{ index $.Values.config.persistence.database.sql.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if $.Values.config.persistence.database.sql.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ $.Values.config.persistence.database.sql.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if $.Values.config.persistence.database.sql.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ $.Values.config.persistence.database.sql.tls.keyFile | quote }} + {{- end }} + {{- end }} + {{- else if eq $dbDriver "mysql" }} + image: {{ $.Values.schema.checkSchema.mysql.image.repository }}:{{ $.Values.schema.checkSchema.mysql.image.tag }} + imagePullPolicy: {{ $.Values.schema.checkSchema.mysql.image.pullPolicy }} + command: + - sh + - -c + - | + # Build connection string based on TLS configuration + build_mysql_cmd() { + local cmd="mariadb -h $DB_HOST -P $DB_PORT -u $DB_USER -p$MYSQL_PWD" + + # Add SSL parameters if TLS is enabled + if [ "$TLS_ENABLED" = "true" ]; then + case "$SSL_MODE" in + "disable"|"false") + cmd="$cmd --skip-ssl" + ;; + "preferred") + ;; + "required"|"true"|"skip-verify") + cmd="$cmd --ssl --ssl-verify-server-cert=off" + ;; + "verify-ca") + cmd="$cmd --ssl --ssl-verify-server-cert" + ;; + "verify-identity") + cmd="$cmd --ssl --ssl-verify-server-cert" + ;; + *) + cmd="$cmd --ssl" + ;; + esac + + # Add SSL certificate parameters if provided + if [ -n "$SSL_CERTFILE" ]; then + cmd="$cmd --ssl-ca=$SSL_CERTFILE" + fi + + if [ -n "$SSL_CLIENT_CERT" ]; then + cmd="$cmd --ssl-cert=$SSL_CLIENT_CERT" + fi + + if [ -n "$SSL_CLIENT_KEY" ]; then + cmd="$cmd --ssl-key=$SSL_CLIENT_KEY" + fi + else + cmd="$cmd --skip-ssl" + fi + + echo "$cmd" + } + + # Wait for MySQL to be ready + echo "Waiting for MySQL to be ready..." + until $(build_mysql_cmd) -e "SELECT 1" >/dev/null 2>&1; do + echo 'MySQL is not ready yet...' + sleep 5 + done + echo "MySQL is ready!" + env: + - name: ES_ENABLED + value: {{ $.Values.config.persistence.elasticsearch.enabled | quote }} + # Basic connection parameters + - name: DB_HOST + value: {{ $.Values.config.persistence.database.sql.hosts | quote }} + - name: DB_PORT + value: {{ $.Values.config.persistence.database.mysql.port | default $.Values.config.persistence.database.sql.port | default 3306 | quote }} + - name: DB_NAME + value: {{ $.Values.config.persistence.database.sql.dbname | quote }} + - name: DB_VISIBILITY_NAME + value: {{ $.Values.config.persistence.database.sql.visibilityDbname | quote }} + - name: DB_USER + value: {{ $.Values.config.persistence.database.sql.user | quote }} + # Authentication parameters + - name: MYSQL_PWD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-frontend-secrets + key: MYSQL_PWD + # TLS Configuration + - name: TLS_ENABLED + value: {{ $.Values.config.persistence.database.sql.tls.enabled | quote }} + {{- if $.Values.config.persistence.database.sql.tls.enabled }} + # SSL Mode + - name: SSL_MODE + value: {{ $.Values.config.persistence.database.sql.tls.sslMode | default "require" | quote }} + # SSL_CERTFILE environment variable (CA certificate) + {{- if $.Values.config.persistence.database.sql.tls.caFile }} + - name: SSL_CERTFILE + value: {{ $.Values.config.persistence.database.sql.tls.caFile | quote }} + {{- else if $.Values.config.persistence.database.sql.tls.caFiles }} + - name: SSL_CERTFILE + value: {{ index $.Values.config.persistence.database.sql.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if $.Values.config.persistence.database.sql.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ $.Values.config.persistence.database.sql.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if $.Values.config.persistence.database.sql.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ $.Values.config.persistence.database.sql.tls.keyFile | quote }} + {{- end }} + {{- end }} + {{- end }} + volumeMounts: + {{- with $.Values.global.tls.volumeMounts }} + {{- toYaml . | nindent 8 }} + {{- end }} + containers: + - name: cadence-schema-server + {{- $globalImage := .Values.global.image | default dict }} + {{- $schemaImage := $.Values.frontend.image | default dict }} + {{- $repository := $schemaImage.repository | default $globalImage.repository }} + {{- $tag := $schemaImage.tag | default $globalImage.tag }} + image: {{ $repository }}:{{ $tag }} + {{- $pullPolicy := $schemaImage.pullPolicy | default $globalImage.pullPolicy | default "IfNotPresent" }} + imagePullPolicy: {{ $pullPolicy }} + {{- if eq $dbDriver "cassandra" }} + # Cassandra Schema Setup + command: + - sh + - -c + - | + set -e + echo "Starting Cadence schema setup for driver: $DB_DRIVER" + echo "=== Setting up Cassandra Schema ===" + + # Build cassandra-tool command with TLS options + build_cassandra_cmd() { + local cmd="cadence-cassandra-tool --ep $DB_HOST" + + # Add authentication + if [ -n "$DB_USER" ]; then + cmd="$cmd -u $DB_USER" + fi + if [ -n "$CASSANDRA_PASSWORD" ]; then + cmd="$cmd -pw $CASSANDRA_PASSWORD" + fi + + # Add protocol version + cmd="$cmd -pv $PROTOCOL_VERSION" + + # Add allowed authenticators from environment variable + if [ -n "$ALLOWED_AUTHENTICATORS" ]; then + cmd="$cmd $ALLOWED_AUTHENTICATORS" + fi + + # Add TLS options if enabled + if [ "$TLS_ENABLED" = "true" ]; then + cmd="$cmd --tls" + if [ -n "$SSL_CERTFILE" ]; then + cmd="$cmd --tls-ca-file $SSL_CERTFILE" + fi + if [ -n "$SSL_CLIENT_CERT" ]; then + cmd="$cmd --tls-cert-file $SSL_CLIENT_CERT" + fi + if [ -n "$SSL_CLIENT_KEY" ]; then + cmd="$cmd --tls-key-file $SSL_CLIENT_KEY" + fi + fi + + echo "$cmd" + } + + # Setup main database schema + echo "Creating main keyspace: $DB_NAME" + if [ "$DATA_CENTER" = "" ]; then + $(build_cassandra_cmd) create -k $DB_NAME --rf $REPLICATION_FACTOR + else + $(build_cassandra_cmd) create -k $DB_NAME --rf $REPLICATION_FACTOR -dc $DATA_CENTER + fi + + echo "Setting up main schema version 0.0" + $(build_cassandra_cmd) -k $DB_NAME setup-schema -v 0.0 || echo "Schema already exists" + + echo "Updating main schema to latest version" + $(build_cassandra_cmd) -k $DB_NAME update-schema -d $CADENCE_HOME/schema/cassandra/cadence/versioned || echo "Rollback is not allowed" + + # Setup visibility database schema (only if ES is not enabled) + if [ "$ES_ENABLED" = "false" ]; then + echo "Creating visibility keyspace: $DB_VISIBILITY_NAME" + if [ "$DATA_CENTER" = "" ]; then + $(build_cassandra_cmd) create -k $DB_VISIBILITY_NAME --rf $REPLICATION_FACTOR + else + $(build_cassandra_cmd) create -k $DB_VISIBILITY_NAME --rf $REPLICATION_FACTOR -dc $DATA_CENTER + fi + + echo "Setting up visibility schema version 0.0" + $(build_cassandra_cmd) -k $DB_VISIBILITY_NAME setup-schema -v 0.0 || echo "Schema already exists" + + echo "Updating visibility schema to latest version" + $(build_cassandra_cmd) -k $DB_VISIBILITY_NAME update-schema -d $CADENCE_HOME/schema/cassandra/visibility/versioned || echo "Rollback is not allowed" + else + echo "Skipping visibility schema setup (Elasticsearch enabled)" + fi + + echo "Schema setup completed successfully!" + env: + # Common environment variables + - name: DB_DRIVER + value: {{ $dbDriver | quote }} + - name: CADENCE_HOME + value: "/etc/cadence" + - name: ES_ENABLED + value: {{ .Values.config.persistence.elasticsearch.enabled | quote }} + # Cassandra specific environment variables + - name: DB_HOST + value: {{ .Values.config.persistence.database.cassandra.hosts | quote }} + - name: DB_NAME + value: {{ .Values.config.persistence.database.cassandra.keyspace | quote }} + - name: PROTOCOL_VERSION + value: {{ .Values.config.persistence.database.cassandra.protoVersion | quote }} + - name: DB_VISIBILITY_NAME + value: {{ .Values.config.persistence.database.cassandra.visibilityKeyspace | quote }} + - name: REPLICATION_FACTOR + value: {{ .Values.config.persistence.database.cassandra.replicationFactor | default 1 | quote }} + - name: DATA_CENTER + value: {{ .Values.config.persistence.database.cassandra.datacenter | quote }} + # Allowed authenticators (build --aa parameters) + - name: ALLOWED_AUTHENTICATORS + value: {{ if .Values.config.persistence.database.cassandra.allowedAuthenticators }}{{ range $index, $auth := .Values.config.persistence.database.cassandra.allowedAuthenticators }}{{ if $index }} {{ end }}--aa {{ $auth | quote }}{{ end }}{{ else }}"--aa org.apache.cassandra.auth.PasswordAuthenticator"{{ end }} + # Authentication parameters (conditional) + {{- if .Values.config.persistence.database.cassandra.user }} + - name: DB_USER + value: {{ .Values.config.persistence.database.cassandra.user | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.password }} + - name: CASSANDRA_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-frontend-secrets + key: CASSANDRA_PASSWORD + {{- end }} + # TLS Configuration + - name: TLS_ENABLED + value: {{ .Values.config.persistence.database.cassandra.tls.enabled | quote }} + {{- if .Values.config.persistence.database.cassandra.tls.enabled }} + # SSL_CERTFILE environment variable (CA certificate) + {{- if .Values.config.persistence.database.cassandra.tls.caFile }} + - name: SSL_CERTFILE + value: {{ .Values.config.persistence.database.cassandra.tls.caFile | quote }} + {{- else if .Values.config.persistence.database.cassandra.tls.caFiles }} + - name: SSL_CERTFILE + value: {{ index .Values.config.persistence.database.cassandra.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if .Values.config.persistence.database.cassandra.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ .Values.config.persistence.database.cassandra.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if .Values.config.persistence.database.cassandra.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ .Values.config.persistence.database.cassandra.tls.keyFile | quote }} + {{- end }} + - name: CASSANDRA_TLS_SERVER_NAME + value: {{ .Values.config.persistence.database.cassandra.tls.serverName | quote }} + {{- end }} + + {{- else if eq $dbDriver "postgres" }} + # PostgreSQL Schema Setup + command: + - sh + - -c + - | + set -e + echo "Starting Cadence schema setup for driver: $DB_DRIVER" + echo "=== Setting up PostgreSQL Schema ===" + + # Build sql-tool command with TLS options + build_postgres_cmd() { + local cmd="cadence-sql-tool --ep $DB_HOST -p $DB_PORT -u $DB_USER -pw $POSTGRES_PWD --plugin postgres" + + # Add TLS options if enabled + if [ "$TLS_ENABLED" = "true" ]; then + cmd="$cmd --tls" + if [ -n "$SSL_CERTFILE" ]; then + cmd="$cmd --tls-ca-file $SSL_CERTFILE" + fi + if [ -n "$SSL_CLIENT_CERT" ]; then + cmd="$cmd --tls-cert-file $SSL_CLIENT_CERT" + fi + if [ -n "$SSL_CLIENT_KEY" ]; then + cmd="$cmd --tls-key-file $SSL_CLIENT_KEY" + fi + fi + + echo "$cmd" + } + + # Create main database + echo "Creating main database: $DB_NAME" + $(build_postgres_cmd) create-database --db $DB_NAME + + echo "Setting up main schema version 0.0" + $(build_postgres_cmd) --db $DB_NAME setup-schema -v 0.0 || echo "Schema already exists" + + echo "Updating main schema to latest version" + $(build_postgres_cmd) --db $DB_NAME update-schema -d $CADENCE_HOME/schema/postgres/cadence/versioned || echo "Rollback is not allowed" + + # Setup visibility database (only if ES is not enabled) + if [ "$ES_ENABLED" = "false" ]; then + echo "Creating visibility database: $DB_VISIBILITY_NAME" + $(build_postgres_cmd) create-database --db $DB_VISIBILITY_NAME + + echo "Setting up visibility schema version 0.0" + $(build_postgres_cmd) --db $DB_VISIBILITY_NAME setup-schema -v 0.0 || echo "Schema already exists" + + echo "Updating visibility schema to latest version" + $(build_postgres_cmd) --db $DB_VISIBILITY_NAME update-schema -d $CADENCE_HOME/schema/postgres/visibility/versioned || echo "Rollback is not allowed" + else + echo "Skipping visibility schema setup (Elasticsearch enabled)" + fi + + echo "Schema setup completed successfully!" + env: + # Common environment variables + - name: DB_DRIVER + value: {{ $dbDriver | quote }} + - name: CADENCE_HOME + value: "/etc/cadence" + - name: ES_ENABLED + value: {{ .Values.config.persistence.elasticsearch.enabled | quote }} + # PostgreSQL specific environment variables + - name: DB_HOST + value: {{ .Values.config.persistence.database.sql.hosts | quote }} + - name: DB_PORT + value: {{ .Values.config.persistence.database.postgres.port | default .Values.config.persistence.database.sql.port | default 5432 | quote }} + - name: DB_NAME + value: {{ .Values.config.persistence.database.sql.dbname | quote }} + - name: DB_VISIBILITY_NAME + value: {{ .Values.config.persistence.database.sql.visibilityDbname | quote }} + - name: DB_USER + value: {{ .Values.config.persistence.database.sql.user | quote }} + # Authentication parameters + - name: POSTGRES_PWD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-frontend-secrets + key: POSTGRES_PWD + # TLS Configuration + - name: TLS_ENABLED + value: {{ .Values.config.persistence.database.sql.tls.enabled | quote }} + {{- if .Values.config.persistence.database.sql.tls.enabled }} + # SSL_CERTFILE environment variable (CA certificate) + {{- if .Values.config.persistence.database.sql.tls.caFile }} + - name: SSL_CERTFILE + value: {{ .Values.config.persistence.database.sql.tls.caFile | quote }} + {{- else if .Values.config.persistence.database.sql.tls.caFiles }} + - name: SSL_CERTFILE + value: {{ index .Values.config.persistence.database.sql.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if .Values.config.persistence.database.sql.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ .Values.config.persistence.database.sql.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if .Values.config.persistence.database.sql.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ .Values.config.persistence.database.sql.tls.keyFile | quote }} + {{- end }} + {{- end }} + + {{- else if eq $dbDriver "mysql" }} + # MySQL Schema Setup + command: + - sh + - -c + - | + set -e + echo "Starting Cadence schema setup for driver: $DB_DRIVER" + echo "=== Setting up MySQL Schema ===" + + # Build sql-tool command with TLS options + build_mysql_cmd() { + local cmd="cadence-sql-tool --ep $DB_HOST -p $DB_PORT -u $DB_USER -pw $MYSQL_PWD --plugin mysql" + + # Add TLS options if enabled + if [ "$TLS_ENABLED" = "true" ]; then + cmd="$cmd --tls" + if [ -n "$SSL_CERTFILE" ]; then + cmd="$cmd --tls-ca-file $SSL_CERTFILE" + fi + if [ -n "$SSL_CLIENT_CERT" ]; then + cmd="$cmd --tls-cert-file $SSL_CLIENT_CERT" + fi + if [ -n "$SSL_CLIENT_KEY" ]; then + cmd="$cmd --tls-key-file $SSL_CLIENT_KEY" + fi + fi + + echo "$cmd" + } + + # Create main database + echo "Creating main database: $DB_NAME" + $(build_mysql_cmd) create-database --db $DB_NAME || echo "Database already exists" + + echo "Setting up main schema version 0.0" + $(build_mysql_cmd) --db $DB_NAME setup-schema -v 0.0 || echo "Schema already exists" + + echo "Updating main schema to latest version" + $(build_mysql_cmd) --db $DB_NAME update-schema -d $CADENCE_HOME/schema/mysql/v8/cadence/versioned || echo "Rollback is not allowed" + + # Setup visibility database (only if ES is not enabled) + if [ "$ES_ENABLED" = "false" ]; then + echo "Creating visibility database: $DB_VISIBILITY_NAME" + $(build_mysql_cmd) create-database --db $DB_VISIBILITY_NAME || echo "Database already exists" + + echo "Setting up visibility schema version 0.0" + $(build_mysql_cmd) --db $DB_VISIBILITY_NAME setup-schema -v 0.0 || echo "Schema already exists" + + echo "Updating visibility schema to latest version" + $(build_mysql_cmd) --db $DB_VISIBILITY_NAME update-schema -d $CADENCE_HOME/schema/mysql/v8/visibility/versioned || echo "Rollback is not allowed" + else + echo "Skipping visibility schema setup (Elasticsearch enabled)" + fi + + echo "Schema setup completed successfully!" + env: + # Common environment variables + - name: DB_DRIVER + value: {{ $dbDriver | quote }} + - name: CADENCE_HOME + value: "/etc/cadence" + - name: ES_ENABLED + value: {{ .Values.config.persistence.elasticsearch.enabled | quote }} + # MySQL specific environment variables + - name: DB_HOST + value: {{ .Values.config.persistence.database.sql.hosts | quote }} + - name: DB_PORT + value: {{ .Values.config.persistence.database.mysql.port | default .Values.config.persistence.database.sql.port | default 3306 | quote }} + - name: DB_NAME + value: {{ .Values.config.persistence.database.sql.dbname | quote }} + - name: DB_VISIBILITY_NAME + value: {{ .Values.config.persistence.database.sql.visibilityDbname | quote }} + - name: DB_USER + value: {{ .Values.config.persistence.database.sql.user | quote }} + # Authentication parameters + - name: MYSQL_PWD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-frontend-secrets + key: MYSQL_PWD + # TLS Configuration + - name: TLS_ENABLED + value: {{ .Values.config.persistence.database.sql.tls.enabled | quote }} + {{- if .Values.config.persistence.database.sql.tls.enabled }} + # SSL_CERTFILE environment variable (CA certificate) + {{- if .Values.config.persistence.database.sql.tls.caFile }} + - name: SSL_CERTFILE + value: {{ .Values.config.persistence.database.sql.tls.caFile | quote }} + {{- else if .Values.config.persistence.database.sql.tls.caFiles }} + - name: SSL_CERTFILE + value: {{ index .Values.config.persistence.database.sql.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if .Values.config.persistence.database.sql.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ .Values.config.persistence.database.sql.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if .Values.config.persistence.database.sql.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ .Values.config.persistence.database.sql.tls.keyFile | quote }} + {{- end }} + {{- end }} + {{- end }} + volumeMounts: + {{- with .Values.global.tls.volumeMounts }} + {{- toYaml . | nindent 8 }} + {{- end }} + {{- with .Values.schema.resources }} + resources: + {{- toYaml . | nindent 10 }} + {{- end }} + volumes: + {{- with .Values.global.tls.volumes }} + {{- toYaml . | nindent 6 }} + {{- end }} +{{- end }} \ No newline at end of file diff --git a/charts/cadence/templates/server-configmap.yaml b/charts/cadence/templates/server-configmap.yaml index bc09c15..679220f 100644 --- a/charts/cadence/templates/server-configmap.yaml +++ b/charts/cadence/templates/server-configmap.yaml @@ -12,4 +12,731 @@ data: {{- else }} # Default dynamic configuration - empty config is valid {} + {{- end }} + + config_template.yaml: |- + # Log configuration + log: + {{- if .Values.config.log.useEnvVars }} + stdout: {{ "{{ default .Env.LOG_STDOUT \"" }}{{ .Values.config.log.stdout | default .Values.global.log.stdout | default true }}{{ "\" }}" }} + level: {{ "{{ default .Env.LOG_LEVEL \"" }}{{ .Values.config.log.level | default .Values.global.log.level | default "info" }}{{ "\" }}" }} + {{- else }} + stdout: {{ .Values.config.log.stdout | default .Values.global.log.stdout | default true }} + level: {{ .Values.config.log.level | default .Values.global.log.level | default "info" }} + {{- end }} + {{- if .Values.config.log.outputFile }} + outputFile: {{ .Values.config.log.outputFile | quote }} + {{- end }} + {{- if .Values.config.log.levelKey }} + levelKey: {{ .Values.config.log.levelKey }} + {{- else }} + levelKey: level + {{- end }} + {{- if .Values.config.log.encoding }} + encoding: {{ .Values.config.log.encoding }} + {{- else }} + encoding: json + {{- end }} + + # Persistence configuration + persistence: + numHistoryShards: {{ .Values.config.persistence.numHistoryShards | default 4 }} + defaultStore: {{ .Values.config.persistence.defaultStore | default "default" | quote }} + {{- if and .Values.config.persistence.visibilityStore (not .Values.config.persistence.elasticsearch.enabled) }} + visibilityStore: {{ .Values.config.persistence.visibilityStore | quote }} + {{- end }} + {{- if and .Values.config.persistence.advancedVisibilityStore .Values.config.persistence.elasticsearch.enabled }} + advancedVisibilityStore: {{ .Values.config.persistence.advancedVisibilityStore | quote }} + {{- end }} + enablePersistenceLatencyHistogramMetrics: {{ .Values.config.persistence.enablePersistenceLatencyHistogramMetrics | default false }} + + # DataStores configuration + datastores: + # Default datastore + {{ .Values.config.persistence.defaultStore | default "default" }}: + {{- if eq .Values.config.persistence.database.driver "cassandra" }} + nosql: + pluginName: "cassandra" + hosts: {{ .Values.config.persistence.database.cassandra.hosts | quote }} + {{- $port := .Values.config.persistence.database.cassandra.port | default 9042 | int }} + {{- if ne $port 9042 }} + port: {{ $port }} + {{- end }} + keyspace: {{ .Values.config.persistence.database.cassandra.keyspace | default "cadence" | quote }} + {{- if .Values.config.persistence.database.cassandra.user }} + user: {{ .Values.config.persistence.database.cassandra.user | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.password }} + password: {{ `{{ .Env.CASSANDRA_PASSWORD }}` }} + {{- end }} + protoVersion: {{ .Values.config.persistence.database.cassandra.protoVersion | default 4 }} + {{- if .Values.config.persistence.database.cassandra.region }} + region: {{ .Values.config.persistence.database.cassandra.region | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.datacenter }} + datacenter: {{ .Values.config.persistence.database.cassandra.datacenter | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.maxConns }} + maxConns: {{ .Values.config.persistence.database.cassandra.maxConns }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.connectTimeout }} + connectTimeout: {{ .Values.config.persistence.database.cassandra.connectTimeout | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.timeout }} + timeout: {{ .Values.config.persistence.database.cassandra.timeout | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.consistency }} + consistency: {{ .Values.config.persistence.database.cassandra.consistency | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.serialConsistency }} + serialConsistency: {{ .Values.config.persistence.database.cassandra.serialConsistency | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.hostSelectionPolicy }} + hostSelectionPolicy: {{ .Values.config.persistence.database.cassandra.hostSelectionPolicy | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.allowedAuthenticators }} + allowedAuthenticators: + {{- range .Values.config.persistence.database.cassandra.allowedAuthenticators }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.connectAttributes }} + connectAttributes: + {{- range $key, $value := .Values.config.persistence.database.cassandra.connectAttributes }} + {{ $key }}: {{ $value | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.tls.enabled }} + tls: + enabled: {{ .Values.config.persistence.database.cassandra.tls.enabled }} + {{- if .Values.config.persistence.database.cassandra.tls.caFile }} + caFile: {{ .Values.config.persistence.database.cassandra.tls.caFile | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.tls.caFiles }} + caFiles: + {{- range .Values.config.persistence.database.cassandra.tls.caFiles }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.tls.certFile }} + certFile: {{ .Values.config.persistence.database.cassandra.tls.certFile | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.tls.keyFile }} + keyFile: {{ .Values.config.persistence.database.cassandra.tls.keyFile | quote }} + {{- end }} + enableHostVerification: {{ .Values.config.persistence.database.cassandra.tls.enableHostVerification | default true }} + {{- if .Values.config.persistence.database.cassandra.tls.requireClientAuth }} + requireClientAuth: {{ .Values.config.persistence.database.cassandra.tls.requireClientAuth }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.tls.serverName }} + serverName: {{ .Values.config.persistence.database.cassandra.tls.serverName | quote }} + {{- end }} + {{- end }} + {{ else -}} + {{/* SQL configuration for both MySQL and PostgreSQL */}} + sql: + {{- if eq .Values.config.persistence.database.driver "mysql" }} + pluginName: "mysql" + {{- $port := .Values.config.persistence.database.mysql.port | default .Values.config.persistence.database.sql.port | default 3306 }} + connectAddr: "{{ .Values.config.persistence.database.sql.hosts }}:{{ $port }}" + {{- else if eq .Values.config.persistence.database.driver "postgres" }} + pluginName: "postgres" + {{- $port := .Values.config.persistence.database.postgres.port | default .Values.config.persistence.database.sql.port | default 5432 }} + connectAddr: "{{ .Values.config.persistence.database.sql.hosts }}:{{ $port }}" + {{- end }} + connectProtocol: "tcp" + databaseName: {{ .Values.config.persistence.database.sql.dbname | default "cadence" | quote }} + user: {{ .Values.config.persistence.database.sql.user | default "cadence" | quote }} + {{- if .Values.config.persistence.database.sql.password }} + {{- if eq .Values.config.persistence.database.driver "mysql" }} + password: {{ `{{ .Env.MYSQL_PWD }}` }} + {{- else if eq .Values.config.persistence.database.driver "postgres" }} + password: {{ `{{ .Env.POSTGRES_PWD }}` }} + {{- end }} + {{- end }} + maxConns: {{ .Values.config.persistence.database.sql.maxConns | default 20 }} + maxIdleConns: {{ .Values.config.persistence.database.sql.maxIdleConns | default 20 }} + maxConnLifetime: {{ .Values.config.persistence.database.sql.maxConnLifetime | default "1h" | quote }} + {{- if ne (.Values.config.persistence.database.sql.numShards | default 1 | int ) 1 }} + nShards: {{ .Values.config.persistence.database.sql.numShards }} + {{- end }} + {{- if .Values.config.persistence.database.sql.encodingType }} + encodingType: {{ .Values.config.persistence.database.sql.encodingType | quote }} + {{- end }} + {{- if .Values.config.persistence.database.sql.decodingTypes }} + decodingTypes: + {{- range .Values.config.persistence.database.sql.decodingTypes }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.sql.useMultipleDatabases }} + useMultipleDatabases: {{ .Values.config.persistence.database.sql.useMultipleDatabases }} + {{- end }} + {{- if .Values.config.persistence.database.sql.multipleDatabasesConfig }} + multipleDatabasesConfig: + {{- range .Values.config.persistence.database.sql.multipleDatabasesConfig }} + - {{ . | toYaml | nindent 14 }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.sql.connectAttributes }} + connectAttributes: + {{- range $key, $value := .Values.config.persistence.database.sql.connectAttributes }} + {{ $key }}: {{ $value | quote }} + {{- end }} + {{- end }} + {{- if eq .Values.config.persistence.database.driver "mysql" }} + {{- if .Values.config.persistence.database.mysql.txIsolationCompat }} + txIsolationCompat: {{ .Values.config.persistence.database.mysql.txIsolationCompat }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.enabled }} + tls: + enabled: {{ .Values.config.persistence.database.sql.tls.enabled }} + {{- if .Values.config.persistence.database.sql.tls.sslMode }} + sslmode: {{ .Values.config.persistence.database.sql.tls.sslMode | quote }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.caFile }} + caFile: {{ .Values.config.persistence.database.sql.tls.caFile | quote }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.caFiles }} + caFiles: + {{- range .Values.config.persistence.database.sql.tls.caFiles }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.certFile }} + certFile: {{ .Values.config.persistence.database.sql.tls.certFile | quote }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.keyFile }} + keyFile: {{ .Values.config.persistence.database.sql.tls.keyFile | quote }} + {{- end }} + enableHostVerification: {{ .Values.config.persistence.database.sql.tls.enableHostVerification | default true }} + {{- if .Values.config.persistence.database.sql.tls.requireClientAuth }} + requireClientAuth: {{ .Values.config.persistence.database.sql.tls.requireClientAuth }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.serverName }} + serverName: {{ .Values.config.persistence.database.sql.tls.serverName | quote }} + {{- end }} + {{- end }} + {{- end }} + {{- if ne .Values.config.persistence.elasticsearch.enabled true }} + # Visibility datastore + {{ if .Values.config.persistence.visibilityStore -}} + {{ .Values.config.persistence.visibilityStore | default "visibility" }}: + {{- if eq .Values.config.persistence.database.driver "cassandra" }} + nosql: + pluginName: "cassandra" + hosts: {{ .Values.config.persistence.database.cassandra.hosts | quote }} + {{- $port := .Values.config.persistence.database.cassandra.port | default 9042 | int }} + {{- if ne $port 9042 }} + port: {{ $port }} + {{- end }} + keyspace: {{ .Values.config.persistence.database.cassandra.visibilityKeyspace | default "cadence_visibility" | quote }} + {{- if .Values.config.persistence.database.cassandra.user }} + user: {{ .Values.config.persistence.database.cassandra.user | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.password }} + password: {{ `{{ .Env.CASSANDRA_PASSWORD }}` }} + {{- end }} + protoVersion: {{ .Values.config.persistence.database.cassandra.protoVersion | default 4 }} + {{- if .Values.config.persistence.database.cassandra.region }} + region: {{ .Values.config.persistence.database.cassandra.region | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.datacenter }} + datacenter: {{ .Values.config.persistence.database.cassandra.datacenter | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.maxConns }} + maxConns: {{ .Values.config.persistence.database.cassandra.maxConns }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.connectTimeout }} + connectTimeout: {{ .Values.config.persistence.database.cassandra.connectTimeout | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.timeout }} + timeout: {{ .Values.config.persistence.database.cassandra.timeout | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.consistency }} + consistency: {{ .Values.config.persistence.database.cassandra.consistency | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.serialConsistency }} + serialConsistency: {{ .Values.config.persistence.database.cassandra.serialConsistency | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.hostSelectionPolicy }} + hostSelectionPolicy: {{ .Values.config.persistence.database.cassandra.hostSelectionPolicy | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.allowedAuthenticators }} + allowedAuthenticators: + {{- range .Values.config.persistence.database.cassandra.allowedAuthenticators }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.connectAttributes }} + connectAttributes: + {{- range $key, $value := .Values.config.persistence.database.cassandra.connectAttributes }} + {{ $key }}: {{ $value | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.tls.enabled }} + tls: + enabled: {{ .Values.config.persistence.database.cassandra.tls.enabled }} + {{- if .Values.config.persistence.database.cassandra.tls.caFile }} + caFile: {{ .Values.config.persistence.database.cassandra.tls.caFile | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.tls.caFiles }} + caFiles: + {{- range .Values.config.persistence.database.cassandra.tls.caFiles }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.tls.certFile }} + certFile: {{ .Values.config.persistence.database.cassandra.tls.certFile | quote }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.tls.keyFile }} + keyFile: {{ .Values.config.persistence.database.cassandra.tls.keyFile | quote }} + {{- end }} + enableHostVerification: {{ .Values.config.persistence.database.cassandra.tls.enableHostVerification | default true }} + {{- if .Values.config.persistence.database.cassandra.tls.requireClientAuth }} + requireClientAuth: {{ .Values.config.persistence.database.cassandra.tls.requireClientAuth }} + {{- end }} + {{- if .Values.config.persistence.database.cassandra.tls.serverName }} + serverName: {{ .Values.config.persistence.database.cassandra.tls.serverName | quote }} + {{- end }} + {{- end }} + {{ else -}} + {{/* SQL configuration for visibility store */}} + sql: + {{- if eq .Values.config.persistence.database.driver "mysql" }} + pluginName: "mysql" + {{- $port := .Values.config.persistence.database.mysql.port | default .Values.config.persistence.database.sql.port | default 3306 }} + connectAddr: "{{ .Values.config.persistence.database.sql.hosts }}:{{ $port }}" + {{- else if eq .Values.config.persistence.database.driver "postgres" }} + pluginName: "postgres" + {{- $port := .Values.config.persistence.database.postgres.port | default .Values.config.persistence.database.sql.port | default 5432 }} + connectAddr: "{{ .Values.config.persistence.database.sql.hosts }}:{{ $port }}" + {{- end }} + connectProtocol: "tcp" + databaseName: {{ .Values.config.persistence.database.sql.visibilityDbname | default "cadence_visibility" | quote }} + user: {{ .Values.config.persistence.database.sql.user | default "cadence" | quote }} + {{- if .Values.config.persistence.database.sql.password }} + {{- if eq .Values.config.persistence.database.driver "mysql" }} + password: {{ `{{ .Env.MYSQL_PWD }}` }} + {{- else if eq .Values.config.persistence.database.driver "postgres" }} + password: {{ `{{ .Env.POSTGRES_PWD }}` }} + {{- end }} + {{- end }} + maxConns: {{ .Values.config.persistence.database.sql.maxConns | default 20 }} + maxIdleConns: {{ .Values.config.persistence.database.sql.maxIdleConns | default 20 }} + maxConnLifetime: {{ .Values.config.persistence.database.sql.maxConnLifetime | default "1h" | quote }} + {{- if ne (.Values.config.persistence.database.sql.numShards | default 1 | int ) 1 }} + nShards: {{ .Values.config.persistence.database.sql.numShards }} + {{- end }} + {{- if .Values.config.persistence.database.sql.encodingType }} + encodingType: {{ .Values.config.persistence.database.sql.encodingType | quote }} + {{- end }} + {{- if .Values.config.persistence.database.sql.decodingTypes }} + decodingTypes: + {{- range .Values.config.persistence.database.sql.decodingTypes }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.sql.useMultipleDatabases }} + useMultipleDatabases: {{ .Values.config.persistence.database.sql.useMultipleDatabases }} + {{- end }} + {{- if .Values.config.persistence.database.sql.multipleDatabasesConfig }} + multipleDatabasesConfig: + {{- range .Values.config.persistence.database.sql.multipleDatabasesConfig }} + - {{ . | toYaml | nindent 14 }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.sql.connectAttributes }} + connectAttributes: + {{- range $key, $value := .Values.config.persistence.database.sql.connectAttributes }} + {{ $key }}: {{ $value | quote }} + {{- end }} + {{- end }} + {{- if eq .Values.config.persistence.database.driver "mysql" }} + {{- if .Values.config.persistence.database.mysql.txIsolationCompat }} + txIsolationCompat: {{ .Values.config.persistence.database.mysql.txIsolationCompat }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.enabled }} + tls: + enabled: {{ .Values.config.persistence.database.sql.tls.enabled }} + {{- if .Values.config.persistence.database.sql.tls.sslMode }} + sslmode: {{ .Values.config.persistence.database.sql.tls.sslMode | quote }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.caFile }} + caFile: {{ .Values.config.persistence.database.sql.tls.caFile | quote }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.caFiles }} + caFiles: + {{- range .Values.config.persistence.database.sql.tls.caFiles }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.certFile }} + certFile: {{ .Values.config.persistence.database.sql.tls.certFile | quote }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.keyFile }} + keyFile: {{ .Values.config.persistence.database.sql.tls.keyFile | quote }} + {{- end }} + enableHostVerification: {{ .Values.config.persistence.database.sql.tls.enableHostVerification | default true }} + {{- if .Values.config.persistence.database.sql.tls.requireClientAuth }} + requireClientAuth: {{ .Values.config.persistence.database.sql.tls.requireClientAuth }} + {{- end }} + {{- if .Values.config.persistence.database.sql.tls.serverName }} + serverName: {{ .Values.config.persistence.database.sql.tls.serverName | quote }} + {{- end }} + {{- end }} + {{- end }} + {{- end }} + {{- end }} + + {{- if .Values.config.persistence.elasticsearch.enabled }} + {{- if eq .Values.config.persistence.advancedVisibilityStore "es-visibility" }} + # Elasticsearch for advanced visibility + es-visibility: + elasticsearch: + disableSniff: {{ .Values.config.persistence.elasticsearch.disableSniff | default true }} + version: {{ .Values.config.persistence.elasticsearch.version | default "v7" | quote }} + url: + {{- if .Values.config.persistence.elasticsearch.protocol }} + scheme: {{ .Values.config.persistence.elasticsearch.protocol | default "http" }} + {{- else if and .Values.config.persistence.elasticsearch.tls.enabled (not .Values.config.persistence.elasticsearch.protocol) }} + scheme: "https" + {{- else }} + scheme: "http" + {{- end }} + host: {{ .Values.config.persistence.elasticsearch.hosts }}:{{ .Values.config.persistence.elasticsearch.port | default 9200 }} + indices: + visibility: {{ .Values.config.persistence.elasticsearch.visibilityIndex | default "cadence-visibility" | quote }} + {{- if .Values.config.persistence.elasticsearch.user }} + username: {{ .Values.config.persistence.elasticsearch.user | quote }} + {{- end }} + {{- if .Values.config.persistence.elasticsearch.password }} + password: {{ `{{ .Env.ES_PWD }}` }} + {{- end }} + {{- if .Values.config.persistence.elasticsearch.awsSigning.enabled }} + awsConfig: + enabled: {{ .Values.config.persistence.elasticsearch.awsSigning.enabled }} + region: {{ .Values.config.persistence.elasticsearch.awsSigning.region | quote }} + service: {{ .Values.config.persistence.elasticsearch.awsSigning.service | default "es" | quote }} + {{- end }} + {{- if .Values.config.persistence.elasticsearch.tls.enabled }} + tls: + enabled: {{ .Values.config.persistence.elasticsearch.tls.enabled }} + {{- if .Values.config.persistence.elasticsearch.tls.caFile }} + caFile: {{ .Values.config.persistence.elasticsearch.tls.caFile | quote }} + {{- end }} + {{- if .Values.config.persistence.elasticsearch.tls.caFiles }} + caFiles: + {{- range .Values.config.persistence.elasticsearch.tls.caFiles }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.persistence.elasticsearch.tls.certFile }} + certFile: {{ .Values.config.persistence.elasticsearch.tls.certFile | quote }} + {{- end }} + {{- if .Values.config.persistence.elasticsearch.tls.keyFile }} + keyFile: {{ .Values.config.persistence.elasticsearch.tls.keyFile | quote }} + {{- end }} + {{- if .Values.config.persistence.elasticsearch.tls.serverName }} + serverName: {{ .Values.config.persistence.elasticsearch.tls.serverName | quote }} + {{- end }} + enableHostVerification: {{ .Values.config.persistence.elasticsearch.tls.enableHostVerification | default true }} + requireClientAuth: {{ .Values.config.persistence.elasticsearch.tls.requireClientAuth | default false }} + {{- end }} + {{- end }} + {{- end }} + + # Ringpop configuration + ringpop: + name: {{ .Values.config.ringpop.name | default "cadence" }} + bootstrapMode: {{ .Values.config.ringpop.bootstrapMode | default "dns" }} + bootstrapHosts: + - {{ include "cadence.fullname" . }}-frontend-headless:{{ .Values.frontend.port | default 7933 }} + - {{ include "cadence.fullname" . }}-history-headless:{{ .Values.history.port | default 7934 }} + - {{ include "cadence.fullname" . }}-matching-headless:{{ .Values.matching.port | default 7935 }} + - {{ include "cadence.fullname" . }}-worker-headless:{{ .Values.worker.port | default 7939 }} + maxJoinDuration: {{ .Values.config.ringpop.maxJoinDuration | default "30s" }} + + # Cluster configuration + clusterGroupMetadata: + failoverVersionIncrement: {{ .Values.config.cluster.failoverVersionIncrement | default 10 }} + primaryClusterName: {{ .Values.config.cluster.primaryClusterName | default "cluster0" }} + currentClusterName: {{ .Values.config.cluster.currentClusterName | default "cluster0" }} + clusterGroup: + {{ .Values.config.cluster.currentClusterName | default "cluster0" }}: + enabled: true + initialFailoverVersion: {{ .Values.config.cluster.initialFailoverVersion | default 0 }} + {{- $transport := .Values.config.cluster.rpcTransport | default "grpc" }} + {{- if eq $transport "grpc" }} + rpcAddress: {{ include "cadence.fullname" . }}-frontend:{{ .Values.frontend.grpcPort | default 7833 }} + {{- else }} + rpcAddress: {{ include "cadence.fullname" . }}-frontend:{{ .Values.frontend.port | default 7933 }} + {{- end }} + rpcTransport: {{ $transport | quote }} + {{- range $clusterName, $clusterConfig := .Values.config.cluster.clusterGroup }} + {{ $clusterName }}: + enabled: {{ $clusterConfig.enabled | default true }} + initialFailoverVersion: {{ $clusterConfig.initialFailoverVersion | default 0 }} + {{- $clusterTransport := $clusterConfig.rpcTransport | default "grpc" }} + {{- if $clusterConfig.rpcAddress }} + rpcAddress: {{ $clusterConfig.rpcAddress | quote }} + {{- else }} + {{- fail (printf "rpcAddress is required for cluster %s" $clusterName) }} + {{- end }} + rpcTransport: {{ $clusterTransport | quote }} + {{- end }} + {{- if .Values.config.cluster.clusterRedirectionPolicy }} + clusterRedirectionPolicy: + policy: {{ .Values.config.cluster.clusterRedirectionPolicy.policy | default "noop" }} + {{- end }} + + # Services configuration + services: + frontend: + rpc: + port: {{ .Values.frontend.port | default 7933 }} + grpcPort: {{ .Values.frontend.grpcPort | default 7833 }} + bindOnIP: {{ `{{ default .Env.POD_IP "0.0.0.0" }}` }} + grpcMaxMsgSize: {{ .Values.config.services.grpcMaxMsgSize | default 4194304 | int }} + metrics: + {{- if eq .Values.config.services.metrics.type "statsd" }} + statsd: + hostPort: {{ .Values.config.services.metrics.statsd.endpoint | quote }} + prefix: {{ .Values.config.services.metrics.statsd.prefixes.frontend | default "cadence-frontend" }} + {{- else if eq .Values.config.services.metrics.type "prometheus" }} + prometheus: + timerType: {{ .Values.config.services.metrics.prometheus.timerType | default "histogram" }} + listenAddress: "0.0.0.0:{{ .Values.metrics.port | default 9090 }}" + {{- end }} + {{- if .Values.config.services.pprof.enabled }} + pprof: + port: {{ .Values.config.services.pprof.ports.frontend | default 6060 }} + {{- end }} + + history: + rpc: + port: {{ .Values.history.port | default 7934 }} + grpcPort: {{ .Values.history.grpcPort | default 7834 }} + bindOnIP: {{ `{{ default .Env.POD_IP "0.0.0.0" }}` }} + grpcMaxMsgSize: {{ .Values.config.services.grpcMaxMsgSize | default 4194304 | int }} + metrics: + {{- if eq .Values.config.services.metrics.type "statsd" }} + statsd: + hostPort: {{ .Values.config.services.metrics.statsd.endpoint | quote }} + prefix: {{ .Values.config.services.metrics.statsd.prefixes.history | default "cadence-history" }} + {{- else if eq .Values.config.services.metrics.type "prometheus" }} + prometheus: + timerType: {{ .Values.config.services.metrics.prometheus.timerType | default "histogram" }} + listenAddress: "0.0.0.0:{{ .Values.metrics.port | default 9090 }}" + {{- end }} + {{- if .Values.config.services.pprof.enabled }} + pprof: + port: {{ .Values.config.services.pprof.ports.history | default 6062 }} + {{- end }} + + matching: + rpc: + port: {{ .Values.matching.port | default 7935 }} + grpcPort: {{ .Values.matching.grpcPort | default 7835 }} + bindOnIP: {{ `{{ default .Env.POD_IP "0.0.0.0" }}` }} + grpcMaxMsgSize: {{ .Values.config.services.grpcMaxMsgSize | default 4194304 | int }} + metrics: + {{- if eq .Values.config.services.metrics.type "statsd" }} + statsd: + hostPort: {{ .Values.config.services.metrics.statsd.endpoint | quote }} + prefix: {{ .Values.config.services.metrics.statsd.prefixes.matching | default "cadence-matching" }} + {{- else if eq .Values.config.services.metrics.type "prometheus" }} + prometheus: + timerType: {{ .Values.config.services.metrics.prometheus.timerType | default "histogram" }} + listenAddress: "0.0.0.0:{{ .Values.metrics.port | default 9090 }}" + {{- end }} + {{- if .Values.config.services.pprof.enabled }} + pprof: + port: {{ .Values.config.services.pprof.ports.matching | default 6061 }} + {{- end }} + + worker: + rpc: + port: {{ .Values.worker.port | default 7939 }} + bindOnIP: {{ `{{ default .Env.POD_IP "0.0.0.0" }}` }} + grpcMaxMsgSize: {{ .Values.config.services.grpcMaxMsgSize | default 4194304 | int }} + metrics: + {{- if eq .Values.config.services.metrics.type "statsd" }} + statsd: + hostPort: {{ .Values.config.services.metrics.statsd.endpoint | quote }} + prefix: {{ .Values.config.services.metrics.statsd.prefixes.worker | default "cadence-worker" }} + {{- else if eq .Values.config.services.metrics.type "prometheus" }} + prometheus: + timerType: {{ .Values.config.services.metrics.prometheus.timerType | default "histogram" }} + listenAddress: "0.0.0.0:{{ .Values.metrics.port | default 9090 }}" + {{- end }} + {{- if .Values.config.services.pprof.enabled }} + pprof: + port: {{ .Values.config.services.pprof.ports.worker | default 6063 }} + {{ end }} + + {{ if .Values.config.kafka.enabled -}} + # Kafka configuration + kafka: + tls: + enabled: {{ .Values.config.kafka.tls.enabled | default false }} + {{- if .Values.config.kafka.tls.caFile }} + caFile: {{ .Values.config.kafka.tls.caFile | quote }} + {{- end }} + {{- if .Values.config.kafka.tls.caFiles }} + caFiles: + {{- range .Values.config.kafka.tls.caFiles }} + - {{ . | quote }} + {{- end }} + {{- end }} + {{- if .Values.config.kafka.tls.certFile }} + certFile: {{ .Values.config.kafka.tls.certFile | quote }} + {{- end }} + {{- if .Values.config.kafka.tls.keyFile }} + keyFile: {{ .Values.config.kafka.tls.keyFile | quote }} + {{- end }} + {{- if .Values.config.kafka.tls.serverName }} + serverName: {{ .Values.config.kafka.tls.serverName | quote }} + {{- end }} + enableHostVerification: {{ .Values.config.kafka.tls.enableHostVerification | default true }} + requireClientAuth: {{ .Values.config.kafka.tls.requireClientAuth | default false }} + sasl: + enabled: {{ .Values.config.kafka.sasl.enabled | default false }} + {{- if .Values.config.kafka.sasl.enabled }} + algorithm: {{ .Values.config.kafka.sasl.mechanism | default "PLAIN" | quote }} + {{- if .Values.config.kafka.sasl.username }} + username: {{ .Values.config.kafka.sasl.username | quote }} + {{- end }} + {{- if .Values.config.kafka.sasl.password }} + password: {{ `{{ .Env.SASL_PASSWORD }}` }} + {{- end }} + {{- end }} + clusters: + default: + brokers: + - {{ .Values.config.kafka.brokers }}:{{ .Values.config.kafka.port | default 9092 }} + topics: + {{ .Values.config.kafka.visibilityTopic | default "cadence-visibility" }}: + cluster: default + {{- if .Values.config.kafka.topicProperties }} + properties: + {{- toYaml .Values.config.kafka.topicProperties | nindent 8 }} + {{- end }} + {{ .Values.config.kafka.visibilityDLQTopic | default "cadence-visibility-dlq" }}: + cluster: default + {{- if .Values.config.kafka.topicProperties }} + properties: + {{- toYaml .Values.config.kafka.topicProperties | nindent 8 }} + {{- end }} + applications: + visibility: + topic: {{ .Values.config.kafka.visibilityTopic | default "cadence-visibility" }} + dlq-topic: {{ .Values.config.kafka.visibilityDLQTopic | default "cadence-visibility-dlq" }} + {{ end }} + + # Archival configuration + archival: + history: + status: {{ .Values.config.archival.history.status | default "disabled" | quote }} + enableRead: {{ .Values.config.archival.history.enableRead | default false }} + provider: + {{- if eq .Values.config.archival.history.provider.type "filestore" }} + filestore: + fileMode: {{ .Values.config.archival.history.provider.filestore.fileMode | default "0644" | quote }} + dirMode: {{ .Values.config.archival.history.provider.filestore.dirMode | default "0755" | quote }} + {{- else if eq .Values.config.archival.history.provider.type "s3" }} + s3store: + region: {{ .Values.config.archival.history.provider.s3store.region | quote }} + {{- if .Values.config.archival.history.provider.s3store.endpoint }} + endpoint: {{ .Values.config.archival.history.provider.s3store.endpoint | quote }} + {{- end }} + s3ForcePathStyle: {{ .Values.config.archival.history.provider.s3store.s3ForcePathStyle | default false }} + {{- else if eq .Values.config.archival.history.provider.type "gcs" }} + gstorage: + credentialsPath: {{ .Values.config.archival.history.provider.gstorage.credentialsPath | quote }} + {{- end }} + visibility: + status: {{ .Values.config.archival.visibility.status | default "disabled" | quote }} + enableRead: {{ .Values.config.archival.visibility.enableRead | default false }} + provider: + {{- if eq .Values.config.archival.visibility.provider.type "filestore" }} + filestore: + fileMode: {{ .Values.config.archival.visibility.provider.filestore.fileMode | default "0644" | quote }} + dirMode: {{ .Values.config.archival.visibility.provider.filestore.dirMode | default "0755" | quote }} + {{- else if eq .Values.config.archival.visibility.provider.type "s3" }} + s3store: + region: {{ .Values.config.archival.visibility.provider.s3store.region | quote }} + {{- if .Values.config.archival.visibility.provider.s3store.endpoint }} + endpoint: {{ .Values.config.archival.visibility.provider.s3store.endpoint | quote }} + {{- end }} + s3ForcePathStyle: {{ .Values.config.archival.visibility.provider.s3store.s3ForcePathStyle | default false }} + {{- else if eq .Values.config.archival.history.provider.type "gcs" }} + gstorage: + credentialsPath: {{ .Values.config.archival.visibility.provider.gstorage.credentialsPath | quote }} + {{- end }} + + # Domain defaults configuration + domainDefaults: + archival: + history: + status: {{ .Values.config.domainDefaults.archival.history.status | default "disabled" | quote }} + {{- if .Values.config.domainDefaults.archival.history.URI }} + URI: {{ .Values.config.domainDefaults.archival.history.URI | quote }} + {{- end }} + visibility: + status: {{ .Values.config.domainDefaults.archival.visibility.status | default "disabled" | quote }} + {{- if .Values.config.domainDefaults.archival.visibility.URI }} + URI: {{ .Values.config.domainDefaults.archival.visibility.URI | quote }} + {{- end }} + + # Blobstore configuration + {{- if .Values.config.blobstore }} + blobstore: + {{- if .Values.config.blobstore.filestore }} + filestore: + outputDirectory: {{ .Values.config.blobstore.filestore.outputDirectory | default "/var/lib/cadence/blobstore" | quote }} + {{- end }} + {{- end }} + + # Public client configuration + {{- if .Values.config.publicClient }} + publicClient: + {{- if .Values.config.publicClient.hostPort }} + hostPort: {{ .Values.config.publicClient.hostPort | quote }} + {{- end }} + transport: {{ .Values.config.publicClient.transport | default "grpc" | quote }} + RefreshInterval: {{ .Values.config.publicClient.refreshInterval | default "10s" | quote }} + {{- end }} + + # Dynamic configuration + {{- if .Values.config.dynamicConfig }} + dynamicconfig: + client: {{ .Values.config.dynamicConfig.client | default "filebased" | quote }} + {{- if eq .Values.config.dynamicConfig.client "filebased" }} + filebased: + filepath: {{ .Values.config.dynamicConfig.filebased.filepath | default "/etc/cadence/config/dynamicconfig/config.yaml" | quote }} + pollInterval: {{ .Values.config.dynamicConfig.filebased.pollInterval | default "60s" | quote }} + {{- end }} + {{- end }} + + {{ if .Values.config.asyncWorkflowQueues.enabled -}} + # Async workflow queues configuration + asyncWorkflowQueues: + {{- range $queueName, $queue := .Values.config.asyncWorkflowQueues }} + {{- if ne $queueName "enabled" }} + {{ $queueName }}: + type: {{ $queue.type | default "kafka" | quote }} + {{- if $queue.config }} + config: + {{- if $queue.config.topic }} + topic: {{ $queue.config.topic | quote }} + {{- end }} + {{- if $queue.config.cluster }} + cluster: {{ $queue.config.cluster | quote }} + {{- end }} + {{- end }} + {{- end }} + {{- end }} {{- end }} \ No newline at end of file diff --git a/charts/cadence/templates/server-deployment.yaml b/charts/cadence/templates/server-deployment.yaml index b08c62a..491d8d8 100644 --- a/charts/cadence/templates/server-deployment.yaml +++ b/charts/cadence/templates/server-deployment.yaml @@ -1,5 +1,6 @@ {{- range $serviceName := (list "frontend" "history" "matching" "worker") }} {{- $service := index $.Values $serviceName }} +{{- $dbDriver := $.Values.config.persistence.database.driver }} --- apiVersion: apps/v1 kind: Deployment @@ -77,28 +78,719 @@ spec: {{- toYaml $serviceTopologySpreadConstraints | nindent 8 }} {{- end }} initContainers: - - name: wait-for-schema - {{- $globalImage := $.Values.global.image | default dict }} - {{- $serviceImage := $service.image | default dict }} - {{- $repository := $serviceImage.repository | default $globalImage.repository }} - {{- $tag := $serviceImage.tag | default $globalImage.tag }} - image: {{ $repository }}:{{ $tag }} - {{- $pullPolicy := $serviceImage.pullPolicy | default $globalImage.pullPolicy | default "IfNotPresent" }} - imagePullPolicy: {{ $pullPolicy }} - command: ["sh", "-c", " - until cqlsh $CASSANDRA_ENDPOINT 9042 -e \" - USE cadence; - SELECT curr_version FROM schema_version WHERE keyspace_name = 'cadence';\" | grep -q {{ $.Values.cassandra.schema.version }} && - cqlsh $CASSANDRA_ENDPOINT 9042 -e \" - USE cadence_visibility; - SELECT curr_version FROM schema_version WHERE keyspace_name = 'cadence_visibility';\" | grep -q {{ $.Values.cassandra.schema.visibility_version }}; + - name: extract-schema-versions + {{- $globalImage := $.Values.global.image | default dict }} + {{- $serviceImage := $service.image | default dict }} + {{- $repository := $serviceImage.repository | default $globalImage.repository }} + {{- $tag := $serviceImage.tag | default $globalImage.tag }} + image: {{ $repository }}:{{ $tag }} + {{- $pullPolicy := $serviceImage.pullPolicy | default $globalImage.pullPolicy | default "IfNotPresent" }} + imagePullPolicy: {{ $pullPolicy }} + {{- $globalContainerSecurityContext := $.Values.global.containerSecurityContext | default dict }} + {{- $serviceContainerSecurityContext := $service.containerSecurityContext | default $globalContainerSecurityContext }} + {{- if $serviceContainerSecurityContext }} + securityContext: + {{- toYaml $serviceContainerSecurityContext | nindent 10 }} + {{- end }} + command: ["/bin/sh"] + args: + - -c + - | + if [ -f "$VERSION_FILE" ]; then + default_version=$(grep 'const Version' "$VERSION_FILE" | awk -F'"' '{print $2}') + visibility_version=$(grep 'const VisibilityVersion' "$VERSION_FILE" | awk -F'"' '{print $2}') + + echo "DEFAULT_VERSION=$default_version" > /shared/schema-versions.env + echo "VISIBILITY_VERSION=$visibility_version" >> /shared/schema-versions.env + + echo "Extracted versions:" + echo " DEFAULT_VERSION=$default_version" + echo " VISIBILITY_VERSION=$visibility_version" + else + echo "Error: version.go file not found at $VERSION_FILE" + exit 1 + fi + env: + {{- if eq $dbDriver "cassandra" }} + - name: VERSION_FILE + value: "/etc/cadence/schema/cassandra/version.go" + {{- else if eq $dbDriver "postgres" }} + - name: VERSION_FILE + value: "/etc/cadence/schema/postgres/version.go" + {{- else if eq $dbDriver "mysql" }} + - name: VERSION_FILE + value: "/etc/cadence/schema/mysql/version.go" + {{- end }} + volumeMounts: + - name: versions-data + mountPath: /shared + - name: wait-for-schema + {{- if eq $dbDriver "cassandra" }} + image: {{ $.Values.schema.checkSchema.cassandra.image.repository }}:{{ $.Values.schema.checkSchema.cassandra.image.tag }} + imagePullPolicy: {{ $.Values.schema.checkSchema.cassandra.image.pullPolicy }} + command: + - sh + - -c + - | + # Waiting for versions file + while [ ! -f /shared/schema-versions.env ]; do + echo "Waiting for schema versions file..." + sleep 2 + done + + # Load extracted versions + export $(cat /shared/schema-versions.env | xargs) + echo "Using extracted versions:" + echo " DEFAULT_VERSION=$DEFAULT_VERSION" + echo " VISIBILITY_VERSION=$VISIBILITY_VERSION" + + # Create .cassandra directory for cqlshrc if it doesn't exist + mkdir -p ~/.cassandra + + # Build cqlshrc configuration file + cat > ~/.cassandra/cqlshrc << EOF + [connection] + hostname = $DB_HOST + port = $DB_PORT + + EOF + + # Add authentication section if user is provided + if [ -n "$DB_USER" ] && [ "$DB_USER" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + [authentication] + username = $DB_USER + EOF + # Add password if provided + if [ -n "$CASSANDRA_PASSWORD" ] && [ "$CASSANDRA_PASSWORD" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + password = $CASSANDRA_PASSWORD + EOF + fi + fi + + # Add SSL configuration if enabled + if [ "$TLS_ENABLED" = "true" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + + [ssl] + EOF + # Add certificate file if specified (CA certificate) + if [ -n "$SSL_CERTFILE" ] && [ "$SSL_CERTFILE" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + certfile = $SSL_CERTFILE + EOF + fi + + # Add client certificate for mutual TLS + if [ -n "$SSL_CLIENT_CERT" ] && [ "$SSL_CLIENT_CERT" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + usercert = $SSL_CLIENT_CERT + EOF + fi + + # Add client private key for mutual TLS + if [ -n "$SSL_CLIENT_KEY" ] && [ "$SSL_CLIENT_KEY" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + userkey = $SSL_CLIENT_KEY + EOF + fi + + # Add validate setting + if [ -n "$SSL_VALIDATE" ] && [ "$SSL_VALIDATE" != "" ]; then + cat >> ~/.cassandra/cqlshrc << EOF + validate = $SSL_VALIDATE + EOF + else + cat >> ~/.cassandra/cqlshrc << EOF + validate = true + EOF + fi + fi + + # Debug: Show generated cqlshrc (remove in production) + echo "Generated cqlshrc:" + cat ~/.cassandra/cqlshrc + echo "---" + + # Build cqlsh command + build_cqlsh_cmd() { + local cmd="cqlsh" + + # Add SSL option if enabled + if [ "$TLS_ENABLED" = "true" ]; then + cmd="$cmd --ssl" + fi + + echo "$cmd" + } + + # Compare database version with cadence schema version + until $(build_cqlsh_cmd) -e " + USE $DB_NAME; + SELECT curr_version FROM schema_version WHERE keyspace_name = '$DB_NAME';" | grep -q "$DEFAULT_VERSION" && + { + if [ "$ES_ENABLED" = "false" ]; then + $(build_cqlsh_cmd) -e " + USE $DB_VISIBILITY_NAME; + SELECT curr_version FROM schema_version WHERE keyspace_name = '$DB_VISIBILITY_NAME';" | grep -q "$VISIBILITY_VERSION" + else + true + fi + } + do + echo 'Waiting for Cassandra schema to be ready...' + sleep 10 + done + env: + - name: ES_ENABLED + value: {{ $.Values.config.persistence.elasticsearch.enabled | quote }} + # Basic connection parameters + - name: DB_HOST + value: {{ $.Values.config.persistence.database.cassandra.hosts | quote }} + - name: DB_PORT + value: {{ $.Values.config.persistence.database.cassandra.port | quote }} + - name: DB_NAME + value: {{ $.Values.config.persistence.database.cassandra.keyspace | quote }} + - name: DB_VISIBILITY_NAME + value: {{ $.Values.config.persistence.database.cassandra.visibilityKeyspace | quote }} + # Authentication parameters (conditional) + {{- if $.Values.config.persistence.database.cassandra.user }} + - name: DB_USER + value: {{ $.Values.config.persistence.database.cassandra.user | quote }} + {{- end }} + {{- if $.Values.config.persistence.database.cassandra.password }} + - name: CASSANDRA_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-{{ $serviceName }}-secrets + key: CASSANDRA_PASSWORD + {{- end }} + # TLS Configuration + - name: TLS_ENABLED + value: {{ $.Values.config.persistence.database.cassandra.tls.enabled | quote }} + {{- if $.Values.config.persistence.database.cassandra.tls.enabled }} + # SSL_CERTFILE environment variable (CA certificate) + {{- if $.Values.config.persistence.database.cassandra.tls.caFile }} + - name: SSL_CERTFILE + value: {{ $.Values.config.persistence.database.cassandra.tls.caFile | quote }} + {{- else if $.Values.config.persistence.database.cassandra.tls.caFiles }} + - name: SSL_CERTFILE + value: {{ index $.Values.config.persistence.database.cassandra.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if $.Values.config.persistence.database.cassandra.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ $.Values.config.persistence.database.cassandra.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if $.Values.config.persistence.database.cassandra.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ $.Values.config.persistence.database.cassandra.tls.keyFile | quote }} + {{- end }} + # SSL_VALIDATE environment variable + - name: SSL_VALIDATE + value: {{ $.Values.config.persistence.database.cassandra.tls.enableHostVerification | quote }} + {{- end }} + {{- else if eq $dbDriver "postgres" }} + image: {{ $.Values.schema.checkSchema.postgres.image.repository }}:{{ $.Values.schema.checkSchema.postgres.image.tag }} + imagePullPolicy: {{ $.Values.schema.checkSchema.postgres.image.pullPolicy }} + command: + - sh + - -c + - | + # Waiting for versions file + while [ ! -f /shared/schema-versions.env ]; do + echo "Waiting for schema versions file..." + sleep 2 + done + + # Load extracted versions + export $(cat /shared/schema-versions.env | xargs) + echo "Using extracted versions:" + echo " DEFAULT_VERSION=$DEFAULT_VERSION" + echo " VISIBILITY_VERSION=$VISIBILITY_VERSION" + + # Build connection string based on TLS configuration + build_psql_cmd() { + local cmd="psql -h $DB_HOST -p $DB_PORT -U $DB_USER" + + # Add SSL mode if TLS is enabled + if [ "$TLS_ENABLED" = "true" ] && [ -n "$SSL_MODE" ]; then + cmd="$cmd --set=sslmode=$SSL_MODE" + + # Add SSL certificate parameters if provided + if [ -n "$SSL_CERTFILE" ]; then + cmd="$cmd --set=sslrootcert=$SSL_CERTFILE" + fi + + if [ -n "$SSL_CLIENT_CERT" ]; then + cmd="$cmd --set=sslcert=$SSL_CLIENT_CERT" + fi + + if [ -n "$SSL_CLIENT_KEY" ]; then + cmd="$cmd --set=sslkey=$SSL_CLIENT_KEY" + fi + fi + + echo "$cmd" + } + + # Wait for PostgreSQL to be ready + echo "Waiting for PostgreSQL to be ready..." + until pg_isready -h $DB_HOST -p $DB_PORT -U $DB_USER; do + echo 'PostgreSQL is not ready yet...' + sleep 5 + done + echo "PostgreSQL is ready!" + + # Check schema versions in both databases + echo "Checking schema versions..." + until + # Check main database schema version + PGPASSWORD=$POSTGRES_PWD $(build_psql_cmd) -d $DB_NAME -t -c " + SELECT curr_version FROM schema_version WHERE db_name = '$DB_NAME';" | grep -q "$DEFAULT_VERSION" && + + # Check visibility database schema version (only if ES is not enabled) + { + if [ "$ES_ENABLED" = "false" ]; then + PGPASSWORD=$POSTGRES_PWD $(build_psql_cmd) -d $DB_VISIBILITY_NAME -t -c " + SELECT curr_version FROM schema_version WHERE db_name = '$DB_VISIBILITY_NAME';" | grep -q "$VISIBILITY_VERSION" + else + true + fi + } do - echo waiting for both cadence and cadence_visibility schema setup; - sleep 10; - done"] - env: - - name: CASSANDRA_ENDPOINT - value: {{ include "cassandra.endpoint" $ }} + echo 'Waiting for PostgreSQL schema to be ready...' + sleep 10 + done + + echo "PostgreSQL schema is ready!" + env: + - name: ES_ENABLED + value: {{ $.Values.config.persistence.elasticsearch.enabled | quote }} + # Basic connection parameters + - name: DB_HOST + value: {{ $.Values.config.persistence.database.sql.hosts | quote }} + - name: DB_PORT + value: {{ $.Values.config.persistence.database.postgres.port | default $.Values.config.persistence.database.sql.port | default 5432 | quote }} + - name: DB_NAME + value: {{ $.Values.config.persistence.database.sql.dbname | quote }} + - name: DB_VISIBILITY_NAME + value: {{ $.Values.config.persistence.database.sql.visibilityDbname | quote }} + - name: DB_USER + value: {{ $.Values.config.persistence.database.sql.user | quote }} + # Authentication parameters + - name: POSTGRES_PWD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-{{ $serviceName }}-secrets + key: POSTGRES_PWD + # TLS Configuration + - name: TLS_ENABLED + value: {{ $.Values.config.persistence.database.sql.tls.enabled | quote }} + {{- if $.Values.config.persistence.database.sql.tls.enabled }} + # SSL Mode + - name: SSL_MODE + value: {{ $.Values.config.persistence.database.sql.tls.sslMode | default "require" | quote }} + # SSL_CERTFILE environment variable (CA certificate) + {{- if $.Values.config.persistence.database.sql.tls.caFile }} + - name: SSL_CERTFILE + value: {{ $.Values.config.persistence.database.sql.tls.caFile | quote }} + {{- else if $.Values.config.persistence.database.sql.tls.caFiles }} + - name: SSL_CERTFILE + value: {{ index $.Values.config.persistence.database.sql.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if $.Values.config.persistence.database.sql.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ $.Values.config.persistence.database.sql.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if $.Values.config.persistence.database.sql.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ $.Values.config.persistence.database.sql.tls.keyFile | quote }} + {{- end }} + {{- end }} + {{- else if eq $dbDriver "mysql" }} + image: {{ $.Values.schema.checkSchema.mysql.image.repository }}:{{ $.Values.schema.checkSchema.mysql.image.tag }} + imagePullPolicy: {{ $.Values.schema.checkSchema.mysql.image.pullPolicy }} + command: + - sh + - -c + - | + # Waiting for versions file + while [ ! -f /shared/schema-versions.env ]; do + echo "Waiting for schema versions file..." + sleep 2 + done + + # Load extracted versions + export $(cat /shared/schema-versions.env | xargs) + echo "Using extracted versions:" + echo " DEFAULT_VERSION=$DEFAULT_VERSION" + echo " VISIBILITY_VERSION=$VISIBILITY_VERSION" + + # Build connection string based on TLS configuration + build_mysql_cmd() { + local cmd="mariadb -h $DB_HOST -P $DB_PORT -u $DB_USER" + + # Add SSL parameters if TLS is enabled + if [ "$TLS_ENABLED" = "true" ]; then + case "$SSL_MODE" in + "disable"|"false") + cmd="$cmd --skip-ssl" + ;; + "preferred") + ;; + "required"|"true"|"skip-verify") + cmd="$cmd --ssl --ssl-verify-server-cert=false" + ;; + "verify-ca") + cmd="$cmd --ssl --ssl-verify-server-cert" + ;; + "verify-identity") + cmd="$cmd --ssl --ssl-verify-server-cert" + ;; + *) + cmd="$cmd --ssl" + ;; + esac + + # Add SSL certificate parameters if provided + if [ -n "$SSL_CERTFILE" ]; then + cmd="$cmd --ssl-ca=$SSL_CERTFILE" + fi + + if [ -n "$SSL_CLIENT_CERT" ]; then + cmd="$cmd --ssl-cert=$SSL_CLIENT_CERT" + fi + + if [ -n "$SSL_CLIENT_KEY" ]; then + cmd="$cmd --ssl-key=$SSL_CLIENT_KEY" + fi + fi + + echo "$cmd" + } + + # Wait for MySQL to be ready + echo "Waiting for MySQL to be ready..." + until mariadb-admin ping -h $DB_HOST -P $DB_PORT -u $DB_USER --password=$MYSQL_PWD --skip-ssl --silent; do + echo 'MySQL is not ready yet...' + sleep 5 + done + echo "MySQL is ready!" + + # Check schema versions in both databases + echo "Checking schema versions..." + until + # Check main database schema version + $(build_mysql_cmd) -D $DB_NAME -e " + SELECT curr_version FROM schema_version WHERE db_name = '$DB_NAME';" | grep -q "$DEFAULT_VERSION" && + + # Check visibility database schema version (only if ES is not enabled) + { + if [ "$ES_ENABLED" = "false" ]; then + $(build_mysql_cmd) -D $DB_VISIBILITY_NAME -e " + SELECT curr_version FROM schema_version WHERE db_name = '$DB_VISIBILITY_NAME';" | grep -q "$VISIBILITY_VERSION" + else + true + fi + } + do + echo 'Waiting for MySQL schema to be ready...' + sleep 10 + done + + echo "MySQL schema is ready!" + env: + - name: ES_ENABLED + value: {{ $.Values.config.persistence.elasticsearch.enabled | quote }} + # Basic connection parameters + - name: DB_HOST + value: {{ $.Values.config.persistence.database.sql.hosts | quote }} + - name: DB_PORT + value: {{ $.Values.config.persistence.database.mysql.port | default $.Values.config.persistence.database.sql.port | default 3306 | quote }} + - name: DB_NAME + value: {{ $.Values.config.persistence.database.sql.dbname | quote }} + - name: DB_VISIBILITY_NAME + value: {{ $.Values.config.persistence.database.sql.visibilityDbname | quote }} + - name: DB_USER + value: {{ $.Values.config.persistence.database.sql.user | quote }} + # Authentication parameters + - name: MYSQL_PWD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-{{ $serviceName }}-secrets + key: MYSQL_PWD + # TLS Configuration + - name: TLS_ENABLED + value: {{ $.Values.config.persistence.database.sql.tls.enabled | quote }} + {{- if $.Values.config.persistence.database.sql.tls.enabled }} + # SSL Mode + - name: SSL_MODE + value: {{ $.Values.config.persistence.database.sql.tls.sslMode | default "require" | quote }} + # SSL_CERTFILE environment variable (CA certificate) + {{- if $.Values.config.persistence.database.sql.tls.caFile }} + - name: SSL_CERTFILE + value: {{ $.Values.config.persistence.database.sql.tls.caFile | quote }} + {{- else if $.Values.config.persistence.database.sql.tls.caFiles }} + - name: SSL_CERTFILE + value: {{ index $.Values.config.persistence.database.sql.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if $.Values.config.persistence.database.sql.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ $.Values.config.persistence.database.sql.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if $.Values.config.persistence.database.sql.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ $.Values.config.persistence.database.sql.tls.keyFile | quote }} + {{- end }} + {{- end }} + {{- end }} + volumeMounts: + - name: versions-data + mountPath: /shared + {{- if $.Values.config.persistence.elasticsearch.enabled }} + # ElasticSearch Schema Validation Init Container + - name: check-elasticsearch-schema + image: {{ $.Values.schema.checkSchema.elasticsearch.image.repository | default "alpine/curl" }}:{{ $.Values.schema.checkSchema.elasticsearch.image.tag | default "latest" }} + imagePullPolicy: {{ $.Values.schema.checkSchema.elasticsearch.image.pullPolicy | default "IfNotPresent" }} + command: + - sh + - -c + - | + # Elasticsearch Schema Validation Script + # This script validates Elasticsearch connectivity and schema readiness + + echo "Starting Elasticsearch schema validation..." + + # Build Elasticsearch connection parameters + build_es_connection() { + # Determine protocol - allow override from values or default based on TLS + if [ -n "$ES_PROTOCOL" ]; then + PROTOCOL="$ES_PROTOCOL" + elif [ "$TLS_ENABLED" = "true" ]; then + PROTOCOL="https" + else + PROTOCOL="http" + fi + + # Build curl options for TLS + CURL_OPTS="" + if [ "$TLS_ENABLED" = "true" ]; then + # Configure SSL verification based on host verification setting + if [ "$ENABLE_HOST_VERIFICATION" = "false" ]; then + CURL_OPTS="$CURL_OPTS -k" + fi + + # Add CA certificate if provided + if [ -n "$SSL_CA_FILE" ]; then + CURL_OPTS="$CURL_OPTS --cacert $SSL_CA_FILE" + fi + + # Add client certificate for mutual TLS if provided + if [ -n "$SSL_CLIENT_CERT" ] && [ -n "$SSL_CLIENT_KEY" ]; then + CURL_OPTS="$CURL_OPTS --cert $SSL_CLIENT_CERT --key $SSL_CLIENT_KEY" + fi + + # Override server name if specified + if [ -n "$SSL_SERVER_NAME" ]; then + CURL_OPTS="$CURL_OPTS --resolve $SSL_SERVER_NAME:$ES_PORT:$ES_HOST" + fi + + # Additional TLS options + if [ "$REQUIRE_CLIENT_AUTH" = "true" ]; then + # Client auth is required, ensure we have client cert + if [ -z "$SSL_CLIENT_CERT" ] || [ -z "$SSL_CLIENT_KEY" ]; then + echo "Error: Client authentication required but client certificate/key not provided" + exit 1 + fi + fi + fi + + # Add authentication if user/password provided + if [ -n "$ES_USER" ] && [ -n "$ES_PWD" ]; then + CURL_OPTS="$CURL_OPTS -u $ES_USER:$ES_PWD" + fi + + # Set global variables + BASE_URL="$PROTOCOL://$ES_HOST:$ES_PORT" + + echo "Connecting to Elasticsearch at: $BASE_URL" + echo "TLS Enabled: $TLS_ENABLED" + if [ "$TLS_ENABLED" = "true" ]; then + echo "Host Verification: $ENABLE_HOST_VERIFICATION" + echo "Client Auth Required: $REQUIRE_CLIENT_AUTH" + fi + } + + # Wait for Elasticsearch to be ready + echo "Waiting for Elasticsearch to be ready..." + build_es_connection + + # Check Elasticsearch health + until curl $CURL_OPTS -s -f "$BASE_URL/_cluster/health?wait_for_status=yellow&timeout=5s" > /dev/null; do + echo "Elasticsearch is not ready yet..." + sleep 10 + done + + echo "Elasticsearch is ready!" + + # Get cluster info for debugging + echo "Elasticsearch cluster information:" + CLUSTER_INFO=$(curl $CURL_OPTS -s "$BASE_URL/") + if [ $? -eq 0 ]; then + echo "$CLUSTER_INFO" | grep -E '"cluster_name"|"version"|"number"' || echo "Could not parse cluster info" + else + echo "Warning: Could not retrieve cluster information" + fi + + # Check if template exists and validate schema + echo "Checking Elasticsearch schema template..." + TEMPLATE_URL="$BASE_URL/_template/cadence-visibility-template" + + # Wait for template to exist + until curl $CURL_OPTS -s -f "$TEMPLATE_URL" > /dev/null; do + echo "Waiting for Cadence visibility template to be ready..." + sleep 10 + done + echo "✓ Cadence visibility template exists" + + # Validate template structure + TEMPLATE_RESPONSE=$(curl $CURL_OPTS -s "$TEMPLATE_URL") + if echo "$TEMPLATE_RESPONSE" | grep -q "cadence-visibility-template"; then + echo "✓ Template structure is valid" + else + echo "⚠ Warning: Template structure may be invalid" + fi + + # Check if visibility index exists + echo "Checking visibility index..." + INDEX_URL="$BASE_URL/$VISIBILITY_INDEX" + + # Wait for index to exist + until curl $CURL_OPTS -s -f "$INDEX_URL" > /dev/null; do + echo "Waiting for visibility index '$VISIBILITY_INDEX' to be ready..." + sleep 10 + done + echo "✓ Visibility index '$VISIBILITY_INDEX' exists" + + # Wait for index to be healthy + until curl $CURL_OPTS -s -f "$INDEX_URL/_stats" > /dev/null; do + echo "Waiting for visibility index to be healthy..." + sleep 5 + done + + INDEX_STATS=$(curl $CURL_OPTS -s "$INDEX_URL/_stats") + echo "✓ Visibility index is accessible and healthy" + # Extract basic stats + DOC_COUNT=$(echo "$INDEX_STATS" | grep -o '"count":[0-9]*' | head -1 | cut -d':' -f2) + if [ -n "$DOC_COUNT" ]; then + echo " - Document count: $DOC_COUNT" + fi + + # Additional checks for different ES versions + echo "Performing version-specific checks for ES $ES_VERSION..." + case "$ES_VERSION" in + "v6") + # Wait for _doc type mapping (ES6 compatibility) + TYPE_URL="$BASE_URL/$VISIBILITY_INDEX/_mapping/_doc" + until curl $CURL_OPTS -s -f "$TYPE_URL" > /dev/null; do + echo "Waiting for ES6 document type mapping..." + sleep 5 + done + echo "✓ ES6 document type mapping exists" + ;; + "v7"|"v8") + # Wait for mapping without type (ES7/8 style) + MAPPING_URL="$BASE_URL/$VISIBILITY_INDEX/_mapping" + until curl $CURL_OPTS -s -f "$MAPPING_URL" > /dev/null; do + echo "Waiting for ES7/8 index mapping..." + sleep 5 + done + echo "✓ ES7/8 index mapping exists" + ;; + *) + echo "⚠ Warning: Unknown ES version: $ES_VERSION" + ;; + esac + + # Final validation summary + echo "" + echo "=== Elasticsearch Schema Validation Summary ===" + echo "Cluster: Ready ✓" + echo "Template: Ready ✓" + echo "Index: Ready ✓" + echo "Mapping: Ready ✓" + echo "Version: $ES_VERSION" + echo "===============================================" + + echo "Elasticsearch schema validation completed successfully!" + exit 0 + env: + # Basic Elasticsearch connection parameters + - name: ES_HOST + value: {{ $.Values.config.persistence.elasticsearch.hosts | quote }} + - name: ES_PORT + value: {{ $.Values.config.persistence.elasticsearch.port | default 9200 | quote }} + - name: ES_PROTOCOL + value: {{ $.Values.config.persistence.elasticsearch.protocol | default "" | quote }} + - name: VISIBILITY_INDEX + value: {{ $.Values.config.persistence.elasticsearch.visibilityIndex | quote }} + - name: ES_VERSION + value: {{ $.Values.config.persistence.elasticsearch.version | quote }} + # Authentication parameters + {{- if $.Values.config.persistence.elasticsearch.user }} + - name: ES_USER + value: {{ $.Values.config.persistence.elasticsearch.user | quote }} + {{- end }} + {{- if $.Values.config.persistence.elasticsearch.password }} + - name: ES_PWD + valueFrom: + secretKeyRef: + name: {{ include "cadence.fullname" $ }}-{{ $serviceName }}-secrets + key: ES_PWD + {{- end }} + # TLS Configuration + - name: TLS_ENABLED + value: {{ $.Values.config.persistence.elasticsearch.tls.enabled | quote }} + {{- if $.Values.config.persistence.elasticsearch.tls.enabled }} + - name: ENABLE_HOST_VERIFICATION + value: {{ $.Values.config.persistence.elasticsearch.tls.enableHostVerification | quote }} + - name: REQUIRE_CLIENT_AUTH + value: {{ $.Values.config.persistence.elasticsearch.tls.requireClientAuth | quote }} + # CA certificate file + {{- if $.Values.config.persistence.elasticsearch.tls.caFile }} + - name: SSL_CA_FILE + value: {{ $.Values.config.persistence.elasticsearch.tls.caFile | quote }} + {{- else if $.Values.config.persistence.elasticsearch.tls.caFiles }} + - name: SSL_CA_FILE + value: {{ index $.Values.config.persistence.elasticsearch.tls.caFiles 0 | quote }} + {{- end }} + # Client certificate for mutual TLS + {{- if $.Values.config.persistence.elasticsearch.tls.certFile }} + - name: SSL_CLIENT_CERT + value: {{ $.Values.config.persistence.elasticsearch.tls.certFile | quote }} + {{- end }} + # Client private key for mutual TLS + {{- if $.Values.config.persistence.elasticsearch.tls.keyFile }} + - name: SSL_CLIENT_KEY + value: {{ $.Values.config.persistence.elasticsearch.tls.keyFile | quote }} + {{- end }} + # Server name override + {{- if $.Values.config.persistence.elasticsearch.tls.serverName }} + - name: SSL_SERVER_NAME + value: {{ $.Values.config.persistence.elasticsearch.tls.serverName | quote }} + {{- end }} + {{- end }} + {{- end }} + {{- with $.Values.global.tls.volumeMounts }} + {{- toYaml . | nindent 8 }} + {{- end }} containers: - name: cadence-{{ $serviceName }} {{- $globalImage := $.Values.global.image | default dict }} @@ -130,29 +822,29 @@ spec: {{- end }} volumeMounts: - name: config - mountPath: /etc/cadence/config/dynamicconfig/config.yaml + mountPath: /etc/cadence/config/config_template.yaml + subPath: config_template.yaml + - name: config + mountPath: {{ $.Values.config.dynamicConfig.filebased.filepath | default "/etc/cadence/config/dynamicconfig/config.yaml" }} subPath: dynamic_config.yaml + {{- with $.Values.global.tls.volumeMounts }} + {{- toYaml . | nindent 8 }} + {{- end }} env: - name: SERVICES value: {{ $serviceName }} - - name: DYNAMIC_CONFIG_FILE_PATH - value: "/etc/cadence/config/dynamicconfig/config.yaml" - - name: PRIMARY_FRONTEND_SERVICE - value: {{ include "cadence.fullname" $ }}-frontend.{{ $.Release.Namespace }}.svc.cluster.local - - name: BROADCAST_ADDRESS + - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - - name: CASSANDRA_SEEDS - value: {{ include "cassandra.endpoint" $ }} + {{- if $.Values.config.log.useEnvVars }} {{- $globalLog := $.Values.global.log | default dict }} {{- $serviceLog := $service.log | default $globalLog }} - name: LOG_LEVEL value: {{ $serviceLog.level | default "info" | quote }} - name: LOG_STDOUT value: {{ $serviceLog.stdout | default true | quote }} - - name: RINGPOP_SEEDS - value: {{ include "cadence.ringpopSeeds" $ }} + {{- end }} {{- $globalEnv := $.Values.global.env | default list }} {{- $serviceEnv := $service.env | default list }} {{- $mergedEnv := concat $globalEnv $serviceEnv }} @@ -161,7 +853,10 @@ spec: {{- end }} {{- $globalSecrets := $.Values.global.secretEnv | default list }} {{- $serviceSecrets := $service.secretEnv | default list }} - {{- $mergedSecrets := concat $globalSecrets $serviceSecrets }} + {{- $userSecrets := concat $globalSecrets $serviceSecrets -}} + {{- include "cadence.databaseSecrets" $ -}} + {{- $databaseSecrets := $.databaseSecrets | default list -}} + {{- $mergedSecrets := concat $userSecrets $databaseSecrets -}} {{- if $mergedSecrets }} envFrom: - secretRef: @@ -175,4 +870,9 @@ spec: - name: config configMap: name: {{ include "cadence.fullname" $ }}-configmap -{{- end }} + - name: versions-data + emptyDir: {} + {{- with $.Values.global.tls.volumes }} + {{- toYaml . | nindent 6 }} + {{- end }} +{{- end }} \ No newline at end of file diff --git a/charts/cadence/templates/server-secret.yaml b/charts/cadence/templates/server-secret.yaml index 3073731..c8033e9 100644 --- a/charts/cadence/templates/server-secret.yaml +++ b/charts/cadence/templates/server-secret.yaml @@ -1,17 +1,21 @@ {{/* This template creates secrets for all Cadence Server services (frontend, history, matching, worker) -It only creates secrets if secretEnv is defined for global or individual services +It creates secrets from secretEnv configuration and automatically adds database/service passwords */}} {{- $services := list "frontend" "history" "matching" "worker" -}} -{{- $hasGlobalSecrets := .Values.global.secretEnv -}} {{- range $serviceName := $services -}} {{- $service := index $.Values $serviceName -}} -{{- $hasServiceSecrets := $service.secretEnv -}} {{- $globalSecrets := $.Values.global.secretEnv | default list -}} {{- $serviceSecrets := $service.secretEnv | default list -}} -{{- $mergedSecrets := concat $globalSecrets $serviceSecrets -}} +{{- $userSecrets := concat $globalSecrets $serviceSecrets -}} + +{{- /* Call helper to populate database secrets */ -}} +{{- include "cadence.databaseSecrets" $ -}} +{{- $databaseSecrets := $.databaseSecrets | default list -}} + +{{- $mergedSecrets := concat $userSecrets $databaseSecrets -}} {{- if $mergedSecrets }} --- diff --git a/charts/cadence/templates/web-deployment.yaml b/charts/cadence/templates/web-deployment.yaml index 2eaad2c..bf1f39d 100644 --- a/charts/cadence/templates/web-deployment.yaml +++ b/charts/cadence/templates/web-deployment.yaml @@ -73,6 +73,28 @@ spec: topologySpreadConstraints: {{- toYaml $serviceTopologySpreadConstraints | nindent 8 }} {{- end }} + {{- if $.Values.web.busybox.enabled }} + initContainers: + - name: wait-frontend + {{- $globalPodSecurityContext := .Values.global.podSecurityContext | default dict }} + {{- $servicePodSecurityContext := .Values.web.podSecurityContext | default $globalPodSecurityContext }} + {{- if $servicePodSecurityContext }} + securityContext: + {{- toYaml $servicePodSecurityContext | nindent 8 }} + {{- end }} + image: "{{ .Values.web.busybox.image.repository }}:{{ .Values.web.busybox.image.tag }}" + imagePullPolicy: {{ .Values.web.busybox.image.pullPolicy }} + command: ['sh', '-c'] + args: + - | + HOST=$(echo "{{ include "cadence.grpcPeers" . }}" | cut -d: -f1) + PORT=$(echo "{{ include "cadence.grpcPeers" . }}" | cut -d: -f2) + until nc -z $HOST $PORT; do + echo "waiting for frontend at $HOST:$PORT..." + sleep 2 + done + echo "frontend is ready!" + {{- end }} containers: - name: cadence-web image: {{ .Values.web.image.repository }}:{{ .Values.web.image.tag }} diff --git a/charts/cadence/values.yaml b/charts/cadence/values.yaml index 21b9fcc..c2fca6b 100644 --- a/charts/cadence/values.yaml +++ b/charts/cadence/values.yaml @@ -8,7 +8,7 @@ global: # -- Global image configuration (shared only by Cadence Server services [frontend, worker, matching and history]) image: repository: "docker.io/ubercadence/server" - tag: "v1.3.1-auto-setup" + tag: "v1.3.1" pullPolicy: IfNotPresent # -- Image pull secrets for private registries @@ -16,21 +16,23 @@ global: # - name: myregistrykey # -- Global environment variables (shared only by Cadence Server services [frontend, worker, matching and history]) - env: - - name: ENABLE_ES - value: "false" - - name: SKIP_SCHEMA_SETUP - value: "true" - - name: RINGPOP_BOOTSTRAP_MODE - value: "dns" - - name: BIND_ON_IP - value: 0.0.0.0 + env: [] + # - name: ENV_VAR + # value: "value" + # Enable this variable if you are using Google Storage. + # - name: GOOGLE_APPLICATION_CREDENTIALS + # value: "absolute/path/to/json/file" # -- Global secret environment variables (shared only by Cadence Server services [frontend, worker, matching and history]) secretEnv: [] # - name: GLOBAL_SECRET_ENV_VAR # value: "value" - + # Enable this variables if you are using S3. + # - name: AWS_ACCESS_KEY_ID + # value: "value" + # - name: AWS_SECRET_ACCESS_KEY + # value: "value" + # -- Global pod security context podSecurityContext: {} # runAsNonRoot: true @@ -57,6 +59,38 @@ global: # -- Global node selector nodeSelector: {} + # -- Global TLS volumes configuration + tls: + # -- Additional volumes for TLS certificates (The mode is important to have the minimum permissions) + volumes: [] + # - name: multiple-ca-certs + # configMap: + # name: cassandra-ca-bundle + # items: + # - key: root-ca.crt + # path: root-ca.pem + # - key: intermediate-ca.crt + # path: intermediate-ca.pem + # - name: example-tls-certs + # secret: + # secretName: example-crt-secret + # items: + # - key: ca.crt + # path: ca.pem + # mode: 0644 + # - key: tls.crt + # path: client.pem + # mode: 0644 + # - key: tls.key + # path: client-key.pem + # mode: 0600 + + # -- Volume mounts for TLS certificates + volumeMounts: [] + # - name: example-tls-certs + # mountPath: /etc/cadence/ssl/ + # readOnly: true + # -- Global logging configuration (shared only by Cadence Server services [frontend, worker, matching and history]) log: # -- Enable stdout logging @@ -412,11 +446,21 @@ web: # -- Image configuration for Web UI image: repository: "docker.io/ubercadence/web" - tag: "v4.0.3" + tag: "v4.0.5" pullPolicy: IfNotPresent imagePullSecrets: [] # - name: myregistrykey + # -- Image configuration for BusyBox to check frontend service + busybox: + enabled: true + image: + repository: "busybox" + tag: "latest" + pullPolicy: IfNotPresent + imagePullSecrets: [] + # - name: myregistrykey + # -- Number of web UI replicas to deploy replicas: 1 @@ -518,14 +562,6 @@ web: # hosts: # - cadence.example.com -# Dynamic configuration -dynamicConfig: - # -- Dynamic config values to be set in the Cadence server - # List of keys can be found at https://pkg.go.dev/github.com/uber/cadence@v1.3.0/common/dynamicconfig/dynamicproperties - values: - history.workflowIDExternalRateLimitEnabled: - - value: true - # Service Account configuration serviceAccount: # -- Enable service account creation @@ -632,36 +668,797 @@ autoscaling: targetCPUUtilizationPercentage: 70 targetMemoryUtilizationPercentage: 80 -############################## -### DATABASE CONFIGURATION ### -############################## +############################################# +########### CADENCE CONFIGURATION ########### +############################################# +config: + # Log configuration + log: + # -- Enable stdout logging (inherits from global.log if not specified) + stdout: ~ + # -- Logging level: debug, info, warn, error (inherits from global.log if not specified) + level: ~ + # -- Output file path for logging (if stdout is false) + outputFile: "" + # -- Log level key name (defaults to "level") + levelKey: level + # -- Log encoding format: json, console (defaults to "json") + encoding: json + # -- Allow using environment variables for log configuration. If enabled, it will use ENV variable of each server component. + useEnvVars: false + + # Ringpop configuration for service discovery + ringpop: + # -- Ringpop cluster name + name: "cadence" + # -- Bootstrap mode: dns, hosts, file + bootstrapMode: "dns" + # -- Maximum duration to wait for joining the ring + maxJoinDuration: "30s" + + # Persistence configuration + persistence: + # -- Number of history shards for partitioning (CANNOT BE CHANGED ONCE SET) + numHistoryShards: 4 + # -- Name of the default datastore + defaultStore: "default" + # -- Name of the visibility datastore (basic visibility) + visibilityStore: "visibility" + # -- Name of the advanced visibility datastore + advancedVisibilityStore: "es-visibility" + # -- Enable persistence latency histogram metrics + enablePersistenceLatencyHistogramMetrics: false + + # Database configuration + database: + # -- Database driver: cassandra, mysql, postgres + driver: "cassandra" + + # Common SQL configuration (applies to both MySQL and PostgreSQL) + sql: + # -- Database host. Can reference Kubernetes services + hosts: "cadence-mysql.cadence.svc.cluster.local" + # -- Database port (will use driver default if not specified) + port: null + # -- Database name for main data + dbname: "cadence" + # -- Database name for visibility data + visibilityDbname: "cadence_visibility" + # -- Database username + user: "cadence" + # -- Database password + password: "" + # -- Maximum number of connections + maxConns: 20 + # -- Maximum number of idle connections + maxIdleConns: 20 + # -- Maximum connection lifetime + maxConnLifetime: "1h" + # -- Number of database shards (default: 1) + numShards: 1 + # -- Encoding type for SQL blobs + encodingType: "thriftrw" + # -- Decoding types for SQL blobs + decodingTypes: ["thriftrw"] + # -- Use multiple databases for sharding + useMultipleDatabases: false + # -- Multiple databases configuration (when useMultipleDatabases is true) + multipleDatabasesConfig: [] + # -- Connection attributes (key-value pairs for connection string) + connectAttributes: {} + # TLS configuration for SQL databases + tls: + # -- Enable TLS + enabled: false + # -- SSL mode (for PostgreSQL: disable, allow, prefer, require, verify-ca, verify-full) + # -- For MySQL: false, true, skip-verify, preferred + sslMode: "" + # -- Path to CA certificate file + caFile: "" + # -- Multiple CA certificate files + caFiles: [] + # -- Path to client certificate file + certFile: "" + # -- Path to client private key file + keyFile: "" + # -- Enable hostname verification (inverse of skipHostVerification) + enableHostVerification: true + # -- Require client authentication for mutual TLS + requireClientAuth: false + # -- Server name for certificate verification + serverName: "" + + # MySQL-specific configuration + mysql: + # -- Default port for MySQL (overrides sql.port if specified) + port: 3306 + # -- Enable transaction isolation compatibility mode + txIsolationCompat: false + + # PostgreSQL-specific configuration + postgres: + # -- Default port for PostgreSQL (overrides sql.port if specified) + port: 5432 + + # Cassandra configuration + cassandra: + # -- Cassandra hosts. Can reference Kubernetes services + hosts: "cadence-cassandra.cadence.svc.cluster.local" + # -- Cassandra port + port: 9042 + # -- Cassandra keyspace for main data + keyspace: "cadence" + # -- Cassandra keyspace for visibility data + visibilityKeyspace: "cadence_visibility" + # -- Number of replicas of cassandra deployed to ensure reliability and fault tolerance across the cluster + replicationFactor: 1 + # -- Cassandra username + user: "cassandra" + # -- Cassandra password + password: "cassandra" + # -- Cassandra protocol version + protoVersion: 4 + # -- AWS region filter for Cassandra (if using AWS Keyspaces) + region: "" + # -- Datacenter filter for Cassandra (Used for schema setup too) + datacenter: "" + # -- Maximum number of connections + maxConns: 10 + # -- Connection timeout + connectTimeout: "10s" + # -- Query timeout + timeout: "1s" + # -- Default consistency level + consistency: "LOCAL_QUORUM" + # -- Serial consistency level + serialConsistency: "LOCAL_SERIAL" + # -- Host selection policy + hostSelectionPolicy: "tokenaware,roundrobin" + # -- Allowed authenticators for custom authentication + allowedAuthenticators: [] + # -- Additional connection attributes + connectAttributes: {} + # TLS configuration for Cassandra + tls: + # -- Enable TLS + enabled: false + # -- CA certificate file to verify server certificates + caFile: "" + # -- Multiple CA certificate files (alternative to caFile) + caFiles: [] + # -- Client certificate file for mutual TLS + certFile: "" + # -- Client private key file for mutual TLS + keyFile: "" + # -- Verify server hostname matches certificate + enableHostVerification: true + # -- Require client certificate authentication + requireClientAuth: false + # -- Override server name for certificate verification + serverName: "localhost" + + # Elasticsearch configuration for advanced visibility + elasticsearch: + # -- Enable Elasticsearch for advanced visibility + enabled: false + # -- Elasticsearch version (v6, use v7 for v7 or higher) + version: "v7" + # -- Elasticsearch username + user: "" + # -- Elasticsearch password + password: "" + # -- Protocol to use (http/https). If not specified, auto-detected based on TLS settings + protocol: "" + # -- Elasticsearch host. + hosts: "elasticsearch-service.elastic-namespace.svc.cluster.local" + # -- Elasticsearch port + port: 9200 + # -- Elasticsearch visibility index name + visibilityIndex: "cadence-visibility" + # -- Enable AWS signing (for AWS Elasticsearch) + awsSigning: + enabled: false + region: "" + service: "es" + # TLS configuration for ElasticSearch + tls: + # -- Enable TLS + enabled: false + # -- CA certificate file to verify server certificates + caFile: "" + # -- Multiple CA certificate files (alternative to caFile) + caFiles: [] + # -- Client certificate file for mutual TLS + certFile: "" + # -- Client private key file for mutual TLS + keyFile: "" + # -- Verify server hostname matches certificate + enableHostVerification: true + # -- Require client certificate authentication + requireClientAuth: false + # -- Override server name for certificate verification + serverName: "" + + # Cluster configuration + cluster: + # -- Version increment used during cluster failover operations + failoverVersionIncrement: 10 + # -- Name of the primary cluster in a multi-cluster setup + primaryClusterName: "cluster0" + # -- Name of the current cluster + currentClusterName: "cluster0" + # -- Whether this cluster is not the primary cluster + isNotPrimary: false + # -- RPC transport protocol (grpc or tchannel) + rpcTransport: "grpc" + # -- Initial failover version for this cluster + initialFailoverVersion: 0 + # -- Cluster group configuration with additional clusters + clusterGroup: + # Example of additional cluster configuration: + # cluster1: + # enabled: true + # initialFailoverVersion: 0 + # rpcAddress: "cadence-cluster1.example.com:7933" + # rpcTransport: "grpc" + # -- Cluster redirection policy for cross-cluster operations + clusterRedirectionPolicy: + # -- Policy for handling cross-cluster requests (noop, selected-apis-forwarding, all-domain-apis-forwarding, selected-apis-forwarding-v2) + policy: "noop" + + # Services configuration + services: + # -- gRPC max message size + grpcMaxMsgSize: 4194304 # 4MB + + # Metrics configuration + metrics: + # -- Metrics type: statsd, prometheus + type: "prometheus" + # StatsD configuration + statsd: + # -- StatsD endpoint. Can reference Kubernetes services + endpoint: "" + # -- Metric prefixes for each service + prefixes: + frontend: "cadence-frontend" + matching: "cadence-matching" + history: "cadence-history" + worker: "cadence-worker" + # Prometheus configuration + prometheus: + # -- Timer type: histogram, summary + timerType: "histogram" + + # Pprof configuration + pprof: + # -- Enable pprof endpoints + enabled: false + # -- Pprof ports for each service + ports: + frontend: 6060 + matching: 6061 + history: 6062 + worker: 6063 + + # Kafka configuration for async workflows + kafka: + # -- Enable Kafka for async workflows + enabled: false + # -- Kafka broker service. Can reference Kubernetes services + brokers: "kafka-service.kafka-namespace.svc.cluster.local" + # -- Kafka port + port: 9092 + # -- Kafka visibility topic name + visibilityTopic: "cadence-visibility" + # -- Kafka visibility DLQ topic name + visibilityDLQTopic: "cadence-visibility-dlq" + # -- Topic properties (optional) + topicProperties: {} + # TLS configuration for Kafka + tls: + # -- Enable TLS + enabled: false + # -- CA certificate file to verify server certificates + caFile: "" + # -- Multiple CA certificate files (alternative to caFile) + caFiles: [] + # -- Client certificate file for mutual TLS + certFile: "" + # -- Client private key file for mutual TLS + keyFile: "" + # -- Verify server hostname matches certificate + enableHostVerification: true + # -- Require client certificate authentication + requireClientAuth: false + # -- Override server name for certificate verification + serverName: "" + # SASL configuration + sasl: + # -- Enable SASL authentication + enabled: false + # -- SASL mechanism: plain, sha512 or sha256 + mechanism: "PLAIN" + # -- SASL username + username: "" + # -- SASL password + password: "" + + # Archival configuration + archival: + # History archival configuration + history: + # -- Archival status: enabled, disabled, paused + status: "disabled" + # -- Enable reading from archives + enableRead: false + # -- Archive providers configuration + provider: + # -- Storage type: filestore, s3, gcs + type: "filestore" + # Filestore archiver + filestore: + # -- File mode for archived files + fileMode: "0644" + # -- Directory mode for archive directories + dirMode: "0755" + # S3 archiver + # - Documentation for S3 here: https://github.com/cadence-workflow/cadence/blob/v1.3.0/common/archiver/s3store/README.md + s3store: + # -- AWS region + region: "" + # -- S3 endpoint (for S3-compatible storage) + endpoint: "" + # -- Force path style URLs + s3ForcePathStyle: false + # Google Cloud Storage archiver + # - Documentation for GStorage here: https://github.com/cadence-workflow/cadence/blob/v1.3.0/common/archiver/gcloud/README.md + gstorage: + # -- Path to service account key file + credentialsPath: "" + + # Visibility archival configuration + visibility: + # -- Archival status: enabled, disabled, paused + status: "disabled" + # -- Enable reading from archives + enableRead: false + # -- Archive providers configuration + provider: + # -- Storage type: filestore, s3, gcs + type: "filestore" + # Filestore archiver + filestore: + # -- File mode for archived files + fileMode: "0644" + # -- Directory mode for archive directories + dirMode: "0755" + # S3 archiver + # - Documentation for S3 here: https://github.com/cadence-workflow/cadence/blob/v1.3.0/common/archiver/s3store/README.md + s3store: + # -- AWS region + region: "" + # -- S3 endpoint (for S3-compatible storage) + endpoint: "" + # -- Force path style URLs + s3ForcePathStyle: false + # Google Cloud Storage archiver + # - Documentation for GStorage here: https://github.com/cadence-workflow/cadence/blob/v1.3.0/common/archiver/gcloud/README.md + gstorage: + # -- Path to service account key file + credentialsPath: "" + + # Domain defaults configuration + domainDefaults: + # -- Default archival settings for new domains + # - Documentation for S3 here: https://github.com/cadence-workflow/cadence/blob/v1.3.0/common/archiver/s3store/README.md + # - Documentation for GStorage here: https://github.com/cadence-workflow/cadence/blob/v1.3.0/common/archiver/gcloud/README.md + archival: + history: + # -- Default history archival status: enabled, disabled + status: "disabled" + # -- Default history archival URI + URI: "" + visibility: + # -- Default visibility archival status: enabled, disabled + status: "disabled" + # -- Default visibility archival URI + URI: "" + + # Blobstore configuration + blobstore: + # Filestore blobstore + filestore: + # -- Output directory for blob storage + outputDirectory: "/etc/cadence/blobstore" + + # Public client configuration + publicClient: + # -- Frontend service address (defaults to current cluster's RPC address) + hostPort: "" + # -- Transport protocol: grpc, tchannel + transport: "grpc" + # -- DNS refresh interval + refreshInterval: "10s" + + # Dynamic configuration + dynamicConfig: + # -- Dynamic config client type: noop, filebased, configstore + client: "filebased" + # File-based dynamic config + filebased: + # -- Path to dynamic config file + filepath: "/etc/cadence/config/dynamicconfig/config.yaml" + # -- Poll interval for config changes + pollInterval: "60s" + + # Async workflow queues configuration + # This is not operative yet. It will be in next releases. + asyncWorkflowQueues: + # -- Enable async workflow queues + enabled: false + # -- Async workflow queue providers + default-queue: + # -- Queue type: kafka + type: "kafka" + # -- Queue configuration + config: + # -- Kafka topic for async workflows + topic: "cadence-async-wf" + # -- Kafka cluster reference + cluster: "default" -# Cassandra configuration -cassandra: - # -- External Cassandra endpoint to connect to. Required when cassandra.deployment.enabled is set to false - endpoint: "" - - schema: - # -- Cassandra schema version of the Cadence keyspace to use - version: "0.42" - # -- Cassandra schema version of the Cadence visibility keyspace to use - visibility_version: "0.9" - - deployment: - # -- When enabled, a single instance Cassandra will be deployed as part of the Helm chart - # -- When disabled, the Cassandra deployment is expected to be provided externally +# Dynamic configuration +dynamicConfig: + # -- Dynamic config values to be set in the Cadence server + # List of keys can be found at https://pkg.go.dev/github.com/uber/cadence/common/dynamicconfig/dynamicproperties + values: + system.minRetentionDays: + - value: 0 + constraints: {} + # Visibility write have to be set in funtion of advanced visibility or not. See https://pkg.go.dev/github.com/uber/cadence/common/dynamicconfig/dynamicproperties#WriteVisibilityStoreName for more information. + system.writeVisibilityStoreName: + - value: "db" + # Visibility read have to be set in funtion of advanced visibility or not. See https://pkg.go.dev/github.com/uber/cadence/common/dynamicconfig/dynamicproperties#ReadVisibilityStoreName for more information. + system.readVisibilityStoreName: + - value: "db" + +############################################################### +#################### SCHEMA CONFIGURATION ##################### +############################################################### +schema: + # Schema validation and deployment configuration + # This job validates and updates the database schema safely: + # - Won't replace existing databases + # - Won't rollback schema versions to prevent data loss + # - Will update schema version if updates are available + # All schema parameters are taken from .Values.config + + # -- Schema job for main database schema management + serverJob: + # -- Enable server schema job enabled: true + # -- Schema job resource allocation + resources: {} + # limits: + # cpu: "500m" + # memory: "1Gi" + # requests: + # cpu: "500m" + # memory: "1Gi" + # -- Schema job affinity rules + affinity: {} + # -- Schema job tolerations + tolerations: [] + # -- Schema job node selector + nodeSelector: {} + + # -- ElasticSearch schema job configuration + elasticSearchJob: + # -- Enable ElasticSearch schema job + enabled: true + # -- ElasticSearch schema job resource allocation + resources: {} + # limits: + # cpu: "500m" + # memory: "1Gi" + # requests: + # cpu: "500m" + # memory: "1Gi" + # -- ElasticSearch schema job affinity rules + affinity: {} + # -- ElasticSearch schema job tolerations + tolerations: [] + # -- ElasticSearch schema job node selector + nodeSelector: {} + + # -- Container images for schema validation + checkSchema: + # -- Cassandra schema validation image + cassandra: + image: + repository: "cassandra" + tag: "4" + pullPolicy: IfNotPresent + # -- MySQL schema validation image + mysql: + image: + repository: "alpine/mysql" + tag: "latest" + pullPolicy: IfNotPresent + # -- PostgreSQL schema validation image + postgres: + image: + repository: "alpine/psql" + tag: "latest" + pullPolicy: IfNotPresent + # -- ElasticSearch schema validation image + elasticsearch: + image: + repository: alpine/curl + tag: "latest" + pullPolicy: IfNotPresent + +################################################################## +#################### DATABASE CONFIGURATION ###################### +################################################################## +# Database deployment configuration +# Enable only if you want to deploy the database components from this chart +# For production environments, consider using external managed databases + +# Cassandra database configuration +# ref: https://github.com/bitnami/charts/tree/main/bitnami/cassandra +# More documentation: https://artifacthub.io/packages/helm/bitnami/cassandra +cassandra: + # -- Enable Cassandra database deployment + enabled: true + # -- Number of Cassandra replicas + replicaCount: 1 + + # Database user configuration + dbUser: + # -- Cassandra username + user: cassandra + # -- Force password setting + forcePassword: false + # -- Cassandra password (leave empty for auto-generation) + password: "cassandra" + + # Cassandra cluster configuration + cluster: + # -- Cassandra cluster name + name: cassandra + # -- Number of seed nodes + seedCount: 1 + # -- Number of tokens per node + numTokens: 256 + # -- Datacenter name + datacenter: dc1 + # -- Rack name + rack: rack1 + # -- Endpoint snitch strategy + endpointSnitch: SimpleSnitch + + # JVM configuration + jvm: + # -- Additional JVM options + extraOpts: "" + # -- Maximum heap size (auto-calculated if empty) + maxHeapSize: "" + # -- New generation heap size (auto-calculated if empty) + newHeapSize: "" + + # -- Cassandra resource allocation + resources: {} + # limits: + # cpu: "500m" + # memory: "1Gi" + # requests: + # cpu: "500m" + # memory: "1Gi" - image: - repository: "cassandra" - tag: "4.1.1" - pullPolicy: IfNotPresent + # Persistence configuration + persistence: + # -- Enable persistent storage + enabled: false + # -- Use existing persistent volume claim + existingClaim: "" + # -- Storage class for data volume + storageClass: "" + # -- Storage class for commit log volume + commitStorageClass: "" + # -- Persistent volume annotations + annotations: {} + # -- Persistent volume access modes + accessModes: + - ReadWriteOnce + # -- Data volume size + size: 8Gi + # -- Commit log volume size + commitLogsize: 2Gi + + # TLS configuration + tls: + # -- Internode encryption level (none, all, dc, rack) + internodeEncryption: none + # -- Enable client-to-node encryption + clientEncryption: false + # -- Auto-generate TLS certificates + autoGenerated: false + # -- Existing secret with TLS certificates + existingSecret: "" + # -- Secret containing TLS passwords + passwordsSecret: "" + # -- Keystore password + keystorePassword: "" + # -- Truststore password + truststorePassword: "" + # -- Secret containing TLS certificates + certificatesSecret: "" + # -- TLS encryption secret name + tlsEncryptionSecretName: "" + # -- Resource preset for TLS containers + resourcesPreset: "nano" + +# PostgreSQL database configuration +# ref: https://github.com/bitnami/charts/tree/main/bitnami/postgresql +# More documentation: https://artifacthub.io/packages/helm/bitnami/postgresql +postgresql: + # -- Enable PostgreSQL database deployment + enabled: false - # -- Resource limits and requests for Cassandra - resources: - limits: - cpu: 1 - memory: 2Gi - requests: - cpu: 500m - memory: 1Gi + # Authentication configuration + auth: + # -- Enable postgres admin user + enablePostgresUser: true + # -- Password for postgres admin user (leave empty for auto-generation) + postgresPassword: "" + # -- Custom username to create + username: "" + # -- Password for custom user (leave empty for auto-generation) + password: "" + # -- Custom database name to create + database: "" + + # Primary database configuration + primary: + # -- Primary database resource allocation + # resources: + # requests: + # cpu: 2 + # memory: 512Mi + # limits: + # cpu: 3 + # memory: 1024Mi + + # Persistence configuration + persistence: + # -- Enable PostgreSQL primary data persistence using PVC + enabled: true + # -- Volume name to assign + volumeName: "data" + # -- Use existing persistent volume claim + existingClaim: "" + # -- Volume mount path + mountPath: /bitnami/postgresql + # -- Volume subdirectory to mount + subPath: "" + # -- Storage class for PostgreSQL primary data volume + storageClass: "" + # -- PVC access modes for PostgreSQL volume + accessModes: + - ReadWriteOnce + # -- PVC storage request size + size: 8Gi + # -- PVC annotations + annotations: {} + # -- PVC labels + labels: {} + # -- Selector to match existing Persistent Volume + selector: {} + # -- Custom PVC data source + dataSource: {} + + # TLS configuration + tls: + # -- Enable TLS traffic support + enabled: false + # -- Auto-generate self-signed TLS certificates + autoGenerated: false + # -- Use server's TLS cipher preferences over client's + preferServerCiphers: true + # -- Secret containing TLS certificates + certificatesSecret: "" + # -- Certificate filename + certFilename: "" + # -- Certificate key filename + certKeyFilename: "" + # -- CA certificate filename for client authentication + certCAFilename: "" + # -- Certificate Revocation List filename + crlFilename: "" + +# MySQL database configuration +# ref: https://github.com/bitnami/charts/tree/main/bitnami/mysql +# More documentation: https://artifacthub.io/packages/helm/bitnami/mysql +mysql: + # -- Enable MySQL database deployment + enabled: false + + # Authentication configuration + auth: + # -- Password for root user (leave empty for auto-generation) + rootPassword: "" + # -- Create the configured database + createDatabase: true + # -- Database name to create + database: "my_database" + # -- Custom username to create + username: "" + # -- Password for custom user (leave empty for auto-generation) + password: "" + # -- MySQL replication username + replicationUser: replicator + # -- MySQL replication user password (leave empty for auto-generation) + replicationPassword: "" + # -- Use existing secret for passwords + # Secret must contain keys: mysql-root-password, mysql-replication-password, mysql-password + existingSecret: "" + # -- Mount credentials as files instead of environment variables + usePasswordFiles: true + + # Primary database configuration + primary: + # -- Primary database resource allocation + # resources: + # requests: + # cpu: 2 + # memory: 512Mi + # limits: + # cpu: 3 + # memory: 1024Mi + + # Persistence configuration + persistence: + # -- Enable persistence using PersistentVolumeClaim + enabled: true + # -- Use existing PersistentVolumeClaim + existingClaim: "" + # -- Volume subdirectory to mount + subPath: "" + # -- Storage class for MySQL primary persistent volume + storageClass: "" + # -- Persistent volume claim annotations + annotations: {} + # -- Persistent volume access modes + accessModes: + - ReadWriteOnce + # -- Persistent volume size + size: 8Gi + # -- Selector to match existing Persistent Volume + selector: {} + + # TLS configuration + tls: + # -- Enable TLS support + enabled: false + # -- Secret containing TLS certificates + existingSecret: "" + # -- Certificate filename + certFilename: tls.crt + # -- Certificate key filename + certKeyFilename: tls.key + # -- CA certificate filename + certCAFilename: "" + # -- CA certificate content + ca: "" + # -- Certificate content + cert: "" + # -- Private key content + key: "" + # Automatic certificate generation + autoGenerated: + # -- Enable automatic certificate generation + enabled: true + # -- Certificate generation mechanism (helm, cert-manager) + engine: helm \ No newline at end of file