[FEATURE]: Reload HTTP server when TLS certificates has been changed

[volker-raschek]

Actual situation

The postgres-exporter is deployed in kubernetes via helm chart. A TLS certificate is mounted into the container file system of the pod via the volumes and volumeMounts spec. The TLS certificate is generated via cert-manager. The TLS certificate is used to start the HTTP server via TLS authentication and encryption.

Prometheus uses a TLS certificate to authenticate itself to the postgres-exporter via TLS authentication. Both TLS certificates, that of the postgres-exporter and prometheus, are issued by the same CA.

When the cert-manager issues a new certificate for the postgres-exporter, it rotates in the file system of the postgres-exporter pod.

I have now searched the documentation and could not find that the postgres-exporter reacts to changes in the TLS certificates and restarts the HTTPS server to use the new TLS certificates.

Workaround

As a workaround, I now use the reloader controller from stakater and define via annotation, that the postgres-exporter should apply a rolling update, when the secret containing the TLS certificates has been changed.

# https://github.com/prometheus-community/helm-charts/blob/8fe8a5775791a456c0e449a8d1ca3570434c9a16/charts/prometheus-postgres-exporter/values.yaml#L224
annotations:
  reloader.stakater.com/auto: "true"

Expected feature

It would be advisable for the application to detect changes to the TLS certificates natively and reload the HTTP server. This would make stakater's reloader controller obsolete.

Furthermore, document in the README or somewhere else of the project, that the application supports reloading TLS certificates natively.

Proposal

Use the fsnotify or cert-watcher package to receive kernel events, when the TLS certificates has been changed and reload the HTTP server.