chore: Address comments from second review round

Reviewers:
  - jschmidt-icinga
  - martialblog

Author: w1ll-i-code

doc/09-object-types.md

@@ -1246,9 +1246,6 @@ you can specify the certificates with the `ca_path`, `cert_path` and `cert_key`
 Writes check result metrics and performance data to an Elasticsearch timeseries datastream.
 This configuration object is available as the [elasticsearch datastream feature](14-features.md#elasticsearchdatastream-writer).
 
-> **Note:**
->
-> This feature is experimental right now and behind a compilation flag.
 
 Example:
 
@@ -1302,9 +1299,72 @@ Changing the behavior of the writer:
   host\_tags\_template      | Array                 | **Optional.** Allows add [tags](https://www.elastic.co/docs/reference/ecs/ecs-base#field-tags) to the document for a Host check result.
   service\_tags\_template   | Array                 | **Optional.** Allows add [tags](https://www.elastic.co/docs/reference/ecs/ecs-base#field-tags) to the document for a Service check result.
   host\_labels\_template    | Dictionary            | **Optional.** Allows add [labels](https://www.elastic.co/docs/reference/ecs/ecs-base#field-labels) to the document for a Host check result.
-  service\_labels\_template | Array                 | **Optional.** Allows add [labels](https://www.elastic.co/docs/reference/ecs/ecs-base#field-labels) to the document for a Service check result.
+  service\_labels\_template | Dictionary            | **Optional.** Allows add [labels](https://www.elastic.co/docs/reference/ecs/ecs-base#field-labels) to the document for a Service check result.
   filter                    | Function              | **Optional.** An expression to filter which check results should be sent to Elasticsearch. Defaults to sending all check results.
 
+#### Macro Usage (Tags, Labels & Namespace)
+
+Macros can be used inside the following template attributes:
+
+- host_tags_template (array of strings)
+- service_tags_template (array of strings)
+- host_labels_template (dictionary of key -> string value)
+- service_labels_template (dictionary of key -> string value)
+- datastream_namespace (string)
+
+Behavior:
+- Tags: Each array element may contain zero or more macros. If at least one macro is missing/unresolvable, the entire tag element is skipped and a debug log entry is written.
+- Labels: Each dictionary value may contain macros. If at least one macro inside the value is missing, that label key/value pair is skipped and a debug log entry is written.
+- Namespace: The datastream_namespace string may contain macros. If a macro is missing or resolves to an empty value, the writer falls back to the default namespace "default".
+- Validation: A template string with an unterminated '$' (e.g. "$host.name") raises a configuration validation error referencing the original string.
+- Macros never partially substitute: either all macros in the string resolve and the rendered value is used, or (for tags/labels) the entry is skipped.
+- Normalization: Performance data metric labels and the resolved datastream namespace undergo normalization: any leading whitespace and leading special characters are trimmed; all remaining special (non-alphanumeric) characters are replaced with an underscore; consecutive underscores are collapsed; leading/trailing underscores are removed. This ensures stable, Elasticsearch-friendly field and namespace names.
+
+Examples:
+
+```
+object ElasticsearchDatastreamWriter "example-datastream" {
+  datastream_namespace = "$host.vars.env$" // Falls back to "default" if $host.vars.env$ is missing
+
+  host_tags_template = [
+    "env-$host.vars.env$",
+    "$host.name$"
+  ]
+
+  service_tags_template = [
+    "svc-$service.name$",
+    "$service.display_name$"
+  ]
+
+  host_labels_template = {
+    os = "$host.vars.os$"
+    fqdn = "$host.name$"
+  }
+
+  service_labels_template = {
+    check_cmd = "$service.check_command$"
+    attempted_env = "$host.vars.missing_env$" // Skipped if missing_env not set
+  }
+
+  filter = {{ service && "production" in host.groups }}
+}
+```
+
+A missing macro example for a host check result:
+- service_tags_template element "svc-$service.name$" is skipped (service not in scope).
+- service_labels_template value "$service.check_command$" is skipped for host check results.
+
+#### Filter Expression
+
+The filter accepts an expression (function literal) and only the variables host and service are available. (service is null / undefined for host check results.)
+
+Examples:
+```
+filter = {{ "production" in host.groups }}
+filter = {{ service && "linux" in host.groups }}
+```
+If the filter returns true, the check result is sent; otherwise it is skipped.
+
 ### ExternalCommandListener <a id="objecttype-externalcommandlistener"></a>
 
 Implements the Icinga 1.x command pipe which can be used to send commands to Icinga.

doc/14-features.md

@@ -445,16 +445,23 @@ or Logstash for additional filtering.
 > **Note**
 >
 > This is a newer alternative to the Elasticsearch Writer above. The Elasticsearch Datastream Writer uses
-> Elasticsearch's datastream feature and follows the Elastic Common Schema (ECS), providing better performance
+> Elasticsearch's data stream feature and follows the Elastic Common Schema (ECS), providing better performance
 > and data organization. Use this writer for new installations. The original Elasticsearch Writer is still
 > available for backward compatibility.
+>
+> OpenSearch: The data stream mode and ECS component template usage differ slightly in OpenSearch. The
+> ElasticsearchDatastreamWriter focuses on Elasticsearch compatibility first. OpenSearch can ingest the data,
+> but you may need to adapt the installed index/component templates manually (e.g. remove time_series mode if
+> unsupported, adjust mappings). The option `manage_index_template` will not work with OpenSearch.
+
 
 This feature sends check results with performance data to an [Elasticsearch](https://www.elastic.co/products/elasticsearch) instance or cluster.
 
 > **Note**
 >
-> This feature requires Elasticsearch to support timeseries datastreams and have the ecs component template installed.
-> This feature was tested successfully with Elasticsearch 8.12 and 9.0.8.
+> This feature requires Elasticsearch to support time series data streams (Elasticsearch 8.x+), and to have the ECS
+> component template installed. It was tested successfully with Elasticsearch 8.12 and 9.0.8.
+
 
 Enable the feature and restart Icinga 2.
 
@@ -473,10 +480,13 @@ The documents for the ElasticsearchDatastreamWriter try to follow the [Elastic C
 version `8.0` as close as possible, with some additional changes to fit the Icinga 2 data model.
 All documents are written to a data stream of the format `metrics-icinga.<check>-<datastream_namespace>`,
 where `<check>` is the name of the checkcommand being executed to keep the number of fields per index low
-and document with the same performance data grouped together. `<datastream_namespace>` is an optional
+and documents with the same performance data grouped together. `<datastream_namespace>` is an optional
 configuration parameter to further separate documents, e.g. by environment like `production` or `development`.
 The `datastream_namespace` can also be used to separate documents e.g. by hostgroups or zones, by using the
 `filter` function to filter the check results and use several writers with different namespaces.
+Time‑series dimensions are applied to `host.name` and (when present) `service.name`, aligning with ECS host and service
+definitions: [ECS host fields](https://www.elastic.co/guide/en/ecs/current/ecs-host.html),
+[ECS service fields](https://www.elastic.co/guide/en/ecs/current/ecs-service.html).
 
 Icinga 2 automatically adds the following threshold metrics
 if existing:
@@ -513,23 +523,22 @@ Will in addition to the above mentioned lines also contain:
 
 #### Filtering check results <a id="elasticsearch-datastream-writer-filtering"></a>
 
-It is possible to filter the check results that are sent to Elasticsearch by using the `filter`
-parameter. This parameter takes a function that is called for each check result and must return
-a boolean value. If the function returns `true`, the check result is sent to Elasticsearch,
-otherwise it is skipped.
+You can filter which check results are sent to Elasticsearch by using the `filter` parameter.
+It takes a function (expression) evaluated for every check result and must return a boolean.
+If the function returns `true`, the check result is sent; otherwise it is skipped.
+
+Only the variables `host` and `service` are available inside this expression.
+For host check results `service` is not set (null/undefined). No other variables (such as
+the raw check result object) are exposed.
 
-This function has access to the check result in the `cr` variable, which contains the check result
-object, the checkable object (host or service) in the `checkable` variable and the host and service
-object in the `host` and `service` variables respectively.
+Example configuration that only sends service check results for hosts in the `linux-server` hostgroup:
 
-An example configuration that only sends check results of services in the `linux-servers` hostgroup
-and with a critical state:
 
 ```
 object ElasticsearchDatastreamWriter "elasticsearchdatastream" {
   ...
   datastream_namespace = "production"
-  filter = {{ service && "linux-server" in host.groups && cr.state == 2 }}
+  filter = {{ service && "linux-server" in host.groups }}
 }
 ```
 

etc/icinga2/features-available/elasticsearchdatastream.conf

@@ -1,6 +1,7 @@
-/** The ElasticsearchDatastreamWriter feature writes Icinga 2 events to an Elasticsearch datastream.
-* This feature requires Elasticsearch 8.12 or later.
-*/
+/*
+ * The ElasticsearchDatastreamWriter feature writes Icinga 2 events to an Elasticsearch datastream.
+ * This feature requires Elasticsearch 8.12 or later.
+ */
 
 object ElasticsearchDatastreamWriter "elasticsearch" {
     host = "127.0.0.1"
@@ -10,18 +11,18 @@ object ElasticsearchDatastreamWriter "elasticsearch" {
     // enable_tls = false
 
     /* The datastream namespace to use. This can be used to separate different
-    *  Icinga instances. Or for letting multiple Writers write to different
-    *  datastreams in the same Elasticsearch cluster. By using the filter option.
-    *  The Elasticsearch datastream name will be
-    *  "metrics-icinga2.{check}-{datastream_namespace}".
-    */
+     * Icinga instances or let multiple Writers write to different
+     * datastreams in the same Elasticsearch cluster by using the filter option.
+     * The Elasticsearch datastream name will be
+     * "metrics-icinga2.{check}-{datastream_namespace}".
+     */
     // datastream_namespace = "default"
 
     /* You can authorize icinga2 through three different methods.
-    * 1. Basic authentication with username and password.
-    * 2. Bearer token authentication with api_token.
-    * 3. Client certificate authentication with cert_path and key_path.
-    */
+     * 1. Basic authentication with username and password.
+     * 2. Bearer token authentication with api_token.
+     * 3. Client certificate authentication with cert_path and key_path.
+     */
     // username = "icinga2"
     // password = "changeme"
 
@@ -31,51 +32,51 @@ object ElasticsearchDatastreamWriter "elasticsearch" {
     // key_path = "/path/to/key.pem"
     // ca_path = "/path/to/ca.pem"
 
-    /* Enable sending the threashold values as additional fields
-    *  with the service check metrics. If set to true, it will
-    *  send warn and crit for every performance data item.
-    */
+    /* Enable sending the threshold values as additional fields
+     * with the service check metrics. If set to true, it will
+     * send warn and crit for every performance data item.
+     */
     // enable_send_thresholds = false
 
     /* The flush settings control how often data is sent to Elasticsearch.
-    *  You can either flush based on a time interval or the number of
-    *  events in the buffer. Whichever comes first will trigger a flush.
-    */
+     * You can either flush based on a time interval or the number of
+     * events in the buffer. Whichever comes first will trigger a flush.
+     */
     // flush_threshold = 1024
     // flush_interval = 10s
 
     /* By default, all endpoints in a zone will activate the feature and start
-    * writing events to the Elasticsearch HTTP API. In HA enabled scenarios,
-    * it is possible to set `enable_ha = true` in all feature configuration
-    * files. This allows each endpoint to calculate the feature authority,
-    * and only one endpoint actively writes events, the other endpoints
-    * pause the feature.
-    */
+     * writing events to the Elasticsearch HTTP API. In HA enabled scenarios,
+     * it is possible to set `enable_ha = true` in all feature configuration
+     * files. This allows each endpoint to calculate the feature authority,
+     * and only one endpoint actively writes events, the other endpoints
+     * pause the feature.
+     */
     // enable_ha = false
 
     /* By default, the feature will create an index template in Elasticsearch
-    * for the datastreams. If you want to manage the index template yourself,
-    * set manage_index_template to false.
-    */
+     * for the datastreams. If you want to manage the index template yourself,
+     * set manage_index_template to false.
+     */
     // manage_index_template = true
 
     /* Additional tags and labels can be added to the host and service
-    * documents by using the host_tags_template, service_tags_template,
-    * host_labels_template and service_labels_template options.
-    * The tags and labels are static and will be added to every document.
-    */
+     * documents by using the host_tags_template, service_tags_template,
+     * host_labels_template and service_labels_template options.
+     * The tags and labels are static and will be added to every document.
+     */
     // host_tags_template = [ "icinga", "$host.vars.os$" ]
     // service_tags_template = [ "icinga", "$service.vars.id$" ]
     // host_labels_template = { "env" = "production", "os" = "$host.vars.os$" }
     // service_labels_template = { "env" = "production", "id" = "$host.vars.id$" }
 
     /* The filter option can be used to filter which events are sent to
-    * Elasticsearch. The filter is a regular Icinga 2 filter expression.
-    * The filter is applied to both host and service events.
-    * If the filter evaluates to true, the event is sent to Elasticsearch.
-    * If the filter is not set, all events are sent to Elasticsearch.
-    * You can use any attribute of the host, service, checkable or
-    * checkresult (cr) objects in the filter expression.
-    */
-    // filter = {{ "host.name == 'myhost' || service.name == 'myservice'" }}
+     * Elasticsearch. The filter is a regular Icinga 2 filter expression.
+     * The filter is applied to both host and service events.
+     * If the filter evaluates to true, the event is sent to Elasticsearch.
+     * If the filter is not set, all events are sent to Elasticsearch.
+     * You can use any attribute of the host, service, checkable or
+     * checkresult (cr) objects in the filter expression.
+     */
+    // filter = {{ host.name == "myhost" || (service && service.name == "myservice") }}
 }

lib/perfdata/CMakeLists.txt

@@ -79,7 +79,6 @@ install_if_not_exists(
   ${ICINGA2_CONFIGDIR}/features-available
 )
 
-
 install(CODE "file(MAKE_DIRECTORY \"\$ENV{DESTDIR}${ICINGA2_FULL_SPOOLDIR}/perfdata\")")
 install(CODE "file(MAKE_DIRECTORY \"\$ENV{DESTDIR}${ICINGA2_FULL_SPOOLDIR}/tmp\")")