Skip to content

Environment variables, loki write and hostName (ingest) as variable for "instance" label #942

Open
@Nachtfalkeaw

Description

@Nachtfalkeaw

Hello,

I am using loki.write to send the logs from flowlogs-pipline IPFIX ingest to my Grafana Loki.

My loki config looks like this:

parameters:
  - name: ipfix_netflow_ingest
    ingest:
      type: collector
      collector:
        hostName: u999fmlab001l     <------- parameter which automatically enters the local hostname or environmentvariable
        port: 2055   
  
  
  - name: loki_write
    write:
      type: loki
      loki:
        url: https://prometheus.sub.domain.de:3100
        tenantID: tenant_02
        batchWait: 15s
        batchSize: 15000
        labels:
          - SamplerAddress
        staticLabels:
          service_name: flowlogs-pipeline
          level: info
          instance: u999fmlab001l     <------- parameter which automatically enters the local hostname or environmentvariable or the "hostName" of ingest
        ignoreList:
          - MplsCount
          - CustomList_1
          - CustomList_2
          - CustomList_3
          - CustomList_4
          - CustomList_5
          - MplsLastTtl
          - CustomInteger_1
          - CustomInteger_2
          - CustomInteger_3
          - CustomInteger_4
          - CustomInteger_5
          - Mpls_1Label
          - Mpls_2Label
          - Mpls_3Label
          - Mpls_4Label
          - Mpls_5Label
          - CustomBytes_1
          - CustomBytes_2
          - CustomBytes_3
          - CustomBytes_4
          - CustomBytes_5
          - MplsLastLabel
          - MplsLabelIp
          - HasMpls
          - Mpls_1Ttl
          - Mpls_2Ttl
          - Mpls_3Ttl
          - Mpls_4Ttl
          - Mpls_5Ttl
        timestampLabel: TimeReceived
        timestampScale: 1s

In Prometheus and in loki the identification of a hostname is often the labekl "instance" but the label name is not that relevant. However for a dynamic/generic flowlogs-pipeline config it would be helpful to have the possibility to add the hostname dynamically into the piplines stages. e.g. the ingest "hostName" is a mandatory field and I see no possibility to add this via a parameter like "constant.hostname" or a env variable like "${HOSTNAME}".

The same is for the loki write stage. I would like to add the label "instance" and it should contain the local hostname of the flowlogs-pipeline system.

In the documentationt here is a hint that this should work or is on the ToDo list?

TODO: environment variables
Supported options are provided by running:

flowlogs-pipeline --help

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions