New input plugin inputs.loki_query

[1tft]

Use Case

For getting consolidated data from different loki data sources (like Openshift built in loki instances).
It would be usefull to get some metrics from logs to save them to another TimeSeries DB, e.g. InfluxDB with longer retention and cheaper storage etc..

Expected behavior

New plugin should work like https://github.com/influxdata/telegraf/tree/master/plugins/inputs/opensearch_query for Loki database endpoint via instant query.

Actual behavior

Not possible to query Loki API.

Additional info

No response

[1tft]

# Derive metrics
[[inputs.loki_query]]
  ## Loki / GEL cluster endpoint(s), including API query path
  ## Multiple urls can be specified as part of the same cluster.  Only one successful call will be made per interval.
  urls = [ "http://node1.loki.example.com:3100/loki/api/v1/query" ] # example for Loki API v1

  ## HTTP method
  # method = "POST"

  ## Loki client timeout, defaults to "5s".
  # timeout = "5s"

  ## HTTP basic authentication details
  # username = "admin"
  # password = "admin"

  ## Optional HTTP headers
  # headers = {"X-Special-Header" = "Special-Value"}

  ## Skip TLS validation.  Useful for local testing and self-signed certs.
  # insecure_skip_verify = false

  ## Optional TLS Config
  # tls_ca = "/etc/telegraf/ca.pem"
  # tls_cert = "/etc/telegraf/cert.pem"
  # tls_key = "/etc/telegraf/key.pem"
  ## Use TLS but skip chain & host verification
  # insecure_skip_verify = false
 
  ## If 'use_system_proxy' is set to true, Telegraf will check env vars such as
  ## HTTP_PROXY, HTTPS_PROXY, and NO_PROXY (or their lowercase counterparts).
  ## If 'use_system_proxy' is set to false (default) and 'http_proxy_url' is
  ## provided, Telegraf will use the specified URL as HTTP proxy.
  # use_system_proxy = false
  # http_proxy_url = "http://localhost:8888"



  [[inputs.loki_query.aggregation]]
    ## Set LogQL query type
    ## Currently only "instant" is supported
    query_type = "instant"
  
    ## measurement name for the results of the aggregation query
    measurement_name = "measurement"

    ## Relative starting time for query
    ## use relative times like "5s", "5h"
    ## for instant queries this is the "evaluation timestamp"
    start = "5m"

    ## Relative ending time for query
    ## use relative times like "5s", "5h"
    ## default is 0s which means current local time
    end = "0s"

    ## Loki query
    ## Evaluation timestamp is automatically added
    logql_query = "sum by (dc, level) ( count_over_time({dc="datacenter01", level="error"} |~ `(?i)(invalid routes detected|too many connections)`"

    ## Field for value (numeric or string)
    # metric_field = "metric"

    ## Fields to be used as tags. Must be string.
    # tags = ["label01", "label02"]

    ## Set to true to not ignore series when the tag(s) above are missing
    # include_missing_tag = false

    ## String value of the tag when the tag does not exist
    ## Required when include_missing_tag is true
    # missing_tag_value = "null"

Maybe its better to name it inputs.logql_query, but this is very simmilar to VictoriaLogs "logsql".