Skip to content

Add multi-target support #1062

@pincher95

Description

@pincher95

✨ Feature request: Multi-target scraping via /probe endpoint

Background

Today each elasticsearch_exporter instance can scrape one cluster – the one given in --es.uri (or default http://localhost:9200).
In environments with dozens of SaaS-managed or shared clusters this forces operators to run one exporter per cluster, which is operationally heavy.

Proposal

  1. New HTTP endpoint /probe

    • Called with ?target=<http[s]://host:port> (and optional auth_module=).
    • On each request the exporter spins up a fresh Prometheus Registry, registers the usual collectors with the supplied target URL and streams the metrics.
  2. Auth modules (config file)

    auth_modules:
      prod_basic:
        type: userpass
        userpass:
          username: metrics
          password: s3cr3t
        options:
          sslmode: disable
    
      staging_key:
        type: apikey
        apikey: BASE64IDKEY==

    – File is loaded once at startup with --config.file=.
    /probe?target=...&auth_module=staging_key injects the creds/header.

  3. Alias on /metrics
    If a request to /metrics already contains target= it is internally routed to the probe handler, preserving full backward compatibility while letting users keep path: /metrics in ServiceMonitors.

  4. No breaking changes
    • Classic single-target /metrics behaviour is untouched.
    • All existing CLI flags (--es.indices, TLS flags, AWS signing, etc.) still apply.

Example Prometheus config

- job_name: es-multi
  metrics_path: /probe          # or /metrics
  params:
    auth_module: [prod_basic]   # default auth module
  static_configs:
    - targets:
        - https://es-prod:9200
    - targets:
        - https://es-staging:9200
      labels:
        __param_auth_module: staging_key
  relabel_configs:
    - source_labels: [__address__]
      target_label: __param_target
    - source_labels: [__param_target]
      target_label: instance
    - target_label: __address__
      replacement: exporter:9114   # exporter service

Acceptance criteria

  • /metrics continues to expose the startup cluster.
  • /probe?target=… returns metrics for any reachable cluster.
  • Auth-module userpass and apikey work as expected.
  • Alias /metrics?target=… behaves identical to /probe.
  • Documentation & example YAML updated.

Feedback and additional requirements welcome!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions