Skip to content

Pipeline: output: prom remote write: style #1965

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 22, 2025
Merged

Pipeline: output: prom remote write: style #1965

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1228548
Pipeline: output: prom remote write: style
esmerel Jul 21, 2025
fbcbb2f
fixing code format I messed up
esmerel Jul 21, 2025
a310292
Update pipeline/outputs/prometheus-remote-write.md
esmerel Jul 21, 2025
fce0984
Apply suggestions from code review
esmerel Jul 22, 2025
3e7f701
Update pipeline/outputs/prometheus-remote-write.md
esmerel Jul 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 48 additions & 46 deletions pipeline/outputs/prometheus-remote-write.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,38 @@
description: An output plugin to submit Prometheus Metrics using the remote write protocol
---

# Prometheus Remote Write

The prometheus remote write plugin allows you to take metrics from Fluent Bit and submit them to a Prometheus server through the remote write mechanism.

Important Note: The prometheus exporter only works with metric plugins, such as Node Exporter Metrics

| Key | Description | Default |
|----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------|
| host | IP address or hostname of the target HTTP Server | 127.0.0.1 |
| http_user | Basic Auth Username | |
| http_passwd | Basic Auth Password. Requires HTTP_user to be set | |
| AWS\_Auth | Enable AWS SigV4 authentication | false |
| AWS\_Service | For Amazon Managed Service for Prometheus, the service name is aps | aps |
| AWS\_Region | Region of your Amazon Managed Service for Prometheus workspace | |
| AWS\_STS\_Endpoint | Specify the custom sts endpoint to be used with STS API, used with the AWS_Role_ARN option, used by SigV4 authentication | |
| AWS\_Role\_ARN | AWS IAM Role to assume, used by SigV4 authentication | |
| AWS\_External\_ID | External ID for the AWS IAM Role specified with `aws_role_arn`, used by SigV4 authentication | |
| port | TCP port of the target HTTP Server | 80 |
| proxy | Specify an HTTP Proxy. The expected format of this value is `http://HOST:PORT`. Note that HTTPS is **not** currently supported. It is recommended not to set this and to configure the [HTTP proxy environment variables](https://docs.fluentbit.io/manual/administration/http-proxy) instead as they support both HTTP and HTTPS. | |
| uri | Specify an optional HTTP URI for the target web server, e.g: /something | / |
| header | Add a HTTP header key/value pair. Multiple headers can be set. | |
| log_response_payload | Log the response payload within the Fluent Bit log | false |
| add_label | This allows you to add custom labels to all metrics exposed through the prometheus exporter. You may have multiple of these fields | |
| workers | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `2` |

## Getting Started

The Prometheus remote write plugin only works with metrics collected by one of the metric input plugins. In the following example, host metrics are collected by the node exporter metrics plugin and then delivered by the prometheus remote write output plugin.
# Prometheus remote write

The _Prometheus remote write_ plugin lets you take metrics from Fluent Bit and submit them to a Prometheus server through the remote write mechanism.

The Prometheus exporter only works with metric plugins, such as Node Exporter Metrics

## Configuration parameters

This plugin supports the following parameters:

| Key | Description | Default |
|-----|-------------|---------|
| `host` | IP address or hostname of the target HTTP server. | `127.0.0.1` |
| `http_user` | Basic Auth username. | _none_ |
| `http_passwd` | Basic Auth Password. Requires `HTTP_user` to be set. | _none_ |
| `AWS_Auth` | Enable AWS SigV4 authentication | `false` |
| `AWS_Service` | For Amazon Managed Service for Prometheus, the service name is `aps`. | `aps` |
| `AWS_Region` | Region of your Amazon Managed Service for Prometheus workspace. | _none_ |
| `AWS_STS_Endpoint` | Specify the custom STS endpoint to be used with STS API, used with the `AWS_Role_ARN` option, used by SigV4 authentication. | _none_ |
| `AWS_Role_ARN` | AWS IAM Role to assume, used by SigV4 authentication. | _none_ |
| `AWS_External_ID` | External ID for the AWS IAM Role specified with `aws_role_arn`, used by SigV4 authentication. | _none_ |
| `port` | TCP port of the target HTTP server. | `80` |
| `proxy` | Specify an HTTP proxy. The expected format of this value is `http://HOST:PORT`. HTTPS isn't supported. Configure the [HTTP proxy environment variables](https://docs.fluentbit.io/manual/administration/http-proxy) instead as they support both HTTP and HTTPS. | _none_ |
| `uri` | Specify an optional HTTP URI for the target web server. For example: `/someuri` | `/` |
| `header` | Add a HTTP header key/value pair. Multiple headers can be set. | _none_ |
| `log_response_payload` | Log the response payload within the Fluent Bit log | `false` |
| `add_label` | This lets you add custom labels to all metrics exposed through the Prometheus exporter. You can have multiple of these fields. | _none_ |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `2` |

## Get started

The Prometheus remote write plugin only works with metrics collected by one of the metric input plugins. In the following example, host metrics are collected by the node exporter metrics plugin and then delivered by the Prometheus remote write output plugin.

{% tabs %}
{% tab title="fluent-bit.yaml" %}
Expand All @@ -40,12 +44,12 @@ The Prometheus remote write plugin only works with metrics collected by one of t
# The following example collects host metrics on Linux and delivers
# them through the Prometheus remote write plugin to new relic :
#
service:
service:
flush: 1
log_level: info

pipeline:
inputs:
inputs:
- name: node_exporter_metrics
tag: node_metrics
scrape_interval: 2
Expand All @@ -64,7 +68,6 @@ pipeline:
add_label:
- app fluent-bit
- color blue

# Note : it would be necessary to replace both YOUR_DATA_SOURCE_NAME and YOUR_LICENSE_KEY
# with real values for this example to work.
```
Expand Down Expand Up @@ -110,12 +113,11 @@ pipeline:

## Examples

The following are examples of using Prometheus remote write with hosted services below
The following are examples of using Prometheus remote write with hosted services:

### Grafana Cloud

With [Grafana Cloud](https://grafana.com/products/cloud/) hosted metrics you will need to use the specific host that is mentioned as well as specify the HTTP username and password given within the Grafana Cloud page.

With [Grafana Cloud](https://grafana.com/products/cloud/) hosted metrics you will need to use the specific host mentioned and specify the HTTP username and password given within the Grafana Cloud page.

{% tabs %}
{% tab title="fluent-bit.yaml" %}
Expand All @@ -126,7 +128,7 @@ pipeline:
outputs:
- name: prometheus_remote_write
match: '*'
host: prometheus-us-central1.grafana.net
host: prometheus-us-central1.grafana.net
uri: /api/prom/push
port: 443
tls: on
Expand Down Expand Up @@ -154,9 +156,9 @@ pipeline:
{% endtab %}
{% endtabs %}

### Logz.io Infrastructure Monitoring
### Logz.io infrastructure monitoring

With Logz.io [hosted prometheus](https://logz.io/solutions/infrastructure-monitoring/) you will need to make use of the header option and add the Authorization Bearer with the proper key. The host and port may also differ within your specific hosted instance.
With Logz.io [hosted Prometheus](https://logz.io/solutions/infrastructure-monitoring/) you will need to make use of the header option and add the Authorization Bearer with the proper key. The host and port can also differ within your specific hosted instance.

{% tabs %}
{% tab title="fluent-bit.yaml" %}
Expand All @@ -167,7 +169,7 @@ pipeline:
outputs:
- name: prometheus_remote_write
match: '*'
host: listener.logz.io
host: listener.logz.io
port: 8053
tls: on
tls.verify: on
Expand All @@ -194,7 +196,7 @@ pipeline:

### Coralogix

With [Coralogix Metrics](https://coralogix.com/platform/metrics/) you may need to customize the URI. Additionally, you will make use of the header key with Coralogix private key.
With [Coralogix Metrics](https://coralogix.com/platform/metrics/) you might need to customize the URI. Additionally, you will make use of the header key with Coralogix private key.

{% tabs %}
{% tab title="fluent-bit.yaml" %}
Expand All @@ -206,7 +208,7 @@ pipeline:
- name: prometheus_remote_write
match: '*'
host: metrics-api.coralogix.com
uri: prometheus/api/v1/write?appLabelName=path&subSystemLabelName=path&severityLabelName=severity
uri: prometheus/api/v1/write?appLabelName=path&subSystemLabelName=path&severityLabelName=severity
port: 443
header: 'Authorization Bearer <CORALOGIX Key>'
tls: on
Expand Down Expand Up @@ -245,7 +247,7 @@ pipeline:
- name: prometheus_remote_write
match: '*'
host: app-tsdb.last9.io
uri: /v1/metrics/82xxxx/sender/org-slug/write
uri: /v1/metrics/82xxxx/sender/org-slug/write
port: 443
tls: on
tls.verify: on
Expand All @@ -272,9 +274,9 @@ pipeline:
{% endtab %}
{% endtabs %}

### Add Prometheus like Labels
### Add Prometheus-like labels

Ordinary prometheus clients add some of the labels as below:
Ordinary Prometheus clients add some of the following labels:

{% tabs %}
{% tab title="fluent-bit.yaml" %}
Expand All @@ -285,7 +287,7 @@ pipeline:
outputs:
- name: prometheus_remote_write
match: your.metric
host: xxxxxxx.yyyyy.zzzz
host: xxxxxxx.yyyyy.zzzz
port: 443
uri: /api/v1/write
header: 'Authorization Bearer YOUR_LICENSE_KEY'
Expand Down Expand Up @@ -320,4 +322,4 @@ pipeline:
{% endtab %}
{% endtabs %}

`instance` label can be emulated with `add_label instance ${HOSTNAME}`. And other labels can be added with `add_label <key> <value>` setting.
The `instance` label can be emulated with `add_label instance ${HOSTNAME}`. And other labels can be added with `add_label <key> <value>` setting.
1 change: 1 addition & 0 deletions vale-styles/FluentBit/Headings.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ exceptions:
- Google Cloud BigQuery
- Google Cloud Platform
- Grafana
- Grafana Cloud
- Graylog Extended Log Format
- gRPC
- HTTP
Expand Down