Skip to content

Commit 492e844

Browse files
adding data_stream stats api docs (#10144)
* adding data_stream stats api docs Signed-off-by: Anton Rubin <anton.rubin@eliatra.com> * fixing vale errors Signed-off-by: Anton Rubin <anton.rubin@eliatra.com> * Apply suggestions from code review Signed-off-by: Nathan Bower <nbower@amazon.com> * Update data-stream-stats.md Signed-off-by: AntonEliatra <anton.rubin@eliatra.com> --------- Signed-off-by: Anton Rubin <anton.rubin@eliatra.com> Signed-off-by: Nathan Bower <nbower@amazon.com> Signed-off-by: AntonEliatra <anton.rubin@eliatra.com> Co-authored-by: Nathan Bower <nbower@amazon.com>
1 parent f7b3288 commit 492e844

File tree

1 file changed

+119
-0
lines changed

1 file changed

+119
-0
lines changed
Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
---
2+
layout: default
3+
title: Data stream stats
4+
parent: Index APIs
5+
nav_order: 74
6+
---
7+
8+
# Data stream stats
9+
10+
The Data Stream Stats API provides statistics about one or more data streams. This includes information such as the number of backing indexes, store size, and maximum timestamp. This API is useful for monitoring storage and indexing activity across data streams.
11+
12+
## Endpoints
13+
```json
14+
GET /_data_stream/_stats
15+
GET /_data_stream/{name}/_stats
16+
```
17+
18+
## Path parameters
19+
20+
The following table lists the available path parameters. All path parameters are optional.
21+
22+
| Parameter | Data type | Description |
23+
| :--- | :--- | :--- |
24+
| `name` | List or String | A comma-separated list of data streams used to limit the request. Wildcard expressions (`*`) are supported. To target all data streams in a cluster, omit this parameter or use `*`. |
25+
26+
## Query parameters
27+
28+
The following table lists the available query parameters. All query parameters are optional.
29+
30+
| Parameter | Data type | Description | Default |
31+
| :--- | :--- | :--- | :--- |
32+
| `error_trace` | Boolean | Whether to include the stack trace of returned errors. | `false` |
33+
| `filter_path` | List or String | Used to reduce the response. This parameter takes a comma-separated list of filters. It supports using wildcards to match any field or part of a field's name. You can also exclude fields with "-". | N/A |
34+
| `human` | Boolean | Whether to return human-readable values for statistics. | `false` |
35+
| `pretty` | Boolean | Whether to pretty format the returned JSON response. | `false` |
36+
| `source` | String | The URL-encoded request definition. Useful for libraries that do not accept a request body for non-POST requests. | N/A |
37+
38+
## Example
39+
40+
Create an index template with a matching pattern and data stream enabled:
41+
42+
```json
43+
PUT /_index_template/template-logs-app
44+
{
45+
"index_patterns": ["logs-app*"],
46+
"data_stream": {}
47+
}
48+
```
49+
{% include copy-curl.html %}
50+
51+
Create the data stream:
52+
53+
```json
54+
PUT /_data_stream/logs-app
55+
```
56+
{% include copy-curl.html %}
57+
58+
Index a document to generate backing indexes:
59+
60+
```json
61+
POST /logs-app/_doc
62+
{
63+
"@timestamp": "2025-06-23T10:00:00Z",
64+
"message": "app started"
65+
}
66+
```
67+
{% include copy-curl.html %}
68+
69+
70+
Retrieve data stream stats:
71+
72+
```json
73+
GET /_data_stream/logs-app/_stats?human=true
74+
```
75+
{% include copy-curl.html %}
76+
77+
## Example response
78+
79+
The response contains storage and shard statistics for each data stream in the cluster:
80+
81+
```json
82+
{
83+
"_shards": {
84+
"total": 2,
85+
"successful": 2,
86+
"failed": 0
87+
},
88+
"data_stream_count": 1,
89+
"backing_indices": 1,
90+
"total_store_size": "16.8kb",
91+
"total_store_size_bytes": 17304,
92+
"data_streams": [
93+
{
94+
"data_stream": "logs-app",
95+
"backing_indices": 1,
96+
"store_size": "16.8kb",
97+
"store_size_bytes": 17304,
98+
"maximum_timestamp": 1750673100000
99+
}
100+
]
101+
}
102+
```
103+
104+
## Response body fields
105+
106+
| Field | Data type | Description |
107+
| `_shards.total` | Integer | The total number of shards involved in the request. |
108+
| `_shards.successful`| Integer | The number of successful shard fetches. |
109+
| `_shards.failed`| Integer | The number of failed shard fetches. |
110+
| `data_stream_count` | Integer | The total number of data streams returned in the response.|
111+
| `backing_indices` | Integer | The total number of backing indexes across all data streams.|
112+
| `total_store_size`| String| A human-readable total size of all data stream storage. Present only if `human=true`. |
113+
| `total_store_size_bytes`| Integer | The total storage used by all data streams, in bytes. |
114+
| `data_streams`| Array | A list of objects, one for each data stream.|
115+
| `data_streams[n].data_stream` | String| The name of the data stream.|
116+
| `data_streams[n].backing_indices` | Integer | The number of backing indexes for the data stream.|
117+
| `data_streams[n].store_size`| String| Human-readable storage used by the data stream. Present only if `human=true`. |
118+
| `data_streams[n].store_size_bytes`| Integer | The total storage used by the data stream, in bytes.|
119+
| `data_streams[n].maximum_timestamp` | Long| The maximum timestamp across all documents in the data stream. |

0 commit comments

Comments
 (0)