From 7fe6cde86eea1e4804639915d62f95794cd11b4f Mon Sep 17 00:00:00 2001 From: Anton Rubin Date: Mon, 14 Apr 2025 12:17:07 +0100 Subject: [PATCH 1/4] Adding retrottle api docs Signed-off-by: Anton Rubin --- _api-reference/tasks/rethrottle.md | 84 ++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 _api-reference/tasks/rethrottle.md diff --git a/_api-reference/tasks/rethrottle.md b/_api-reference/tasks/rethrottle.md new file mode 100644 index 00000000000..cbc0d2e1a45 --- /dev/null +++ b/_api-reference/tasks/rethrottle.md @@ -0,0 +1,84 @@ +--- +layout: default +title: Rethrottle +parent: Tasks API +nav_order: 40 +--- + +# Rethrottle + +You can use the following APIs to dynamically change the `requests_per_second` for [`_reindex`]({{site.url}}{{site.baseurl}}/api-reference/document-apis/reindex/), [`_update_by_query`]({{site.url}}{{site.baseurl}}/api-reference/document-apis/update-by-query/), or [`_delete_by_query`]({{site.url}}{{site.baseurl}}/api-reference/document-apis/delete-by-query/) operations that are already running. + +## Endpoints + +```json +POST /_delete_by_query/{task_id}/_rethrottle +POST /_reindex/{task_id}/_rethrottle +POST /_update_by_query/{task_id}/_rethrottle +``` + +## Path parameters + +Parameter | Data type | Description +:--- | :--- | :--- +`task_id` | String | The unique identifier for the running task that you want to rethrottle. + +## Query parameters + +Parameter | Data type | Description +:--- | :--- | :--- +`requests_per_second` | Float | The new throttle value to apply to the task. Use `-1` to disable throttling. Optional. + +### Example request: Re-throttle a running delete by query task + +```bash +curl -X POST "localhost:9200/_delete_by_query//_rethrottle?requests_per_second=10" -H 'Content-Type: application/json' +``` + +### Example request: Re-throttle a running reindex task + +```bash +curl -X POST "localhost:9200/_reindex//_rethrottle?requests_per_second=20" -H 'Content-Type: application/json' +``` + +### Example request: Re-throttle a running update by query task + +```bash +curl -X POST "localhost:9200/_update_by_query//_rethrottle?requests_per_second=5" -H 'Content-Type: application/json' +``` + +## Response body fields + +The response provides detailed task and node-level information about the rethrottled operation. + +| Field | Data type | Description | +| :--- | :--- | :--- | +| `nodes` | Object | A map of node IDs to details about the task on each node. | +| `nodes..name` | String | The name of the node where the task is running. | +| `nodes..transport_address` | String | The transport address of the node. | +| `nodes..host` | String | The host IP address. | +| `nodes..ip` | String | The IP address and port. | +| `nodes..roles` | Array | The roles assigned to the node. | +| `nodes..attributes` | Object | Node-level attributes. | +| `nodes..tasks` | Object | A map of task IDs to detailed information about each task. | +| `nodes..tasks..type` | String | The task type, such as `transport`. | +| `nodes..tasks..action` | String | The specific action being performed (e.g., `reindex`). | +| `nodes..tasks..status` | Object | Current status of the task. | +| `nodes..tasks..status.total` | Integer | Total number of documents to process. | +| `nodes..tasks..status.created` | Integer | Number of documents created. | +| `nodes..tasks..status.updated` | Integer | Number of documents updated. | +| `nodes..tasks..status.deleted` | Integer | Number of documents deleted. | +| `nodes..tasks..status.batches` | Integer | Number of batches processed. | +| `nodes..tasks..status.version_conflicts` | Integer | Number of version conflicts. | +| `nodes..tasks..status.noops` | Integer | Number of no-op updates. | +| `nodes..tasks..status.retries` | Object | Retry stats for bulk and search operations. | +| `nodes..tasks..status.requests_per_second` | Float | Current throttle rate in requests per second. | +| `nodes..tasks..status.throttled_millis` | Integer | Time in milliseconds the task was throttled. | +| `nodes..tasks..status.throttled_until_millis` | Integer | Time in milliseconds the task is expected to remain throttled. | +| `nodes..tasks..description` | String | A human-readable description of the task. | +| `nodes..tasks..start_time_in_millis` | Integer | Task start time in epoch milliseconds. | +| `nodes..tasks..running_time_in_nanos` | Integer | Task runtime in nanoseconds. | +| `nodes..tasks..cancellable` | Boolean | Whether the task can be cancelled. | +| `nodes..tasks..cancelled` | Boolean | Whether the task has been cancelled. | +| `nodes..tasks..headers` | Object | Optional HTTP headers associated with the task. | +| `nodes..tasks..resource_stats` | Object | Statistics about resource usage. | \ No newline at end of file From c461394e5b5d99aa9ec0834b8d5f5bf74b1886e8 Mon Sep 17 00:00:00 2001 From: AntonEliatra Date: Mon, 30 Jun 2025 19:56:38 +0100 Subject: [PATCH 2/4] Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: AntonEliatra --- _api-reference/tasks/rethrottle.md | 32 +++++++++++++++--------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/_api-reference/tasks/rethrottle.md b/_api-reference/tasks/rethrottle.md index cbc0d2e1a45..9dae2cbef14 100644 --- a/_api-reference/tasks/rethrottle.md +++ b/_api-reference/tasks/rethrottle.md @@ -54,7 +54,7 @@ The response provides detailed task and node-level information about the rethrot | Field | Data type | Description | | :--- | :--- | :--- | | `nodes` | Object | A map of node IDs to details about the task on each node. | -| `nodes..name` | String | The name of the node where the task is running. | +| `nodes..name` | String | The name of the node on which the task is running. | | `nodes..transport_address` | String | The transport address of the node. | | `nodes..host` | String | The host IP address. | | `nodes..ip` | String | The IP address and port. | @@ -62,23 +62,23 @@ The response provides detailed task and node-level information about the rethrot | `nodes..attributes` | Object | Node-level attributes. | | `nodes..tasks` | Object | A map of task IDs to detailed information about each task. | | `nodes..tasks..type` | String | The task type, such as `transport`. | -| `nodes..tasks..action` | String | The specific action being performed (e.g., `reindex`). | -| `nodes..tasks..status` | Object | Current status of the task. | -| `nodes..tasks..status.total` | Integer | Total number of documents to process. | -| `nodes..tasks..status.created` | Integer | Number of documents created. | -| `nodes..tasks..status.updated` | Integer | Number of documents updated. | -| `nodes..tasks..status.deleted` | Integer | Number of documents deleted. | -| `nodes..tasks..status.batches` | Integer | Number of batches processed. | -| `nodes..tasks..status.version_conflicts` | Integer | Number of version conflicts. | -| `nodes..tasks..status.noops` | Integer | Number of no-op updates. | +| `nodes..tasks..action` | String | The specific action being performed (for example, `reindex`). | +| `nodes..tasks..status` | Object | The current status of the task. | +| `nodes..tasks..status.total` | Integer | The total number of documents to process. | +| `nodes..tasks..status.created` | Integer | The number of documents created. | +| `nodes..tasks..status.updated` | Integer | The number of documents updated. | +| `nodes..tasks..status.deleted` | Integer | The number of documents deleted. | +| `nodes..tasks..status.batches` | Integer | The number of batches processed. | +| `nodes..tasks..status.version_conflicts` | Integer | The number of version conflicts. | +| `nodes..tasks..status.noops` | Integer | The number of no-op updates. | | `nodes..tasks..status.retries` | Object | Retry stats for bulk and search operations. | | `nodes..tasks..status.requests_per_second` | Float | Current throttle rate in requests per second. | -| `nodes..tasks..status.throttled_millis` | Integer | Time in milliseconds the task was throttled. | -| `nodes..tasks..status.throttled_until_millis` | Integer | Time in milliseconds the task is expected to remain throttled. | +| `nodes..tasks..status.throttled_millis` | Integer | The time, in milliseconds, the task was throttled. | +| `nodes..tasks..status.throttled_until_millis` | Integer | The time, in milliseconds, the task is expected to remain throttled. | | `nodes..tasks..description` | String | A human-readable description of the task. | -| `nodes..tasks..start_time_in_millis` | Integer | Task start time in epoch milliseconds. | -| `nodes..tasks..running_time_in_nanos` | Integer | Task runtime in nanoseconds. | -| `nodes..tasks..cancellable` | Boolean | Whether the task can be cancelled. | -| `nodes..tasks..cancelled` | Boolean | Whether the task has been cancelled. | +| `nodes..tasks..start_time_in_millis` | Integer | The task start time in epoch milliseconds. | +| `nodes..tasks..running_time_in_nanos` | Integer | The task runtime in nanoseconds. | +| `nodes..tasks..cancellable` | Boolean | Whether the task can be canceled. | +| `nodes..tasks..cancelled` | Boolean | Whether the task has been canceled. | | `nodes..tasks..headers` | Object | Optional HTTP headers associated with the task. | | `nodes..tasks..resource_stats` | Object | Statistics about resource usage. | \ No newline at end of file From e4307134107ce54d0c38ba463cf55eb732be0c79 Mon Sep 17 00:00:00 2001 From: Anton Rubin Date: Mon, 30 Jun 2025 20:13:14 +0100 Subject: [PATCH 3/4] addressing PR comments Signed-off-by: Anton Rubin --- _api-reference/tasks/rethrottle.md | 96 ++++++++++++++++++++++++++++-- 1 file changed, 90 insertions(+), 6 deletions(-) diff --git a/_api-reference/tasks/rethrottle.md b/_api-reference/tasks/rethrottle.md index 9dae2cbef14..c03e2088543 100644 --- a/_api-reference/tasks/rethrottle.md +++ b/_api-reference/tasks/rethrottle.md @@ -31,20 +31,104 @@ Parameter | Data type | Description ### Example request: Re-throttle a running delete by query task -```bash -curl -X POST "localhost:9200/_delete_by_query//_rethrottle?requests_per_second=10" -H 'Content-Type: application/json' +```json +POST /_delete_by_query//_rethrottle?requests_per_second=10 ``` +{% include copy-curl.html %} + ### Example request: Re-throttle a running reindex task -```bash -curl -X POST "localhost:9200/_reindex//_rethrottle?requests_per_second=20" -H 'Content-Type: application/json' +```json +POST /_reindex//_rethrottle?requests_per_second=20 ``` +{% include copy-curl.html %} ### Example request: Re-throttle a running update by query task -```bash -curl -X POST "localhost:9200/_update_by_query//_rethrottle?requests_per_second=5" -H 'Content-Type: application/json' +```json +POST /_update_by_query//_rethrottle?requests_per_second=5 +``` +{% include copy-curl.html %} + +## Example response + +The following response provides details regarding the active `update_by_query` task: + +``` +{ + "nodes": { + "bvv8SKpiRhOhF9_Bu8gZ7w": { + "name": "opensearch-node1", + "transport_address": "172.18.0.4:9300", + "host": "172.18.0.4", + "ip": "172.18.0.4:9300", + "roles": [ + "cluster_manager", + "data", + "ingest", + "remote_cluster_client" + ], + "attributes": { + "shard_indexing_pressure_enabled": "true" + }, + "tasks": { + "bvv8SKpiRhOhF9_Bu8gZ7w:640": { + "node": "bvv8SKpiRhOhF9_Bu8gZ7w", + "id": 640, + "type": "transport", + "action": "indices:data/write/update/byquery", + "status": { + "total": 4785, + "updated": 1000, + "created": 0, + "deleted": 0, + "batches": 1, + "version_conflicts": 0, + "noops": 0, + "retries": { + "bulk": 0, + "search": 0 + }, + "throttled_millis": 0, + "requests_per_second": 50, + "throttled_until_millis": 2146 + }, + "description": "update-by-query [test-rethrottle] updated with Script{type=inline, lang='painless', idOrCode='ctx._source.new_field = 'updated'', options={}, params={}}", + "start_time_in_millis": 1751310547697, + "running_time_in_nanos": 9567425129, + "cancellable": true, + "cancelled": false, + "headers": { + "X-Opaque-Id": "1b911516-44cd-4920-8c1e-79368ea7cdfd" + }, + "resource_stats": { + "average": { + "cpu_time_in_nanos": 0, + "memory_in_bytes": 0 + }, + "total": { + "cpu_time_in_nanos": 0, + "memory_in_bytes": 0 + }, + "min": { + "cpu_time_in_nanos": 0, + "memory_in_bytes": 0 + }, + "max": { + "cpu_time_in_nanos": 0, + "memory_in_bytes": 0 + }, + "thread_info": { + "thread_executions": 0, + "active_threads": 0 + } + } + } + } + } + } +} ``` ## Response body fields From e245865b00e9223364a23f99ff96e2de181a0229 Mon Sep 17 00:00:00 2001 From: Nathan Bower Date: Tue, 1 Jul 2025 11:51:33 -0400 Subject: [PATCH 4/4] Apply suggestions from code review Signed-off-by: Nathan Bower --- _api-reference/tasks/rethrottle.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/_api-reference/tasks/rethrottle.md b/_api-reference/tasks/rethrottle.md index c03e2088543..cdd14759013 100644 --- a/_api-reference/tasks/rethrottle.md +++ b/_api-reference/tasks/rethrottle.md @@ -29,7 +29,7 @@ Parameter | Data type | Description :--- | :--- | :--- `requests_per_second` | Float | The new throttle value to apply to the task. Use `-1` to disable throttling. Optional. -### Example request: Re-throttle a running delete by query task +### Example request: Rethrottle a running delete by query task ```json POST /_delete_by_query//_rethrottle?requests_per_second=10 @@ -37,14 +37,14 @@ POST /_delete_by_query//_rethrottle?requests_per_second=10 {% include copy-curl.html %} -### Example request: Re-throttle a running reindex task +### Example request: Rethrottle a running reindex task ```json POST /_reindex//_rethrottle?requests_per_second=20 ``` {% include copy-curl.html %} -### Example request: Re-throttle a running update by query task +### Example request: Rethrottle a running update by query task ```json POST /_update_by_query//_rethrottle?requests_per_second=5 @@ -133,7 +133,7 @@ The following response provides details regarding the active `update_by_query` t ## Response body fields -The response provides detailed task and node-level information about the rethrottled operation. +The response provides detailed task- and node-level information about the rethrottled operation. | Field | Data type | Description | | :--- | :--- | :--- | @@ -157,8 +157,8 @@ The response provides detailed task and node-level information about the rethrot | `nodes..tasks..status.noops` | Integer | The number of no-op updates. | | `nodes..tasks..status.retries` | Object | Retry stats for bulk and search operations. | | `nodes..tasks..status.requests_per_second` | Float | Current throttle rate in requests per second. | -| `nodes..tasks..status.throttled_millis` | Integer | The time, in milliseconds, the task was throttled. | -| `nodes..tasks..status.throttled_until_millis` | Integer | The time, in milliseconds, the task is expected to remain throttled. | +| `nodes..tasks..status.throttled_millis` | Integer | The time, in milliseconds, that the task was throttled. | +| `nodes..tasks..status.throttled_until_millis` | Integer | The time, in milliseconds, that the task is expected to remain throttled. | | `nodes..tasks..description` | String | A human-readable description of the task. | | `nodes..tasks..start_time_in_millis` | Integer | The task start time in epoch milliseconds. | | `nodes..tasks..running_time_in_nanos` | Integer | The task runtime in nanoseconds. |