Skip to content

[9.1] Adds new 'none' and 'recursive' chunking strategies to Inference APIs #4841

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 22, 2025
Merged

[9.1] Adds new 'none' and 'recursive' chunking strategies to Inference APIs #4841

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3c0c472
Adds new chunking strategies
kosabogi Jul 8, 2025
f856d38
Adds external link
kosabogi Jul 14, 2025
4f6cbf5
Adds a sentence to point to the link
kosabogi Jul 14, 2025
732d600
Resolves merge conflict
kosabogi Jul 16, 2025
1dc0035
Update specification/inference/_types/Services.ts
kosabogi Jul 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions specification/_doc_ids/table.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ ccr-put-follow,https://www.elastic.co/docs/api/doc/elasticsearch/operation/opera
ccr-resume-auto-follow-pattern,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ccr-resume-auto-follow-pattern,https://www.elastic.co/guide/en/elasticsearch/reference/8.18/ccr-resume-auto-follow-pattern.html,
ccs-network-delays,https://www.elastic.co/docs/solutions/search/cross-cluster-search#ccs-network-delays,,
ccs-privileges,https://www.elastic.co/docs/deploy-manage/remote-clusters/remote-clusters-cert#remote-clusters-privileges-ccs,,
chunking-strategies,https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#chunking-strategies,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to add a Description field here to avoid the default link text?

Chunking strategies probably works

Sorry if you've already planned to do this :)

Copy link
Contributor Author

@kosabogi kosabogi Jul 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this a few times, but it didn’t work, unfortunately.

I added a description to the table.csv file like this:

chunking-strategies,https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#chunking-strategies,https://www.elastic.co/guide/en/kibana/8.18/inference-endpoints.html,Chunking strategies documentation

However, I didn’t see any results in the elasticsearch-openapi.json file, neither the previous URL nor the link description:

"strategy": {
  "externalDocs": {
    "url": "https://www.elastic.co/docs/explore-analyze/elastic-inference/inference-api#chunking-strategies"
  },
  "description": "The chunking strategy: `sentence`, `word`, `none` or `recursive`.\n\n * If `strategy` is set to `recursive`, you must also specify:\n\n- `max_chunk_size`\n- either `separators` or `separator_group`\n\nLearn more about different chunking strategies in the External documentation.",
  "default": "sentence",
  "type": "string"
}

To understand what’s going on, I tested the exact same case that @lcawl referenced here:
#4772 (comment)

t worked perfectly for me too. The OpenAPI file was updated exactly as Lisa described in her comment.

My only guess is that the example Lisa tested wasn't for a parameter-level externalDocs link, but rather for a general API description, which might be treated differently by the tooling?
Could that explain the difference? Any ideas? Or am I missing something here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

images

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only guess is that the example Lisa tested wasn't for a parameter-level externalDocs link, but rather for a general API description, which might be treated differently by the tooling?
Could that explain the difference? Any ideas? Or am I missing something here?

That might be it, I think you can only have one ext-id per API too IIRC, don't know if there's a previous one defined for this API.

There's as many exceptions as there are rules I fear 😠

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also know it that way that you can have only one ext-id - but here, the ext-id seems to work, what doesn’t work is the unique link text 🤔
@lcawl, do you happen to have any idea why this might not be working?

clean-up-snapshot-repo,https://www.elastic.co/docs/deploy-manage/tools/snapshot-and-restore/self-managed#snapshots-repository-cleanup,,
clear-repositories-metering-archive-api,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-nodes-clear-repositories-metering-archive,https://www.elastic.co/guide/en/elasticsearch/reference/8.18/clear-repositories-metering-archive-api.html,
clear-scroll-api,https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-clear-scroll,https://www.elastic.co/guide/en/elasticsearch/reference/8.18/clear-scroll-api.html,
Expand Down
4 changes: 4 additions & 0 deletions specification/_types/mapping/ChunkingSettings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ import { OverloadOf } from '@spec_utils/behaviors'
export class ChunkingSettings implements OverloadOf<InferenceChunkingSettings> {
strategy: string

separator_group: string

separators: string[]

max_chunk_size: integer

overlap?: integer
Expand Down
29 changes: 28 additions & 1 deletion specification/inference/_types/Services.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -288,8 +288,35 @@ export class InferenceChunkingSettings {
*/
sentence_overlap?: integer
/**
* The chunking strategy: `sentence` or `word`.
* This parameter is only applicable when using the `recursive` chunking strategy.
*
* Sets a predefined list of separators in the saved chunking settings based on the selected text type.
* Values can be `markdown` or `plaintext`.
*
* Using this parameter is an alternative to manually specifying a custom `separators` list.
*/
separator_group: string
/**
* A list of strings used as possible split points when chunking text with the `recursive` strategy.
*
* Each string can be a plain string or a regular expression (regex) pattern.
* The system tries each separator in order to split the text, starting from the first item in the list.
*
* After splitting, it attempts to recombine smaller pieces into larger chunks that stay within
* the `max_chunk_size` limit, to reduce the total number of chunks generated.
*/
separators: string[]
/**
* The chunking strategy: `sentence`, `word`, `none` or `recursive`.
*
* * If `strategy` is set to `recursive`, you must also specify:
*
* - `max_chunk_size`
* - either `separators` or`separator_group`
*
* Learn more about different chunking strategies in the linked documentation.
* @server_default sentence
* @ext_doc_id chunking-strategies
*/
strategy?: string
}
Expand Down
4 changes: 4 additions & 0 deletions specification/inference/put/examples/request/InferencePutExample1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,8 @@ value: |-
"model_id": "rerank-english-v3.0",
"api_key": "{{COHERE_API_KEY}}"
}
"chunking_settings": {
"strategy": "recursive",
"max_chunk_size": 200,
"separator_group": "markdown"
}