Set INDICES_MAX_CLAUSE_COUNT dynamically #13568

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

msfroh merged 12 commits into opensearch-project:main from harshavamsi:dynamic_max_clause

Jun 18, 2024

Contributor

harshavamsi commented May 6, 2024 •

edited

Loading

Description

Changes the INDICES_MAX_CLAUSE_COUNT_SETTING to be dynamic instead of being static.

This PR is virtually identical to #1527, but there was earlier push back around making this an updateable dynamic setting due to the assumption that users would abuse this setting rather than optimize their underlying query. Lucene has since updated the way the clause counts work and this is now enforced at the indexSearcher level -- https://issues.apache.org/jira/browse/LUCENE-8811.

More info can be found in #12549 around the motivation for this change, but I ended up raising a PR to make a simple dynamic setting.

Related Issues

Resolves #1526 #12549

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)
Public documentation issue/PR created https://github.com/opensearch-project/documentation-website/pull/7391/files

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.


          Set INDICES_MAX_CLAUSE_COUNT dynamically

dd140e2

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

Contributor

github-actions bot commented May 6, 2024

❌ Gradle check result for dd140e2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

jainankitk reviewed

View reviewed changes

server/src/main/java/org/opensearch/search/SearchService.java Show resolved Hide resolved

harshavamsi force-pushed the dynamic_max_clause branch from 6ccaa23 to 28d5525 Compare

May 7, 2024 00:24


          Adds new test to verify clause count is updated

4ff1d13

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

harshavamsi force-pushed the dynamic_max_clause branch from 28d5525 to 4ff1d13 Compare

May 7, 2024 00:27

Contributor

github-actions bot commented May 7, 2024

❌ Gradle check result for 28d5525: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          Changelog

6321ca8

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

harshavamsi marked this pull request as ready for review

May 7, 2024 00:58

harshavamsi requested review from Bukhtawar, CEHENKLE, Rishikesh1159, VachaShah, anasalkouz, andrross, dblock, dbwiddis, dreamer-89, gbbafna, kotwanikunal, mch2, msfroh, nknize, owaiskazi19, reta, sachinpkale, saratvemulapalli, shwetathareja, sohami and tlfeng as code owners

May 7, 2024 00:58

github-actions bot added the v3.0.0 label


          Fix changelog

e715516

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

harshavamsi force-pushed the dynamic_max_clause branch from b221209 to e715516 Compare

June 13, 2024 21:20

Contributor

github-actions bot commented Jun 13, 2024

❌ Gradle check result for b221209: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

harshavamsi mentioned this pull request

[DOC]Change the docs around max_clause_count opensearch-project/documentation-website#7385

Closed

4 tasks

This was referenced Jun 13, 2024

[AUTOCUT] Gradle Check Flaky Test Report for IndicesRequestCacheIT #14288

Closed

[AUTOCUT] Gradle Check Flaky Test Report for SearchWithRandomIOExceptionsIT #14306

Open

Contributor

github-actions bot commented Jun 13, 2024

❌ Gradle check result for e715516: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

reta approved these changes

View reviewed changes

Contributor Author

harshavamsi commented Jun 14, 2024

Opened https://github.com/opensearch-project/documentation-website/pull/7391/files

opensearch-ci-bot mentioned this pull request

[AUTOCUT] Gradle Check Flaky Test Report for RemoteSplitIndexIT #14296

Open

prudhvigodithi mentioned this pull request

[AUTOCUT] Gradle Check Flaky Test Report for IndicesRequestCacheIT prudhvigodithi/OpenSearch#27

Closed

Contributor

github-actions bot commented Jun 17, 2024

❌ Gradle check result for e715516: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor Author

harshavamsi commented Jun 17, 2024

❌ Gradle check result for e715516: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

#14300
#12651
@msfroh can we merge?

This was referenced Jun 17, 2024

[AUTOCUT] Gradle Check Flaky Test Report for TieredSpilloverCacheStatsIT #14300

Closed

[AUTOCUT] Gradle Check Flaky Test Report for AzureBlobContainerRetriesTests #14318

Open

Contributor

github-actions bot commented Jun 18, 2024

✅ Gradle check result for e715516: SUCCESS

msfroh merged commit c5c6024 into opensearch-project:main

opensearch-trigger-bot bot pushed a commit that referenced this pull request


          Set INDICES_MAX_CLAUSE_COUNT dynamically (#13568)

231daad

---------

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
(cherry picked from commit c5c6024)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

opensearch-trigger-bot bot mentioned this pull request

[Backport 2.x] Set INDICES_MAX_CLAUSE_COUNT dynamically #14436

Merged

harshavamsi deleted the dynamic_max_clause branch

June 18, 2024 18:38

dblock pushed a commit that referenced this pull request


          Set INDICES_MAX_CLAUSE_COUNT dynamically (#13568) (#14436)

8148e4d

---------


(cherry picked from commit c5c6024)

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

harshavamsi added a commit to harshavamsi/OpenSearch that referenced this pull request


          Set INDICES_MAX_CLAUSE_COUNT dynamically (opensearch-project#13568)

d4d8666

---------

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

kkewwei pushed a commit to kkewwei/OpenSearch that referenced this pull request


          Set INDICES_MAX_CLAUSE_COUNT dynamically (opensearch-project#13568) (o…

8b4619b

…pensearch-project#14436)

---------

(cherry picked from commit c5c6024)

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: kkewwei <kkewwei@163.com>

BrewTestBot mentioned this pull request

opensearch 2.16.0 Homebrew/homebrew-core#180446

Merged

wdongyu pushed a commit to wdongyu/OpenSearch that referenced this pull request


          Set INDICES_MAX_CLAUSE_COUNT dynamically (opensearch-project#13568)

7aaee0c

---------

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

msfroh mentioned this pull request

[Feature Request] Make rewrite-level max_clause_count check configurable at the index and/or query level #16723

Open

opensearch-ci-bot mentioned this pull request

[AUTOCUT] Gradle Check Flaky Test Report for S3BlobContainerRetriesTests #17551

Open

This was referenced Jan 15, 2026

docs: add query-improvements report for v2.16.0 tkykenmt/opensearch-feature-explorer#2421

Merged

[feature] Query Improvements tkykenmt/opensearch-feature-explorer#2255

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

jainankitk jainankitk approved these changes

reta reta approved these changes

anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

andrross Awaiting requested review from andrross andrross is a code owner

Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

dblock Awaiting requested review from dblock

dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

dreamer-89 Awaiting requested review from dreamer-89

gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

mch2 Awaiting requested review from mch2 mch2 is a code owner

msfroh Awaiting requested review from msfroh msfroh is a code owner

nknize Awaiting requested review from nknize

owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

sohami Awaiting requested review from sohami sohami is a code owner

tlfeng Awaiting requested review from tlfeng

VachaShah Awaiting requested review from VachaShah

jed326 Awaiting requested review from jed326 jed326 is a code owner

peternied Awaiting requested review from peternied

Labels

backport 2.x enhancement Search v2.16.0 v3.0.0