Upgrade lucene to version 10.2.1 #17961
Open
+45
−46
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
❌ Gradle check result for 5e74113: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 93c4e0c: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 9a34fb8: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 84276e4: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
@expani see https://github.com/opensearch-project/OpenSearch/pull/17960/files#diff-53173a30404a65ce7f35073d65258143cfa2d47947ff8e0817337ff3d37e01f3L368 -- let's make use of the |
|
@harshavamsi I was thinking if you can merge the constant scorer change with some context as to why it helps. I can rebase it once merged and fix Lucene 10.2.0 upgrade stuff. I want to focus on test failures in this PR. Like this one |
sounds good, i'll add some context |
|
❌ Gradle check result for 48d60bc: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
Failure due to known flaky test #15806 with different seeds Tried the same with current mainline and it fails as well. So, don't think it's related to Lucene 10.2.0 upgrade. |
expani
requested review from
anasalkouz,
andrross,
ashking94,
Bukhtawar,
CEHENKLE,
cwperks,
dbwiddis,
gbbafna,
jed326,
kotwanikunal,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
sachinpkale,
saratvemulapalli and
shwetathareja
as code owners
Benchmark Baseline Comparison ResultsBenchmark Results for Job: https://build.ci.opensearch.org/job/benchmark-compare/93/
|
|
❌ Gradle check result for 048405a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 048405a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 048405a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
reproducible with full data ingestion. |
…h current ConstantScoreSupplier Signed-off-by: expani <anijainc@amazon.com>
|
{"run-benchmark-test": "id_4"} |
|
❌ Gradle check result for fd42095: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
@rishabh6788 Tried a fix for now. fd42095 |
Signed-off-by: expani <anijainc@amazon.com>
|
❌ Gradle check result for cefd174: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Seems just flaky as I have seen this before. |
|
The Jenkins job url is https://build.ci.opensearch.org/job/benchmark-pull-request/3041/ . Final results will be published once the job is completed. |
|
❌ Gradle check result for cefd174: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for cefd174: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Benchmark ResultsBenchmark Results for Job: https://build.ci.opensearch.org/job/benchmark-pull-request/3041/
|
Benchmark Baseline Comparison ResultsBenchmark Results for Job: https://build.ci.opensearch.org/job/benchmark-compare/95/
|
|
❌ Gradle check result for cefd174: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
range, asc_sort_timestamp and sort_numeric_asc are regressing heavily, rest look okay. |
Thanks for summarizing @rishabh6788 I will check if any of it is because of Lucene 10.2.x changes. Is the Benchmark Big5 Index Snapshot also updated to use Lucene 10.2.1 ? Did the 10.2.0 snapshot show variance after creation ? This might be a case of performance variance for sort and range queries relying on BKD due to #18168 ? |
I think lucene snapshot havent been published since 8 months ago |
|
But since you are using official version of lucene, why need snapshots? |
@peterzhuamazon I meant the index snapshot since |
|
Invalid comment format or config id. Please refer to https://github.com/opensearch-project/OpenSearch/blob/main/PERFORMANCE_BENCHMARKS.md on how to run benchmarks on pull requests. |
|
❌ Gradle check result for cefd174: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for cefd174: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for cefd174: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Upgrading to Lucene 10.2.1
https://lucene.apache.org/core/10_2_1/changes/Changes.html
Performance Testing Areas