[Backport 2.x] Fix local segments stats update in RemoteStoreRefreshListener (#8758) #9268

ashking94 · 2023-08-11T15:32:17Z

Description

Backport 2416d45 from #8758.

opensearch-trigger-bot · 2023-08-11T15:58:47Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git]
Components skipped due to git failures: [https://github.com/opensearch-project/opensearch-oci-object-storage.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git]

BUILD SUCCESSFUL in 26m 7s

github-actions · 2023-08-11T16:01:08Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/22485/
CommitID: fae2e85
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

ashking94 · 2023-08-11T17:32:00Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌

URL: https://build.ci.opensearch.org/job/gradle-check/22485/

CommitID: fae2e85
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

org.opensearch.client.ResponseException: method [PUT], host [http://127.0.0.1:38277], URI [/test-index-segrep], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"unknown value for [index.codec] must be one of [default, best_compression, zstd, zstd_no_dict] but was: zlib"}],"type":"illegal_argument_exception","reason":"unknown value for [index.codec] must be one of [default, best_compression, zstd, zstd_no_dict] but was: zlib"},"status":400}

opensearch-trigger-bot · 2023-08-12T08:41:05Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git]
Components skipped due to git failures: [https://github.com/opensearch-project/opensearch-oci-object-storage.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 25m 10s

github-actions · 2023-08-12T09:05:47Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/22571/
CommitID: 2acda32
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot · 2023-08-12T09:47:16Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git]
Components skipped due to git failures: [https://github.com/opensearch-project/opensearch-oci-object-storage.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 25m 28s

github-actions · 2023-08-12T10:04:15Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/22575/
CommitID: 62d15e6
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot · 2023-08-12T22:15:08Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer.git]
Components skipped due to git failures: [https://github.com/opensearch-project/opensearch-oci-object-storage.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/ml-commons.git]

BUILD SUCCESSFUL in 25m 18s

github-actions · 2023-08-12T22:35:51Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/22586/
CommitID: c8a4925
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

…arch-project#8758) Signed-off-by: Ashish Singh <ssashish@amazon.com>

opensearch-trigger-bot · 2023-08-13T07:00:21Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/performance-analyzer.git]
Components skipped due to git failures: [https://github.com/opensearch-project/opensearch-oci-object-storage.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 25m 17s

github-actions · 2023-08-13T07:24:35Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      2 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.classMethod
      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

URL: https://build.ci.opensearch.org/job/gradle-check/22588/
CommitID: 7f7fd33
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2023-08-13T07:25:06Z

Codecov Report

Merging #9268 (7f7fd33) into 2.x (d30199f) will increase coverage by 0.09%.
The diff coverage is 79.74%.

@@             Coverage Diff              @@
##                2.x    #9268      +/-   ##
============================================
+ Coverage     70.80%   70.89%   +0.09%     
- Complexity    57491    57558      +67     
============================================
  Files          4759     4759              
  Lines        271848   271867      +19     
  Branches      40093    40090       -3     
============================================
+ Hits         192481   192748     +267     
+ Misses        62846    62584     -262     
- Partials      16521    16535      +14

Files Changed	Coverage Δ
...nsearch/index/shard/CheckpointRefreshListener.java	`84.61% <ø> (+7.69%)`	⬆️
...earch/index/store/RemoteSegmentStoreDirectory.java	`87.70% <0.00%> (ø)`
...index/shard/CloseableRetryableRefreshListener.java	`76.78% <67.64%> (+3.45%)`	⬆️
...rch/index/remote/RemoteSegmentTransferTracker.java	`89.56% <86.36%> (+0.36%)`	⬆️
...search/index/shard/RemoteStoreRefreshListener.java	`85.71% <95.45%> (+2.56%)`	⬆️

... and 464 files with indirect coverage changes

ashking94 added the skip-changelog label Aug 11, 2023

ashking94 changed the title ~~Fix local segments stats update in RemoteStoreRefreshListener (#8758)~~ [Backport 2.x] Fix local segments stats update in RemoteStoreRefreshListener (#8758) Aug 12, 2023

ashking94 force-pushed the backport-9137 branch from fae2e85 to 2acda32 Compare August 12, 2023 08:15

ashking94 marked this pull request as ready for review August 12, 2023 08:15

ashking94 force-pushed the backport-9137 branch from 2acda32 to 62d15e6 Compare August 12, 2023 09:21

ashking94 force-pushed the backport-9137 branch from 62d15e6 to c8a4925 Compare August 12, 2023 21:49

Fix local segments stats update in RemoteStoreRefreshListener (opense…

7f7fd33

…arch-project#8758) Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 force-pushed the backport-9137 branch from c8a4925 to 7f7fd33 Compare August 13, 2023 06:34

sachinpkale approved these changes Aug 14, 2023

View reviewed changes

sachinpkale merged commit 77df735 into opensearch-project:2.x Aug 14, 2023

ashking94 mentioned this pull request Dec 26, 2023

[Backport 2.x] Fix local segments stats update in RemoteStoreRefreshListener #9137

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Backport 2.x] Fix local segments stats update in RemoteStoreRefreshListener (#8758) #9268

[Backport 2.x] Fix local segments stats update in RemoteStoreRefreshListener (#8758) #9268

Uh oh!

ashking94 commented Aug 11, 2023

Uh oh!

opensearch-trigger-bot bot commented Aug 11, 2023

Uh oh!

github-actions bot commented Aug 11, 2023

Uh oh!

ashking94 commented Aug 11, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

opensearch-trigger-bot bot commented Aug 12, 2023

Uh oh!

github-actions bot commented Aug 12, 2023

Uh oh!

opensearch-trigger-bot bot commented Aug 12, 2023

Uh oh!

github-actions bot commented Aug 12, 2023

Uh oh!

opensearch-trigger-bot bot commented Aug 12, 2023

Uh oh!

github-actions bot commented Aug 12, 2023

Uh oh!

opensearch-trigger-bot bot commented Aug 13, 2023

Uh oh!

github-actions bot commented Aug 13, 2023

Uh oh!

codecov bot commented Aug 13, 2023

Uh oh!

Uh oh!

[Backport 2.x] Fix local segments stats update in RemoteStoreRefreshListener (#8758) #9268

[Backport 2.x] Fix local segments stats update in RemoteStoreRefreshListener (#8758) #9268

Uh oh!

Conversation

ashking94 commented Aug 11, 2023

Description

Uh oh!

opensearch-trigger-bot bot commented Aug 11, 2023

Uh oh!

github-actions bot commented Aug 11, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

ashking94 commented Aug 11, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

opensearch-trigger-bot bot commented Aug 12, 2023

Uh oh!

github-actions bot commented Aug 12, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

opensearch-trigger-bot bot commented Aug 12, 2023

Uh oh!

github-actions bot commented Aug 12, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

opensearch-trigger-bot bot commented Aug 12, 2023

Uh oh!

github-actions bot commented Aug 12, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

opensearch-trigger-bot bot commented Aug 13, 2023

Uh oh!

github-actions bot commented Aug 13, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

codecov bot commented Aug 13, 2023

Codecov Report

Uh oh!

Uh oh!