[AUTOCUT] Gradle Check Flaky Test Report for MetricAggregatorTests

[opensearch-ci-bot]

Flaky Test Report for MetricAggregatorTests

Noticed the MetricAggregatorTests has some flaky, failing tests that failed during post-merge actions.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
16e9fed	18221	57830	org.opensearch.search.aggregations.startree.MetricAggregatorTests.testStarTreeDocValues
18c64db	18112	57314	org.opensearch.search.aggregations.startree.MetricAggregatorTests.testStarTreeDocValues
1b48dbd	18090	57280	org.opensearch.search.aggregations.startree.MetricAggregatorTests.testStarTreeDocValues
92b90fa	18220	58259	org.opensearch.search.aggregations.startree.MetricAggregatorTests.testStarTreeDocValues

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the MetricAggregatorTests class are:

• 17275
• 16481
• 18106
• 17100
• 18070
• 15289
• 17759
• 17855
• 17941
• 17989
• 18054
• 18088
• 18116
• 18119
• 18124
• 18162

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[andrross]

This is easily reproducible:

REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.search.aggregations.startree.MetricAggregatorTests.testStarTreeDocValues" -Dtests.seed=9F7B34E51FC4BB95 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=en-DK -Dtests.timezone=America/Kentucky/Louisville -Druntime.java=21

MetricAggregatorTests > testStarTreeDocValues FAILED
    java.lang.AssertionError: expected:<16.0> but was:<265.0>
        at __randomizedtesting.SeedInfo.seed([9F7B34E51FC4BB95:29A33C629A7BAF56]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:555)
        at org.junit.Assert.assertEquals(Assert.java:685)
        at org.opensearch.search.aggregations.startree.MetricAggregatorTests.lambda$verifyAggregation$15(MetricAggregatorTests.java:430)
        at org.opensearch.search.aggregations.startree.MetricAggregatorTests.testCase(MetricAggregatorTests.java:492)
        at org.opensearch.search.aggregations.startree.MetricAggregatorTests.testCase(MetricAggregatorTests.java:446)
        at org.opensearch.search.aggregations.startree.MetricAggregatorTests.testStarTreeDocValuesInternal(MetricAggregatorTests.java:267)
        at org.opensearch.search.aggregations.startree.MetricAggregatorTests.testStarTreeDocValues(MetricAggregatorTests.java:163)

[andrross]

FYI @expani @sandeshkr419

[ajleong623]

I can look into this.

[ajleong623]

@andrross I ran ./gradlew ':server:test' --tests "org.opensearch.search.aggregations.startree.MetricAggregatorTests.testStarTreeDocValues" -Dtests.seed=9F7B34E51FC4BB95 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=en-DK -Dtests.timezone=America/Kentucky/Louisville -Druntime.java=21 in my OpenSearch repository, but I did not end up having an error.

Were you able to reproduce it on your end? I notice that the distribution my computer uses is different since the Linux distribution is used in the build where the test failed.

Is there any documentation on the possible "-Dtests" configurations I can use, or how I can run the tests on a linux distribution?

[andrross]

@ajleong623 Yeah I can reproduce it with that seed. Did you remove the AwaitsFix tag on the test that disables it?

OpenSearch/server/src/test/java/org/opensearch/search/aggregations/startree/MetricAggregatorTests.java

Line 140 in ae78944

@AwaitsFix(bugUrl = "https://github.com/opensearch-project/OpenSearch/issues/18110")

[ajleong623]

@andrross I see what happened. I was running it in an outdated repository originally before the issue showed up, but I just cloned a new repository, and I removed the tag then ran the test. It was reproduced. Thank you so much for the suggestion.

[ajleong623]

Just as a quick update, I am still working on this. At the moment, I noticed that the should boolean query is where the error is happening, so maybe that filter is not being applied in the star tree filter. I am diving into how the star tree index processes the documents and trying to see why more docs are being returned. I hope to have the pr out by the end of this week. This is one of the few issues I am working on currently.