Skip to content

Support isDistinctFrom and isNotDistinctFrom operators #3357

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

pengpeng-lu
Copy link
Contributor

No description provided.

}
} else if (comparand == null) {
return 1;
return toComparable(fieldValue).compareTo(toComparable(comparand));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the original line 237~244 is wrong? If any of the left or right is null, the comparison result should be unknown? Also in the original code, we call: if(value == null) return null before we call this compare method, so the block of if(fieldValue == null) was never reached.

@@ -707,28 +713,44 @@ public static Type invertComparisonType(@Nonnull final Comparisons.Type type) {
@Nullable
@SpotBugsSuppressWarnings("NP_BOOLEAN_RETURN_NULL")
public static Boolean evalComparison(@Nonnull Type type, @Nullable Object value, @Nullable Object comparand) {
if (value == null) {
return null;
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as I mentioned in another comment, because of this is called first, we'll never reach the if(fieldValue == null) block in method compare. I removed this block here, and added to cases below.

Copy link
Contributor Author

@pengpeng-lu pengpeng-lu May 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right now, if type = TEXT_CONTAINS_ALL, or anything that is not in the switch block, and value == null, this method returns null. Is this the expected behavior? I think we wanted to throw exception in this case?

@@ -184,6 +184,13 @@ private static Optional<QueryPredicate> promoteOperandsAndCreatePredicate(@Nulla
@Nonnull Value leftChild,
@Nonnull Value rightChild,
@Nonnull final Comparisons.Type comparisonType) {
if (leftChild.getResultType().getTypeCode() == Type.TypeCode.NULL && rightChild.getResultType().getTypeCode() == Type.TypeCode.NULL) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused of the difference between TypeCode.Unknown and TypeCode.NULL, should we use NULL or Unknown here?

IS_DISTINCT_FROM_SID(Comparisons.Type.IS_DISTINCT_FROM, Type.TypeCode.STRING, Type.TypeCode.UUID, (l, r) -> Comparisons.evalComparison(Comparisons.Type.IS_DISTINCT_FROM, PromoteValue.PhysicalOperator.stringToUuidValue((String) l), r)),
IS_DISTINCT_FROM_UID(Comparisons.Type.IS_DISTINCT_FROM, Type.TypeCode.UNKNOWN, Type.TypeCode.UUID, (l, r) -> true),
IS_DISTINCT_FROM_IDU(Comparisons.Type.IS_DISTINCT_FROM, Type.TypeCode.UUID, Type.TypeCode.UNKNOWN, (l, r) -> true),
IS_DISTINCT_FROM_NN(Comparisons.Type.IS_DISTINCT_FROM, Type.TypeCode.NULL, Type.TypeCode.NULL, (l, r) -> false),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, should we use NULL or Unknown here?

Copy link
Collaborator

@arnaud-lacurie arnaud-lacurie Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understood from conversations with @normen662 and @hatyo, UNKNOWN is used by the legacy planner, and NULL is used by the cascades planner to represent the literal NULL. You will want to add both to this map, as you've done here.
You will also want to add tests in BooleanValueTest.
Currently missing from this list is NULL on the right hand side of all the other types, and NULL on the left hand side.

@pengpeng-lu pengpeng-lu added the enhancement New feature or request label May 16, 2025
@@ -417,7 +417,7 @@ protected void testParameterComparison(String name, String field, Object val1, C
final Bindings bindings = Bindings.newBuilder()
.set("fooParam", val2)
.build();
if (val1 != null && val2 != null && (type == Comparisons.Type.IN && !(val2 instanceof List) || type.name().startsWith("TEXT_"))) {
if (val1 != null && val2 != null && (type == Comparisons.Type.IN && !(val2 instanceof List)) || type.name().startsWith("TEXT_")) {
Copy link
Contributor Author

@pengpeng-lu pengpeng-lu May 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this was a bug. Why it worked before: when type = TEXT_CONTAINS_ALL, it gets into assertEquals and eventually, Comparisons.evalComparison, and because value == null, it returns null.

@pengpeng-lu pengpeng-lu requested review from hatyo and removed request for arnaud-lacurie May 30, 2025 15:41
Comment on lines 130 to 131
IS_DISTINCT_FROM: 'IS DISTINCT FROM';
IS_NOT_DISTINCT_FROM: 'IS NOT DISTINCT FROM';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unusual, you should split the tokens to enable tokenizing them correctly disregarding whitespaces.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, modified.

@@ -881,6 +881,209 @@ void aliasingTableToResolveAmbiguityWorks() throws Exception {
}
}

@Test
Copy link
Contributor

@hatyo hatyo May 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please move this to YAML instead? newly added SQL features should be added to YAML be default unless there is a good reason for writing the tests in Java/Junit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

Copy link
Contributor

@hatyo hatyo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels like there should be some changes added to ExpressionVisitor.java that I do not see in this PR, but I am not sure.

@pengpeng-lu
Copy link
Contributor Author

It feels like there should be some changes added to ExpressionVisitor.java that I do not see in this PR, but I am not sure.

is there any specific queries that you are concerned about? I think this is a feature on the "operator" level, not "expression" level.

@pengpeng-lu pengpeng-lu requested a review from hatyo June 3, 2025 00:10
@arnaud-lacurie arnaud-lacurie requested a review from MMcM June 3, 2025 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants