Streaming aggregate cursor now requires 4.1.9.0 or later for continuation deserialization #3246
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…tion deserialization This cleans up the `StreamingAggregateCursor` and its helper classes to remove the `createDefaultIfEmpty` option. We'd been carrying that option around as we introduced that field in FoundationDB#3092 and wanted to preserve behavior when upgrading from older versions. The intention had been that all new plans would set the option starting in 4.1, but a bug (fixed in FoundationDB#3211; see FoundationDB#3096) means that we didn't actually enable it until 4.1.9.0. That means that after this change, we'd require 4.1.9.0 or newer in order to safely continue these queries. In theory, we could wait with this change, but 4.1.9.0 has enough fixes that I actually think our recommendation should be that anyone upgrading from 4.0 or below go straight to 4.1.9.0, and then they can proceed safely to a newer version with this change. I was able to validate that this change is compatible with 4.1.9.0 via the cross-version tests run during PRB. When I ran the full mixed mode tests with the `aggregate-index-tests.yamsql`, I found that only 4.1.9.0 worked, which is expected. This resolves FoundationDB#3092.
pengpeng-lu
approved these changes
Apr 15, 2025
normen662
reviewed
Apr 16, 2025
| @@ -371,7 +361,7 @@ public PRecordQueryStreamingAggregationPlan toProto(@Nonnull final PlanSerializa | |||
| builder.setGroupingKeyAlias(groupingKeyAlias.getId()) | |||
| .setAggregateAlias(aggregateAlias.getId()) | |||
| .setCompleteResultValue(completeResultValue.toValueProto(serializationContext)) | |||
| .setIsCreateDefaultOnEmpty(isCreateDefaultOnEmpty); | |||
| .setIsCreateDefaultOnEmpty(false); | |||
There was a problem hiding this comment.
Can we mark this as deprecated in the proto file?
There was a problem hiding this comment.
In theory... We almost want something else, because unlike traditional deprecation where we would want to stop parsing this field entirely, here, we want to enforce that it is always set to false (never true and never unset). In general, I don't think there is a way to properly deprecate these fields unless we (1) add in something like a "reject if code version of generating instance is less than x" check to allow us to reject older versions or (2) are okay with behavior silently changing between incompatible versions instead of (as here) the old version being rejected.
This was referenced Apr 24, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This cleans up the
StreamingAggregateCursorand its helper classes to remove thecreateDefaultIfEmptyoption. We'd been carrying that option around as we introduced that field in #3092 and wanted to preserve behavior when upgrading from older versions. The intention had been that all new plans would set the option starting in 4.1, but a bug (fixed in #3211; see #3096) means that we didn't actually enable it until 4.1.9.0. That means that after this change, we'd require 4.1.9.0 or newer in order to safely continue these queries. In theory, we could wait with this change, but 4.1.9.0 has enough fixes that I actually think our recommendation should be that anyone upgrading from 4.0 or below go straight to 4.1.9.0, and then they can proceed safely to a newer version with this change.I was able to validate that this change is compatible with 4.1.9.0 via the cross-version tests run during PRB. When I ran the full mixed mode tests with the
aggregate-index-tests.yamsql, I found that only 4.1.9.0 worked, which is expected.This resolves #3107.