Skip to content

feat(storage): add variant shredding for parquet#19422

Draft
dantengsky wants to merge 7 commits intodatabendlabs:mainfrom
dantengsky:feat/variant-shredding
Draft

feat(storage): add variant shredding for parquet#19422
dantengsky wants to merge 7 commits intodatabendlabs:mainfrom
dantengsky:feat/variant-shredding

Conversation

@dantengsky
Copy link
Member

@dantengsky dantengsky commented Feb 7, 2026

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

  • Enable Parquet VARIANT encoding + inline shredding for Variant columns in Fuse
  • Add read-path unshred, stats gating, and EXPLAIN inline indication with JSONB compatibility
  • Add unit + sqllogic coverage for typed_value decoding and inline shredding

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Feb 7, 2026
@dantengsky dantengsky added the ci-cloud Build docker image for cloud test label Feb 7, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Feb 7, 2026

🤖 CI Job Analysis

Workflow: 21800558292

📊 Summary

  • Total Jobs: 85
  • Failed Jobs: 2
  • Retryable: 0
  • Code Issues: 2

NO RETRY NEEDED

All failures appear to be code/test issues requiring manual fixes.

🔍 Job Details

  • linux / test_private_tasks: Not retryable (Code/Test)
  • linux / sqllogic / ee (parquet): Not retryable (Code/Test)

🤖 About

Automated analysis using job annotations to distinguish infrastructure issues (auto-retried) from code/test issues (manual fixes needed).

@dantengsky dantengsky force-pushed the feat/variant-shredding branch 4 times, most recently from c02ceda to 62f0cdd Compare February 8, 2026 08:30
fix(storage): stabilize variant shredding read path
@dantengsky dantengsky force-pushed the feat/variant-shredding branch from 62f0cdd to 6b94ea6 Compare February 8, 2026 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cloud Build docker image for cloud test pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant