Skip to content

feat: enable block stream write #18285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
+2,943 −656
Open

feat: enable block stream write #18285

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
ecdc91f
improve stream write
zhyass Jun 30, 2025
6cc6e4b
fix virtual column builder
zhyass Jul 4, 2025
a7d9b3c
add block statistics
zhyass Jul 13, 2025
46dda49
remove null in bloom index builder
zhyass Jul 17, 2025
c05ddff
use metahll
zhyass Jul 18, 2025
87ee941
avoid large string
zhyass Jul 18, 2025
6dd1065
fix
zhyass Jul 19, 2025
dafe936
use segment level stats
zhyass Jul 29, 2025
b39280a
add test
zhyass Jul 31, 2025
0f579dc
remove unused code
zhyass Aug 1, 2025
e94aafb
Merge branch 'main' into feat_stream
BohuTANG Aug 2, 2025
1cb3b87
fix review comment
zhyass Aug 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions src/common/storage/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,11 @@ edition = { workspace = true }
storage-hdfs = ["databend-common-meta-app/storage-hdfs"]

[dependencies]
ahash = { workspace = true }
anyhow = { workspace = true }
arrow-schema = { workspace = true }
async-backtrace = { workspace = true }
borsh = { workspace = true }
chrono = { workspace = true }
dashmap = { workspace = true, features = ["serde"] }
databend-common-ast = { workspace = true }
Expand Down
2 changes: 2 additions & 0 deletions src/common/storage/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ pub use stage::STDIN_FD;
mod copy;
mod histogram;
mod merge;
mod meta_hll;
mod metrics_layer;
mod multi_table_insert;
mod statistics;
Expand All @@ -83,6 +84,7 @@ pub use histogram::Histogram;
pub use histogram::HistogramBucket;
pub use histogram::DEFAULT_HISTOGRAM_BUCKETS;
pub use merge::MutationStatus;
pub use meta_hll::MetaHLL;
pub use multi_table_insert::MultiTableInsertStatus;
pub use statistics::Datum;
pub use statistics::F64;
Loading
Loading