Skip to content

Reuse Rows allocation in SortPreservingMergeStream / RowCursorStream #15720

Closed
@Dandandan

Description

@Dandandan

Is your feature request related to a problem or challenge?

While reviewing our Sort code, I found Rows is being allocated within RowCursorStream for each batch (via RowConverter::convert_columns)
This involves a number allocations (exponentially allocating based on Vec) for each batch.
We should avoid this and reuse Rows for the stream.

Describe the solution you'd like

Find a way to reuse Rows for the entire stream.

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestperformanceMake DataFusion faster

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions