Skip to content

Parquet metadata size accounting overcounts Arc<T> #8897

@alamb

Description

@alamb

Describe the bug

While testing the upgrade to arrow 57.1.0 in DataFusion

I noticed that the reported size of the memory used by metadata grew by almost 33%

Image

I know that @adamreeve fixed the heap accounting for encryption

But I don't think that would increase usage by 33%

I went back and reviewed the code and I found at least one issue where the heap size is double counted for Arc<T> I think

Expected behavior
Heap usage should be accurate

Additional context

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions