Skip to content

Raise an appropriate warning when a column has exclusively None values and is essentially dropped #2632

Closed
@anuunchin

Description

@anuunchin

Since dlt is data-driven, it cannot infer the data type for a column if all values are None, unless the user explicitly provides a type hint. This silent dropping has caused confusion for users before.

Because it is impossible to infer the data type from None, we now raise an explicit warning that lists the columns which could not be typed and were therefore silently dropped.

The warning should come from the normalizers.

Metadata

Metadata

Assignees

Labels

QoLQuality of Life

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions