Feat/iceberg advanced partitioning

[rakesh-tmdc]

Hey Team,

I’ve been using dlt for the past 3–4 months, mostly with Apache Iceberg as the destination. Recently, I needed support for Iceberg partitioning, especially for more advanced use cases like time and bucket partitions.

I’ve implemented support for these in a way that’s fully compatible with existing column-level partition configurations:
Still works with earlier formats like:

{ "region": { "partition": true }, "category": { "partition": true } }

Now also supports advanced options like:
{ "date_added": { "partition": { "type": "year", "index": 1, "name": "yearly_partition" } }, "user_id": { "partition": { "type": "bucket", "index": 2, "bucket_count": 32, "name": "user_bucket" } }, "region": { "partition": { "type": "identity", "index": 3 } } }

Would love feedback from the team!

[netlify]

✅ Deploy Preview for dlt-hub-docs canceled.

Name	Link
🔨 Latest commit	5916083
🔍 Latest deploy log	https://app.netlify.com/projects/dlt-hub-docs/deploys/68b70681267a61000844a333

[burnash]

Hi @rakesh-tmdc, thanks for the contribution, this looks good and useful. In dlt+ we already have an iceberg_adapter() with iceberg_partition helpers for these transforms. We're open to moving this adapter module to open source dlt so your PR can reuse it and stay fully compatible with our existing semantics/docs.

If you're up for it, we can extract the adapter and have your changes delegate partition spec parsing/validation to it to keep behavior consistent across catalogs.

[rakesh-tmdc]

Thanks @burnash , glad to hear this is useful! Extracting the iceberg_adapter and its partition helpers into open source sounds like a great idea — I’d definitely prefer to reuse that instead of duplicating logic.

Once it’s available, I can rework my PR so that partition spec parsing/validation delegates to the adapter, which should keep things consistent. Just let me know when/where the adapter lands, and I’ll update accordingly.