Skip to content

Commit c699dc3

Browse files
committed
Docs adjustments and athena iceberg test fix [ci skip]
1 parent a51c20b commit c699dc3

File tree

2 files changed

+5
-2
lines changed

2 files changed

+5
-2
lines changed

docs/website/docs/dlt-ecosystem/destinations/iceberg.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ The [S3-compatible](./filesystem.md#using-s3-compatible-storage) interface for G
120120
The `az` [scheme](./filesystem.md#supported-schemes) is not supported when using the `iceberg` table format. Please use the `abfss` scheme. This is because `pyiceberg`, which dlt used under the hood, currently does not support `az`.
121121

122122
## Table format `merge` support (**experimental**)
123-
The [`upsert`](../../general-usage/merge-loading.md#upsert-strategy) merge strategy is supported for `iceberg`.
123+
The [`upsert`](../../general-usage/merge-loading.md#upsert-strategy) merge strategy is supported for `iceberg`. This strategy requires that the input data contains no duplicate rows based on the key columns, and that the target table also does not contain duplicates on those keys.
124124

125125
:::caution
126126
The `upsert` merge strategy for the filesystem destination with Iceberg table format is **experimental**.

tests/load/pipeline/test_merge_disposition.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -440,7 +440,10 @@ def r(data):
440440
table_data = load_tables_to_dicts(p, "parent", "parent__child", exclude_system_cols=True)
441441
if merge_strategy == "upsert":
442442
# merge keys will not apply and parent will not be deleted
443-
if destination_config.table_format in ["delta", "iceberg"]:
443+
if (
444+
destination_config.table_format in ["delta", "iceberg"]
445+
and destination_config.destination_type != "athena"
446+
):
444447
# delta merges cannot delete from nested tables
445448
assert table_counts == {
446449
"parent": 3, # id == 3 not deleted (not present in the data)

0 commit comments

Comments
 (0)