Row with NULLs during replication from DuckDB local file to Postgres

[nicokant]

Sling version:

1.5.14

What is the Operating System?

Linux

Do you have a CLI Pro/Platform subscription?

Yes

Description of the issue

Incremental replication fails because of NULL constraints on the target table.

The strange thing is that the source table contains only valid data (the field created is populated using now() for every row.

I've tried to debug adding the SLING_KEEP_TEMP and the temporary table has the first row containing just NULL for every column.

It seems that there is something strange while moving the data from duckdb to postgresql that make appear the NULLs row

Replication Configuration

source: DUCKDB
target: SEAPOP_DATABASE

streams:
  sjofugl_person:
    mode: incremental
    object: public.seapop_person
    primary_key: id
#    target_options:
#      table_keys:
#        primary: [id]

env:
  SLING_KEEP_TEMP: true
  SLING_SHOW_PROGRESS: true

Log Output

2026-04-13 15:37:05 INF Sling CLI | https://slingdata.io
2026-04-13 15:37:05 DBG Sling version: 1.5.14 (linux amd64)
2026-04-13 15:37:05 DBG CLI Pro token validated
2026-04-13 15:37:05 INF Starting Seapop sync pipeline
2026-04-13 15:37:05 INF Creating uuid columns in sjofugl tables for later replication to Seapop Portal
2026-04-13 15:37:05 DBG executing step "step-03-replication" (type: replication)
2026-04-13 15:37:05 INF Sling Replication | DUCKDB -> SEAPOP_DATABASE | sjofugl_person
2026-04-13 15:37:05 DBG Sling version: 1.5.14 (linux amd64)
2026-04-13 15:37:05 DBG type is db-db
2026-04-13 15:37:05 DBG using: {"columns":null,"mode":"incremental","select":null,"transforms":null}
2026-04-13 15:37:05 DBG using source options: {"empty_as_null":false,"datetime_format":"AUTO","max_decimals":-1}
2026-04-13 15:37:05 DBG using target options: {"datetime_format":"auto","file_max_rows":0,"max_decimals":-1,"use_bulk":true,"add_new_columns":true,"adjust_column_type":false,"column_casing":"normalize","table_tmp":"\"public\".\"seapop_person_tmp\""}
2026-04-13 15:37:05 INF connecting to source database (duckdb)
2026-04-13 15:37:06 DBG opened "duckdb" connection (conn-duckdb-duckdb-N7K)
2026-04-13 15:37:06 INF connecting to target database (postgres)
2026-04-13 15:37:06 DBG opened "postgres" connection (conn-postgres-seapop_database-0wA)
2026-04-13 15:37:06 INF reading from source database
2026-04-13 15:37:06 DBG applying column casing (normalize) for target type (postgres)
2026-04-13 15:37:06 DBG applying column casing (normalize) for target type (postgres)
2026-04-13 15:37:06 DBG column type change for precision (string to integer)
2026-04-13 15:37:06 DBG column type change for scale (string to integer)
2026-04-13 15:37:06 DBG applying column casing (normalize) for target type (postgres)
2026-04-13 15:37:06 DBG truncated column names (exceeds max length of 63 for "postgres")
2026-04-13 15:37:06 DBG    _90m________________________________________________________________00m__90m => _90m___________________________________________________________
2026-04-13 15:37:06 DBG select * from "main"."sjofugl_person"
2026-04-13 15:37:06 DBG applying column casing (normalize) for target type (postgres)
2026-04-13 15:37:06 INF writing to target database [mode: incremental]
2026-04-13 15:37:06 DBG drop table if exists "public"."seapop_person_tmp" [seapop_database-0wA]
2026-04-13 15:37:06 DBG table "public"."seapop_person_tmp" dropped
2026-04-13 15:37:06 DBG create table if not exists "public"."seapop_person_tmp" ("created" varchar(65500),
  "modified" varchar(65500),
  "sjofugldb_id" varchar(65500),
  "first_name" varchar(65500),
  "last_name" varchar(65500),
  "belongs_to_id" varchar(65500),
  "email" varchar(65500),
  "phone" varchar(65500),
  "ringer_license" varchar(65500),
  "ringer_group" varchar(65500),
  "user_id" varchar(65500),
  "id" varchar(65500)) [seapop_database-0wA]
2026-04-13 15:37:06 INF created table "public"."seapop_person_tmp"
2026-04-13 15:37:06 INF streaming data
2026-04-13 15:37:06 DBG column type change for phone (string to integer)
2026-04-13 15:37:06 DBG select count(*) cnt from "public"."seapop_person_tmp" [seapop_database-0wA]
2026-04-13 15:37:06 DBG merging from temporary table "public"."seapop_person_tmp" to final table "public"."seapop_person" with primary keys [id], strategy: update_insert
2026-04-13 15:37:06 DBG inserting created [character varying(65500)] into created [timestamp with time zone]
2026-04-13 15:37:06 DBG inserting modified [character varying(65500)] into modified [timestamp with time zone]
2026-04-13 15:37:06 DBG inserting sjofugldb_id [character varying(65500)] into sjofugldb_id [character varying]
2026-04-13 15:37:06 DBG inserting first_name [character varying(65500)] into first_name [character varying]
2026-04-13 15:37:06 DBG inserting last_name [character varying(65500)] into last_name [character varying]
2026-04-13 15:37:06 DBG inserting belongs_to_id [character varying(65500)] into belongs_to_id [uuid]
2026-04-13 15:37:06 DBG inserting email [character varying(65500)] into email [character varying(254)]
2026-04-13 15:37:06 DBG inserting phone [character varying(65500)] into phone [character varying]
2026-04-13 15:37:06 DBG inserting ringer_license [character varying(65500)] into ringer_license [character varying]
2026-04-13 15:37:06 DBG inserting ringer_group [character varying(65500)] into ringer_group [character varying]
2026-04-13 15:37:06 DBG inserting user_id [character varying(65500)] into user_id [bigint]
2026-04-13 15:37:06 DBG inserting id [character varying(65500)] into id [uuid]
2026-04-13 15:37:06 DBG create temporary table temp6mngf as
with src_table as (
  select "created"::timestamp with time zone as "created", "modified"::timestamp with time zone as "modified", "sjofugldb_id" as "sjofugldb_id", "first_name" as "first_name", "last_name" as "last_name", "belongs_to_id"::uuid as "belongs_to_id", "email" as "email", "phone" as "phone", "ringer_license" as "ringer_license", "ringer_group" as "ringer_group", "user_id"::bigint as "user_id", "id"::uuid as "id" from "public"."seapop_person_tmp"
)
, updates as (
  update "public"."seapop_person" tgt
  set "created" = src."created"::timestamp with time zone, "modified" = src."modified"::timestamp with time zone, "sjofugldb_id" = src."sjofugldb_id", "first_name" = src."first_name", "last_name" = src."last_name", "belongs_to_id" = src."belongs_to_id"::uuid, "email" = src."email", "phone" = src."phone", "ringer_license" = src."ringer_license", "ringer_group" = src."ringer_group", "user_id" = src."user_id"::bigint
  from src_table src
  where src."id"::uuid = tgt."id"
  returning tgt.*
)
select * from updates [seapop_database-0wA]
2026-04-13 15:37:06 DBG create unique index if not exists temp6mngf_idx on temp6mngf ("id") [seapop_database-0wA]
2026-04-13 15:37:06 DBG with src_table as (
  select "created"::timestamp with time zone as "created", "modified"::timestamp with time zone as "modified", "sjofugldb_id" as "sjofugldb_id", "first_name" as "first_name", "last_name" as "last_name", "belongs_to_id"::uuid as "belongs_to_id", "email" as "email", "phone" as "phone", "ringer_license" as "ringer_license", "ringer_group" as "ringer_group", "user_id"::bigint as "user_id", "id"::uuid as "id" from "public"."seapop_person_tmp"
)
insert into "public"."seapop_person"
("created", "modified", "sjofugldb_id", "first_name", "last_name", "belongs_to_id", "email", "phone", "ringer_license", "ringer_group", "user_id", "id")
select "created"::timestamp with time zone as "created", "modified"::timestamp with time zone as "modified", "sjofugldb_id" as "sjofugldb_id", "first_name" as "first_name", "last_name" as "last_name", "belongs_to_id"::uuid as "belongs_to_id", "email" as "email", "phone" as "phone", "ringer_license" as "ringer_license", "ringer_group" as "ringer_group", "user_id"::bigint as "user_id", "id"::uuid as "id" from src_table src
where not exists (
  select 1
  from temp6mngf upd
  where src."id"::uuid = upd."id"
) [seapop_database-0wA]
2026-04-13 15:37:06 DBG closed "duckdb" connection (conn-duckdb-duckdb-N7K)
2026-04-13 15:37:06 DBG closed "postgres" connection (conn-postgres-seapop_database-0wA)
2026-04-13 15:37:06 INF execution failed

2026-04-13 15:37:06 DBG failure while executing step "step-03-replication" (type: replication) => 
--------------------------- sjofugl_person ---------------------------
--- task_run.go:125 func2 ---
~ Could not WriteToDb
--- task_run.go:866 runDbToDb ---
~ error merging data from temp to final table
--- task_run_write.go:450 WriteToDb ---
~ could not merge from temp into final
--- task_run_write.go:989 mergeData ---
~ could not merge from temp into final
--- task_run_write.go:1145 performMerge ---
~ could not merge
--- database.go:2914 MergeWithStrategy ---
~ Could not merge
--- transaction.go:526 MergeWithStrategy ---

--- task_run.go:125 func2 ---
--- task_run.go:866 runDbToDb ---
--- task_run_write.go:450 WriteToDb ---
--- task_run_write.go:989 mergeData ---
--- task_run_write.go:1145 performMerge ---
--- database.go:2914 MergeWithStrategy ---
--- transaction.go:521 MergeWithStrategy ---
~ Error executing query
--- transaction.go:153 ExecMultiContext ---
~ Error executing: with src_table as (
  select "created"::timestamp with time zone as "created", "modified"::timestamp with time zone as "modified", "sjofugldb_id" as "sjofugldb_id", "first_name" as "first_name", "last_name" as "last_name", "belongs_to_id"::uuid as "belongs_to_id", "email" as "email", "phone" as "phone", "ringer_license" as "ringer_license", "ringer_group" as "ringer_group", "user_id"::bigint as "user_id", "id"::uuid as "id" from "public"."seapop_person_tmp"
)
insert into "public"."seapop_person"
("created", "modified", "sjofugldb_id", "first_name", "last_name", "belongs_to_id", "email", "phone", "ringer_license", "ringer_group", "user_id", "id")
select "created"::timestamp with time zone as "created", "modified"::timestamp with time zone as "modified", "sjofugldb_id" as "sjofugldb_id", "first_name" as "first_name", "last_name" as "last_name", "belongs_to_id"::uuid as "belongs_to_id", "email" as "email", "phone" as "phone", "ringer_license" as "ringer_license", "ringer_group" as "ringer_group", "user_id"::bigint as "user_id", "id"::uuid as "id" from src_table src
where not exists (
  select 1
  from temp6mngf upd
  where src."id"::uuid = upd."id"
)
--- transaction.go:139 ExecContext ---
pq: null value in column "created" of relation "seapop_person" violates not-null constraint

fatal:
--- proc.go:285 main ---
--- sling_cli.go:532 main ---
--- sling_cli.go:573 cliInit ---
--- cli.go:287 CliProcess ---
--- sling_run.go:287 processRun ---
--- sling_run.go:709 runPipeline ---
~ failure running pipeline (see docs @ https://docs.slingdata.io)
--- pipeline.go:232 Execute ---
~ error executing step: step-03-replication
--- pipeline.go:349 Execute ---
~ error running replication
--- hooks..go:2224 Execute ---

--------------------------- sjofugl_person ---------------------------
--- task_run.go:125 func2 ---
~ Could not WriteToDb
--- task_run.go:866 runDbToDb ---
~ error merging data from temp to final table
--- task_run_write.go:450 WriteToDb ---
~ could not merge from temp into final
--- task_run_write.go:989 mergeData ---
~ could not merge from temp into final
--- task_run_write.go:1145 performMerge ---
~ could not merge
--- database.go:2914 MergeWithStrategy ---
~ Could not merge
--- transaction.go:526 MergeWithStrategy ---
--- task_run.go:125 func2 ---
--- task_run.go:866 runDbToDb ---
--- task_run_write.go:450 WriteToDb ---
--- task_run_write.go:989 mergeData ---
--- task_run_write.go:1145 performMerge ---
--- database.go:2914 MergeWithStrategy ---
--- transaction.go:521 MergeWithStrategy ---
~ Error executing query
--- transaction.go:153 ExecMultiContext ---
~ Error executing: with src_table as (
  select "created"::timestamp with time zone as "created", "modified"::timestamp with time zone as "modified", "sjofugldb_id" as "sjofugldb_id", "first_name" as "first_name", "last_name" as "last_name", "belongs_to_id"::uuid as "belongs_to_id", "email" as "email", "phone" as "phone", "ringer_license" as "ringer_license", "ringer_group" as "ringer_group", "user_id"::bigint as "user_id", "id"::uuid as "id" from "public"."seapop_person_tmp"
)
insert into "public"."seapop_person"
("created", "modified", "sjofugldb_id", "first_name", "last_name", "belongs_to_id", "email", "phone", "ringer_license", "ringer_group", "user_id", "id")
select "created"::timestamp with time zone as "created", "modified"::timestamp with time zone as "modified", "sjofugldb_id" as "sjofugldb_id", "first_name" as "first_name", "last_name" as "last_name", "belongs_to_id"::uuid as "belongs_to_id", "email" as "email", "phone" as "phone", "ringer_license" as "ringer_license", "ringer_group" as "ringer_group", "user_id"::bigint as "user_id", "id"::uuid as "id" from src_table src
where not exists (
  select 1
  from temp6mngf upd
  where src."id"::uuid = upd."id"
)
--- transaction.go:139 ExecContext ---
pq: null value in column "created" of relation "seapop_person" violates not-null constraint

Adding target_options__table_keys__primary

2026-04-13 15:38:28 INF Sling CLI | https://slingdata.io
2026-04-13 15:38:28 DBG Sling version: 1.5.14 (linux amd64)
2026-04-13 15:38:28 DBG CLI Pro token validated
2026-04-13 15:38:28 INF Starting Seapop sync pipeline
2026-04-13 15:38:28 INF Creating uuid columns in sjofugl tables for later replication to Seapop Portal
2026-04-13 15:38:28 DBG executing step "step-03-replication" (type: replication)
2026-04-13 15:38:28 INF Sling Replication | DUCKDB -> SEAPOP_DATABASE | sjofugl_person
2026-04-13 15:38:28 DBG Sling version: 1.5.14 (linux amd64)
2026-04-13 15:38:28 DBG type is db-db
2026-04-13 15:38:28 DBG using: {"columns":null,"mode":"incremental","select":null,"transforms":null}
2026-04-13 15:38:28 DBG using source options: {"empty_as_null":false,"datetime_format":"AUTO","max_decimals":-1}
2026-04-13 15:38:28 DBG using target options: {"datetime_format":"auto","file_max_rows":0,"max_decimals":-1,"use_bulk":true,"add_new_columns":true,"adjust_column_type":false,"column_casing":"normalize","table_keys":{"primary":["id"]},"table_tmp":"\"public\".\"seapop_person_tmp\""}
2026-04-13 15:38:28 INF connecting to source database (duckdb)
2026-04-13 15:38:28 DBG opened "duckdb" connection (conn-duckdb-duckdb-Iy9)
2026-04-13 15:38:28 INF connecting to target database (postgres)
2026-04-13 15:38:28 DBG opened "postgres" connection (conn-postgres-seapop_database-wjH)
2026-04-13 15:38:28 INF reading from source database
2026-04-13 15:38:28 DBG applying column casing (normalize) for target type (postgres)
2026-04-13 15:38:28 DBG applying column casing (normalize) for target type (postgres)
2026-04-13 15:38:28 DBG column type change for precision (string to integer)
2026-04-13 15:38:28 DBG column type change for scale (string to integer)
2026-04-13 15:38:29 DBG applying column casing (normalize) for target type (postgres)
2026-04-13 15:38:29 DBG truncated column names (exceeds max length of 63 for "postgres")
2026-04-13 15:38:29 DBG    _90m________________________________________________________________00m__90m => _90m___________________________________________________________
2026-04-13 15:38:29 DBG select * from "main"."sjofugl_person"
2026-04-13 15:38:29 DBG applying column casing (normalize) for target type (postgres)
2026-04-13 15:38:29 INF writing to target database [mode: incremental]
2026-04-13 15:38:29 DBG drop table if exists "public"."seapop_person_tmp" [seapop_database-wjH]
2026-04-13 15:38:29 DBG table "public"."seapop_person_tmp" dropped
2026-04-13 15:38:29 DBG create table if not exists "public"."seapop_person_tmp" ("created" varchar(65500),
  "modified" varchar(65500),
  "sjofugldb_id" varchar(65500),
  "first_name" varchar(65500),
  "last_name" varchar(65500),
  "belongs_to_id" varchar(65500),
  "email" varchar(65500),
  "phone" varchar(65500),
  "ringer_license" varchar(65500),
  "ringer_group" varchar(65500),
  "user_id" varchar(65500),
  "id" varchar(65500), primary key ("id")) [seapop_database-wjH]
2026-04-13 15:38:29 INF created table "public"."seapop_person_tmp"
2026-04-13 15:38:29 INF streaming data
2026-04-13 15:38:29 DBG column type change for phone (string to integer)
2026-04-13 15:38:29 WRN &pq.Error{Severity:"ERROR", Code:"23502", Message:"null value in column \"id\" of relation \"seapop_person_tmp\" violates not-null constraint", Detail:"Failing row contains (null, null, null, null, null, null, null, null, null, null, null, null).", Hint:"", Position:"", InternalPosition:"", InternalQuery:"", Where:"COPY seapop_person_tmp, line 1: \"\\N\t\\N\t\\N\t\\N\t\\N\t\\N\t\\N\t\\N\t\\N\t\\N\t\\N\t\\N\"", Schema:"public", Table:"seapop_person_tmp", Column:"id", DataTypeName:"", Constraint:"", File:"execMain.c", Line:"2003", Routine:"ExecConstraints"}
2026-04-13 15:38:29 DBG closed "duckdb" connection (conn-duckdb-duckdb-Iy9)
2026-04-13 15:38:29 DBG closed "postgres" connection (conn-postgres-seapop_database-wjH)
2026-04-13 15:38:29 INF execution failed

2026-04-13 15:38:29 DBG failure while executing step "step-03-replication" (type: replication) => 
--------------------------- sjofugl_person ---------------------------
--- proc.go:285 main ---
--- sling_cli.go:532 main ---
--- sling_cli.go:573 cliInit ---
--- cli.go:287 CliProcess ---
--- sling_run.go:287 processRun ---
--- sling_run.go:709 runPipeline ---
--- pipeline.go:232 Execute ---
--- pipeline.go:349 Execute ---
--- hooks..go:2197 Execute ---
--- sling_run..go:88 RunReplication ---
--- sling_run.go:627 replicationRun ---
--- sling_run.go:481 runTask ---
--- task_run.go:189 Execute ---


--- task_run.go:125 func2 ---
--- task_run.go:866 runDbToDb ---
--- task_run_write.go:349 WriteToDb ---
--- database.go:2744 BulkImportFlow ---
~ could not bulk import
--- database.go:2731 func1 ---
~ could not copy data
--- database_postgres.go:475 BulkImportStream ---
~ could not close statement
--- database_postgres.go:466 func2 ---
pq: null value in column "id" of relation "seapop_person_tmp" violates not-null constraint

context canceled

--- task_run.go:125 func2 ---
~ Could not WriteToDb
--- task_run.go:866 runDbToDb ---
~ could not insert into "public"."seapop_person_tmp"
--- task_run_write.go:355 WriteToDb ---

--- task_run.go:125 func2 ---
--- task_run.go:866 runDbToDb ---
--- task_run_write.go:349 WriteToDb ---
--- database.go:2744 BulkImportFlow ---
~ could not bulk import
--- database.go:2731 func1 ---
~ could not copy data
--- database_postgres.go:475 BulkImportStream ---
~ could not close statement
--- database_postgres.go:466 func2 ---
pq: null value in column "id" of relation "seapop_person_tmp" violates not-null constraint

context canceled

fatal:
--- proc.go:285 main ---
--- sling_cli.go:532 main ---
--- sling_cli.go:573 cliInit ---
--- cli.go:287 CliProcess ---
--- sling_run.go:287 processRun ---
--- sling_run.go:709 runPipeline ---
~ failure running pipeline (see docs @ https://docs.slingdata.io)
--- pipeline.go:232 Execute ---
~ error executing step: step-03-replication
--- pipeline.go:349 Execute ---
~ error running replication
--- hooks..go:2224 Execute ---

--------------------------- sjofugl_person ---------------------------
--- proc.go:285 main ---
--- sling_cli.go:532 main ---
--- sling_cli.go:573 cliInit ---
--- cli.go:287 CliProcess ---
--- sling_run.go:287 processRun ---
--- sling_run.go:709 runPipeline ---
--- pipeline.go:232 Execute ---
--- pipeline.go:349 Execute ---
--- hooks..go:2197 Execute ---
--- sling_run..go:88 RunReplication ---
--- sling_run.go:627 replicationRun ---
--- sling_run.go:481 runTask ---
--- task_run.go:189 Execute ---


--- task_run.go:125 func2 ---
--- task_run.go:866 runDbToDb ---
--- task_run_write.go:349 WriteToDb ---
--- database.go:2744 BulkImportFlow ---
~ could not bulk import
--- database.go:2731 func1 ---
~ could not copy data
--- database_postgres.go:475 BulkImportStream ---
~ could not close statement
--- database_postgres.go:466 func2 ---
pq: null value in column "id" of relation "seapop_person_tmp" violates not-null constraint

--- task_run.go:125 func2 ---
~ Could not WriteToDb
--- task_run.go:866 runDbToDb ---
~ could not insert into "public"."seapop_person_tmp"
--- task_run_write.go:355 WriteToDb ---
--- task_run.go:125 func2 ---
--- task_run.go:866 runDbToDb ---
--- task_run_write.go:349 WriteToDb ---
--- database.go:2744 BulkImportFlow ---
~ could not bulk import
--- database.go:2731 func1 ---
~ could not copy data
--- database_postgres.go:475 BulkImportStream ---
~ could not close statement
--- database_postgres.go:466 func2 ---
pq: null value in column "id" of relation "seapop_person_tmp" violates not-null constraint

[nicokant]

I actually noticed that it's not an additional row, but the first row has been transformed to NULLs, all the others are fine.

[flarco]

hey @nicokant I could not reproduce this error.

Looking at the log, it's strange to see _90m________________________________________________________________00m__90m

I wonder if duckdb is not formatting the output CSV correctly. Could you try with setting env var DUCKDB_USE_ARROW=true?

[nicokant]

Tried, but nothing have changed. Do you have any suggestions? or commands I can try to run to get more info/debug logs?

[flarco]

It's a strange one. Can you perhaps some me some data that you are able to reproduce with? so I can reproduce as well?
A small duckdb file would work. If you prefer sending via email, send to support@slingdata.io

[nicokant]

I'll try to reproduce with some example data as the one I'm using now are sensitive