client.insert fails for column of type Variant(Tuple)

[ggman12]

Describe the bug

Steps to reproduce

Expected behaviour

Code example

import os
from datetime import datetime

client = clickhouse_connect.get_client(
    host=os.environ['CLICKHOUSE_HOST'],
    user='default',
    password=os.environ['CLICKHOUSE_PASSWORD'],
    secure=True,
)

query = """
CREATE TABLE test_variant_table
(
    icao String,
    time DateTime64(3),
    altitude Variant(Int32, String),
    aircraft 
    Variant(Tuple(
        a Int32,
        b Int32,
    )),
) ENGINE = ReplacingMergeTree()
PARTITION BY toYYYYMM(time)
ORDER BY (icao, time)
SETTINGS 
index_granularity = 8192;
"""
client.command("SET enable_variant_type = 1;")
client.command(query)

column_names = ['icao','time','altitude','aircraft']
row = (
    "ABCDEFG",
    datetime(2025, 6, 29, 16, 0, 0, 123000),
    30,
    (310,320)
)
client.insert(
    table='test_variant_table',
    data=[row],
    column_names=column_names,
    
)

clickhouse-connect and/or ClickHouse server logs

---------------------------------------------------------------------------
DatabaseError                             Traceback (most recent call last)
Cell In[16], line 39
     32 column_names = ['icao','time','altitude','aircraft']
     33 row = (
     34     "ABCDEFG",
     35     datetime(2025, 6, 29, 16, 0, 0, 123000),
     36     30,
     37     (310,320)
     38 )
---> 39 client.insert(
     40     table='test_variant_table',
     41     data=[row],
     42     column_names=column_names,
     43     
     44 )

File ~(redacted)/lib/python3.12/site-packages/clickhouse_connect/driver/client.py:676, in Client.insert(self, table, data, column_names, database, column_types, column_type_names, column_oriented, settings, context, transport_settings)
    674         raise ProgrammingError('Attempting to insert new data with non-empty insert context') from None
    675     context.data = data
--> 676 return self.data_insert(context)

File ~(redacted)/lib/python3.12/site-packages/clickhouse_connect/driver/httpclient.py:284, in HttpClient.data_insert(self, context)
    282 params.update(self._validate_settings(context.settings))
    283 headers = dict_copy(headers, context.transport_settings)
...
--> 402 raise OperationalError(err_str) if retried else DatabaseError(err_str) from None

DatabaseError: HTTPDriver for (redacted) received ClickHouse error code 70
 Code: 70. DB::Exception: Cannot convert type String to Variant(Tuple(a Int32, b Int32)). Conversion to Variant allowed only for types from this Variant: while converting column "aircraft" from type String to type Variant(Tuple(a Int32, b Int32)): While executing Native. (CANNOT_CONVERT_TYPE) (version 25.4.1.37390 (official build))

Configuration

Environment

• clickhouse-connect version: 0.8.18
• Python version: 3.12.3
• Operating system: Ubuntu 24.04

ClickHouse server

• ClickHouse Server version: 25.4.1.37390
• ClickHouse Server non-default settings, if any:
• CREATE TABLE statements for tables involved:
• Sample data for these tables, use clickhouse-obfuscator if necessary

[joe-clickhouse]

Thanks for opening this. I'll look into it a bit more, but at first glance, it seems like we need to improve robustness of write_column_data. Currently, we're blindly converting all the values to their string representations which ClickHouse is fine with for simple types like ints and strs but it appears that it'll break down for more complex types (like Tuple). E.g. the Python tuple (310, 320) becomes '(310, 320)' which ClickHouse can't parse back in the Tuple type.

[genzgd]

Please include your ClickHouse version, as string to Variant conversion may have changed/improved in later releases.

We may also need to enable the ClickHouse setting cast_string_to_variant_use_inference