Flags in connection string get into Client() with invalid types

[akurdyukov]

If Airflow defined connection URL like sqlite://user:password@server:9440/database?secure=true&verify=false flags secure and verify go to Client as strings with leads to invalid behavior in clickhouse_driver.

[ne1r0n]

Hi @akurdyukov, just wondering, is there any reason not to use JSON Booleans type specifying mentioned flags in connection extra.
For example:

{
  "secure": true,
  "verify": false
}

[akurdyukov]

Call to connection_kwargs = conn.extra_dejson.copy() returns strings, not bool, unfortunately.

[ne1r0n]

hmm, strange. What airflow and python version you are using?
extra_dejson does simple json.loads and you can see that it converts to bool type if you check the following

Python 3.12.2 (main, Feb  6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import json
>>> json.loads('{"boolean": true}')
{'boolean': True}

Also I did a debug in airflow:

[akurdyukov]

The problems shows itself when you configure connection with URL like sqlite://user:password@server:9440/database?secure=true&verify=false in env variable of Airflow (https://airflow.apache.org/docs/apache-airflow/stable/howto/connection.html#storing-connections-in-environment-variables). JSON is created by URL parser.

[ne1r0n]

Thanks, now understood your case.