-
Notifications
You must be signed in to change notification settings - Fork 88
Description
From what I can tell, if Proton is based on / extends ClickHouse, it would make a lot of sense to add support for retrieving data in the Apache Arrow format as ClickHouse already supports using Arrow for performance reasons. This could also extend to adding support for Arrow Flight SQL (which would provide ADBC support for free).
Alongside generalized Arrow support, and the recent announcements of ODBC and JDBC drivers, it would be extremely beneficial and performant to also add an ADBC driver. To be fair, as I mentioned above, if Arrow FlightSQL support is provided, then the existing ADBC driver for Arrow Flight SQL could just be used directly (and would also allow using the existing Arrow FlightSQL JDBC/ODBC drivers.
The generalized reason for supporting ADBC is that it allows bypassing the transposition of column-oriented data to and from row-orientations just to pass across ODBC/JDBC interfaces (which are inherently row-oriented).
This could be provided either by contributing a driver to the https://github.com/apache/arrow-adbc repository, or by simply maintaining a driver here (if it is decided to not go the route of using Arrow FlightSQL). I can absolutely assist with any work involved here, though don't have the bandwidth to do it myself. I just thought it would be a useful suggestion to make here.