Skip to content

Implement the Go ADBC Interface and build script #865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,37 @@ You should see data like the following:
└─────────┴─────────┴──────────────────┴──────────────────┘
```

### Using the ADBC Driver

To use the ADBC driver for Arrow Flight SQL, follow these steps:

1. Ensure that the `USE_ADBC` configuration option is enabled in your `config.h.in` file.
2. Implement the ADBC driver in your code by including the necessary headers and creating an instance of the `ADBCDriver` class.
3. Use the `ExecuteQuery` method of the `ADBCDriver` class to execute SQL queries and retrieve results as Arrow Tables.

Example:

```cpp
#include <ADBC/ADBCDriver.h>
#include <arrow/table.h>
#include <iostream>

int main() {
ADBC::ADBCDriver driver;
std::string query = "SELECT * FROM test_table";
std::shared_ptr<arrow::Table> result;

try {
result = driver.ExecuteQuery(query);
std::cout << "Query executed successfully. Number of rows: " << result->num_rows() << std::endl;
} catch (const std::runtime_error& e) {
std::cerr << "Error executing query: " << e.what() << std::endl;
}

return 0;
}
```

### ⏩ What's next?
To see more examples of using Timeplus Proton, check out the [examples](https://github.com/timeplus-io/proton/tree/develop/examples) folder.

Expand Down
98 changes: 98 additions & 0 deletions src/ADBC/ADBCDriver.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
#include "ADBCDriver.h"
#include <adbc.h>
#include <adbc_driver_manager.h>
#include <arrow/flight/sql/client.h>
#include <arrow/flight/sql/driver.h>
#include <arrow/flight/sql/server.h>
#include <arrow/flight/sql/types.h>
#include <arrow/flight/sql/utils.h>
#include <arrow/flight/types.h>
#include <arrow/flight/utils.h>
#include <arrow/io/api.h>
#include <arrow/ipc/api.h>
#include <arrow/result.h>
#include <arrow/status.h>
#include <arrow/table.h>
#include <arrow/util/logging.h>
#include <arrow/util/memory.h>
#include <arrow/util/uri.h>
#include <iostream>
#include <memory>
#include <string>

namespace ADBC {

class ADBCDriver {
public:
ADBCDriver() {
// Initialize the ADBC driver manager
AdbcStatusCode status = AdbcDriverManagerInitialize();
Comment on lines +28 to +29

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this function defined? There is no AdbcDriverManagerInitialize() in adbc_driver_manager.h`

if (status != ADBC_STATUS_OK) {
std::cerr << "Failed to initialize ADBC driver manager: " << AdbcStatusCodeToString(status) << std::endl;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AdbcStatusCodeMessage not AdbcStatusCodeToString

throw std::runtime_error("Failed to initialize ADBC driver manager");
}

// Register the Arrow Flight SQL driver
status = AdbcDriverManagerRegisterDriver("arrow-flight-sql", &arrow_flight_sql_driver);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function doesn't exist either, did you mean AdbcLoadDriver?

Also, if you are only using the flightsql driver, why use the driver manager? Why not just link against libadbc_driver_flightsql.so directly?

if (status != ADBC_STATUS_OK) {
std::cerr << "Failed to register Arrow Flight SQL driver: " << AdbcStatusCodeToString(status) << std::endl;
throw std::runtime_error("Failed to register Arrow Flight SQL driver");
}
}

~ADBCDriver() {
// Cleanup the ADBC driver manager
AdbcDriverManagerCleanup();
}

std::shared_ptr<arrow::Table> ExecuteQuery(const std::string& query) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should return an std::shared_ptr<arrow::RecordBatchReader> to stream the results instead

// Create a connection to the Arrow Flight SQL server
AdbcConnection connection;
AdbcStatusCode status = AdbcConnectionInitialize(&connection, "arrow-flight-sql", nullptr);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AdbcConnectionNew and AdbcConnectionInit?

if (status != ADBC_STATUS_OK) {
std::cerr << "Failed to initialize ADBC connection: " << AdbcStatusCodeToString(status) << std::endl;
throw std::runtime_error("Failed to initialize ADBC connection");
}

// Execute the query
AdbcStatement statement;
status = AdbcStatementInitialize(&statement, &connection);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AdbcStatementNew and AdbcStatementInit

if (status != ADBC_STATUS_OK) {
std::cerr << "Failed to initialize ADBC statement: " << AdbcStatusCodeToString(status) << std::endl;
throw std::runtime_error("Failed to initialize ADBC statement");
}

status = AdbcStatementSetSqlQuery(&statement, query.c_str());
if (status != ADBC_STATUS_OK) {
std::cerr << "Failed to set SQL query: " << AdbcStatusCodeToString(status) << std::endl;
throw std::runtime_error("Failed to set SQL query");
}

AdbcResult result;
status = AdbcStatementExecuteQuery(&statement, &result);
if (status != ADBC_STATUS_OK) {
std::cerr << "Failed to execute query: " << AdbcStatusCodeToString(status) << std::endl;
throw std::runtime_error("Failed to execute query");
}

// Convert the result to an Arrow Table
std::shared_ptr<arrow::Table> table;
arrow::Status arrow_status = arrow::flight::sql::ResultToTable(result, &table);
if (!arrow_status.ok()) {
std::cerr << "Failed to convert result to Arrow Table: " << arrow_status.ToString() << std::endl;
throw std::runtime_error("Failed to convert result to Arrow Table");
}
Comment on lines +71 to +84

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

execute query uses an ArrowArrayStream there is no AdbcResult


// Cleanup
AdbcResultRelease(&result);
AdbcStatementRelease(&statement);
AdbcConnectionRelease(&connection);

return table;
}

private:
AdbcDriver arrow_flight_sql_driver;
};

} // namespace ADBC
37 changes: 37 additions & 0 deletions src/ADBC/ADBCDriver.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#pragma once

#include <adbc.h>
#include <adbc_driver_manager.h>
Comment on lines +3 to +4

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

include paths should be <arrow-adbc/adbc.h> and <arrow-adbc/adbc_driver_manager.h>

#include <arrow/flight/sql/client.h>
#include <arrow/flight/sql/driver.h>
#include <arrow/flight/sql/server.h>
#include <arrow/flight/sql/types.h>
#include <arrow/flight/sql/utils.h>
#include <arrow/flight/types.h>
#include <arrow/flight/utils.h>
#include <arrow/io/api.h>
#include <arrow/ipc/api.h>
#include <arrow/result.h>
#include <arrow/status.h>
#include <arrow/table.h>
#include <arrow/util/logging.h>
#include <arrow/util/memory.h>
#include <arrow/util/uri.h>
#include <iostream>
#include <memory>
#include <string>

namespace ADBC {

class ADBCDriver {
public:
ADBCDriver();
~ADBCDriver();

std::shared_ptr<arrow::Table> ExecuteQuery(const std::string& query);

private:
AdbcDriver arrow_flight_sql_driver;
};

} // namespace ADBC
65 changes: 65 additions & 0 deletions src/ADBC/go/adbc_driver.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
package main
Copy link

@zeroshade zeroshade Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this file seems wrong? there's nothing here and it's using the wrong library version


import (
"context"
"database/sql"
"database/sql/driver"
"fmt"
"github.com/apache/arrow/go/v8/arrow"
"github.com/apache/arrow/go/v8/arrow/array"
"github.com/apache/arrow/go/v8/arrow/flight"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql"
"github.com/apache/arrow/go/v8/arrow/memory"
"github.com/apache/arrow/go/v8/arrow/scalar"
"github.com/apache/arrow/go/v8/arrow/util"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver"
"github.com/apache/arrow/go/v8/arrow/flight/flightsql/adbc/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/driver/manager/
8 changes: 8 additions & 0 deletions src/ADBC/go/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash

# Set the Go environment variables
export GO111MODULE=on
export CGO_ENABLED=1

# Build the Go ADBC Interface as a C shared object
go build -o adbc_driver.so -buildmode=c-shared adbc_driver.go
2 changes: 2 additions & 0 deletions src/Common/config.h.in
Original file line number Diff line number Diff line change
Expand Up @@ -79,3 +79,5 @@
#cmakedefine01 USE_ENCRYPT_DECRYPT_FUNCS
#cmakedefine01 USE_ARG_MIN_MAX_FUNCS
/// proton: ends.

#cmakedefine01 USE_ADBC
Loading