Skip to content

Add Google Test suite for regression testing baseline.#276

Open
fdcastel wants to merge 2 commits intoFirebirdSQL:masterfrom
fdcastel:add-tests-from-v4-pr
Open

Add Google Test suite for regression testing baseline.#276
fdcastel wants to merge 2 commits intoFirebirdSQL:masterfrom
fdcastel:add-tests-from-v4-pr

Conversation

@fdcastel
Copy link
Member

Summary

Adds a comprehensive Google Test suite (375 tests across 38 test suites) that exercises the Firebird ODBC driver through the standard ODBC API via the Driver Manager. This establishes a regression testing baseline on the current, unmodified driver — per the plan discussed in PR #275.

No driver source code is modified by this PR.

Motivation

Before making any code changes (bugfixes, build system, CI), we need a testing baseline that documents what the current driver can do and where improvements are needed. This test suite:

  1. Verifies existing functionality — 230 tests pass, confirming data types, parameter binding, cursors, catalog functions, etc. all work correctly.
  2. Documents known gaps — 145 tests are marked GTEST_SKIP() with explanations of what improvement they require (e.g., "Requires Phase 7 (OC-1): SQLCopyDesc null records crash fix").
  3. Provides a regression gate — future PRs can remove individual GTEST_SKIP() markers as fixes are merged, turning them into pass/fail tests.

What's included

Infrastructure

File Description
tests/CMakeLists.txt Standalone CMake project; fetches Google Test v1.14.0 via FetchContent
tests/test_main.cpp GTest entry point
tests/test_helpers.h OdbcConnectedTest fixture, TempTable RAII guard, error helpers
tests/README.md Build/run instructions, test categories, connection string format

Test files (30 files)

Category A — Tests that pass on vanilla master (16 files)

File Description
test_data_types.cpp All SQL data types: INTEGER, VARCHAR, NUMERIC, DATE, TIME, TIMESTAMP, etc.
test_result_conversions.cpp SQLGetData type conversions
test_param_conversions.cpp SQLBindParameter type conversions
test_prepare.cpp SQLPrepare / SQLExecute lifecycle
test_cursors.cpp Cursor behavior across statements
test_cursor_commit.cpp Cursor behavior across transactions
test_cursor_name.cpp SQLSetCursorName / SQLGetCursorName
test_data_at_execution.cpp SQL_DATA_AT_EXEC / SQLPutData
test_array_binding.cpp Column-wise + row-wise parameter arrays
test_bindcol.cpp Dynamic unbind/rebind mid-fetch
test_descrec.cpp SQLGetDescRec for all column types
test_blob.cpp BLOB read/write (small, large, NULL)
test_multi_statement.cpp Multiple statement handles on one connection
test_stmthandles.cpp Stress test: 100+ simultaneous statement handles
test_wchar.cpp SQL_C_WCHAR bind/fetch and truncation
test_escape_sequences.cpp ODBC escape sequence passthrough

Category B — Mixed pass/skip (6 files)

These files contain tests that pass and tests that are GTEST_SKIP()'d because they depend on driver improvements not yet merged.

File Pass Skip Skip reason
test_descriptor.cpp 6 7 SQLCopyDesc crash, SetDescCount allocation
test_connect_options.cpp 6 30 CONNECTION_TIMEOUT, ASYNC_ENABLE, QUERY_TIMEOUT, RESET_CONNECTION
test_errors.cpp 11 7 SQL_DIAG_ROW_COUNT, string truncation indicator
test_catalogfunctions.cpp 26 3 SQLGetTypeInfo ordering, GUID searchability
test_server_version.cpp 4 2 FB4+ type info detection
test_scrollable_cursor.cpp 5 4 Scrollable cursor edge cases

Category C — All tests SKIP'd (8 files)

These files test features that don't exist on vanilla master. Every test is GTEST_SKIP()'d with an explanation.

File Tests Feature required
test_null_handles.cpp 65 Crash prevention: NULL handle validation
test_savepoint.cpp 4 Savepoint transaction isolation
test_conn_settings.cpp 3 ConnSettings DSN attribute
test_odbc38_compliance.cpp 12 ODBC 3.8 (SQL_OV_ODBC3_80, RESET_CONNECTION)
test_guid_and_binary.cpp 14 SQL_GUID type mapping, FB4+ types
test_odbc_string.cpp 26 OdbcString class (guarded with __has_include)
test_phase7_crusher_fixes.cpp Not compiled (documentation only, tests duplicated in Cat B files)
test_phase11_typeinfo_timeout_pool.cpp Not compiled (documentation only, tests duplicated in Cat B files)

How to build and run

Prerequisites

  • The Firebird ODBC driver must be pre-built (from Builds/MsVc2022.win/OdbcFb.sln) and registered as an ODBC data source
  • A Firebird 3.0+ database must be accessible
  • CMake ≥ 3.14

Build

cmake -B build-tests -S tests
cmake --build build-tests --config Release

Run

rem Windows
set FIREBIRD_ODBC_CONNECTION=Driver={Firebird ODBC Driver};Dbname=localhost:C:\path\to\test.fdb;Uid=SYSDBA;Pwd=masterkey;
ctest --test-dir build-tests --output-on-failure -C Release

Expected output

[==========] 375 tests from 38 test suites ran.
[  PASSED  ] 230 tests.
[  SKIPPED ] 145 tests.
[  FAILED  ] 0 tests.

Without FIREBIRD_ODBC_CONNECTION set, all 375 tests skip gracefully.

Design decisions

  1. Standalone CMake project — Since there's no root CMakeLists.txt yet (that's a future PR), the test suite is self-contained with its own project() declaration. It will integrate via add_subdirectory(tests) when a root CMake build is added.

  2. Tests via Driver Manager, not direct DLL calls — Tests link against odbc32/odbccp32 (the Windows ODBC Driver Manager) and connect using a standard connection string. This tests the driver as applications actually use it. The one exception is test_null_handles.cpp (which loads the DLL directly to test entry-point validation) — all its tests are SKIP'd.

  3. GTEST_SKIP() preserves test code — Rather than deleting tests that don't pass on vanilla, each is wrapped with GTEST_SKIP() << "reason". The test code remains intact so that future PRs can simply remove the SKIP line to activate the test.

  4. No driver code changes — This PR adds only the tests/ directory. No existing source files are modified.

Checklist

  • Tests build with CMake 3.14+ / MSVC 2022
  • 230 tests pass against vanilla Firebird ODBC driver
  • 145 tests skip with clear explanations
  • 0 test failures
  • Graceful SKIP when no database connection is available
  • README with build and run instructions

@fdcastel
Copy link
Member Author

Pushed fixes for four issues identified in @irodushka’s review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments