Skip to content

Conversation

@burmecia
Copy link
Member

@burmecia burmecia commented Nov 8, 2025

What kind of change does this PR introduce?

This PR is add a configurable stream_buffer_size option to ClickHouse FDW.

What is the current behavior?

The stream buffer size is hard coded to 1024.

What is the new behavior?

  • Add stream_buffer_size option to allow users to configure the internal streaming buffer size
  • Implement validation to enforce limits between 1 and 100000 rows
  • Values outside the valid range are automatically clamped to nearest bound
  • Update documentation to reflect new option and its limits
  • Default value remains 1024 rows for backward compatibility

Additional context

N/A

Copilot AI review requested due to automatic review settings November 8, 2025 04:02
@burmecia burmecia added the clickhouse Clickhouse related issues label Nov 8, 2025
@burmecia burmecia changed the title feat(clickhouse): add configurable stream_buffer_size option to ClickHouse FDW feat(clickhouse): [WRA-9] add configurable stream_buffer_size option to ClickHouse FDW Nov 8, 2025
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a configurable stream_buffer_size option to the ClickHouse FDW, allowing users to tune the internal buffer size for streaming data. The version is bumped from 0.1.8 to 0.1.9.

  • Added stream_buffer_size foreign table option with validation (clamped between 1 and 100,000, defaults to 1024)
  • Reorganized documentation to group all foreign table options together
  • Updated changelog and documentation

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
wrappers/src/fdw/clickhouse_fdw/clickhouse_fdw.rs Bumped version to 0.1.9 and implemented stream_buffer_size option parsing with validation
wrappers/src/fdw/clickhouse_fdw/tests.rs Added stream_buffer_size option to test table definition
wrappers/src/fdw/clickhouse_fdw/README.md Added changelog entry for version 0.1.9
docs/catalog/clickhouse.md Documented stream_buffer_size option and reorganized foreign table options section

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@burmecia burmecia requested a review from imor November 10, 2025 02:10
@burmecia burmecia merged commit 303da1d into main Nov 10, 2025
6 checks passed
@burmecia burmecia deleted the bo/chore/clickhouse-steam-buffer-size branch November 10, 2025 04:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clickhouse Clickhouse related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants