out_bigquery: make HTTP buffer size configurable

[tsubaron]

Replace hardcoded HTTP buffer size in the BigQuery output plugin with a configurable value from ctx->buffer_size.

Previously, the buffer size was fixed at 4192 bytes:

flb_http_buffer_size(c, 4192);

This has now been updated to:

flb_http_buffer_size(c, ctx->buffer_size);

so that users can adjust the buffer size via configuration if needed.

We understand that the buffer size was previously hardcoded because BigQuery API responses are generally assumed to be of fixed size.

However, in practice we have observed cases where the plugin encounters buffer overflow errors. We suspect this happens when the BigQuery API returns larger-than-expected error responses in certain situations.

Making the buffer size configurable allows users to increase it as needed to inspect the full API response, which is especially useful for debugging and monitoring unexpected behaviors in production environments.

This is a small, backward-compatible change that does not alter existing behavior unless the buffer_size option is explicitly configured. The default value remains 4192 bytes. For these reasons, we believe this enhancement is suitable for inclusion in the next patch release (4.0.x).

Fixes #9061

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change

[SERVICE]
    flush        1
    daemon       off
    log_level    debug

[INPUT]
    name         dummy
    dummy        {"message":"Hello BigQuery"}

[OUTPUT]
    name                       bigquery
    match                      *
    project_id                 your_gcp_project_id
    dataset_id                 your_dataset_id
    table_id                   your_table_id
    google_service_credentials /path/to/service_account.json
    buffer_size                8192

• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found
  [N/A] This change only adjusts the buffer size configuration; no memory allocations or changes in lifecycle management were introduced.

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• Run local packaging test showing all targets (including any new ones) build.
  [N/A]
• Set ok-package-test label to test for all targets (requires maintainer to do).
  [N/A]

Documentation

• Documentation required for this feature

See: fluent/fluent-bit-docs#1915

Backporting

• Backport to latest stable release.
  This enhancement is backward-compatible and small in scope. We believe it is suitable for inclusion in the next patch release (4.0.x).

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.