Skip to content

Feature Request: Support Raw String Option for HTTP Batch Mode #134

@bpb

Description

@bpb

In some use cases, apis require batched data to be submitted as a raw string rather than wrapped in a JSON array.

Currently the only way to send batched raw data strings is to use the single request mode and create a pre aggregated roll up payload as a string before sinking. This operation is a bit obtuse.

BodyPublisher publisher;
// By default, Java's BodyPublishers.ofByteArrays(elements) will just put Jsons
// into the HTTP body without any context.
// What we do here is we pack every Json/byteArray into Json Array hence '[' and ']'
// at the end, and we separate every element with comma.
elements.add(BATCH_START_BYTES);
for (HttpSinkRequestEntry entry : reqeustBatch) {
elements.add(entry.element);
elements.add(BATCH_ELEMENT_DELIM_BYTES);
}
elements.set(elements.size() - 1, BATCH_END_BYTES);
publisher = BodyPublishers.ofByteArrays(elements);

Please consider adding this functionality in the near future.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions