-
Notifications
You must be signed in to change notification settings - Fork 50
Open
Description
In some use cases, apis require batched data to be submitted as a raw string rather than wrapped in a JSON array.
Currently the only way to send batched raw data strings is to use the single request mode and create a pre aggregated roll up payload as a string before sinking. This operation is a bit obtuse.
Lines 111 to 122 in b95e7fb
BodyPublisher publisher; | |
// By default, Java's BodyPublishers.ofByteArrays(elements) will just put Jsons | |
// into the HTTP body without any context. | |
// What we do here is we pack every Json/byteArray into Json Array hence '[' and ']' | |
// at the end, and we separate every element with comma. | |
elements.add(BATCH_START_BYTES); | |
for (HttpSinkRequestEntry entry : reqeustBatch) { | |
elements.add(entry.element); | |
elements.add(BATCH_ELEMENT_DELIM_BYTES); | |
} | |
elements.set(elements.size() - 1, BATCH_END_BYTES); | |
publisher = BodyPublishers.ofByteArrays(elements); |
Please consider adding this functionality in the near future.
Metadata
Metadata
Assignees
Labels
No labels