gRPC and Protobuf support for remote user-defined operations #283
Description
Is your feature request related to a problem? Please describe.
Data movement from VDMS to remote servers for a remote UDF involves multiple filesystem read/write operations, which could result in bottlenecks, especially for data-intensive use cases. Moreover, this flow results in multiple read/write operations even at the remote server end.
Describe the solution you'd like
We plan to update the workflow so that instead of files, we work with protobufs, reducing the file I/O operations for both VDMS and the remote server. With protobuf as the data format, gRPC would be a better alternative to HTTP. This solution would also make writing UDFs easier, as the user would not need to worry about file read/write or the format of the files.