Cannot update netty pipeline with outbound channel handlers

[stijnopheide]

Currently transforming the netty pipeline with outbound channel handlers on a TCP server does not work. Aleph expects that you only put onto the stream something that can be converted to a ByteBuf by aleph.netty/to-byte-buf.

For incoming channel handlers you can change this behavior by setting :raw-stream? true, however that does not change the behavior on outbound channel handlers. See https://github.com/ztellman/aleph/blob/master/src/aleph/tcp.clj#L47.

Probably this same problem exists for TCP clients.

The use case is adding frame splitters/creators for Protobuf messages (https://netty.io/4.1/api/io/netty/handler/codec/protobuf/ProtobufDecoder.html) which is implemented as netty encoders/decoders. In the TCP handler you would receive a decoded Protobuf message and you have to return a Protobuf message, not a coercable-to-ByteBuf.

[KingMob]

Yeah, by and large, Aleph uses Netty, but doesn't support you using Netty very well. There may have been good reasons for this (like the ref-counting management required when using your own channel handlers), but I'm not privy to Zach's mind.

The general expectation is the main handler you supply takes care of everything. It would be nice to better support channel handlers, though, so if you want to make a PR for UDP and TCP that adds a :coerce-fn key to the options maps, I'd support that.

[arnaudgeiser]

Starting to work on this.
We worked a lot with GRPC (and thus Protobuf) during last year, I consider this feature request valuable.

[ohbadiah]

Any update on this, out of curiosity?

[arnaudgeiser]

It's a small change but I had a hard time writing integration tests around GRPC.
Not much progress since I decided to give it a shot one year ago.

[ohbadiah]

@arnaudgeiser is there a branch I could take a look at?

[ohbadiah]

I mentioned this issue in Clojurians slack here.

[arnaudgeiser]

I don't have much but the following branch : https://github.com/clj-commons/aleph/compare/protobuf?expand=1