feat(http) Request Continuation support

[mathieucarbou]

Request Continuation is the ability to pause the processing of a request (the actual sending over the network) to be able to let another task commit the response on the network later.

This is a common supported use case amongst web servers.

A uage example is described in the following discussion:

#34

Currently, the only supported way is to use chunk encoding and return RESPONSE_TRY_AGAIN from the callback until the processing is done somewhere else, then send the response in the chunk buffer, when the processing is completed.

[willmmiles]

Oh, I implemented something like this for the WLED fork!

The original goal was to implement a deferred request queue so low-memory devices wouldn't OOM trying to handle too many parallel operations. I started with a simple queue in the onClient callback, and a queue-servicing function to actually handle replies. Later, when I ran in to lock contention on a shared resource in a handler function, I added AsyncWebRequest::deferResponse() so a request could be left in the queue so it could be retried later. The queue gets pumped on the completion of any response, in poll() callbacks, and could be explicitly triggered from outside (ie. when the lock was released). I didn't bother with tracking which unblock events mapped to what requests since the queue was FIFO and it wasn't necessary for our application -- any still-blocked requests would re-block, so the queue processor would move on to the next one.

I don't know if any of that code is directly useful, unfortunately. Our fork is too far back, and I fear there are too many code size compromises in this version for us to consider switching to this one (>10kb code size increase when we're evaluating PRs by the byte to fit in existing partition maps!) -- but if there's anything I can to do help, let me know; or you find anything there useful, you're welcome to it.

[mathieucarbou]

@willmmiles : thanks! I was not planning to go that far for this implementation (I don't think I need to) but I will test the limits. With the throttling that @vortigont added in AsyncTCP I suppose it will be fine... But I suspect also that the max limit will be 16 (config from lwip esp-idf) - we'll see ;-)
In any case, if you are willing one day to try out these new libs with WLED and report back the issues, that would be helpful!

[mathieucarbou]

@me-no-dev @vortigont : FYI request continuation is implemented and ready for review!

Added an example: https://github.com/ESP32Async/ESPAsyncWebServer/blob/feat/continuation/examples/RequestContinuation/RequestContinuation.ino

And doc: https://github.com/ESP32Async/ESPAsyncWebServer/tree/feat/continuation?tab=readme-ov-file#request-continuation

Ran the perf test and some others too.

[mathieucarbou]

@willmmiles : fyi I did some testing with this implementation and the RequestContinuationComplete example.

for i in {1..16}; do curl --connect-timeout 120 http://192.168.4.1/ & done

works perfectly, which is expected (16 concurrent slots lwip)

for i in {1..32}; do curl --connect-timeout 120 http://192.168.4.1/ & done

works also perfectly: we can see the results arriving by batches of 16, but slower

for i in {1..48}; do curl --connect-timeout 180 http://192.168.4.1/ & done

same as before, slower but all are served by batches of 16.

So I think this more than enough.
An app should not have so many paused requests like that, otherwise this is a use case for SSE or WebSocket.