Connection timeouts not enforced in AsyncClient under heavy load

[zzrcxb]

Hi,

I am experiencing a problem with the AsyncClient module when stress testing my Flask SocketIO service. Specifically, I am using
await asyncio.gather(*[sio.connect(url, transports='websocket', wait_timeout=5) for sio in many_clients]).
However, when my service becomes heavily loaded, some of the sio.connect calls never return and block the execution, even after waiting for minutes, despite specifying a timeout of 5 seconds.

I have tried inspecting the code, and it seems that the execution is getting stuck at this line. I'm wondering if I have misused the API, or if the line needs to be wrapped in asyncio.wait_for.

Any help or suggestions would be greatly appreciated!

[miguelgrinberg]

The Socket.IO connection has two different phases. The first phase is about establishing the low-level Engine.IO transport. The second phase connects the Socket.IO namespace(s). The wait timeout applies to the second phase. I guess the definition can be extended to apply to the whole connection, but that is not how this is currently designed to work.

Unfortunately in an asynchronous application you can't add too much CPU load. I'm not sure how many clients you are starting, but from your description it sounds like maybe you are doing more than what your process can handle reasonably well.

[zzrcxb]

Thanks for you quick reply!

I tried to wrap the eio.connect with asyncio.wait_for and it worked well. Therefore, I am wondering if you are interested in a pull request that extends wait timeout to cover both phases. Right now, I am wrapping the entire sio.connect as a workaround.

[miguelgrinberg]

Unfortunately it's not as simple as you paint it, so I think I'm not going to do this.

One problem is that the default 1 second wait timeout is likely insufficient in many high-latency connections, so making this change can introduce failures in code that currently works fine. And increasing the wait timeout can change the timing of existing applications. Given that this is (I think) a highly unlikely situation I think not making any changes is best.

Unless you think this a bug and that the hang happens under reasonable load, and not as a result of you trying to start too many clients. Then you are welcome to show me how to reproduce it and I can debug and attempt to fix the hang.

[zzrcxb]

Got it. Although I doubt it's because I launched too many clients. I tried the normal synchronous client with ThreadExecutorPool and it worked fine (same number of clients). I guess it's a tricky scenario to reliably reproduce, so I will just stick to my workaround and leave the discussion here for future references.