python-socketio: can two AsyncClient run in the same asyncio loop?

[emanuele3d]

Apologies if the question is not stricly related to Flask-SocketIO: this page redirects here.

I am effectively trying to handle two (or more) socketio.AsyncClient objects within the same asyncio loop.

So far I have been unsuccessful because it seems that each client needs a asyncClient.wait() line or a while(True): await asyncClient.sleep(0) codeblock after asyncClient.connect() to keep the connection up and running. But both options effectively block the thread and the code to instantiate and run the next AsyncClient is never reached.

So far it seems to me I can only run multiple clients using multiple threads and therefore multiple asyncio loops. Is this correct or is there a way for multiple AsyncClients to interleave their activities within the same asyncio loop?

Alternatively, couldn't the AsyncClient instance keep the connection open until it is garbage-collected instead of closing it immediately after the connect() statement if there are no wait() or sleep() statements following it?

I posted this question also on StackOverflow: it is worded differently but it has some code and is effectively aiming at the same goal. Another question (not by me) seems to be heading in the same general direction.

[miguelgrinberg]

Sorry about the bad discussions link, it should have directed you to the discussions on the other repository, I just corrected it.

The sio.wait() call is not mandatory, it is just there as a convenience, because in many cases once you start the client your main thread/task has nothing else to do other than waiting for the connection to end. I haven't tested this, but it would be perfectly fine to connect the two clients and then wait on both one after the other:

    sio1.connect(...)
    sio2.connect(...)
    sio1.wait()
    sio2.wait()

I haven't tested this myself, but I don't see what it wouldn't work. Or maybe I should say that if this does not work, it is very likely that there is a bug I need to fix. :)

I also replied to your SO question, which is related to this problem.

[emanuele3d]

Hmmm.. I can't mark this answer as the answer... I'm not sure why...

[emanuele3d]

Thank you Miguel for your reply. Let me know if you'd like to move this discussion on the intended location rather than here.

I followed your suggestion with this code:

class OneClientTwoServerTests:

    def __init__(self):
        self._start_two_servers_of_your_choice()
        
        loop = asyncio.get_event_loop()
        loop.run_until_complete(self._start_clients())

    async def _start_clients(self):

        asyncClient1 = socketio.AsyncClient()
        await asyncClient1.connect('http://127.0.0.1:9101')

        asyncClient2 = socketio.AsyncClient()
        await asyncClient2.connect('http://127.0.0.1:9102')

        # wait for both
        print('starting wait for clientS')
        await asyncClient1.wait()
        print('started wait for client 1')
        await asyncClient2.wait()
        print('started wait for client 2')
        await asyncClient2.wait()

The output stops at "starting wait for clientS", which makes me think the first wait() call blocks the thread. Should I start the async method in a different way rather than through the run_until_complete() call? I will try this afternoon to change the async method to only start one client, so that I have to call it twice. Perhaps wait() only stops the execution of a single method call and having the same call scheduled twice would allow asyncio to keep the two clients in parallel?

[miguelgrinberg]

The wait method is blocking, in the sense that it only returns when the client is disconnected. It does not block the asyncio loop though, all the background tasks (your own and those from the Socket.IO clients both) will continue to run while this method waits.

So at this point I'm not really sure if there are any remaining problems. If you wait for the two clients one after the other, your main task will stay running and will allow the clients to complete whatever they need to do before the loop stops. This is how working exactly as designed.

[emanuele3d]

Ok, I think I'm getting there... what I have to make sure is that I make N wait() calls, one per client, -after- all the code in which I initialize and connect all the clients.

And looking at the source code for wait(), it doesn't matter that the first wait() call blocks all the others: eventually all of them, as their respective clients disconennect, will terminate their execution even if the clients disconnect in an order that is different from how the wait() calls are listed in the code.

Wow. All this took quite a bit of time to figure out. It's not quite obvious from the documentation that a wait() call blocks but that it's generally not a problem. Might be worth a small article on Medium or some other outlet like that, to make an example with multiple clients within a containing object and why the consecutive wait() calls at the end work despite being blocking. I'll see if I can get around it at some point.

Thank you for your help Miguel!

[miguelgrinberg]

And looking at the source code for wait(), it doesn't matter that the first wait() call blocks all the others: eventually all of them, as their respective clients disconennect, will terminate their execution even if the clients disconnect in an order that is different from how the wait() calls are listed in the code.

Right! In fact, if you have your own logic to determine when the application should exit, you can wait on your own thing and not call wait. The wait method is provided as a convenience, but it is entirely optional.