scaling socketIO with multiple workers on Heroku

[WeekiCode]

Hi,

Flask/socketIO does not work in Heroku with gunicorn workers >1. Application works fine on Heroku with worker=1. I get various socket failure with workers=2 (see log below [engineio_logger=True]). You have given various answers that say to use multiple copies of gunicorn, instead of multiple workers. Heroku support says they do not support multiple gunicorns in one Dyno. Not sure I believe them. Do you know of people using socketIO in Heroku that have scaled? Is there another way of running multiple flask/socketIO servers in order to scale?
heroku socketIO log.txt

[bluthen]

This is only an answer to the last question. I've never used Heroku.
The workaround could be very application dependent. I've made a couple applications were we use multiple socketio servers.

• We use sticky sessions and had a application were we didn't need to share information between other clients
• We had a application were the sharing of information among other clients in only in small groups, so you can use sticky sessions based on a custom cookie were all the people in the small group get the same cookie and hit the same socket io server.

Otherwise you need to have some way for the servers to share information as well as sticky sessions. A message queue with: Redis, kafka, or something else.
https://flask-socketio.readthedocs.io/en/latest/deployment.html#using-multiple-workers

[WeekiCode]

In Heroku sticky sessions are called "affinity" and enabled with the command: heroku features:enable http-session-affinity --app xxxxxxxx

I enabled it and it ran fine with a single worker. When I added a second worker, I started to get a lot of: Invalid session 50EQeR4s0LS2v9-SAAAA. I got this message for more than one ID. From the app's perspective, the items that depend on socketIO did not work at all.

Log file is attached...
after stick and 2 workers change.txt

Any ideas?

[miguelgrinberg]

I suggest you read the documentation wrt using multiple workers. The link @bluthen has provided is correct.

Running multiple gunicorn processes in a single Heroku dyno is not something they want you to do. I have never tried, but it may work if you create a bash script that launches all the Gunicorns as background processes, and then put this bash script in your Procfile. In any case, are you sure you need multiple workers? If you are using gevent or eventlet, then a single worker is often enough, especially for constrained platforms such as Heroku.

[miguelgrinberg]

My answer is the same as before. There are no universal numbers, it all depends on your application, so the only way you'll find answers is by stress-testing your application to find what its limits are. There is really no other way.

If I understand it correctly, eventlet runs one green thread at a time.

This is not related to eventlet. Python runs one thread at a time. If you have N CPUs, the only way to utilize all of them concurrently is to run N copies of your Python process, or else use a specialized package such as multiprocessing to create a child process for each CPU.

I am assuming that the real work is put off to Celery

This is highly dependent on the application. Not all applications need Celery.

I have no idea if I am worried about something that I should not be

I think your worries are reasonable. You should find actual answers through testing to understand if the worries are justified or not.

[WeekiCode]

Is it safe to assume the Gunicorn provides that ability to add N CPU's to the equation?

[miguelgrinberg]

If you are asking as a general question, then yes, Gunicorn can start multiple worker processes. But you already know that this is insufficient for Socket.IO, since this is what started this discussion.

[WeekiCode]

I meant, staring multiple gunicorn's (1 worker each), but we have discussed that in relation to Heroku. I am not sure how to scale with Heroku beyond what one eventlet can do.

Thanks very much for your patience and time!

[miguelgrinberg]

In Heroku the unit is the dyno. To scale, you add dynos.

[WeekiCode]

thanks @bluthen..... I will try it tomorrow and report back