Problem with uWSGI deployment (again)

[denisvitez]

Hello,

in the past I got the SocketIO to successfully work with my "production" deployment using uWSGI and Nginx. But now I have problems setting it up again and unfortunately the test device where the server was running died due to eMMC failure so I can't check the configuration to cross reference with my docs (if I didn't update something after tweaking on the device). The previous version was running using Python 3.8 but I have now updated to Python 3.11 and also updated the libraries to the latest version.

Python version:
3.11.2

Installed package versions:

Flask-SocketIO==5.5.1
python-socketio==5.12.1
python-engineio==4.11.2
uwsgi==2.0.28
gevent==24.11.1
gevent-websocket==0.10.1

I am running the application with uwsgi using the following command:
uwsgi --ini hosting.ini

And this is my hosting.ini:

[uwsgi]
chdir = /apps/api/src
module = wsgi:app
master = true
processes = 1
http-websockets = true
http = 0.0.0.0:5001
vacuum = true
die-on-term = true
lazy-apps=true

gevent = 1000
gevent-early-monkey-patch = true

I create the application object as following:

def create_app():
    flask_app = Flask(__name__)
    socketio.init_app(
        flask_app,
        message_queue='redis://',
        cors_allowed_origins='*',
        logger=True,
        engineio_logger=True,
        async_mode="gevent"
    )
    return flask_app

And then start it in the wsgi module as:

from application.main import create_app, socketio
from application import blueprint

app = create_app()
app.register_blueprint(blueprint)

app.app_context().push()

if __name__ == "__main__":
    socketio.run(app, host='0.0.0.0')

However when I start my uWSGI server I am met with the following exception:
uwsgi[4090]: RuntimeError: The gevent-websocket server is not configured appropriately. See the Deployment section of the documentation for more information.

I have also tried using the gevent_uwsgi as the async mode, but that results in a different exception, which is I assume related to the uwsgi not SocketIO? Also no idea why I should be getting the SSL errors, since I'm not using WSS or HTTPS.

Jan 15 15:33:01 M207-sn815 uwsgi[4380]: 6i4gbEZjwbSW1gHhAAAB: Received request to upgrade to websocket
Jan 15 15:33:01 M207-sn815 uwsgi[4380]: you need to build uWSGI with SSL support to use the websocket handshake api function !!!
Jan 15 15:33:01 M207-sn815 uwsgi[4380]: Traceback (most recent call last):
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:   File "/app/api/prod_env/lib/python3.11/site-packages/flask/app.py", line 1536, in __call__
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:     return self.wsgi_app(environ, start_response)
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:   File "/app/api/prod_env/lib/python3.11/site-packages/flask_socketio/__init__.py", line 42, in __call__
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:     return super().__call__(environ, start_response)
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:   File "/app/api/prod_env/lib/python3.11/site-packages/engineio/middleware.py", line 63, in __call__
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:     return self.engineio_app.handle_request(environ, start_response)
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:   File "/app/api/prod_env/lib/python3.11/site-packages/socketio/server.py", line 434, in handle_request
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:     return self.eio.handle_request(environ, start_response)
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:   File "/app/api/prod_env/lib/python3.11/site-packages/engineio/server.py", line 286, in handle_request
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:     packets = socket.handle_get_request(
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:               ^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:   File "/app/api/prod_env/lib/python3.11/site-packages/engineio/socket.py", line 92, in handle_get_request
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:     return getattr(self, '_upgrade_' + transport)(environ,
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:   File "/app/api/prod_env/lib/python3.11/site-packages/engineio/socket.py", line 151, in _upgrade_websocket
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:     return ws(environ, start_response)
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:   File "/app/api/prod_env/lib/python3.11/site-packages/engineio/async_drivers/gevent_uwsgi.py", line 35, in __call__
Jan 15 15:33:01 M207-sn815 uwsgi[4380]:     uwsgi.websocket_handshake()
Jan 15 15:33:01 M207-sn815 uwsgi[4380]: OSError: unable to complete websocket handshake

If anyone is using uwsgi and can share their working configuration or insight on something I am doing wrong, it will be much appreciated :)

[miguelgrinberg]

• Uninstall gevent-websocket, since it isn't used when deploying with uWSGI (actually it isn't used anymore for any configurations).
• Change the async mode to async_mode='gevent_uwsgi'

Hopefully that'll do it.

[denisvitez]

Well I have removed gevent-websocket package from my venv and used gevent_uwsgi as the async mode. But I'm still getting the error:
OSError: unable to complete websocket handshake thrown by the engineio package.

[miguelgrinberg]

The changes I suggested above should have eliminated this message from your logs:

uwsgi[4090]: RuntimeError: The gevent-websocket server is not configured appropriately. See the Deployment section of the documentation for more information.

But now that I look at your log again, there is another one that you need to address yourself:

Jan 15 15:33:01 M207-sn815 uwsgi[4380]: you need to build uWSGI with SSL support to use the websocket handshake api function !!!

[denisvitez]

Yes, I have noticed that in the log, but I was under the impression that I did rebuild uwsgi. However, it turned out that I was wrong. I had to do the following to solve the issue:
sudo apt-get install libssl-dev

And then in the virutalenv I had to remove the package and reinstall it with the --no-cache-dir flag to force the rebuild.
pip install uwsgi --no-cache-dir

So this must have been the two steps missing in my documentation:

1. Use gevent_uwsgi instead of gevent
2. Install libssl-dev before installing uwsgi package with pip

There are no exceptions in the logs now, and the socket.io connection is established with the clients. I have to solve the actually getting message part now. Before the upgrade my code was using the:
socketio.emit('updateInput', socket_msg, broadcast=True)

Which I have now changed to (I think I've seen it here on some issue/discussion):
socketio.emit('updateInput', socket_msg, to='/')

I can see in the API logs that the message was emitted, but I don't get any data on my clients as I did, is there something I'm missing when migrating from the "broadcast" property?

Thank you for all the help so far!

[miguelgrinberg]

The to='/' is wrong. If you want to send to everyone, then remove the to argument. Sending to everyone is the default for socketio.emit().

[denisvitez]

I removed the to='/' and now everything is working. Thank you!