Access to flask socketio msg queue and difference with celery msg queue

[MarioCiranni]

Hi,
Ive looked up on the web a lot for a satisfying answer about how to access the socketio msg queue but no particular avail.
Im still not sure how can I get access to the socketio msg queue in a flask application that implements websocket with flask-socketio library. @miguelgrinberg IIve also noticed that in your talk back at the PyCon in 2016 you clearly distinguish between socketio msg queue and celery msg queue and how they have different purposes, but still I am not sure how they are used and what makes them different.

Im bringing up this topic because I am developing a webserver. On the backend I need to process a stream of images coming from the webcam and feed it to a ML model. For I had some lagging issue when displaying the image back on the browser, which I take is due to the CPU intensive work done on the backend which causes delay in the response back to the client, I just tought that I could get access to the msg queue / buffer of the socket and just keep the last image or the last n images in order to speed up execution and avoid this CPU bottleneck. Yet, I am not sure which might be the right decision to make in this regard. Ive seen that in case of CPU intensive tasks usually Celery it is used in conjunction with Flask but I do not understand completely fits in my case or I could opt for a (maybe worse but) simpler solution like just accessing the socketio msg queue and drop some images to speed up execution as I was saying above.

I would appreciate so much if you could get a feedback on this.
Thanks,

Mario

[miguelgrinberg]

Can you explain what you are trying to do? The message queue is used internally, you are not supposed to use it directly.

[MarioCiranni]

server and client communicate via webosocket with flask-socketio. Client sends stream of images from camera to the server, the server does some image processing on each and every single image in the stream and returns a score. Problem is the delay to get back the score on the client side sometimes is too much. I imagined the bidirectional communication between client and server as a FIFO queue. What I thought was to reduce the size of the FIFO queue (like dropping some images and keeping only n images) when the number of images is too high or something similar to this. So the idea is to lighten the buffer dimension to avoid the delay issue.
That is what I thought and the reason I started investigating msg queues in the first place. I dont know if it is more clear now. Maybe I am missing some important points here since you are telling me that the msg queue is not even supposed to be used directly.
Maybe in flask-socketio the reasoning on this has to be tweaked to obtain the same result I dont know.
Any hint or help would be highly appreciated, thank you.

[miguelgrinberg]

Así said before, the queue is managed internally, as it is used for all the servers to communicate. Yzou can always drop events when your work queue starts to get back up. Maybe I don't understand what you are saying, but direct access to the socket.io queue is not going to give you what you want, since client messages don't go in this queue.

[MarioCiranni]

ok so which is the queue client messages go then?

[miguelgrinberg]

I'm sorry but I still think you are not thinking about this in the right way. This package uses a message queue for the servers to communicate and coordinate. Messages that clients send arrive through the network, not through a message queue.