How to use the streaming_callback callback function of OpenAIChatGenerator to output sse or websocket to the frontend? #8160
Unanswered
springrain
asked this question in
Questions
Replies: 1 comment
-
|
This is hard to achieve in the single threaded request/chunk response execution model of OpenAIChatGenerator. You need to launch another thread for each request or wait for async implementation of OpenAIChatGenerator which should be coming soon. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
How to use the streaming_callback callback function of OpenAIChatGenerator to output sse or websocket to the frontend? Can you provide an example of a test? I'm having some problems with the implementation, thank you very much.
Beta Was this translation helpful? Give feedback.
All reactions