Skip to content

Stream POC #200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed

Stream POC #200

wants to merge 7 commits into from

Conversation

ewilliams-cloudera
Copy link
Collaborator

POC demonstrating using streaming responses against a direct llm call. Additional work will be necessary to get this fully fleshed out to use. Next steps include:

  1. the Python side should kick off evals, question rewrites, etc after the streamed response is finished
  2. The simple Python endpoint bypasses functionality for using the data source, which will need to be added
  3. The UI will need to use the mutation
  4. Will need to add support for query config
  5. Will need to replace the current loading chat UI with the streamed response
  6. Once the streamed response is done, the UI will need to request the full payload.

@mliu-cloudera mliu-cloudera deleted the mob/streaming branch May 5, 2025 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants