-
Notifications
You must be signed in to change notification settings - Fork 72
feat: ollama full support for golem and wasm #8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@@ -1,6 +1,6 @@ | |||
[package] | |||
name = "golem-llm-anthropic" | |||
version = "0.0.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not adjust the other creates in the project please
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add automated tests or at least a video showcasing the examples passing against a local ollama instance
…racing execution flow in OllamaApi and EventStream
@Rutik7066 Others are ahead of you by a significant amount, I will close this, but feel free to re-open if you address all of these issues and follow instructions precisely. Thanks! |
@jdegoes could you please open this PR I am working on this. I will update the lastest change I have implemented ndjson alongside eventsource. |
…it for improved stream management
0081403
to
7113ac6
Compare
@mschuwalow PR is ready for 🚀 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, aside the few minor comments. I like how you handled the ndjson and sse stream differences.
llm/src/event_source/stream.rs
Outdated
NdJsonStream(NdJsonStream), | ||
} | ||
|
||
pub trait Stream { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As Stream is already a commonly used trait, I would prefer a name that causes less collision
@mschuwalow added the changes and ci fixes as well. CI run link https://github.com/Rutik7066/golem-llm/actions/runs/15250412169/job/42886202542 |
Yes, with this implementation it would be easier to add different types of stream and will be more maintainable. |
Thank you guys @vigoo @mschuwalow 🎉🍾🥂 |
Adds Ollama Support
GOLEAM_OLLAMA_BASE_URL
with the-e
flag.provider_metadata_json
, which is highly beneficial for consumers.StreamError
and a commonStream
interface that is identical to the existingEventStream
. This is extended byEventStream
andNdJsonStream
, and we can easily add more streaming formats while ensuring minimal code changes, better code maintainability, and adherence to best practices.Test Video Link
Closes #6
Fixes #6
/claim #6