Add local Ollama model support

[varshith257]

Closes #6
/claim #6

This PR adds local model support via Ollama allowing golem-llm to run without requiring an external API key and all 6 tests are passed

InShot_20250517_170445425.mp4

[varshith257]

@jdegoes @vigoo Can you review this please?

Just on a note for reviewers, PR 12 under bounty issue contains a most of my code directly copied from my earlier implementation without any attribution. Since this is a bounty issue, I’d like to flag this for fairness. Happy to provide more context on this

[mschuwalow]

Please switch to the native ollama api and add automated tests in ci.

[mschuwalow]

Suggested change

	description = "WebAssembly component for working with Ollama APIs, integrated into Golem Cloud local model support"
	description = "WebAssembly component for working with Ollama APIs, with special support for Golem Cloud"

[mschuwalow]

tool_choice parameter does not seem to be supported https://github.com/ollama/ollama/blob/main/docs/openai.md?plain=1#L245

[mschuwalow]

You are basing this on their openai compatible api, which is warned against https://github.com/ollama/ollama/blob/7edfdd2f5f48a7be035cec23b4acd12f7c112e1c/docs/openai.md#openai-compatibility

Please use ollama's native api instead https://github.com/ollama/ollama/blob/7edfdd2f5f48a7be035cec23b4acd12f7c112e1c/docs/api.md?plain=1#L499

[Nanashi-lab]

@Nanashi-lab It looks like this implementation was at least partially copied from this one. We do not allow copying from existing work and require that all contributions be original.

@jdegoes It is the otherway around, My Implementation was on 12th, He has forced pushed commit on 16th. He has forced pushed twice, so it hard to see what the original commit was.

Among his branches he has, branch test-25, and two weeks ago when the PR was made his code was -> Link

You can read the whole branch history here - Branch History I made local copy just in case.

When the PR was made, he did not have support for Image or Tool Calling, He was using the default api at /api/chat , No Streaming Implementation that I can see, as /api/chat uses ndjson and sse (Which is what ollama expects).

Now - After the 16th force push

1. Uses openai compatible Api after my PR
2. Supports Image in the same way as mine, download it then convert into base64 and then passes it
3. Our codes look awfully same.

This was original PR comment left two weeks ago

This PR adds local model support via Ollama allowing golem-llm to run without requiring an external API key.
NOTE

There are few case of ollama and here the thoughts:

    Simple chat (generate or chat) is fully supported
    Tool calls and image inputs explicitly return NotSupported errors
    Streaming is implemented using stream_chat with an SSE handler
    Streaming tool use falls back safely, returning NotSupported
    Crash recovery keeps stream resume and retry work as expected

I think we need to focus review on fallback behavior for unsupported features and verify that streaming integration is as expected

You find this under edits, you can says it doesnt support tool calls, or image inputs, He does says streaming is implemented using stream_chat with an sse handler. (I am not sure what that means, but ndjson to sse is no where near the code.)

After his force push on 16th, my PR was on 12th. The code has changed to look like mine. The force push is also not due to rebasing, as that would leave commit history intact.

[varshith257]

@Nanashi-lab I didn't get it what you intended.But on the day of I submitted this PR I got done the work and commented for review in golem slack except of testing. You can check there with @vigoo and your PR came very late of my submitted PR and i have checked on the day your PR got submitted and i checked most of my code presented in your PR. I have reported same there in channel

The only change we got difference with other contributor and me is that I initially completed my PR with Ollama nativeapi and later while testing there has been few incompatible issues with native ollama API with golem llm(not supported of tool-calls and few more inconsitencies) and i got busy with my gradution works and as per reviewers i have been here after LambdaConf and then tested again with guildinees from @mschuwalow in issue and made few changes to accomplish with Openai compat.

Note for reviewers:

There has been the patterns of other llms but you can watch my implementation is slightly differed from there and same my code you can watch on #12 even names of most vars and more

[Nanashi-lab]

Other Specific examples

In his two week old PR (Made before mine)

    Ok(OllamaChatRequest {
        model: config.model,
        messages: ollama_messages,
        tools,
        format: options.get("format").map(|f| f.to_string()),
        options: request_options,
        template: options.get("template").map(|t| t.to_string()),
        keep_alive: options.get("keep_alive").map(|k| k.to_string()),
        stream: false,
    })
}

In my PR (12th) and his current PR (16th)

    Ok(OllamaChatRequest {
        model: config.model,
        messages: ollama_messages,
        tools,
        tool_choice,
        response_format: options.get("response_format").cloned(),
        temperature: config.temperature,
        top_p: options.get("top_p").and_then(|v| v.parse().ok()),
        stop: config.stop_sequences,
        frequency_penalty: options
            .get("frequency_penalty")
            .and_then(|v| v.parse().ok()),
        presence_penalty: options.get("presence_penalty").and_then(|v| v.parse().ok()),
        seed: options.get("seed").and_then(|v| v.parse().ok()),
        max_tokens: config.max_tokens,
        keep_alive: options.get("keep_alive").cloned(),
        stream: false,
    })

He himself has admitted he, he was using native API (Which 2 weeks ago, which turned out to the right way to do this ticket), mine clearly uses Openai compatable API, (which he has copied one to one, force pushed (not a rebase), which turns out is not the right way to solve the ticket)

[jdegoes]

@Nanashi-lab Thank you for providing these examples.

Note that in the future, if you want a definitive baseline for establishing authorship of some bounty issue, you can email a ZIP of your code to contact@ziverge.com.

In the meantime, we will discuss and propose a resolution to this dispute.

[mschuwalow]

@Nanashi-lab @varshith257

There is definitely a lot of overlap in your implementations, and I think it's very likely that one of you copied from the other.
As there were some force pushes done by @varshith257, I cannot easily tell right now whose implementation came first.

As both of you are still missing some aspects of the bounty, I ask you to implement the missing features. Do this without copying and without force pushes please. Whoever ends up having the first complete implementation will be awarded the bounty.
In case there are more accusations of copying, we'll use the github push timestamps to decide who was first.
If the push timestamps cannot be used due to force pushes, we'll assume the other person came first.

The following features are missing:

• Using the native ollama api (which implies adding durability for ndjson streams)
• Automated tests as part of the ci pipeline using the smallest possible models