Skip to content

Conversation

@matmorel
Copy link

@matmorel matmorel commented Jul 23, 2025

Addresses one of the issues raised in #686

Problem

Ollama LLM can return chunk in parts when it's too long.

Solution

Improve the json_responses_chunk_handler method to handle incomplete JSON chunks in stream. If a chunk does not end with }, it is considered incomplete and buffered until the next chunk arrives. This prevents JSON parsing errors and ensures all responses are processed correctly.

This PR is heavily inspired from #995 by @berkcaputcu, my implementation doesn't rely on exceptions and also includes additional specs.

@matmorel matmorel changed the title Ollama handle multiline JSON Ollama fix handle multiline JSON Jul 23, 2025
@matmorel matmorel changed the title Ollama fix handle multiline JSON Ollama fix handle incomplete JSON chunks in stream Jul 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant