Skip to content

Add straming_callback to run() method of OllamaGenerator #315

@r-gg-b

Description

@r-gg-b

Currently, you can only set the streaming_callback on OllamaGenerator through the init method, which sets it for the entire object. This is problematic if you aim to re-use this object to handle concurrent requests, because the streaming callback typically references different chunk collectors (for different requests).

As such one cannot have concurrent processing of requests by the same pipeline.

Suggested implementation:

Just add streaming_callback argument to the run() method, so that it can be dynamically set with each request. (like OpenAIGenerator for instance)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions