Skip to content

feat: OllamaChatGenerator - add Toolset support #1765

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
May 22, 2025
Merged

feat: OllamaChatGenerator - add Toolset support #1765

merged 8 commits into from
May 22, 2025

Conversation

vblagoje
Copy link
Member

@vblagoje vblagoje commented May 19, 2025 •
edited
Loading

Why:

Enhance OllamaChatGenerator by allowing it to accept a Toolset instance in addition to a list of Tool objects. This brings it in line with other chat generators and improves consistency and flexibility in tool usage.

What:

  • Updated OllamaChatGenerator to support tools parameter as either a List[Tool] or a Toolset
  • Adjusted serialization logic to handle Toolset using serialize_tools_or_toolset
  • Modified deserialization to support both formats using deserialize_tools_or_toolset_inplace

How can it be used:

The OllamaChatGenerator can now be used with a Toolset for better tool management and compatibility across generators:

toolset = Toolset(tools)
generator = OllamaChatGenerator(model="qwen3:4b", tools=toolset)

It also allows seamless deserialization and execution with Toolset as part of the component configuration.

How did you test it:

  • Added tests for:

    • Initialization with Toolset
    • Serialization and deserialization of Toolset-based components
    • run() execution flow using a Toolset and mocked response from Ollama client
    • Verified backward compatibility with plain list of tools

Manually:

  • I've tested qwen3:4b with itinerary agent and bunch of MCPToolset and it kinda worked. Well in some basic form expected from such a relatively small model running on my local CPU.

Notes for the reviewer:

Please confirm:

  • Serialization/deserialization logic is sound and consistent with other generators
  • ToolCall.id fallback generation doesn't break traceability or downstream logic
  • Toolset support is fully covered by new test cases without affecting existing behavior

Note:

@github-actions github-actions bot added integration:ollama type:documentation Improvements or additions to documentation labels May 19, 2025
@vblagoje vblagoje marked this pull request as ready for review May 21, 2025 08:09
@vblagoje vblagoje requested a review from a team as a code owner May 21, 2025 08:09
@vblagoje vblagoje requested review from mpangrazzi and anakin87 and removed request for a team May 21, 2025 08:09
@vblagoje
Copy link
Member Author

vblagoje commented May 21, 2025 •
edited
Loading

@mpangrazzi let's have @anakin87 look at these changes - he's familiar with the code and the context of #1780

@vblagoje vblagoje removed the request for review from mpangrazzi May 21, 2025 08:10
@sjrl
Copy link
Contributor

sjrl commented May 21, 2025

@vblagoje lets make sure to pin Haystack in the deps to when we added Toolset

@anakin87
Copy link
Member

anakin87 commented May 21, 2025 •
edited
Loading

Looks good in general.

I would only update this aspect in the PR description

Automatically generates UUIDs for ToolCall.id if missing (e.g., in Ollama tool responses)

I understand that this is something you had to introduce to make Langfuse work, but in the current version of this PR we are not introducing this change.

@vblagoje
Copy link
Member Author

Looks good in general.

I would only update this aspect in the PR description

Automatically generates UUIDs for ToolCall.id if missing (e.g., in Ollama tool responses)

I understand that this is something you had to introduce to make Langfuse work, but in the current version of this PR we are not introducing this change.

Right, spot on! I had to remove it pending a proper resolution of #1780

anakin87
anakin87 requested changes May 21, 2025
View reviewed changes
Copy link
Member

@anakin87 anakin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from pinning Haystack (see #1765 (comment)), looks good

@vblagoje
Copy link
Member Author

Apart from pinning Haystack (see #1765 (comment)), looks good

It is already @sjrl and @anakin87 . See the project toml

anakin87
anakin87 approved these changes May 22, 2025
View reviewed changes
Copy link
Member

@anakin87 anakin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, sorry. Looks good!

@vblagoje vblagoje merged commit 9c3b13c into main May 22, 2025
6 checks passed
@vblagoje vblagoje deleted the ollama_chat_generator_toolset branch May 22, 2025 07:24
Amnah199 pushed a commit that referenced this pull request Jun 4, 2025
@vblagoje @Amnah199
feat: OllamaChatGenerator - add Toolset support (#1765)
02fc098 
* Add Toolset support to OllamaChatGenerator

* Lint

* Lambdas are not serializable

* Lint

* Generate tool call id if not available

* Lint

* Revert back to not using ToolCall id

* Lint
Amnah199 added a commit that referenced this pull request Jun 4, 2025
@Amnah199 @denisw
feat: Allow full metadata field customization (#1893)
06345b2 
* feat(azure-ai-search): Allow full metadata field customization

So far, the `metadata_fields` init parameter only allowed a few custom
simple value types to be mapped (e.g., no nested metadata) and also
hardcoded the fields to be only `filterable` (but not `searchable`
or `facetable`, for instance).

For full flexibility, allow an Azure AI Search `SearchField` instance
to be passed as mapping instead of a Python type.

* PR comments

* feat: Add OpenRouter integration (#1723)

* Add openrouter integration

* Add tests for chat generator and support extra headers

* Add async tests

* Fix config files

* Add example

* Fixes

* Fix read me

* PR comments

* Small fixes

* Updated labeler and README

* Update docstrings

* Add user agent to Azure AI Search(#1743)

* docs: update changelog for integrations/azure_ai_search (#1745)

* Update changelog for integrations/azure_ai_search

* Update CHANGELOG.md

---------


Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* docs: ChatMessage examples (#1752)

* feat: Support Llama API as a Chat Generator (#1742)

* init: llama-api chat generator

* docs: update comments for LlamaChatGenerator

* feat: add keyword only *

* fix: replace streaming_callback type

* fix: add Toolset for tools

* fix: rm unused typing

* docs: add meta header

* docs: fix comments to llama api

* docs: add meta header

* docs: add meta header

* fix: rename LlamaChat to MetaLlamaChat

* docs: add meta header

* docs: align doc format

* add workflow for nightly tests

* add meta_llama to labeler

* add new integration to repo readme overview table

* replace .llama.chat. with .meta_llama.chat.

* fmt

* replace llama with meta_llama in pydocs

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update changelog for integrations/meta_llama (#1754)

Co-authored-by: julian-risch <4181769+julian-risch@users.noreply.github.com>

* chore(deps): bump fossas/fossa-action from 1.6.0 to 1.7.0 (#1750)

Bumps [fossas/fossa-action](https://github.com/fossas/fossa-action) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/fossas/fossa-action/releases)
- [Commits](fossas/fossa-action@v1.6.0...v1.7.0)

---
updated-dependencies:
- dependency-name: fossas/fossa-action
  dependency-version: 1.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update how skipping works (#1756)

* test: Ollama - make test_run_with_response_format more robust (#1757)

* feat: adapt `OllamaGenerator` metadata to OpenAI format (#1753)

* feat: adapt Ollama metadata to OpenAI format in `OllamaGenerator`

* Add: `OllamaGenerator` support in Langfuse

* Ran Linters

* Revert "Add: `OllamaGenerator` support in Langfuse"

This reverts commit 1f399e0.

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>

* Add: `OllamaGenerator` support in Langfuse (#1759)

* Update changelog for integrations/ollama (#1761)

Co-authored-by: sjrl <10526848+sjrl@users.noreply.github.com>

* Update changelog for integrations/langfuse (#1762)

Co-authored-by: sjrl <10526848+sjrl@users.noreply.github.com>

* docs: update changelog for integrations/openrouter (#1763)

* Update changelog for integrations/openrouter


---------

Co-authored-by: Amnah199 <13835656+Amnah199@users.noreply.github.com>
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* chore: fix README for meta-llama (#1766)

* chore(deps): bump aws-actions/configure-aws-credentials (#1751)

Bumps [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials) from 4.2.0 to 4.2.1.
- [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases)
- [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md)
- [Commits](aws-actions/configure-aws-credentials@f24d719...b475783)

---
updated-dependencies:
- dependency-name: aws-actions/configure-aws-credentials
  dependency-version: 4.2.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* ci: Bedrock - improve worfklow; skip tests from CI (#1773)

* feat: OllamaChatGenerator - add Toolset support (#1765)

* Add Toolset support to OllamaChatGenerator

* Lint

* Lambdas are not serializable

* Lint

* Generate tool call id if not available

* Lint

* Revert back to not using ToolCall id

* Lint

* Update changelog for integrations/ollama (#1775)

Co-authored-by: vblagoje <458335+vblagoje@users.noreply.github.com>

* feat: MCPTool and MCPToolset async resource management improvements (#1758)

* Add MCPClientSessionManager to connect/close mcp clients

* Update and refactor mcp tests

* More descriptive connection error raising

* Proper test cleanup

* Testing CI windows

* linting

* Improve connection error raise

* PR feedback

* Proper naming, and more precise cleanup sequence

---------

Co-authored-by: Michele Pangrazzi <xmikex83@gmail.com>

* test: add service_tier to test_convert_anthropic_chunk_to_streaming_chunk (#1778)

* fix: Bring Mistral integration up to date with changes made to OpenAIChatGenerator and OpenAI Embedders (#1774)

* Bringing Mistral up to date

* Fix Mistral Embedders to be deserializable

* Fix lint

* Fix lint

* Bump minimum haystack version

* Update changelog for integrations/mistral (#1781)

Co-authored-by: sjrl <10526848+sjrl@users.noreply.github.com>

* feat: Add `to_dict` to `STACKITDocumentEmbedder` and `STACKITTextEmbedder` and more init parameters from underlying OpenAI classes (#1779)

* Add to_dicts and more tests

* Bumpy haystack version

* Add changes to chat generator as well

* Update changelog for integrations/stackit (#1782)

Co-authored-by: sjrl <10526848+sjrl@users.noreply.github.com>

* feat: add run_async for CohereChatGenerator (#1689)

* CohereChatGenerator async support

* Tests and linter fixes

* fix

* refinements

* refactor + tests reorgani    |             ^^^^^ T201

* rename test

* remove markers

* reformat

* fix

* minor fixes

* Trigger CI

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>

* Update changelog for integrations/cohere (#1784)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* docs: update changelog for integrations/google_ai (#1812)

* Update changelog for integrations/google_ai

* Update CHANGELOG.md

---------

Co-authored-by: wochinge <7667273+wochinge@users.noreply.github.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* fix: Fix exposing Qdrant api-key in `metadata` field when running `to_dict` (#1813)

* Add to_dict test

* Add more type hints

* More type hints

* Add fix for exposing api key in metadata when running to_dict

* Add unit test

* PR comments

* Update changelog for integrations/qdrant (#1814)

Co-authored-by: sjrl <10526848+sjrl@users.noreply.github.com>

* ci: check lowest direct dependencies (#1788)

* ci: check lowest direct dependencies

* try single quotes

* debug

* debugging

* try chroma

* no bedrock

* retry

* explicit option

* don't run tests

* debug 1

* try output file

* more

* no deepeval

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* build: add pins for Anthropic (#1811)

* build: add pins for Anthropic

* rm file incorrectly added

* Update changelog for integrations/anthropic (#1815)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* build: add pins for Vertex (#1810)

* Update changelog for integrations/google_vertex (#1816)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* build: add pins for Cohere (#1817)

* Update changelog for integrations/cohere (#1829)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* build: remove pin for Deepeval (#1826)

* Update changelog for integrations/deepeval (#1830)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* feat: Add streamable-http transport MCP support (#1777)

* Add streamable-http transport

* Improve error message for tool invocation

* Add streamable MCPTool example, update examples

* Improve examples

* Add unit tests

* Update integrations/mcp/examples/mcp_client.py

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* initialize vars outside try block

* Small fix

* Fix linting

---------

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Update changelog for integrations/mcp (#1831)

Co-authored-by: vblagoje <458335+vblagoje@users.noreply.github.com>

* build: pining lower versions of haystack and `aiohttp` for `ElasticSearch` (#1827)

* pining lower versions

* adding missing comma

* pinning to >=2.4.0

* pinning to >=2.3.0

* pinning aiohttp to >=3.0.0

* pinning aiohttp to >=2.0.0

* pinning aiohttp to >=2.5.0

* pinning aiohttp to >=2.6.0

* pinning aiohttp to >=3.0.0

* pinning aiohttp to >=3.1.0

* pinning aiohttp to >=3.2.0

* pinning aiohttp to >=3.3.0

* pinning aiohttp to >=3.10.0

* pinning aiohttp to >=3.9.0

* pinning aiohttp to >=3.8.0

* reverting back aiohttp to 3.9.0

* Update changelog for integrations/elasticsearch (#1834)

Co-authored-by: davidsbatista <7937824+davidsbatista@users.noreply.github.com>

* build: add Jina pins (#1836)

* Update changelog for integrations/jina (#1838)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* build: add Langfuse pins (#1837)

* Update changelog for integrations/langfuse (#1839)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* build: pin version for `pymongo` and `haystack` in MongoDB integration (#1832)

* pinning to older version o haystack and mongodb

* pinining haystack and pymongo

* wip

* fixing format

* adding missing CI job

* making sure lowest version of pymongo has the async client

* making sure lowest version of pymongo has the async client

* versioning

* haysack 2.9

* haysack 2.10

* haysack 2.11

* Remove failing test. No need to have it here since it's already tested in haystack main. (#1842)

* ci: Missing labels for stackit and anthropic (#1844)

* Missing labels for stackit and anthropic

* PR comments

* build: app pins for MCP (#1845)

* Update changelog for integrations/mongodb_atlas (#1840)

Co-authored-by: davidsbatista <7937824+davidsbatista@users.noreply.github.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>

* docs: update changelog for integrations/mcp (#1848)

* Update changelog for integrations/mcp

* Update CHANGELOG.md

---------

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* build: add pins for Pgvector (#1849)

* Update changelog for integrations/pgvector (#1850)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* build: add pins for Optimum (#1847)

* build: add pins for Optimum

* try with python 3.13

* don't call HF on unit tests

* Update changelog for integrations/optimum (#1852)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* build: add pins for Qdrant (#1853)

* build: add pins for Pinecone (#1851)

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* Update changelog for integrations/pinecone (#1855)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* docs: update changelog for integrations/qdrant (#1856)

* Update changelog for integrations/qdrant

* Update CHANGELOG.md

---------

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* chore: review license compliance workflow (#1843)

* chore: review license compliance workflow

* refactor

* deepeval

* build: add pins for Ragas (#1854)

* feat: Add GitHub integration with components, tools, and prompts (#1637)

* add agent_prompts and github_components

* rename to github_haystack

* remove github-haystack

* renamed integration, added components dir

* add tests, pydoc, update pyproject.toml

* add workflow

* fmt

* fmt

* lint

* ruff

* fmt

* lint:all

* replace StrEnum for py 3.9+ compatibility

* move files

* fix tests

* lint

* fix pydoc and extend init files

* Add integration:github to labeler.yml

* unify how we set GITHUB_TOKEN in tests

* fix 3 usage examples. 3 remaining

* remove empty lines from prompts

* GitHub capitalization

* add license header

* all caps for prompts

* add GitHubFileEditorTool

* enforce kwargs instead of positional args

* use _get_request_headers and base_headers consistently

* lint

* rename GitHubRepositoryViewer to GitHubRepoViewer

* lint

* add pipeline serialization test

* extend pipeline to_dict test

* set default branch of repo viewer

* lint

* add four more tools

* lint

* rename prompts

* add tests for four more tools

* rename context prompt

* add outputs_to_state as param to GitHubFileEditorTool

* add outputs_to_state as param to GitHubRepoViewerTool

* set default outputs_to_state for GitHubRepoViewerTool

* extract serialize_handlers to utils; don't use mutable defaults

* replace init_parameters with data for serde in FileEditor, RepoViewer

* add outputs_to_state to GitHubIssueCommenterTool; replace init_parameters with data

* add outputs_to_state to GitHubIssueViewerTool; replace init_parameters with data

* add outputs_to_state to GitHubPRCreatorTool; replace init_parameters with data

* move param docstrings to init methods

* use generate_qualified_class_name instead of hardcoded name

* test with lowest supported version

* don't test http_client_kwargs for compatibility with Haystack 2.12

* build: pinning `unstructured` to lowest working versions (#1841)

* finding lowest working versions

* adding missing CI job

* adding missing limitation

* feat: AnthropicChatGenerator - add Toolset support (#1787)

* AnthropicChatGenerator - add Toolset support

* Use new serialization method for tools

* Update haystack dep to 2.13.1 which includes Toolset

* Small update

* buid: add pins for Snowflake + small refactoring (#1860)

* Update changelog for integrations/snowflake (#1862)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* Update changelog for integrations/ragas (#1857)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* Update changelog for integrations/unstructured (#1861)

Co-authored-by: davidsbatista <7937824+davidsbatista@users.noreply.github.com>

* build: add pins for Nvidia (#1846)

* Update changelog for integrations/nvidia (#1863)

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>

* build: add pins for Google AI (#1828)

* docs: update changelog for integrations/google_ai (#1864)

* Update changelog for integrations/google_ai

* Update CHANGELOG.md

---------

Co-authored-by: anakin87 <44616784+anakin87@users.noreply.github.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update changelog for integrations/anthropic (#1865)

Co-authored-by: vblagoje <458335+vblagoje@users.noreply.github.com>

* docs: Update changelog for integrations/github (#1858)

Co-authored-by: julian-risch <4181769+julian-risch@users.noreply.github.com>

* feat: adding an `HybridRetriever` as a `Supercomponent` having `OpenSearch` as the document store (#1701)

* adding tests

* linting and typing

* adding env variable

* env variable

* extending docstring

* removing generation part

* updating tests

* adding a run test with mocked sentence_transformers

* fixing format

* refactor: use `component_to_dict` in OpenSearchHybridRetriever (#1866)

* Update changelog for integrations/opensearch (#1867)

Co-authored-by: davidsbatista <7937824+davidsbatista@users.noreply.github.com>

* oshr-docs (#1868)

* refactor: OpenSearchHybridRetriever use `deserialize_chatgenerator_inplace` (#1870)

* test to use deserialize_chatgenerator_inplace

* removing unused imports

* using deserialize_chatgenerator_inplace

* Update integrations/opensearch/src/haystack_integrations/components/retrievers/opensearch/open_search_hybrid_retriever.py

* Update changelog for integrations/opensearch (#1874)

Co-authored-by: davidsbatista <7937824+davidsbatista@users.noreply.github.com>

* feat: add run_async support for CohereTextEmbedder (#1873)

* feat: add run_async support for CohereTextEmbedder

* fix: review comments

* feat: Add Google GenAI GoogleGenAIChatGenerator (#1875)

* Initial work

* Remove utils

* Add async support

* Async test issue

* Simplify async tests

* Linting

* Improve comment

* Linting

* Improve pyproject.toml

* Add new google genai integration to workflow

* Add labeler

* Add pydoc

* Pin deps

* Pin google-genai dep

* Update integrations/google_genai/src/haystack_integrations/components/generators/google_genai/chat/chat_generator.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* Update integrations/google_genai/src/haystack_integrations/components/generators/google_genai/chat/chat_generator.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* PR feedback

* Add system message comment

* Leave only minimal working examples in README

* Update integrations/google_genai/src/haystack_integrations/components/generators/google_genai/chat/chat_generator.py

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update integrations/google_genai/src/haystack_integrations/components/generators/google_genai/chat/chat_generator.py

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Linting

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update changelog for integrations/google_genai (#1886)

Co-authored-by: vblagoje <458335+vblagoje@users.noreply.github.com>

* feat: Use Langfuse local to_openai_dict_format function to serialize messages (#1885)

* Use Langfuse local to_openai_dict_format function to serialize messages

* Linting

* PR feedback

* Add detailed tracing for GoogleGenAIChatGenerator (#1887)

* docs: update changelog for integrations/langfuse (#1888)

* Update changelog for integrations/langfuse

* Update CHANGELOG.md

---------

Co-authored-by: vblagoje <458335+vblagoje@users.noreply.github.com>
Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>

* try reenabling pinecone tests (#1871)

* PR comments

* Small updates

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Denis Washington <denis.washington@futurice.com>
Co-authored-by: Denis Washington <denis@denisw.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anakin87 anakin87 anakin87 approved these changes

Assignees
No one assigned
Labels
integration:ollama type:documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add Toolset support in OllamaChatGenerator
3 participants
@vblagoje @sjrl @anakin87