🐛 Toolcalls and Text in one response

[Lommix]

Bug description

Hello,

I am currently building an agent TUI on top of this wonderful crate and noticed, that each response can only be either a tool call or a text. But as far as I am aware, many APIs can send a message and tool call in the same response. This is critical, when ordering the agents to comment each tool call for better precision.

This should be true for gemini and claude.

Adapter

gemini

Model

No response

Suggested Resolution

No response

[jeremychone]

Thanks for reporting this.

Yes, this is a limitation of the current API that needs to be addressed.

The good news is we are in the process of a version update, so the timing is good.

I’ll probably do a live session about this. In short, the current plan is to have ChatResponse.content: Vec<MessageContent> and keep or add some convenient functions to get the text version.

P.S. I changed it from Bug to Feature since this is more like a new feature, which some might argue should have been there from the start.

[Lommix]

Sounds good!

This is by far the most important feature (in my opinion) to push this crate to the top.
Currently, complex agent workflows are just not possible without this.

Let me know if you need any help.

[jeremychone]

@Lommix That makes sense. I didn't add it at first to keep things simple, but now it's time to include it.

If you could provide a simple example of a prompt using either a Gemini or OpenAI model to reproduce this, that would be great.

[Lommix]

I don't have a guarantee prompt. But it should be pretty simple.
Just state the following in the system prompt:

With every tool call you make, you must add a comment explaining what the tool call is doing and why you are using it. This
comment should be concise and informative.

Did a quick test. Works great with gemini

[jeremychone]

@Lommix Perfect!