Skip to content

Commit ff86ac9

Browse files
feat: update host multi-agent docs (#1335)
1 parent 6ad55a1 commit ff86ac9

File tree

2 files changed

+148
-0
lines changed

2 files changed

+148
-0
lines changed

content/en/docs/eino/core_modules/flow_integration_components/multi_agent_hosting.md

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -343,3 +343,77 @@ HandOff to answer_with_journal with argument {"reason":"To find out the user's m
343343
Answer:
344344
You got up at 7:00 in the morning.
345345
```
346+
347+
## FAQ
348+
349+
### Host direct answer does not have streaming effect
350+
351+
Host Multi-Agent provides a configuration for `StreamToolCallChecker` to determine whether the Host outputs directly.
352+
353+
Different models may output tool calls in different ways in streaming mode: some models (e.g., OpenAI) output tool calls directly; some models (e.g., Claude) output text first and then output tool calls. Therefore, different methods are needed for the determination. This field is used to specify a function for determining whether the model's streaming output contains tool calls.
354+
355+
It is optional. If not filled, the determination of whether the "non-empty package" contains tool calls is used:
356+
357+
```go
358+
func firstChunkStreamToolCallChecker(_ context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
359+
defer sr.Close()
360+
361+
for {
362+
msg, err := sr.Recv()
363+
if err == io.EOF {
364+
return false, nil
365+
}
366+
if err != nil {
367+
return false, err
368+
}
369+
370+
if len(msg.ToolCalls) > 0 {
371+
return true, nil
372+
}
373+
374+
if len(msg.Content) == 0 { // skip empty chunks at the front
375+
continue
376+
}
377+
378+
return false, nil
379+
}
380+
}
381+
```
382+
383+
The above default implementation is applicable when the Tool Call Message output by the model contains only Tool Calls.
384+
385+
The default implementation is not applicable when there is a non-empty content chunk before outputting the Tool Call. In this case, a custom tool call checker is required as follows:
386+
387+
```go
388+
toolCallChecker := func(ctx context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
389+
defer sr.Close()
390+
for {
391+
msg, err := sr.Recv()
392+
if err != nil {
393+
if errors.Is(err, io.EOF) {
394+
// finish
395+
break
396+
}
397+
398+
return false, err
399+
}
400+
401+
if len(msg.ToolCalls) > 0 {
402+
return true, nil
403+
}
404+
}
405+
return false, nil
406+
}
407+
```
408+
409+
This custom `StreamToolCallChecker` may need to check **all packages** for the presence of ToolCalls in extreme cases, resulting in the loss of the "streaming judgment" effect. If you want to maintain the "streaming judgment" effect as much as possible, the suggested solution is:
410+
411+
Try to add a prompt to constrain the model not to output additional text when making tool calls, for example: "If you need to call a tool, output the tool directly without outputting text."
412+
413+
Different models may be affected by the prompt differently. You need to adjust the prompt and verify the effect in actual use.
414+
415+
### Host picks multiple Specialists simultaneously
416+
417+
The Host provides the selection of Specialists in the form of Tool Calls, so it may select multiple Specialists simultaneously in the form of a Tool Call list. At this time, the Host Multi-Agent will route the request to these multiple Specialists at the same time. After the multiple Specialists complete their tasks, the Summarizer node will summarize multiple Messages into one Message as the final output of the Host Multi-Agent.
418+
419+
Users can customize the behavior of the Summarizer by configuring the Summarizer, specifying a ChatModel and a SystemPrompt. If not specified, the Host Multi-Agent will concatenate the output Message Contents of multiple Specialists and return the result.

content/zh/docs/eino/core_modules/flow_integration_components/multi_agent_hosting.md

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -343,3 +343,77 @@ HandOff to answer_with_journal with argument {"reason":"To find out the user's m
343343
Answer:
344344
You got up at 7:00 in the morning.
345345
```
346+
347+
## FAQ
348+
349+
### Host 直接输出时没有流式
350+
351+
Host Multi-Agent 提供了一个 StreamToolCallChecker 的配置,用于判断 Host 是否直接输出。
352+
353+
不同的模型在流式模式下输出工具调用的方式可能不同: 某些模型(如 OpenAI) 会直接输出工具调用;某些模型 (如 Claude) 会先输出文本,然后再输出工具调用。因此需要使用不同的方法来判断,这个字段用来指定判断模型流式输出中是否包含工具调用的函数。
354+
355+
可选填写,未填写时使用“非空包”是否包含工具调用判断:
356+
357+
```go
358+
func firstChunkStreamToolCallChecker(_ context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
359+
defer sr.Close()
360+
361+
for {
362+
msg, err := sr.Recv()
363+
if err == io.EOF {
364+
return false, nil
365+
}
366+
if err != nil {
367+
return false, err
368+
}
369+
370+
if len(msg.ToolCalls) > 0 {
371+
return true, nil
372+
}
373+
374+
if len(msg.Content) == 0 { // skip empty chunks at the front
375+
continue
376+
}
377+
378+
return false, nil
379+
}
380+
}
381+
```
382+
383+
上述默认实现适用于:模型输出的 Tool Call Message 中只有 Tool Call。
384+
385+
默认实现不适用的情况:在输出 Tool Call 前,有非空的 content chunk。此时,需要自定义 tool Call checker 如下:
386+
387+
```go
388+
toolCallChecker := func(ctx context.Context, sr *schema.StreamReader[*schema.Message]) (bool, error) {
389+
defer sr.Close()
390+
for {
391+
msg, err := sr.Recv()
392+
if err != nil {
393+
if errors.Is(err, io.EOF) {
394+
// finish
395+
break
396+
}
397+
398+
return false, err
399+
}
400+
401+
if len(msg.ToolCalls) > 0 {
402+
return true, nil
403+
}
404+
}
405+
return false, nil
406+
}
407+
```
408+
409+
上面这个自定义 StreamToolCallChecker,在极端情况下可能需要判断**所有包**是否包含 ToolCall,从而导致“流式判断”的效果丢失。如果希望尽可能保留“流式判断”效果,解决这一问题的建议是:
410+
411+
> 💡
412+
> 尝试添加 prompt 来约束模型在工具调用时不额外输出文本,例如:“如果需要调用 tool,直接输出 tool,不要输出文本”。
413+
> 不同模型受 prompt 影响可能不同,实际使用时需要自行调整 prompt 并验证效果。
414+
415+
### Host 同时选择多个 Specialist
416+
417+
Host 以 Tool Call 的形式给出对 Specialist 的选择,因此可能以 Tool Call 列表的形式同时选中多个 Specialist。此时 Host Multi-Agent 会同时将请求路由到这多个 Specialist,并在多个 Specialist 完成后,通过 Summarizer 节点总结多条 Message 为一条 Message,作为 Host Multi-Agent 的最终输出。
418+
419+
用户可通过配置 Summarizer,指定一个 ChatModel 以及 SystemPrompt,来定制化 Summarizer 的行为。如未指定,Host Multi-Agent 会将多个 Specialist 的输出 Message Content 拼接后返回。

0 commit comments

Comments
 (0)