-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Description
** Please make sure you read the contribution guide and file the issues in the right place. **
Contribution guide.
Describe the bug
When static_instruction and instruction are both used, the instruction ends up inserted in the middle of the context. In llm_flow/contents.py, _add_instructions_to_user_content chooses the insertion point using only if llm_request.contents[i].role != "user".
However, FunctionResponse can have the role user (i dont know it is always). With the current reverse scan:
for i in range(len(llm_request.contents) - 1, -1, -1):
if llm_request.contents[i].role != "user":
insert_index = i + 1
breakthe chosen insert_index falls between a FunctionCall (assistant/tool_use) and its corresponding FunctionResponse (user/tool_result), producing:
FunctionCall – instruction – FunctionResponse
This ordering violates Anthropic’s requirements: (https://docs.claude.com/en/docs/agents-and-tools/tool-use/implement-tool-use)
- Tool result blocks must immediately follow their corresponding tool use blocks.
- In the user message containing tool results, tool_result blocks must come first, and any text must come after all tool results.
As a result, Claude-based agents fail to invoke tools.
Root cause
- The insertion heuristic looks only at
role != "user", not at semantic boundaries (tool_use ↔ tool_result adjacency).
I’m prepared to contribute a fix to this project.
To Reproduce
Always, use tool with Antrhophic LLM and static_instruction and instruction
venv/lib/python3.13/site-packages/litellm/llms/bedrock/chat/converse_handler.py", line 237, in async_completion
raise BedrockError(status_code=error_code, message=err.response.text)
litellm.llms.bedrock.common_utils.BedrockError: {"message":"The number of toolResult blocks at messages.4.content exceeds the number of toolUse blocks of previous turn."}