Skip to content

Mixing static_instruction and instruction prevents tool use in Anthropic-family LLMs. #3229

@UlookEE

Description

@UlookEE

** Please make sure you read the contribution guide and file the issues in the right place. **
Contribution guide.

Describe the bug

When static_instruction and instruction are both used, the instruction ends up inserted in the middle of the context. In llm_flow/contents.py, _add_instructions_to_user_content chooses the insertion point using only if llm_request.contents[i].role != "user".

However, FunctionResponse can have the role user (i dont know it is always). With the current reverse scan:

for i in range(len(llm_request.contents) - 1, -1, -1):
    if llm_request.contents[i].role != "user":
        insert_index = i + 1
        break

the chosen insert_index falls between a FunctionCall (assistant/tool_use) and its corresponding FunctionResponse (user/tool_result), producing:

FunctionCall  –  instruction  –  FunctionResponse

This ordering violates Anthropic’s requirements: (https://docs.claude.com/en/docs/agents-and-tools/tool-use/implement-tool-use)

  • Tool result blocks must immediately follow their corresponding tool use blocks.
  • In the user message containing tool results, tool_result blocks must come first, and any text must come after all tool results.

As a result, Claude-based agents fail to invoke tools.


Root cause

  • The insertion heuristic looks only at role != "user", not at semantic boundaries (tool_use ↔ tool_result adjacency).

I’m prepared to contribute a fix to this project.

To Reproduce
Always, use tool with Antrhophic LLM and static_instruction and instruction

venv/lib/python3.13/site-packages/litellm/llms/bedrock/chat/converse_handler.py", line 237, in async_completion
    raise BedrockError(status_code=error_code, message=err.response.text)
litellm.llms.bedrock.common_utils.BedrockError: {"message":"The number of toolResult blocks at messages.4.content exceeds the number of toolUse blocks of previous turn."}

Metadata

Metadata

Assignees

Labels

models[Component] Issues related to model support

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions