fix: prevent codebase_investigator schema validation infinite retry loop#23113
fix: prevent codebase_investigator schema validation infinite retry loop#23113kunal-10-cloud wants to merge 4 commits intogoogle-gemini:mainfrom
Conversation
…oop (google-gemini#17648) - Add explicit input interface section to agent system prompt - Implement pre-validation with MAX 3 retries per turn in local-executor - Enhance error messages with missing fields and valid examples in subagent-tool - Prevents API quota exhaustion from infinite validation failures
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a critical issue where the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces a robust mechanism to prevent infinite retry loops during agent schema validation by adding pre-validation with a retry limit, and also improves error messages and clarifies input requirements for the codebase_investigator agent. However, a critical bug was identified where the wrong property name is used to access the tool's schema, rendering the validation logic ineffective and leaving the system vulnerable to Denial of Service (DoS) and API quota exhaustion. Additionally, the review addresses code duplication and fixes other bugs in the new validation logic to ensure it works as intended and improves maintainability.
I am having trouble creating individual review comments. Click here to see my feedback.
packages/core/src/agents/local-executor.ts (80-85)
The regular expression in extractMissingFieldsFromError does not correctly parse the error messages from the ajv schema validator. The ajv.errorsText() method produces messages like params must have required property 'objective', which your regex /'([^']+)'\s*(?:is required|must be defined)/gi will not match.
Additionally, the logic to extract the field name from the match is incorrect. m.replace(/['"]/g, '').trim() on a match like "'objective' is required" would result in "objective is required", not just "objective".
A more robust approach would be to parse the structured error array from ajv before it's converted to a string. If you must parse the string, the regex needs to be corrected for ajv's output.
function extractMissingFieldsFromError(error: string): string[] {
// Match patterns like "must have required property 'objective'" from ajv
const requiredPropertyRegex = /must have required property '([^']+)'/gi;
const matches = [...error.matchAll(requiredPropertyRegex)];
return matches.map((match) => match[1]);
}
packages/core/src/agents/local-executor.ts (375)
A critical bug exists in the pre-validation logic: it incorrectly attempts to access the tool's schema using inputSchema instead of the correct parameterSchema property used by DeclarativeTool and its subclasses. This oversight means toolSchema will always be undefined, effectively bypassing validation for all tools. Consequently, the intended mitigation against infinite retry loops and API quota exhaustion is ineffective, leaving the system vulnerable to Denial of Service (DoS) if a model repeatedly generates invalid tool calls. Additionally, using (tool as any).inputSchema is not type-safe and could lead to runtime errors.
const toolSchema = (tool as any).parameterSchema;
packages/core/src/agents/local-executor.ts (77-115)
The helper functions extractMissingFieldsFromError and createExampleInput are duplicated in packages/core/src/agents/subagent-tool.ts. This violates the DRY (Don't Repeat Yourself) principle and can lead to maintenance issues. For instance, a bug identified in extractMissingFieldsFromError in this file is also present in the duplicated version. Please refactor these functions into a shared utility file, for example, in packages/core/src/utils/, to ensure they are defined in a single place.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request effectively addresses an infinite retry loop in the codebase_investigator agent by introducing pre-validation of tool arguments. The approach of adding a retry limit per turn is a solid strategy to prevent API quota exhaustion. The improved error messages will also greatly aid in debugging. My review includes two high-severity suggestions. First, I've noted duplicated helper functions for error parsing, which should be refactored into a shared utility for better maintainability. I've also identified a bug in the error parsing logic and suggested a fix. Second, there's an unsafe property access on the tool object to retrieve its schema, which could lead to runtime errors; I've provided a safer alternative, aligning with the rule for correct toolRequest.args parsing.
Note: Security Review did not run due to the size of the PR.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
@jacob314, @SandyTao520 , @abhipatel12, @jerop, @mattKorwel, @gsquared94, @sehoon38, @bdmorgan, could you please review this pr and let me know if any changes are required |
Summary
Fixes infinite schema validation retry loop in codebase_investigator agent. When the required
objectiveparameter is missing, the model enters an infinite validation cycle, exhausting API quota. This fix adds pre-validation with retry limiting (max 3 per turn) and enhanced error messages showing exactly what fields are missing and how to provide them.Details
Root Cause
The 380+ line system prompt obscures input interface requirements, causing the model to fail to recognize that
objectiveis mandatory. Validation failures retry silently and indefinitely.Three-Phase Solution
Phase 1: Clarify Agent Configuration (codebase-investigator.ts)
CRITICAL: Input Interfacesection at START of system promptPhase 2: Pre-Validation with Retry Limiting (local-executor.ts)
MAX_VALIDATION_RETRIES_PER_TURN = 3constantPhase 3: Enhanced Error Messages (subagent-tool.ts)
Design Decisions
Related Issues
Fixes #17648
How to Validate
1. Unit Tests
2.Linting
3. Manual Testing
-Invoke codebase_investigator without objective parameter
-Verify error message shows: MISSING FIELDS, EXPECTED SCHEMA, VALID EXAMPLE
-Verify agent doesn't retry infinitely (max 3 retries per turn)
4. Integration Testing
-Missing objective field: should error after 3 attempts with helpful message
-Malformed input: should show expected schema
-Valid input: should process normally
Pre-Merge Checklist