Test and Update Agents Module #122
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![codegen-sh[bot]](https://avatars.githubusercontent.com/in/321279?s=60&v=4){kind=small}
# Motivation The **Codegen on OSS** package provides a pipeline that: - **Collects repository URLs** from different sources (e.g., CSV files or GitHub searches). - **Parses repositories** using the codegen tool. - **Profiles performance** and logs metrics for each parsing run. - **Logs errors** to help pinpoint parsing failures or performance bottlenecks. <!-- Why is this change necessary? --> # Content <!-- Please include a summary of the change --> see [codegen-on-oss/README.md](https://github.com/codegen-sh/codegen-sdk/blob/acfe3dc07b65670af33b977fa1e7bc8627fd714e/codegen-on-oss/README.md) # Testing <!-- How was the change tested? --> `uv run modal run modal_run.py` No unit tests yet 😿 # Please check the following before marking your PR as ready for review - [ ] I have added tests for my changes - [x] I have updated the documentation or added new documentation as needed
Original commit by Tawsif Kamal: Revert "Revert "Adding Schema for Tool Outputs"" (codegen-sh#894) Reverts codegen-sh#892 --------- Co-authored-by: Rushil Patel <rpatel@codegen.com> Co-authored-by: rushilpatel0 <171610820+rushilpatel0@users.noreply.github.com>
Original commit by Ellen Agarwal: fix: Workaround for relace not adding newlines (codegen-sh#907)
Reviewer's GuideThis PR introduces a full suite of unit and integration tests for every component of the Agents module (including langchain extensions), enriches module and README documentation with comprehensive usage examples and API references, and updates release configuration metadata. Sequence Diagram for Basic Agent Interaction (Agent.run)sequenceDiagram
actor User
participant A as Agent
participant API as CodegenAPI
participant AT as AgentTask
User->>A: __init__(token="your_api_token", org_id=123)
User->>A: run(prompt="Which github repos can you currently access?")
A->>API: POST /agent_run (payload: prompt)
API-->>A: JobDetails { id: "job_xyz", status: "queued" }
A->>AT: __init__(id="job_xyz", status="queued", agent_ref=A)
A-->>User: agent_task (AgentTask instance)
User->>AT: refresh()
AT->>API: GET /job_status (id="job_xyz")
API-->>AT: JobDetails { id: "job_xyz", status: "completed", result: "Accessible repos..." }
note right of AT: Updates internal status and result
User->>AT: result (accesses updated result)
Sequence Diagram for CodeAgent Interaction (CodeAgent.run)sequenceDiagram
actor User
participant CoA as CodeAgent
participant CB as Codebase
participant ACfg as AgentConfig
participant LLMS as LLMService
User->>ACfg: __init__(keep_first_messages=2, ...)
User->>CoA: __init__(codebase=CB, agent_config=ACfg, ...)
User->>CoA: run(prompt="Create a function to parse JSON", image_urls=["img1.png"])
CoA->>LLMS: Generate code (prompt, images, config)
note right of CoA: May use tools, manage state, apply config
LLMS-->>CoA: Generated code result
CoA-->>User: "def parse_json(...): ..."
User->>CoA: get_agent_trace_url()
CoA-->>User: "https://langsmith.com/.../trace_id"
Sequence Diagram for Message Tracing with MessageStreamTracersequenceDiagram
actor Developer
participant MessageSource as Agent/Component
participant Stream as Message Stream
participant Tracer as MessageStreamTracer
participant Logger as CustomLogger
Developer->>Logger: new CustomLogger()
Developer->>Tracer: new MessageStreamTracer(logger=Logger)
Developer->>MessageSource: operation_yielding_messages()
MessageSource-->>Stream: yield UserMessage("Hello")
MessageSource-->>Stream: yield AIMessage("Response part 1")
loop For each message in stream
Developer->>Tracer: process_stream(Stream)
Tracer->>Stream: next()
Stream-->>Tracer: message_chunk (e.g., UserMessage)
Tracer->>Logger: log(message_chunk)
Tracer-->>Developer: processed_message_chunk
end
Developer->>Tracer: get_traces()
Tracer-->>Developer: [trace_data_1, trace_data_2]
Class Diagram for Agent Hierarchy and Core ComponentsclassDiagram
class Agent {
+token: str
+org_id: int
+base_url: str
+run(prompt: str) AgentTask
+get_status() dict
}
class AgentTask {
+id: str
+org_id: int
+status: str
+result: any
+refresh() None
}
class ChatAgent {
+codebase: Codebase
+model_provider: str
+model_name: str
+memory: bool
+run(prompt: str, thread_id: str) str
+chat(prompt: str, thread_id: str) tuple[str, str]
+get_chat_history(thread_id: str) list
}
class CodeAgent {
+codebase: Codebase
+model_provider: str
+model_name: str
+memory: bool
+agent_config: AgentConfig
+logger: ExternalLogger
+run(prompt: str, image_urls: list[str]) str
+get_agent_trace_url() str
+get_tools() list[BaseTool]
+get_state() dict
}
class Codebase {
<<Dependency>>
# Path or other initialization params
}
class AgentConfig {
<<Configuration>>
# Attributes like keep_first_messages, max_messages
}
class ExternalLogger {
<<Interface>>
# Methods like log()
}
class BaseTool {
<<Dependency>>
# Name or other attributes
}
Agent "1" -- "1" AgentTask : creates/manages
ChatAgent ..> Codebase : uses
CodeAgent ..> Codebase : uses
CodeAgent ..> AgentConfig : uses (optional)
CodeAgent ..> ExternalLogger : uses (optional)
CodeAgent ..> BaseTool : uses (optional)
Class Diagram for Agent Configuration (AgentConfig)classDiagram
class AgentConfig {
<<TypedDict>>
+keep_first_messages: int
+max_messages: int
}
note for AgentConfig "Configuration for CodeAgent behavior, such as message summarization rules."
Class Diagram for Message Tracing and Logging UtilitiesclassDiagram
class MessageStreamTracer {
+logger: ExternalLogger
+process_stream(message_stream: Iterable) Iterable
+get_traces() list
}
class ExternalLogger {
<<Interface>>
+log(data: Any) None
}
class CustomLogger {
+logs: list
+log(data: Any) None
}
class UserMessage {
+content: str
# Potentially other attributes like 'type'
}
class AIMessage {
# Similar to UserMessage, represents AI response part
+content: str
}
MessageStreamTracer o-- "1" ExternalLogger : uses
CustomLogger ..|> ExternalLogger : implements
MessageStreamTracer ..> UserMessage : processes
MessageStreamTracer ..> AIMessage : processes
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Join our Discord community for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
I see a check failed - I'm on it! 🫡 |
|
✅ Fixed the failing mypy checks in this commit. The issues were related to type annotations in the agents module:
All mypy checks should now pass successfully. 💻 View my work • React 👍 or 👎 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This PR implements comprehensive tests and documentation updates for the Agents module in the Codegen SDK, addressing the requirements in issue ZAM-409.
Changes
Added Tests
test_agent.py: Enhanced existing tests for theAgentclasstest_chat_agent.py: New tests for theChatAgentclasstest_code_agent.py: New tests for theCodeAgentclasstest_data.py: New tests for the message data classestest_loggers.py: New tests for theExternalLoggerprotocoltest_tracer.py: New tests for theMessageStreamTracerclasstest_utils.py: New tests for theAgentConfigTypedDicttest_constants.py: New tests for the constantstest_agents_module.py: Integration tests for the module as a wholetest_agent.py(langchain extension): Tests for the langchain agent extensionDocumentation Updates
__init__.pywith usage examplesREADME.mdwith comprehensive documentation:Code Improvements
AgentclassChatAgentandCodeAgentTesting
All tests pass successfully, providing good coverage of the Agents module functionality.
Related Issues
Closes ZAM-409
💻 View my work • About Codegen
Summary by Sourcery
Implement comprehensive test coverage for the Agents module, enhance documentation with detailed usage and API references, fix type hints and error handling, and expose new agent classes in the main module exports.
New Features:
Bug Fixes:
Enhancements:
Build:
Documentation:
Tests: