Feature Request: Enhanced Cost Efficiency with Scenario Recording and Replay #16
Description
First and foremost, a massive congratulations and thank you to the developers for creating this mind-blowing tool. The "Web Agent" is an incredible contribution, and its potential is truly inspiring.
I would like to propose a feature that could significantly enhance the cost-efficiency of the agent.
The Proposal: Scenario Recording and Replay for Known Scenarios
To reduce the costs associated with AI model calls, the Web Agent could implement a system for recording and replaying interactions for known scenarios.
How it would work:
Recording: In a "recording mode," the agent, guided by the AI, would execute a task for the first time. The sequence of actions and decisions would be saved as a "scenario."
Replay: When the agent is asked to perform the same task again, it would first recognize the known scenario and enter "replay mode." In this mode, it would follow the saved sequence of actions without invoking the AI.
AI Intervention on Failure: If at any point during the replay the scenario fails (e.g., a UI element has changed, an expected outcome is not met), the agent would seamlessly transition back to AI control. The AI would then analyze the situation, determine the new appropriate course of action, and, if successful, the new sequence could be saved to update the existing scenario.
Benefits:
Significant Cost Reduction: This would drastically reduce the number of API calls to the language model for repetitive tasks, leading to substantial cost savings.
Increased Speed: Replaying a recorded scenario would likely be faster than waiting for real-time AI inference for each step.
Maintained Robustness: The AI would still be available to handle exceptions and changes in the web environment, ensuring the agent's adaptability is not compromised.
This feature would make the Web Agent even more powerful and practical for a wide range of automated web tasks, especially those that are performed regularly.
Thank you for considering this feature request and for your outstanding work on this project.