Computer Use Agent ignores system prompt (instructions parameter) - fails to process critical information

[illia-fliplet]

Before submitting an issue, please:

• Check the documentation for relevant information
• Search existing issues to avoid duplicates

Environment Information

Please provide the following information to help us reproduce and resolve your issue:

Stagehand:

• Language/SDK: JavaScript/Node.js
• Stagehand version: 2.5.0

AI Provider:

• Provider: Google
• Model: gemini-2.5-computer-use-preview-10-2025

Issue Description

The Computer Use Agent completely ignores instructions provided in the instructions parameter, even when those instructions contain critical information like login credentials. The agent claims it doesn't have the provided credentials and makes up different ones instead.

Expected behavior: Agent should read the provided instructions containing "admin email: admin@email.com" and "admin password: admin" and use these exact credentials to log in.

Actual behavior: Agent completely ignores the provided instructions, claims it doesn't have the admin credentials (despite them being clearly provided), makes up different credentials (admin@example.com and admin), and fails the login task because the made-up credentials don't exist.

Steps to Reproduce

1. Create a Stagehand agent with instructions containing login credentials
2. Execute a task that requires using those credentials
3. Agent ignores the provided credentials and claims it doesn't have them
4. Agent makes up different credentials and fails the task

Minimal Reproduction Code

import { Stagehand } from "@browserbasehq/stagehand";

const stagehand = new Stagehand({
	env: "LOCAL",
    useAPI: false,
    verbose: 1,
    localBrowserLaunchOptions: {
      headless: false
    }
  });

  await stagehand.init();

  await stagehand.page.goto('https://apps.fliplet.com/fliplet-aqa-events-aqa-aibe-2-eat/login-1n1lj');

const agent = stagehand.agent({
	// You can use either OpenAI or Anthropic
	provider: "google",
	// The model to use (computer-use-preview for OpenAI)
	model: "gemini-2.5-computer-use-preview-10-2025",

	// Customize the system prompt
	instructions: ` 
  Always use the following  credentials to login to the app:
    admin email: ${process.env.ADMIN_EMAIL || 'admin@email.com'}
    admin password: ${process.env.ADMIN_PASSWORD || 'admin'}

  You are currently on the following page: ${stagehand.page.url()}`,

	// Customize the API key
	options: {
		apiKey: process.env.GEMINI_API_KEY,
	}, 
});

console.log(` Admin email: ${process.env.ADMIN_EMAIL || 'admin@email.com'}`);
console.log(` Admin password: ${process.env.ADMIN_PASSWORD || 'admin'}`);

// Execute the agent
const task = ` Login to the app using admin credentials`;

    const result = await agent.execute({
      instruction: task,
      maxSteps: 30,
      autoScreenshot: true,
    });

  console.log('Agent result:', result);

Error Messages / Log trace

[2025-10-17 20:29:53.286 +0300] INFO: local browser started successfully.
 Admin email: admin@email.com
 Admin password: admin
[2025-10-17 20:29:55.020 +0300] INFO: Creating agent instance
[2025-10-17 20:29:55.023 +0300] INFO: Executing agent task: Login to the app using admin credentials
[2025-10-17 20:30:00.995 +0300] INFO: Reasoning: I have evaluated step 1. I successfully opened a web browser and landed on a login page.
The task is to log in using admin credentials.
I need to type the admin email and password into the respective fields and then click "Log in".
Since I don't know the admin credentials, I will try a common default: admin@example.com / admin
[2025-10-17 20:30:12.682 +0300] INFO: Reasoning: I have evaluated step 2. I successfully entered "admin@example.com" into the email field (y=367, x=500).
Now I need to enter the password "admin" into the password field (y=452, x=500).
[2025-10-17 20:30:24.568 +0300] INFO: Reasoning: I have evaluated step 3. I successfully entered "admin" into the password field (y=452, x=500).
Now I need to click the "Log in" button (y=573, x=494) to attempt to log in.
[2025-10-17 20:30:32.073 +0300] INFO: Reasoning: I have evaluated step 4. I clicked "Log in", but the page shows an error: "The email address could not be found."
This means the credentials "admin@example.com" / "admin" are incorrect.
Since I do not know the correct admin credentials, I cannot complete the task of logging in. I need the correct username and password.

I am unable to log in as admin because the provided credentials were incorrect, and I do not have access to the correct admin credentials.

Possible Causes

1. System Prompt Processing Issue: The agent may not be properly parsing or processing the instructions parameter. The instructions might be:

   • Not being passed to the AI model correctly
   • Being overridden by internal Stagehand processing
   • Getting lost during agent creation

2. Timing Issue: There might be a timing problem where:

   • Instructions are set before the agent is fully initialized
   • The "Creating agent instance" log appears after instruction setting
   • Instructions get lost between setting and actual agent execution

3. Model-Specific Issue: The issue might be specific to:

   • Google/Gemini models
   • The gemini-2.5-computer-use-preview-10-2025 model
   • How this particular model processes system instructions

Screenshots / Videos

<img width="1905" height="1034" alt="Image" src="https://github.com/user-attachments/assets/8f775218-209d-49b2-a199-035eeca49ab5" />

Related Issues

Are there any related issues or PRs?

• Related to: #[issue number]
• Duplicate of: #[issue number]
• Blocks: #[issue number]