Skip to content

Disable mutating tools in read-only mode #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

diksipav
Copy link

@diksipav diksipav commented Jul 20, 2025

What kind of change does this PR introduce?

Improvement: #112

What is the current behavior?

Read-only mode executes SQL as a read-only Postgres user (execute_sql tool), and
apply_migration tool is disabled in read-only mode.

What is the new behavior?

Alongside execute_sql and apply_migration all other mutating tools are disabled in read-only mode.

Copy link
Collaborator

@gregnr gregnr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @diksipav 👍 can we please also add some tests for each of these to validate that the errors are thrown in read only mode?

@diksipav diksipav marked this pull request as draft July 20, 2025 19:53
@diksipav
Copy link
Author

diksipav commented Jul 21, 2025

Hey @gregnr it seems to me that for e2e testing a real ANTHROPIC_API_KEY is used and we are calling the LLM, right? Also one of the latest (and most expensive models is used). I am testing now with my own api key, but maybe we can at least test with a cheaper model? Also not sure in this case should I test every option or for example to test one tool per each feature group in read-only mode.

Or to not write full e2e tests but to just test tool calls? Maybe to extend server.test.ts.

@diksipav diksipav marked this pull request as ready for review July 21, 2025 15:40
@diksipav diksipav requested a review from gregnr July 21, 2025 17:57
Copy link
Collaborator

@gregnr gregnr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diksipav I think there is no need for e2e tests in this case, let's just stick to unit. The logic is pretty straightforward and probably not worth the cost of running through a real LLM.

The tests you've added to server.test.ts look good.

@@ -179,7 +179,6 @@ export const mockManagementApi = [
name: z.string(),
region: z.string(),
organization_id: z.string(),
db_pass: z.string(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this intentional?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I deleted db_pass because it is not used anywhere in the mock function. I can put it back?

I will also fix the tests message and remove e2e test I previously wrote.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right that we don't use it in the mock implementation, but we still want to include it during request body validation. We're mocking the real management API with this endpoint which does require db_pass, so it's important we still check for it and throw if the client forgot to pass it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, great. I already put it back in the last commit : )

@@ -9,12 +9,14 @@ const DEFAULT_TEST_MODEL = 'claude-3-7-sonnet-20250219';

type SetupOptions = {
projectId?: string;
readOnly?: boolean;
features?: string[];
Copy link
Author

@diksipav diksipav Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can leave or remove changes to utils.ts since we are not using them for now.
Edit: I removed them.

@diksipav diksipav requested a review from gregnr July 23, 2025 07:36
@gregnr
Copy link
Collaborator

gregnr commented Jul 26, 2025

Looks good @diksipav 👍 you can ignore the failed E2E tests since these are just due to the missing API key. We'll merge this soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants