Skip to content

feat: use GoogleCompletionRequest.systemInstruction for system prompt #989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 29, 2025

Conversation

PhilKes
Copy link
Contributor

@PhilKes PhilKes commented Apr 13, 2025

Depends on carlrobertoh/llm-client#61

  • Uses Google API's systemInstruction field for system prompts

@PhilKes
Copy link
Contributor Author

PhilKes commented Apr 13, 2025

Since I havent been following the development of this plugin the past months, I am not sure but I am guessing up until now thought tokens of reasoning models are also just printed to the user in the chat panel etc, right?
carlrobertoh/llm-client#61 adds the thought fields to the response contents as well, so you could possibly mark the thought content response appropriately, like OpenAI or Google AI Studio does in the chat.

@carlrobertoh
Copy link
Owner

IIRC the thinking output is never displayed. At least when I initially introduced the Gemini thinking models, the support wasn't there yet (or was intentionally removed or something).

@PhilKes
Copy link
Contributor Author

PhilKes commented Apr 18, 2025

IIRC the thinking output is never displayed. At least when I initially introduced the Gemini thinking models, the support wasn't there yet (or was intentionally removed or something).

Well currently a response from e.g. Gemini 2.5 Pro looks like this:

Sometimes the thinking output seems to be in the same code block as the answer.
I guess it would be best to use the new thought boolean to distinguish thought tokens from the actual completion

@PhilKes
Copy link
Contributor Author

PhilKes commented Apr 18, 2025

Can you release llm-client v0.8.39 so we can merge this? 🙂

@carlrobertoh
Copy link
Owner

IIRC the thinking output is never displayed. At least when I initially introduced the Gemini thinking models, the support wasn't there yet (or was intentionally removed or something).

Well currently a response from e.g. Gemini 2.5 Pro looks like this:

Sometimes the thinking output seems to be in the same code block as the answer. I guess it would be best to use the new `thought` boolean to distinguish thought tokens from the actual completion

Yes, we need to use the new parameter. I believe that response is simply describing the CoT coming from the default system prompt itself - "First think step-by-step - describe your plan for what to build in pseudocode, written out in great detail."

@PhilKes PhilKes force-pushed the google-system-instruction branch from e7dc4f3 to 8de410f Compare April 27, 2025 14:19
@PhilKes
Copy link
Contributor Author

PhilKes commented Apr 27, 2025

Solved the conflicts

@carlrobertoh carlrobertoh merged commit 9f3d1cf into carlrobertoh:master Apr 29, 2025
2 checks passed
carlrobertoh added a commit that referenced this pull request Apr 30, 2025
…#989)

* feat: use GoogleCompletionRequest.systemInstruction for system prompt

* fix(test): req body assertion

---------

Co-authored-by: Carl-Robert Linnupuu <carlrobertoh@gmail.com>
carlrobertoh added a commit that referenced this pull request Apr 30, 2025
…#989)

* feat: use GoogleCompletionRequest.systemInstruction for system prompt

* fix(test): req body assertion

---------

Co-authored-by: Carl-Robert Linnupuu <carlrobertoh@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants