copilot: Add support for tool_calls for gpt-4.1, gpt-4o, o4-mini #29369
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cla-bot
bot
added
the
cla-signed
imumesh18
changed the title
imumesh18
force-pushed
the
copilot-agent-models
branch
from
269d04a to
94331de
Compare
|
@notpeter can you please review this and help merge it? |
|
Hey @bennetbo, can you please review this and help me get it merged? |
Signed-off-by: Umesh Yadav <umesh4257@gmail.com>
imumesh18
force-pushed
the
copilot-agent-models
branch
from
94331de to
2997e55
Compare
|
I've confirmed this and implemented it in #29027. I tried changing the tool schema to the same one the Google provider does for models of that vendor, but to no avail. Not certain exactly what would need to be done to fix those. o3-mini also should be a model which supports tools, at least according to the API's data. |
|
Nevermind, the issue with Google models I had the other day was the same one you found in VSCode; it appears to be resolved now on the server side. #29027 will now enable tool calling support for all Copilot models of all vendors. |
|
Is there a reason why more models which support tool calling are not added here? I should mention #29855 |
This PR is aimed at enabling tools for the models that Microsoft VSCode lists in its Copilot agent mode, with the exception of Google's models, which require a different tool schema. #29027 proposes changes which will enable tools for Google models, as well as all current and future models which support tools. These only pertain to the Copilot model provider. Other providers are being worked on elsewhere. For instance, #29563 proposes adding tool support for ollama models which support them. These implementations take effort; they're (generally) not one line changes or match case additions like you see in this specific PR. |
bennetbo
approved these changes
May 5, 2025
JosephTLyons
pushed a commit
that referenced
this pull request
May 5, 2025
) Github Copilot currently supports following models for agent mode with tool calls. Currently we are only supporting anthropic models and not openai and gemini. This PR add support for the openai models. I have tested it and it works for all of them. For gemini models it seems there is a issues from copilot side so not adding that in this PR as enabling gemini model breaks it in the ask mode as well. <img width="392" alt="image" src="https://github.com/user-attachments/assets/fb7a4148-e48c-45c5-9ff9-c02f71217dfb" /> - [x] GPT-4.1 - [x] GPT-4.0 - [x] o4-mini Release Notes: - agent: Add tool calling support for gpt-4.1, gpt-4o, o4-mini when using Copilot Chat as a provider Signed-off-by: Umesh Yadav <umesh4257@gmail.com>
This was referenced May 5, 2025
bennetbo
added a commit
that referenced
this pull request
May 12, 2025
I noticed the discussion in #28881, and had thought of exactly the same a few days prior. This implementation should preserve existing functionality fairly well. I've added a dependency (serde_with) to allow the deserializer to skip models which cannot be deserialized, which could occur if a future provider, for instance, is added. Without this modification, such a change could break all models. If extra dependencies aren't desired, a manual implementation could be used instead. - Closes #29369 Release Notes: - Dynamically detect available Copilot Chat models, including all models with tool support --------- Co-authored-by: AidanV <aidanvanduyne@gmail.com> Co-authored-by: imumesh18 <umesh4257@gmail.com> Co-authored-by: Bennet Bo Fenner <bennet@zed.dev> Co-authored-by: Agus Zubiaga <hi@aguz.me>
JosephTLyons
pushed a commit
that referenced
this pull request
May 12, 2025
I noticed the discussion in #28881, and had thought of exactly the same a few days prior. This implementation should preserve existing functionality fairly well. I've added a dependency (serde_with) to allow the deserializer to skip models which cannot be deserialized, which could occur if a future provider, for instance, is added. Without this modification, such a change could break all models. If extra dependencies aren't desired, a manual implementation could be used instead. - Closes #29369 Release Notes: - Dynamically detect available Copilot Chat models, including all models with tool support --------- Co-authored-by: AidanV <aidanvanduyne@gmail.com> Co-authored-by: imumesh18 <umesh4257@gmail.com> Co-authored-by: Bennet Bo Fenner <bennet@zed.dev> Co-authored-by: Agus Zubiaga <hi@aguz.me>
JosephTLyons
pushed a commit
that referenced
this pull request
May 12, 2025
I noticed the discussion in #28881, and had thought of exactly the same a few days prior. This implementation should preserve existing functionality fairly well. I've added a dependency (serde_with) to allow the deserializer to skip models which cannot be deserialized, which could occur if a future provider, for instance, is added. Without this modification, such a change could break all models. If extra dependencies aren't desired, a manual implementation could be used instead. - Closes #29369 Release Notes: - Dynamically detect available Copilot Chat models, including all models with tool support --------- Co-authored-by: AidanV <aidanvanduyne@gmail.com> Co-authored-by: imumesh18 <umesh4257@gmail.com> Co-authored-by: Bennet Bo Fenner <bennet@zed.dev> Co-authored-by: Agus Zubiaga <hi@aguz.me>
dive
pushed a commit
to dive/zed
that referenced
this pull request
May 12, 2025
…es#29027) I noticed the discussion in zed-industries#28881, and had thought of exactly the same a few days prior. This implementation should preserve existing functionality fairly well. I've added a dependency (serde_with) to allow the deserializer to skip models which cannot be deserialized, which could occur if a future provider, for instance, is added. Without this modification, such a change could break all models. If extra dependencies aren't desired, a manual implementation could be used instead. - Closes zed-industries#29369 Release Notes: - Dynamically detect available Copilot Chat models, including all models with tool support --------- Co-authored-by: AidanV <aidanvanduyne@gmail.com> Co-authored-by: imumesh18 <umesh4257@gmail.com> Co-authored-by: Bennet Bo Fenner <bennet@zed.dev> Co-authored-by: Agus Zubiaga <hi@aguz.me>
osiewicz
pushed a commit
that referenced
this pull request
May 15, 2025
I noticed the discussion in #28881, and had thought of exactly the same a few days prior. This implementation should preserve existing functionality fairly well. I've added a dependency (serde_with) to allow the deserializer to skip models which cannot be deserialized, which could occur if a future provider, for instance, is added. Without this modification, such a change could break all models. If extra dependencies aren't desired, a manual implementation could be used instead. - Closes #29369 Release Notes: - Dynamically detect available Copilot Chat models, including all models with tool support --------- Co-authored-by: AidanV <aidanvanduyne@gmail.com> Co-authored-by: imumesh18 <umesh4257@gmail.com> Co-authored-by: Bennet Bo Fenner <bennet@zed.dev> Co-authored-by: Agus Zubiaga <hi@aguz.me>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Github Copilot currently supports following models for agent mode with tool calls. Currently we are only supporting anthropic models and not openai and gemini. This PR add support for the openai models. I have tested it and it works for all of them. For gemini models it seems there is a issues from copilot side so not adding that in this PR as enabling gemini model breaks it in the ask mode as well.
GPT-4.1
GPT-4.0
o4-mini
Release Notes: