Skip to content

Add GuidedChoice to ChatCompletionRequest #1034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions chat.go
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,16 @@
return nil
}

// ChatCompletionRequestExtensions contains third-party OpenAI API extensions (e.g., vendor-specific implementations like vLLM).

Check failure on line 251 in chat.go

View workflow job for this annotation

GitHub Actions / Sanity check

The line is 128 characters long, which exceeds the maximum of 120 characters. (lll)
type ChatCompletionRequestExtensions struct {
// GuidedChoice is a vLLM-specific extension that restricts the model's output
// to one of the predefined string choices provided in this field. This feature
// is used to constrain the model's responses to a controlled set of options,
// ensuring predictable and consistent outputs in scenarios where specific
// choices are required.
GuidedChoice []string `json:"guided_choice,omitempty"`
Copy link
Preview

Copilot AI Jul 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding validation for the GuidedChoice field to ensure it contains at least one option when provided, as an empty array would not be meaningful for constraining model output.

Copilot uses AI. Check for mistakes.

}

// ChatCompletionRequest represents a request structure for chat completion API.
type ChatCompletionRequest struct {
Model string `json:"model"`
Expand Down Expand Up @@ -309,6 +319,8 @@
ChatTemplateKwargs map[string]any `json:"chat_template_kwargs,omitempty"`
// Specifies the latency tier to use for processing the request.
ServiceTier ServiceTier `json:"service_tier,omitempty"`
// Embedded struct for non-OpenAI extensions
ChatCompletionRequestExtensions
}

type StreamOptions struct {
Expand Down
Loading