anthropic passthrough mode #577
Conversation
# Conflicts: # core/schemas/bifrost.go # transports/bifrost-http/integrations/anthropic.go # transports/bifrost-http/integrations/anthropic/types.go # transports/bifrost-http/integrations/utils.go
Feature: Add antrhopic passthrough mode
📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds Anthropic Passthrough provider and wiring: new provider implementation, factory registration, schema/context keys for original request data and raw SSE, proxy/transport utilities, HTTP transport updates to preserve headers/path/body for OAuth passthrough, and streaming handler changes to forward raw SSE events and headers. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant C as Client
participant HT as HTTP Transport
participant BF as Bifrost Core
participant PR as AnthropicPassthrough Provider
participant A as Anthropic API
rect rgba(230,240,255,0.6)
note over HT: OAuth + Anthropic path detected
C->>HT: POST /anthropic/v1/messages (OAuth)
HT->>HT: Store original body/headers/path in context
HT->>BF: Bifrost ChatRequest (Provider=AnthropicPassthrough)
BF->>PR: ChatCompletion(ctx with originals)
PR->>A: POST /v1/messages (forward headers/body)
A-->>PR: Response (status, headers, body)
PR-->>BF: BifrostResponse + RawHeaders
BF-->>HT: Response passthrough bytes + headers
HT-->>C: HTTP 200 with Anthropic headers/body
end
sequenceDiagram
autonumber
participant C as Client
participant HT as HTTP Transport (SSE)
participant BF as Bifrost Core
participant PR as AnthropicPassthrough Provider
participant A as Anthropic SSE
rect rgba(255,245,230,0.6)
note over HT: Streaming passthrough with raw SSE
C->>HT: POST (stream=true)
HT->>BF: Stream request (ctx carries originals)
BF->>PR: ChatCompletionStream
PR-->>A: Open SSE stream (forward headers/body)
A-->>PR: SSE events (raw)
PR-->>BF: BifrostStream events (RawSSEEvent set)
BF-->>HT: RawSSEEvent frames
HT-->>C: Write raw SSE lines (no extra [DONE])
PR->>PR: Collect usage/finish/message_id
PR-->>BF: Final telemetry (no extra event emission)
BF->>HT: Stream end (post-hooks)
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (3 warnings)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
core/bifrost.go(2 hunks)core/providers/anthropic_passthrough.go(1 hunks)core/providers/utils.go(4 hunks)core/schemas/bifrost.go(4 hunks)core/utils.go(1 hunks)transports/bifrost-http/handlers/inference.go(2 hunks)transports/bifrost-http/integrations/anthropic.go(2 hunks)transports/bifrost-http/integrations/utils.go(9 hunks)
🧰 Additional context used
🧬 Code graph analysis (7)
core/utils.go (1)
core/schemas/bifrost.go (3)
Ollama(53-53)SGL(55-55)AnthropicPassthrough(48-48)
transports/bifrost-http/integrations/anthropic.go (3)
transports/bifrost-http/integrations/utils.go (3)
RouteConfig(142-153)RequestConverter(81-81)ResponseConverter(85-85)core/schemas/providers/anthropic/types.go (1)
AnthropicMessageRequest(35-47)core/schemas/bifrost.go (2)
BifrostRequest(15-27)BifrostResponse(231-246)
transports/bifrost-http/integrations/utils.go (2)
core/schemas/bifrost.go (5)
Anthropic(47-47)AnthropicPassthrough(48-48)BifrostContextKeyOriginalRequest(116-116)BifrostContextKeyOriginalHeaders(117-117)BifrostContextKeyOriginalPath(118-118)transports/bifrost-http/handlers/inference.go (1)
ChatRequest(165-169)
core/bifrost.go (2)
core/schemas/bifrost.go (1)
AnthropicPassthrough(48-48)core/providers/anthropic_passthrough.go (1)
NewAnthropicPassthroughProvider(62-102)
core/schemas/bifrost.go (1)
ui/lib/types/config.ts (1)
ModelProvider(165-167)
core/providers/utils.go (4)
ui/lib/types/config.ts (1)
ProxyConfig(113-118)core/schemas/provider.go (6)
ProxyConfig(79-84)NoProxy(69-69)HTTPProxy(71-71)Socks5Proxy(73-73)EnvProxy(75-75)PostHookRunner(191-191)core/schemas/logger.go (1)
Logger(28-55)core/schemas/bifrost.go (2)
BifrostResponse(231-246)BifrostContextKeyStreamEndIndicator(115-115)
core/providers/anthropic_passthrough.go (5)
core/schemas/logger.go (1)
Logger(28-55)core/schemas/provider.go (5)
DefaultRequestTimeoutInSeconds(14-14)ErrProviderJSONMarshaling(26-26)PostHookRunner(191-191)ErrProviderRequest(24-24)DefaultStreamBufferSize(17-17)core/schemas/bifrost.go (14)
ModelProvider(42-42)AnthropicPassthrough(48-48)BifrostError(537-546)BifrostContextKeyOriginalRequest(116-116)BifrostContextKeyOriginalPath(118-118)BifrostContextKeyOriginalHeaders(117-117)BifrostResponse(231-246)BifrostChatRequest(182-188)ChatCompletionRequest(95-95)BifrostResponseExtraFields(490-500)RequestType(90-90)BifrostStream(525-529)ChatCompletionStreamRequest(96-96)LLMUsage(288-294)core/schemas/providers/anthropic/types.go (2)
AnthropicMessageError(279-282)AnthropicStreamEvent(233-241)core/schemas/account.go (1)
Key(8-17)
| rawBytes := resp.Body() | ||
|
|
||
| if resp.StatusCode() != fasthttp.StatusOK { | ||
|
|
||
| var errorResp anthropic.AnthropicMessageError | ||
|
|
||
| bifrostErr := handleProviderAPIError(resp, &errorResp) | ||
| bifrostErr.Error.Type = &errorResp.Error.Type | ||
| bifrostErr.Error.Message = errorResp.Error.Message | ||
|
|
||
| return nil, nil, nil, bifrostErr | ||
| } | ||
|
|
||
| decodedBody := rawBytes | ||
| contentEncoding := string(resp.Header.Peek("Content-Encoding")) | ||
|
|
||
| if contentEncoding == "gzip" { | ||
| var err error | ||
| decodedBody, err = resp.BodyGunzip() | ||
| if err != nil { | ||
| return nil, nil, nil, newBifrostOperationError(schemas.ErrProviderJSONMarshaling, fmt.Errorf("failed to decompress gzip response: %w", err), provider.GetProviderKey()) | ||
| } | ||
| } | ||
|
|
||
| httpHeaders := make(http.Header) | ||
| for k, v := range resp.Header.All() { | ||
| httpHeaders.Add(string(k), string(v)) | ||
| } | ||
|
|
||
| return rawBytes, decodedBody, httpHeaders, nil |
There was a problem hiding this comment.
Copy the fasthttp body before releasing the response.
fasthttp.AcquireResponse reuses its internal buffer once you Release it (which happens via the defer at Line 152). Returning rawBytes := resp.Body() hands callers a slice backed by that recycled buffer, so after the function returns the contents become undefined and can flip to unrelated data. Take a defensive copy of the body (and reuse that copy for decodedBody) before releasing the response object.
Apply this diff:
- rawBytes := resp.Body()
-
- decodedBody := rawBytes
+ rawBytes := append([]byte(nil), resp.Body()...)
+
+ decodedBody := rawBytes
contentEncoding := string(resp.Header.Peek("Content-Encoding"))Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In core/providers/anthropic_passthrough.go around lines 179 to 208, the code
returns resp.Body() which is a slice backed by fasthttp's internal buffer that
will be recycled when the response is released; make a defensive copy of the raw
body bytes immediately after reading resp.Body() (e.g., allocate a new slice and
copy into it) and use that copy for both rawBytes and as the starting point for
decodedBody (replace decodedBody with the copy, then overwrite it when
gunzipping into a new slice or reuse the copy as appropriate); ensure
httpHeaders construction remains the same and return the copied rawBytes and the
decodedBody copy so callers don't observe corrupted data after the response is
released.
| httpHeaders := make(http.Header) | ||
| for k, v := range resp.Header.All() { | ||
| httpHeaders.Add(string(k), string(v)) | ||
| } |
There was a problem hiding this comment.
Use VisitAll to materialize headers.
fasthttp.ResponseHeader doesn’t expose an All() helper, so this won’t compile. Iterate with VisitAll to populate the http.Header.
Apply this diff:
- httpHeaders := make(http.Header)
- for k, v := range resp.Header.All() {
- httpHeaders.Add(string(k), string(v))
- }
+ httpHeaders := make(http.Header)
+ resp.Header.VisitAll(func(k, v []byte) {
+ httpHeaders.Add(string(k), string(v))
+ })📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| httpHeaders := make(http.Header) | |
| for k, v := range resp.Header.All() { | |
| httpHeaders.Add(string(k), string(v)) | |
| } | |
| httpHeaders := make(http.Header) | |
| resp.Header.VisitAll(func(k, v []byte) { | |
| httpHeaders.Add(string(k), string(v)) | |
| }) |
🤖 Prompt for AI Agents
In core/providers/anthropic_passthrough.go around lines 203 to 206, the code
attempts to call resp.Header.All() which doesn't exist on
fasthttp.ResponseHeader; replace that iteration with
resp.Header.VisitAll(func(k, v []byte) { httpHeaders.Add(string(k), string(v))
}) so headers are materialized correctly into the http.Header (convert byte
slices to strings when adding).
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
fasthttp ResponseHeader VisitAll deprecated alternative
💡 Result:
Use the new iterator/collection APIs instead — replace ResponseHeader.VisitAll with ResponseHeader.All. (Similarly: VisitAllInOrder → AllInOrder, VisitAllCookie → Cookies, VisitAllTrailer → Trailers.) [1]
Sources: fasthttp docs. [1] https://pkg.go.dev/github.com/valyala/fasthttp (see ResponseHeader methods).
@hazyone, you're absolutely right! I apologize for the confusion. VisitAll is indeed deprecated in favor of the newer iterator-based API. The original code is actually correct and should work as-is with modern versions of fasthttp:
httpHeaders := make(http.Header)
for k, v := range resp.Header.All() {
httpHeaders.Add(string(k), string(v))
}The All() method returns an iterator that can be used with Go's range loops (available in Go 1.23+), yielding byte slices for both keys and values. Please disregard my previous suggestion to use VisitAll.
(\\_/)
(•_•) *scratches ear sheepishly*
/ > 📚
| for key, value := range ctx.Request.Header.All() { | ||
| keyStr := string(key) | ||
| headers[keyStr] = append(headers[keyStr], string(value)) | ||
| } | ||
| return headers |
There was a problem hiding this comment.
Fix: fasthttp header iteration uses VisitAll, not All().
fasthttp.RequestHeader has no All() method, so this addition breaks compilation. Use VisitAll (or similar) to collect headers instead of calling All().
-func extractHeaders(ctx *fasthttp.RequestCtx) map[string][]string {
- headers := make(map[string][]string)
- for key, value := range ctx.Request.Header.All() {
- keyStr := string(key)
- headers[keyStr] = append(headers[keyStr], string(value))
- }
- return headers
-}
+func extractHeaders(ctx *fasthttp.RequestCtx) map[string][]string {
+ headers := make(map[string][]string)
+ ctx.Request.Header.VisitAll(func(key, value []byte) {
+ k := string(key)
+ headers[k] = append(headers[k], string(value))
+ })
+ return headers
+}🤖 Prompt for AI Agents
In transports/bifrost-http/integrations/utils.go around lines 250 to 254, the
code calls ctx.Request.Header.All() which does not exist on
fasthttp.RequestHeader and breaks compilation; replace the All() iteration with
Header.VisitAll and inside the VisitAll callback convert the key and value byte
slices to strings and append them into the headers map (headers[string(k)] =
append(headers[string(k)], string(v))) so all header occurrences are collected
correctly.
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
fasthttp RequestHeader VisitAll deprecated alternative method
💡 Result:
Use the new iterator APIs (All / AllInOrder /Cookies /Trailers) instead of the deprecated VisitAll family. Example (RequestHeader):
- RequestHeader.All() — iterate all headers (unordered)
- RequestHeader.AllInOrder() — iterate headers in receive order
- RequestHeader.Cookies() — replace VisitAllCookie
- RequestHeader.Trailers() — replace VisitAllTrailer
Example usage:
h.All()(func(key, value []byte) bool {
// use key, value (copy if you need them after iteration)
return true // continue
})
(See fasthttp docs: RequestHeader.All / AllInOrder and deprecation note for VisitAll.) [1]
Sources
[1] fasthttp RequestHeader docs (All, AllInOrder, VisitAll deprecated).
@hazyone, you're absolutely right! VisitAll is deprecated. The original code in the PR was actually attempting to use the newer All() method, but with incorrect syntax.
The All() method returns an iterator function, so it needs to be called with a callback. Here's the corrected implementation:
func extractHeaders(ctx *fasthttp.RequestCtx) map[string][]string {
headers := make(map[string][]string)
ctx.Request.Header.All()(func(key, value []byte) bool {
k := string(key)
headers[k] = append(headers[k], string(value))
return true // continue iteration
})
return headers
}The key difference is that All() returns an iterator function, which you then immediately call with a callback function that receives key and value as byte slices. The callback should return true to continue iteration.
|
@hazyone apologies - when I was testing out we realized that we dont need this change - as we already have integrations in place (unlike passthrough requirement of litellm). @Pratham-Mishra04 will share the video but to use Bifrost with Claude code for example just change set env variable ANTHROPIC_BASE_URL=http(s)://<bifrost_base_url>/anthropic and it works. He is currently fixing some logging issues |
|
Hey @hazyone, here's the video for Claude code setup - https://www.loom.com/share/5ab9a2354eec46998c0510dc0100fbf3?sid=1b65d658-b361-4c6a-998f-4d11f1e2e344 We have a few patches going in with #579 on a new version |
|
@Pratham-Mishra04 I guess this would work only in case of using Anthropic API key. My PR is about using their MAX subscription, because they have Authorization with oauth2 and slightly different payloads. |
Author
@hazyone
Summary
Add support for Anthropic OAuth passthrough mode to enable Claude Code integration. This allows Bifrost to handle OAuth tokens (sk-ant-oat-*) by preserving the original request structure and headers.
Changes
AnthropicPassthroughProviderto handle OAuth authentication modeType of change
Affected areas
How to test
Test with Claude Code by configuring it to use Bifrost as a proxy:
Breaking changes
Related issues
Enables Claude Code integration with Bifrost
Security considerations
Checklist
docs/contributing/README.mdand followed the guidelines