Skip to content

independent test cases #669

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

independent test cases #669

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
248 changes: 248 additions & 0 deletions core/providers/anthropic_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,248 @@
package providers

import (
"context"
"os"
"testing"

"github.com/maximhq/bifrost/core/schemas"
)

func TestAnthropicChatCompletion(t *testing.T) {
apiKey := os.Getenv("ANTHROPIC_API_KEY")
if apiKey == "" {
t.Skip("ANTHROPIC_API_KEY not set")
}

provider := NewAnthropicProvider(&schemas.ProviderConfig{
NetworkConfig: schemas.NetworkConfig{
BaseURL: "https://api.anthropic.com",
DefaultRequestTimeoutInSeconds: 30,
},
ConcurrencyAndBufferSize: schemas.ConcurrencyAndBufferSize{
Concurrency: 10,
},
}, newTestLogger())

ctx := context.Background()
key := schemas.Key{Value: apiKey}

request := &schemas.BifrostChatRequest{
Provider: schemas.Anthropic,
Model: "claude-3-haiku-20240307",
Input: []schemas.ChatMessage{
Comment on lines +30 to +33
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Verify Anthropic model IDs (may have changed).

Please confirm latest model slugs for Messages and Responses as of 2025‑10‑23.

Also applies to: 88-115, 150-162, 205-218

🌐 Web query:

What are Anthropic’s current model IDs for Haiku and Sonnet (Messages API and Responses API), and which “latest” aliases should be used?

💡 Result:

  • Claude Haiku (current model ID): claude-haiku-4-5-20251001 — use alias: claude-haiku-4-5. [1][2]
  • Claude Sonnet (current model ID): claude-sonnet-4-5-20250929 — use alias: claude-sonnet-4-5. [1][2]

Sources: Anthropic / Claude model overview and API client docs. [1] (docs.claude.com) [2] (docs.claude.com)

Citations:

🏁 Script executed:

#!/bin/bash
# Inspect the test file at the specified line ranges
echo "=== Lines 30-33 ==="
sed -n '30,33p' core/providers/anthropic_test.go

echo ""
echo "=== Lines 88-115 ==="
sed -n '88,115p' core/providers/anthropic_test.go

echo ""
echo "=== Lines 150-162 ==="
sed -n '150,162p' core/providers/anthropic_test.go

echo ""
echo "=== Lines 205-218 ==="
sed -n '205,218p' core/providers/anthropic_test.go

Length of output: 1780

🏁 Script executed:

#!/bin/bash
# Inspect the test file at the specified line ranges
echo "=== Lines 30-33 ==="
sed -n '30,33p' core/providers/anthropic_test.go

echo ""
echo "=== Lines 88-115 ==="
sed -n '88,115p' core/providers/anthropic_test.go

echo ""
echo "=== Lines 150-162 ==="
sed -n '150,162p' core/providers/anthropic_test.go

echo ""
echo "=== Lines 205-218 ==="
sed -n '205,218p' core/providers/anthropic_test.go

Length of output: 1780

Update all Anthropic model IDs to current versions (claude-haiku-4-5 and claude-sonnet-4-5).

Claude Haiku's current model ID is claude-haiku-4-5-20251001 and Claude Sonnet's current model ID is claude-sonnet-4-5-20250929. Update:

  • Lines 30-33, 88-115, 150-162: Change "claude-3-haiku-20240307" to "claude-haiku-4-5" (or full version ID)
  • Lines 205-218: Change "claude-3-5-sonnet-20241022" to "claude-sonnet-4-5" (or full version ID)
🤖 Prompt for AI Agents 
In core/providers/anthropic_test.go around lines 30-33, 88-115, 150-162 and
205-218, the tests use outdated Anthropic model IDs; replace occurrences of
"claude-3-haiku-20240307" with the current haiku model ID ("claude-haiku-4-5" or
the full version "claude-haiku-4-5-20251001") and replace occurrences of
"claude-3-5-sonnet-20241022" with the current sonnet model ID
("claude-sonnet-4-5" or the full version "claude-sonnet-4-5-20250929"); update
the string literals in those lines so tests reference the new model IDs
consistently.

{
Role: "user",
Content: &schemas.ChatMessageContent{ContentStr: stringPtr("Say hello in one word")},
},
},
Params: &schemas.ChatParameters{
Temperature: float64Ptr(0.7),
MaxCompletionTokens: intPtr(10),
},
}

resp, err := provider.ChatCompletion(ctx, key, request)
if err != nil {
t.Fatalf("ChatCompletion failed: %v", err)
}

if resp == nil {
t.Fatal("Expected non-nil response")
}
if len(resp.Choices) == 0 {
t.Fatal("Expected at least one choice")
}
if resp.Choices[0].Message.Content == nil {
t.Fatal("Expected message content")
}
t.Logf("Response: %v", resp.Choices[0].Message.Content)
}

func TestAnthropicChatCompletionWithTools(t *testing.T) {
apiKey := os.Getenv("ANTHROPIC_API_KEY")
if apiKey == "" {
t.Skip("ANTHROPIC_API_KEY not set")
}

provider := NewAnthropicProvider(&schemas.ProviderConfig{
NetworkConfig: schemas.NetworkConfig{
BaseURL: "https://api.anthropic.com",
DefaultRequestTimeoutInSeconds: 30,
},
ConcurrencyAndBufferSize: schemas.ConcurrencyAndBufferSize{
Concurrency: 10,
},
}, newTestLogger())

ctx := context.Background()
key := schemas.Key{Value: apiKey}

props := map[string]interface{}{
"location": map[string]interface{}{
"type": "string",
"description": "The city name",
},
}

request := &schemas.BifrostChatRequest{
Provider: schemas.Anthropic,
Model: "claude-3-haiku-20240307",
Input: []schemas.ChatMessage{
{
Role: "user",
Content: &schemas.ChatMessageContent{ContentStr: stringPtr("What's the weather in San Francisco?")},
},
},
Params: &schemas.ChatParameters{
Temperature: float64Ptr(0.7),
MaxCompletionTokens: intPtr(100),
Tools: []schemas.ChatTool{
{
Type: schemas.ChatToolTypeFunction,
Function: &schemas.ChatToolFunction{
Name: "get_weather",
Description: stringPtr("Get the current weather"),
Parameters: &schemas.ToolFunctionParameters{
Type: "object",
Properties: &props,
Required: []string{"location"},
},
},
},
},
},
}

resp, err := provider.ChatCompletion(ctx, key, request)
if err != nil {
t.Fatalf("ChatCompletion with tools failed: %v", err)
}

if resp == nil {
t.Fatal("Expected non-nil response")
}
if len(resp.Choices) == 0 {
t.Fatal("Expected at least one choice")
}
t.Logf("Tool calls: %d", len(resp.Choices[0].Message.ToolCalls))
}

func TestAnthropicChatCompletionStream(t *testing.T) {
apiKey := os.Getenv("ANTHROPIC_API_KEY")
if apiKey == "" {
t.Skip("ANTHROPIC_API_KEY not set")
}

provider := NewAnthropicProvider(&schemas.ProviderConfig{
NetworkConfig: schemas.NetworkConfig{
BaseURL: "https://api.anthropic.com",
DefaultRequestTimeoutInSeconds: 30,
},
ConcurrencyAndBufferSize: schemas.ConcurrencyAndBufferSize{
Concurrency: 10,
},
}, newTestLogger())

ctx := context.Background()
key := schemas.Key{Value: apiKey}

request := &schemas.BifrostChatRequest{
Provider: schemas.Anthropic,
Model: "claude-3-haiku-20240307",
Input: []schemas.ChatMessage{
{
Role: "user",
Content: &schemas.ChatMessageContent{ContentStr: stringPtr("Count from 1 to 3")},
},
},
Params: &schemas.ChatParameters{
Temperature: float64Ptr(0.7),
},
}

streamChan, err := provider.ChatCompletionStream(ctx, mockPostHookRunner, key, request)
if err != nil {
t.Fatalf("ChatCompletionStream failed: %v", err)
}

count := 0
for chunk := range streamChan {
if chunk == nil {
continue
}
if chunk.BifrostError != nil {
t.Fatalf("Stream error: %v", chunk.BifrostError)
}
count++
}

if count == 0 {
t.Fatal("Expected at least one chunk")
}
t.Logf("Received %d chunks", count)
}

func TestAnthropicResponses(t *testing.T) {
apiKey := os.Getenv("ANTHROPIC_API_KEY")
if apiKey == "" {
t.Skip("ANTHROPIC_API_KEY not set")
}

provider := NewAnthropicProvider(&schemas.ProviderConfig{
NetworkConfig: schemas.NetworkConfig{
BaseURL: "https://api.anthropic.com",
DefaultRequestTimeoutInSeconds: 30,
},
ConcurrencyAndBufferSize: schemas.ConcurrencyAndBufferSize{
Concurrency: 10,
},
}, newTestLogger())

ctx := context.Background()
key := schemas.Key{Value: apiKey}

userRole := schemas.ResponsesInputMessageRoleUser
request := &schemas.BifrostResponsesRequest{
Provider: schemas.Anthropic,
Model: "claude-3-5-sonnet-20241022",
Input: []schemas.ResponsesMessage{
{
Role: &userRole,
Content: &schemas.ResponsesMessageContent{ContentStr: stringPtr("What is 2+2?")},
},
},
Params: &schemas.ResponsesParameters{
MaxOutputTokens: intPtr(100),
},
}

resp, err := provider.Responses(ctx, key, request)
if err != nil {
t.Fatalf("Responses failed: %v", err)
}

if resp == nil {
t.Fatal("Expected non-nil response")
}
if len(resp.Output) == 0 {
t.Fatal("Expected at least one output message")
}
t.Logf("Response output messages: %d", len(resp.Output))
}

func TestAnthropicGetProviderKey(t *testing.T) {
provider := NewAnthropicProvider(&schemas.ProviderConfig{
NetworkConfig: schemas.NetworkConfig{
BaseURL: "https://api.anthropic.com",
},
ConcurrencyAndBufferSize: schemas.ConcurrencyAndBufferSize{
Concurrency: 10,
},
}, newTestLogger())

key := provider.GetProviderKey()
if key != schemas.Anthropic {
t.Errorf("Expected provider key %s, got %s", schemas.Anthropic, key)
}
}
Loading