[Bug]: Bifrost strips provider-specific fields (e.g., extra_content), breaking Gemini 3 Pro function calling

[kmskrishna]

Prerequisites

• I have searched existing issues and discussions to avoid duplicates
• I am using the latest version (or have tested against main/nightly)

Description

Description

When using Gemini 3 Pro (gemini-3-pro-preview) through Bifrost with function calling, requests fail with:

400 Bad Request: "Function call is missing a thought_signature in functionCall parts. 
This is required for tools to work correctly, and missing thought_signature may lead 
to degraded model performance."

Root Cause

Bifrost's ChatAssistantMessageToolCall struct doesn't include the extra_content field, causing it to be silently dropped during JSON unmarshaling when proxying responses from Google's Gemini API.

File: core/schemas/chatcompletions.go (lines 483-488)

type ChatAssistantMessageToolCall struct {
    Index    uint16                               `json:"index"`
    Type     *string                              `json:"type,omitempty"`
    ID       *string                              `json:"id,omitempty"`
    Function ChatAssistantMessageToolCallFunction `json:"function"`
    // ❌ Missing: extra_content field
}

What Google Returns

When Gemini generates a function call, the response includes thought signatures in extra_content:

{
  "role": "assistant",
  "tool_calls": [{
    "id": "function-call-123",
    "type": "function",
    "function": {
      "name": "get_weather",
      "arguments": "{\"location\":\"Paris\"}"
    },
    "extra_content": {
      "google": {
        "thought_signature": "<encrypted_signature>"
      }
    }
  }]
}

What Bifrost Returns to Client

{
  "role": "assistant",
  "tool_calls": [{
    "id": "function-call-123",
    "type": "function",
    "function": {
      "name": "get_weather",
      "arguments": "{\"location\":\"Paris\"}"
    }
    // ❌ extra_content is missing
  }]
}

Why This Breaks Gemini 3 Pro

According to Google's documentation:

When using Gemini 3 Pro, you must pass back thought signatures during function calling, otherwise you will get a validation error (4xx status code).

Thought signatures must be:

1. Received from Gemini in the first function call response
2. Preserved and sent back in subsequent requests
3. Included in the exact same position (first function call in parallel calls, each step in sequential calls)

Model-specific behavior:

• Gemini 3 Pro: Thought signatures are MANDATORY - validation fails without them
• Gemini 2.5 models: Thought signatures are OPTIONAL - works even if missing

The Data Flow Problem

1. Google's API returns response with extra_content
2. Bifrost unmarshals into ChatAssistantMessageToolCall struct
3. Go's json.Unmarshal silently drops extra_content (no struct field)
4. Bifrost marshals back to JSON without extra_content
5. Client receives incomplete data
6. Client sends back to Bifrost (missing thought signature)
7. Bifrost forwards to Google
8. Google rejects with 400 error

Reproduction Steps

1. Configure Bifrost with Gemini 3 Pro

curl -X POST http://localhost:8080/api/providers \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "gemini",
    "keys": [{"value": "YOUR_API_KEY", "models": ["gemini-3-pro-preview"], "weight": 1.0}]
  }'

2. Make initial request with tools

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/gemini-3-pro-preview",
    "messages": [{"role": "user", "content": "What is the weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {"type": "string"}
          },
          "required": ["location"]
        }
      }
    }]
  }'

Result: ✅ Success - Returns function call (but without extra_content)

3. Send function result back

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini/gemini-3-pro-preview",
    "messages": [
      {"role": "user", "content": "What is the weather in Paris?"},
      {
        "role": "assistant",
        "tool_calls": [{
          "id": "function-call-123",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"location\":\"Paris\"}"
          }
        }]
      },
      {
        "role": "tool",
        "tool_call_id": "function-call-123",
        "content": "{\"temperature\":\"15C\",\"condition\":\"sunny\"}"
      }
    ],
    "tools": [...]
  }'

Result: ❌ 400 Bad Request - Missing thought_signature

Proposed Solution

Add extra_content field to schema

File: core/schemas/chatcompletions.go

type ChatAssistantMessageToolCall struct {
    Index        uint16                               `json:"index"`
    Type         *string                              `json:"type,omitempty"`
    ID           *string                              `json:"id,omitempty"`
    Function     ChatAssistantMessageToolCallFunction `json:"function"`
    ExtraContent map[string]interface{}               `json:"extra_content,omitempty"` // Add this
}

Implementation Notes

1. Preserve on receive: When Bifrost receives response from Google, extra_content will be captured
2. Pass through to client: Client receives complete data including extra_content
3. Preserve on send: When client sends back, extra_content is forwarded to Google
4. No validation needed: Bifrost doesn't need to understand or validate thought signatures, just preserve them

Additional Considerations

For the Gemini provider specifically (core/providers/gemini/), when converting from native Gemini format:

File: core/providers/gemini/types.go already has:

type Part struct {
    ThoughtSignature []byte `json:"thoughtSignature,omitempty"` // Line 892
    // ...
}

File: core/providers/gemini/chat.go should populate extra_content when converting:

// Around line 76-85
toolCall := schemas.ChatAssistantMessageToolCall{
    Index: uint16(len(toolCalls)),
    ID:    schemas.Ptr(callID),
    Type:  schemas.Ptr(string(schemas.ChatToolChoiceTypeFunction)),
    Function: schemas.ChatAssistantMessageToolCallFunction{
        Name:      &name,
        Arguments: string(jsonArgs),
    },
}

// Add: Preserve thought signature if present
if len(part.ThoughtSignature) > 0 {
    toolCall.ExtraContent = map[string]interface{}{
        "google": map[string]interface{}{
            "thought_signature": string(part.ThoughtSignature),
        },
    }
}

toolCalls = append(toolCalls, toolCall)

Impact

Current state:

• ❌ Gemini 3 Pro function calling fails completely
• ✅ Gemini 2.5 Pro works (thought signatures optional for 2.5)
• ❌ Any other provider-specific fields in extra_content or similar are also dropped

After fix:

• ✅ Gemini 3 Pro function calling works
• ✅ Future-proof for other provider extensions
• ✅ No breaking changes for existing functionality

References

• Gemini Thought Signatures Documentation
• OpenAI Tool Calls Spec (supports additional properties)

Steps to reproduce

Added in the description

Expected behavior

Bifrost should preserve the extra_content field from Gemini's response and pass it through to clients, allowing them to send it back in subsequent requests so Gemini 3 Pro function calling works correctly.

Actual behavior

Bifrost strips the extra_content field during JSON unmarshaling because ChatAssistantMessageToolCall struct lacks this field, causing clients to receive incomplete tool call data and subsequent requests to fail with 400 error from Gemini 3 Pro.

Affected area(s)

Core (Go)

Version

latest

Environment

## Environment

- **Bifrost version:** Latest `main` branch
- **Model:** `gemini-3-pro-preview`
- **Error:** 400 Bad Request on second+ function calling turns

Relevant logs/output

{"is_bifrost_error":false,"status_code":400,"error":{"message":"Function call is missing a thought_signature in functionCall parts. This is required for tools to work correctly, and missing thought_signature may lead to degraded model performance. Additional data, function call `default_api:get_weather` , position 2. please refer to https://ai.google.dev/gemini-api/docs/thought-signatures for more details.\n"},"extra_fields":{"provider":"gemini","model_requested":"gemini-3-pro-preview","request_type":"chat_completion"}}

Regression?

No response

Severity

High (major functionality broken)

[akshaydeo]

It seems like Gemini 3.0 has changed interface :). We are looking into it