Skip to content

Conversation

@Sameerlite
Copy link
Collaborator

Title

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • I have added a screenshot of my new test passing locally
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

Sameerlite and others added 30 commits November 19, 2025 11:55
Fixes #16810

## Problem

When using completion() with models that have mode: "responses" (like o3-pro,
gpt-5-codex), the response_format parameter with JSON schemas was being ignored
or incorrectly handled, causing:
- Large schemas (>512 chars) to fail with "metadata.schema_dict_json: string too long" error
- Structured outputs to be silently dropped
- Users' code to break unexpectedly

## Root Cause

The completion -> responses bridge in
litellm/completion_extras/litellm_responses_transformation/transformation.py
was missing the conversion of response_format (Chat Completion format) to
text.format (Responses API format).

The inverse bridge (responses -> completion) already had this conversion
implemented in commit 29f0ed2, but the completion -> responses direction
was incomplete.

## Solution

Added _transform_response_format_to_text_format() method that converts:
- response_format with json_schema → text.format with json_schema
- response_format with json_object → text.format with json_object
- response_format with text → text.format with text

Updated transform_request() to detect and convert response_format parameter
before sending to litellm.responses().

## Changes

- Added _transform_response_format_to_text_format() method (lines 592-647)
- Modified transform_request() to handle response_format (lines 199-203)
- Added comprehensive tests to validate the conversion

## Testing

- 5 new unit tests covering all conversion scenarios
- Real API test with OpenAI confirming large schemas (>512 chars) work
- No more metadata.schema_dict_json errors

## Impact

Users can now use completion() with models that have mode: "responses" and:
- Use large JSON schemas without hitting metadata 512 char limit
- Get proper structured outputs
- Have their existing code continue working
* add AWS fields for KeyManagementSettings

* docs IAM roles

* use aws iam auth on secret manager v2

* fix: load_aws_secret_manager

* test_secret_manager_with_iam_role_settings
* feat: mcp prompts support

* feat: mcp resources support
Updated pydantic version to 2.11.0 for compatibility.
…16898)

* TestPromptRequest

* add prompts/test endpoint for testing prompt

* TestPromptTestEndpoint

* feat: working v1 of this ui

* workig prompt endpoints

* add chat ui for prompts

* add conversation panel

* add init chat ui
* TestPromptRequest

* add prompts/test endpoint for testing prompt

* TestPromptTestEndpoint

* feat: working v1 of this ui

* workig prompt endpoints

* add chat ui for prompts

* add conversation panel

* add init chat ui

* allow clicking edit prompt

* fix use get_base_prompt_id

* add endpoints for viewing prompt versions

* TestPromptVersioning

* add getPromptVersions

* add VersionHistorySidePanel

* allow viewing version history

* add version history
* fix images being dropped from tool results for bedrock

* type fixes
* though signature tool call id

* [stripe] refactor and tests

* [stripe] remove md and move to factory

* [stripe] remove redudant test

* [stripe] ran black formatting

* [stripe] add thought signature docs

* [stripe] remove unused import
* Attempt CI/CD Fix

* Adding test for coverage

* Adding max depth to copilot and vertex

* Fixing mypy lint and docker database

* Fixing UI build issues

* Update playwright test
…16929)

* add _get_prompt_data_from_dotprompt_content

* fix pre call hook for prompt template

* fix: get_latest_version_prompt_id

* fix get_latest_version_prompt_id

* test_get_latest_version_prompt_id
#16932)

* add _get_prompt_data_from_dotprompt_content

* fix pre call hook for prompt template

* fix: get_latest_version_prompt_id

* fix get_latest_version_prompt_id

* test_get_latest_version_prompt_id

* fx info and delete lookup for prompts

* refactor prompt table
…ish of showing version history (#16941)

* add _get_prompt_data_from_dotprompt_content

* fix pre call hook for prompt template

* fix: get_latest_version_prompt_id

* fix get_latest_version_prompt_id

* test_get_latest_version_prompt_id

* fx info and delete lookup for prompts

* refactor prompt table

* - rename to prompt studio

* fix get_prompt_info

* fix endpoints

* add PromptCodeSnippets

* prompt info view

* add prompt info view

* show correct version for prompts

* fix version selector

* fix endpoints and version

* fix get_prompt_info

* fix version display
Change model identifier from cerebras/openai/gpt-oss-120b to
cerebras/gpt-oss-120b to match Cerebras API requirements.

The Cerebras API only accepts 'gpt-oss-120b' as the model ID, not
'openai/gpt-oss-120b'. The previous name was causing "Model does not
exist" errors when users tried to use it.

Tested with real API calls to confirm:
- cerebras/gpt-oss-120b → sends 'gpt-oss-120b' → ✅ works
- cerebras/openai/gpt-oss-120b → sends 'openai/gpt-oss-120b' → ❌ fails

Fixes #16924
* new model - add together_ai/zai-org/GLM-4.6

* together_ai/zai-org/GLM-4.6
…ody" (#16943)

* add search_tool_name in litellm params

* test_search_tool_name_in_all_litellm_params

* bump config
* docs: fix mcp url format

* fix: update Cursor MCP example to use url instead of server_url
Add gemini-3-pro-image-preview model configuration for Google's new
image generation model (aka "Nano Banana Pro 🍌").

Model details:
- Input: $2.00/1M tokens (text), $0.0011/image
- Output: $12.00/1M tokens (text), $0.134/image (1K/2K)
- Context: 65k input / 32k output tokens
- Capabilities: structured outputs, web search, caching, thinking
- No function calling support
- Available on both Gemini API and Vertex AI

Added variants:
- gemini-3-pro-image-preview (base, uses Vertex AI)
- gemini/gemini-3-pro-image-preview (Gemini API)
- vertex_ai/gemini-3-pro-image-preview (Vertex AI)

Source: https://ai.google.dev/gemini-api/docs/pricing
Fixes: #16925
* feat: Add support for Grok 4.1 Fast models

Add new xAI Grok 4.1 Fast models optimized for high-performance agentic tool calling:

- xai/grok-4-1-fast (alias for grok-4-1-fast-reasoning)
- xai/grok-4-1-fast-reasoning (with reasoning capabilities)
- xai/grok-4-1-fast-reasoning-latest
- xai/grok-4-1-fast-non-reasoning (without reasoning for faster responses)
- xai/grok-4-1-fast-non-reasoning-latest

Features:
- Context window: 2,000,000 tokens
- Pricing: $0.20/1M input, $0.50/1M output tokens
- Cached tokens: $0.05/1M tokens
- Supports: Function calling, Structured outputs, Vision, Audio input, Web search, Reasoning

Fixes #16927

* docs: Add comprehensive Grok models documentation

- Add 'Supported Models' section highlighting new Grok 4.1 Fast models
- Include comparison guide for reasoning vs non-reasoning models
- Add complete model family table (Grok 4.1, 4, 3, Code, 2)
- Add features legend explaining capabilities
- Remove pricing details (link to xAI docs instead for current rates)
- Improve documentation clarity and consistency

Related to #16927

* docs: Minor corrections to xai.md
…sponse (#16875)

This fix addresses the same issue that was resolved for OpenAI video in PR #16708.

The GeminiVideoConfig class was importing BaseVideoConfig only within TYPE_CHECKING,
causing it to be 'Any' at runtime. This prevented the async_transform_video_content_response
method from being available during video content downloads.

Changes:
- Moved BaseVideoConfig import from TYPE_CHECKING to top-level imports
- Added test_gemini_video_config_has_async_transform() to verify the fix
- Ensures GeminiVideoConfig properly inherits BaseVideoConfig at runtime

Fixes video generation errors for Gemini Veo models:
'GeminiVideoConfig' object has no attribute 'async_transform_video_content_response'
* add DOCKER_MODEL_RUNNER

* add DockerModelRunnerChatConfig Transorm

* add docker_model_runner

* add docker_model_runner

* docs docker model runner

* add DockerModelRunnerChatConfig

* add docker_model_runner to providers

* test_completion_hits_correct_url_and_body

* fix sidebar

* TestDockerModelRunnerIntegration

* test_completion_with_custom_engine_and_host

* docs docker model runner

* docs fix
ishaan-jaff and others added 25 commits November 25, 2025 12:20
)

* test_bedrock_openai_imported_model

* AmazonBedrockOpenAIConfig

* add openai route for bedrock

* docs fix

* fix code qa check
* include server_tool_use in streaming usage

* add test
* fix transcription exception handling

* reraise the exception
* init RAG api types

* add RAG endpoints

* init main.py for RAG ingest API

* init RecursiveCharacterTextSplitter

* add BaseRAGIngestion

* fix OpenAIRAGIngestion

* fix img handler

* init OpenAIRAGIngestion

* init BedrockRAGIngestion

* init BedrockRAGIngestion

* init rag tests

* init BedrockVectorStoreOptions

* implement BedrockRAGIngestion

* add BaseRAGAPI

* add endpoint for RAG ingest

* add ingest RAG endpoints

* add test doc

* add parse_rag_ingest_request

* update endpoints

* docs add docs for new RAG API

* fix qa check

* fix linting

* docs ficx

* docs

* add max depth checks

* docs anthropic
…rmat-bridge-conversion

fix: Support response_format parameter in completion -> responses bridge
- Automatically pass LiteLLM virtual key context as X-LiteLLM-* headers
- Includes key_alias, user_id, team_id, org_id, and user_email
- No configuration required - always enabled for application/user tracking
- Excludes sensitive data (metadata, API tokens) for security
- Add comprehensive tests (30 tests, all passing)
- Update documentation with header details
This should allow postgres to perform a more efficient index scan instead of a sequential table scan.

These two queries consistently show up in the longest-running ones in our instance, and are a major latency source for the usage page on the admin UI.
…meters (#17019)

- Add model identifier to FLASH_IMAGE_PREVIEW_MODEL_IDENTIFIERS
- Add imageSize parameter support (1K, 2K, 4K) with GeminiImageSize type
- Add tests for imageSize parameter transformation
- Update documentation with new model
[Feature] UI - Disable edit, delete, info, for dynamically generated spend tags
[Feature] UI - Org Admin Team Permissions Fix
@vercel
Copy link

vercel bot commented Nov 26, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
litellm Ready Ready Preview Comment Nov 26, 2025 4:18pm

@Sameerlite Sameerlite marked this pull request as ready for review November 26, 2025 16:18
f"MCP list_resource_templates - MCP servers from context: {mcp_servers}"
)
verbose_logger.debug(
f"MCP list_resource_templates - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}"

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
This expression logs
sensitive data (password)
as clear text.

Copilot Autofix

AI 8 days ago

To fix this problem, the code should avoid logging any part of the authentication header dictionary which may contain sensitive information, even its keys (since they may contain identifying data or can be used to enumerate the types of auth mechanisms present).
The best way to fix is to either remove the log statement at line 466 entirely, or replace it with a generic message that does not output any data from mcp_server_auth_headers. If visibility is required for debugging, log only that auth headers are present, or the count (number of headers), without listing names or values.
Edit only line 466 of litellm/proxy/_experimental/mcp_server/server.py to remove the risky log statement or make it generic, ensuring you do not log sensitive data.

No new imports or methods are needed.


Suggested changeset 1
litellm/proxy/_experimental/mcp_server/server.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/proxy/_experimental/mcp_server/server.py b/litellm/proxy/_experimental/mcp_server/server.py
--- a/litellm/proxy/_experimental/mcp_server/server.py
+++ b/litellm/proxy/_experimental/mcp_server/server.py
@@ -463,7 +463,7 @@
                 f"MCP list_resource_templates - MCP servers from context: {mcp_servers}"
             )
             verbose_logger.debug(
-                f"MCP list_resource_templates - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}"
+                "MCP list_resource_templates - MCP server auth headers present: %s", bool(mcp_server_auth_headers)
             )
 
             resource_templates = await _list_mcp_resource_templates(
EOF
@@ -463,7 +463,7 @@
f"MCP list_resource_templates - MCP servers from context: {mcp_servers}"
)
verbose_logger.debug(
f"MCP list_resource_templates - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}"
"MCP list_resource_templates - MCP server auth headers present: %s", bool(mcp_server_auth_headers)
)

resource_templates = await _list_mcp_resource_templates(
Copilot is powered by AI and may make mistakes. Always verify output.
f"MCP list_resource_templates - User API Key Auth from context: {user_api_key_auth}"
)
verbose_logger.debug(
f"MCP list_resource_templates - MCP servers from context: {mcp_servers}"

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
This expression logs
sensitive data (password)
as clear text.

Copilot Autofix

AI 8 days ago

The best way to fix the issue is to avoid logging the full contents of mcp_servers directly. Instead, log only non-sensitive, high-level information, such as the count of items, or specific non-sensitive attributes if required (e.g., server "aliases" but never credentials/keys/hostnames). This can be performed by changing the affected verbose_logger.debug call on line 463 in litellm/proxy/_experimental/mcp_server/server.py to redact sensitive details, such as logging only the type of value (if present), or the count/summary information, rather than the full data structure.

Specific steps:

  • Edit the affected debug line (verbose_logger.debug(...)) in the list_resource_templates endpoint.
  • Replace it with a log statement that does NOT include the full content of mcp_servers, but instead logs, e.g., the number of servers (len(mcp_servers) if a list), or "present/absent" if None.
  • No changes to imports or additional methods are necessary.
  • This change ensures sensitive details are never written to logs, following the project logging convention.

Suggested changeset 1
litellm/proxy/_experimental/mcp_server/server.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/proxy/_experimental/mcp_server/server.py b/litellm/proxy/_experimental/mcp_server/server.py
--- a/litellm/proxy/_experimental/mcp_server/server.py
+++ b/litellm/proxy/_experimental/mcp_server/server.py
@@ -460,7 +460,7 @@
                 f"MCP list_resource_templates - User API Key Auth from context: {user_api_key_auth}"
             )
             verbose_logger.debug(
-                f"MCP list_resource_templates - MCP servers from context: {mcp_servers}"
+                f"MCP list_resource_templates - MCP servers present: {'Yes' if mcp_servers else 'No'}, number of servers: {len(mcp_servers) if mcp_servers else 0}"
             )
             verbose_logger.debug(
                 f"MCP list_resource_templates - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}"
EOF
@@ -460,7 +460,7 @@
f"MCP list_resource_templates - User API Key Auth from context: {user_api_key_auth}"
)
verbose_logger.debug(
f"MCP list_resource_templates - MCP servers from context: {mcp_servers}"
f"MCP list_resource_templates - MCP servers present: {'Yes' if mcp_servers else 'No'}, number of servers: {len(mcp_servers) if mcp_servers else 0}"
)
verbose_logger.debug(
f"MCP list_resource_templates - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}"
Copilot is powered by AI and may make mistakes. Always verify output.
raw_headers,
) = get_auth_context()
verbose_logger.debug(
f"MCP list_resource_templates - User API Key Auth from context: {user_api_key_auth}"

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
This expression logs
sensitive data (password)
as clear text.

Copilot Autofix

AI 8 days ago

To fix clear-text logging of sensitive information, you should never log raw authentication objects or credentials. Instead:

  1. Identify the region to fix:
    In litellm/proxy/_experimental/mcp_server/server.py, at line 460, the code logs the entire user_api_key_auth object.
  2. Best way:
    Only log non-sensitive, high-level information. For example, if user_api_key_auth contains or has an identifier (such as a username, safe account id, or just that the key exists), log only that non-sensitive part—or omit the log altogether if it is not necessary.
  3. Implement:
    Change the logging statement at line 460 to either:
    • Not log the user_api_key_auth at all, or
    • Log a sanitized/minimal/safe detail (e.g., type, presence, or a masked identifier).
    • This does not require new library imports since you already use logging and string formatting.
    • If the class has something like .user_id or .username, use that; otherwise log only that key is present.
Suggested changeset 1
litellm/proxy/_experimental/mcp_server/server.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/proxy/_experimental/mcp_server/server.py b/litellm/proxy/_experimental/mcp_server/server.py
--- a/litellm/proxy/_experimental/mcp_server/server.py
+++ b/litellm/proxy/_experimental/mcp_server/server.py
@@ -457,7 +457,7 @@
                 raw_headers,
             ) = get_auth_context()
             verbose_logger.debug(
-                f"MCP list_resource_templates - User API Key Auth from context: {user_api_key_auth}"
+                f"MCP list_resource_templates - User API Key Auth present: {user_api_key_auth is not None}"
             )
             verbose_logger.debug(
                 f"MCP list_resource_templates - MCP servers from context: {mcp_servers}"
EOF
@@ -457,7 +457,7 @@
raw_headers,
) = get_auth_context()
verbose_logger.debug(
f"MCP list_resource_templates - User API Key Auth from context: {user_api_key_auth}"
f"MCP list_resource_templates - User API Key Auth present: {user_api_key_auth is not None}"
)
verbose_logger.debug(
f"MCP list_resource_templates - MCP servers from context: {mcp_servers}"
Copilot is powered by AI and may make mistakes. Always verify output.
f"MCP list_resources - MCP servers from context: {mcp_servers}"
)
verbose_logger.debug(
f"MCP list_resources - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}"

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
This expression logs
sensitive data (password)
as clear text.

Copilot Autofix

AI 8 days ago

To fix this problem, we should avoid logging any part of the mcp_server_auth_headers dictionary that could contain sensitive information. The best approach is to remove this debug log line entirely, or to ensure only sanitized, non-sensitive, static data is logged instead (such as indicating presence of headers, how many keys, or listing only known-safe key names). Specifically, replace or remove line 428 in list_resources() to ensure no sensitive data—header keys or values—are ever written to logs.

Implementation:

  • Remove or heavily sanitize line 428 of list_resources in litellm/proxy/_experimental/mcp_server/server.py.
  • If logging presence of headers is helpful, you may log the number of headers or a static string instead.
  • No additional imports or dependencies are required.
Suggested changeset 1
litellm/proxy/_experimental/mcp_server/server.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/proxy/_experimental/mcp_server/server.py b/litellm/proxy/_experimental/mcp_server/server.py
--- a/litellm/proxy/_experimental/mcp_server/server.py
+++ b/litellm/proxy/_experimental/mcp_server/server.py
@@ -424,8 +424,9 @@
             verbose_logger.debug(
                 f"MCP list_resources - MCP servers from context: {mcp_servers}"
             )
+            # Avoid logging potentially sensitive auth header keys.
             verbose_logger.debug(
-                f"MCP list_resources - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}"
+                f"MCP list_resources - MCP server auth headers present: {len(mcp_server_auth_headers) if mcp_server_auth_headers else 0}"
             )
 
             resources = await _list_mcp_resources(
EOF
@@ -424,8 +424,9 @@
verbose_logger.debug(
f"MCP list_resources - MCP servers from context: {mcp_servers}"
)
# Avoid logging potentially sensitive auth header keys.
verbose_logger.debug(
f"MCP list_resources - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}"
f"MCP list_resources - MCP server auth headers present: {len(mcp_server_auth_headers) if mcp_server_auth_headers else 0}"
)

resources = await _list_mcp_resources(
Copilot is powered by AI and may make mistakes. Always verify output.
f"MCP list_resources - User API Key Auth from context: {user_api_key_auth}"
)
verbose_logger.debug(
f"MCP list_resources - MCP servers from context: {mcp_servers}"

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
This expression logs
sensitive data (password)
as clear text.

Copilot Autofix

AI 8 days ago

To address the clear-text logging of potentially sensitive data, we should avoid printing the full contents of mcp_servers directly. Instead, we should:

  1. Log only the count (number of elements) or a generic placeholder if mcp_servers is present, rather than printing full values.
  2. Optionally, if needed for diagnostics, ensure that only non-sensitive attributes (e.g., server aliases, not auth data) are logged.
  3. The fix is specific to line 425, where the log statement should be updated.
  4. No additional imports or methods are needed, only an edit to the log statement.

Suggested changeset 1
litellm/proxy/_experimental/mcp_server/server.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/proxy/_experimental/mcp_server/server.py b/litellm/proxy/_experimental/mcp_server/server.py
--- a/litellm/proxy/_experimental/mcp_server/server.py
+++ b/litellm/proxy/_experimental/mcp_server/server.py
@@ -422,7 +422,7 @@
                 f"MCP list_resources - User API Key Auth from context: {user_api_key_auth}"
             )
             verbose_logger.debug(
-                f"MCP list_resources - MCP servers from context: {mcp_servers}"
+                f"MCP list_resources - MCP servers from context: ({len(mcp_servers) if mcp_servers else 0} servers)"
             )
             verbose_logger.debug(
                 f"MCP list_resources - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}"
EOF
@@ -422,7 +422,7 @@
f"MCP list_resources - User API Key Auth from context: {user_api_key_auth}"
)
verbose_logger.debug(
f"MCP list_resources - MCP servers from context: {mcp_servers}"
f"MCP list_resources - MCP servers from context: ({len(mcp_servers) if mcp_servers else 0} servers)"
)
verbose_logger.debug(
f"MCP list_resources - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}"
Copilot is powered by AI and may make mistakes. Always verify output.
raw_headers,
) = get_auth_context()
verbose_logger.debug(
f"MCP list_prompts - User API Key Auth from context: {user_api_key_auth}"

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (password)
as clear text.
This expression logs
sensitive data (password)
as clear text.

Copilot Autofix

AI 8 days ago

The best way to fix this problem is to avoid logging sensitive details. Instead, log only non-sensitive metadata such as existence, type, or user id—never the actual key or credential values.

  • In file litellm/proxy/_experimental/mcp_server/server.py, in the list_prompts async function, change the log statement at line 342 to redact the sensitive details, e.g., log only whether user_api_key_auth exists or (if it has a user_id or similar attribute) log that instead.
  • Do not log the string representation of the full object.
  • No changes elsewhere or additional imports are needed.

Suggested changeset 1
litellm/proxy/_experimental/mcp_server/server.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/proxy/_experimental/mcp_server/server.py b/litellm/proxy/_experimental/mcp_server/server.py
--- a/litellm/proxy/_experimental/mcp_server/server.py
+++ b/litellm/proxy/_experimental/mcp_server/server.py
@@ -339,7 +339,7 @@
                 raw_headers,
             ) = get_auth_context()
             verbose_logger.debug(
-                f"MCP list_prompts - User API Key Auth from context: {user_api_key_auth}"
+                f"MCP list_prompts - User API Key Auth from context: {'present' if user_api_key_auth is not None else 'missing'}"
             )
             verbose_logger.debug(
                 f"MCP list_prompts - MCP servers from context: {mcp_servers}"
EOF
@@ -339,7 +339,7 @@
raw_headers,
) = get_auth_context()
verbose_logger.debug(
f"MCP list_prompts - User API Key Auth from context: {user_api_key_auth}"
f"MCP list_prompts - User API Key Auth from context: {'present' if user_api_key_auth is not None else 'missing'}"
)
verbose_logger.debug(
f"MCP list_prompts - MCP servers from context: {mcp_servers}"
Copilot is powered by AI and may make mistakes. Always verify output.
f"Authorization={urllib.parse.quote(f'Bearer {api_key}')}"
if api_key is not None:
otlp_auth_headers = f"Authorization=Bearer {api_key}"
elif "app.phoenix.arize.com" in endpoint:

Check failure

Code scanning / CodeQL

Incomplete URL substring sanitization High

The string
app.phoenix.arize.com
may be at an arbitrary position in the sanitized URL.

Copilot Autofix

AI 8 days ago

To fix this, we should parse the endpoint URL using Python's urllib.parse.urlparse, extract its hostname, and check if it matches the trusted host (app.phoenix.arize.com) or its allowed subdomains (using explicit matching). The substring check ("app.phoenix.arize.com" in endpoint) should be replaced with code that parses endpoint and performs the check on the hostname field. The only lines to change are those that check for "app.phoenix.arize.com" in the endpoint string, which is line 61 and 86 in your code. We'll import urllib.parse.urlparse at the top, and replace both usages with parsed hostname checks.


Suggested changeset 1
litellm/integrations/arize/arize_phoenix.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/integrations/arize/arize_phoenix.py b/litellm/integrations/arize/arize_phoenix.py
--- a/litellm/integrations/arize/arize_phoenix.py
+++ b/litellm/integrations/arize/arize_phoenix.py
@@ -1,5 +1,6 @@
 import os
 from typing import TYPE_CHECKING, Any, Union
+from urllib.parse import urlparse
 
 from litellm._logging import verbose_logger
 from litellm.integrations.arize import _utils
@@ -58,7 +59,11 @@
                 protocol = "otlp_grpc"
             else:
                 # Phoenix Cloud endpoints (app.phoenix.arize.com) include the space in the URL
-                if "app.phoenix.arize.com" in collector_endpoint:
+                collector_hostname = urlparse(collector_endpoint).hostname
+                if collector_hostname is not None and (
+                    collector_hostname == "app.phoenix.arize.com"
+                    or collector_hostname.endswith(".app.phoenix.arize.com")
+                ):
                     endpoint = collector_endpoint
                     protocol = "otlp_http"
                 # For other HTTP endpoints, ensure they have the correct path
@@ -83,11 +88,16 @@
         otlp_auth_headers = None
         if api_key is not None:
             otlp_auth_headers = f"Authorization=Bearer {api_key}"
-        elif "app.phoenix.arize.com" in endpoint:
-            # Phoenix Cloud requires an API key
-            raise ValueError(
-                "PHOENIX_API_KEY must be set when using Phoenix Cloud (app.phoenix.arize.com)."
-            )
+        else:
+            endpoint_hostname = urlparse(endpoint).hostname
+            if endpoint_hostname is not None and (
+                endpoint_hostname == "app.phoenix.arize.com"
+                or endpoint_hostname.endswith(".app.phoenix.arize.com")
+            ):
+                # Phoenix Cloud requires an API key
+                raise ValueError(
+                    "PHOENIX_API_KEY must be set when using Phoenix Cloud (app.phoenix.arize.com)."
+                )
 
         project_name = os.environ.get("PHOENIX_PROJECT_NAME", "litellm-project")
 
EOF
@@ -1,5 +1,6 @@
import os
from typing import TYPE_CHECKING, Any, Union
from urllib.parse import urlparse

from litellm._logging import verbose_logger
from litellm.integrations.arize import _utils
@@ -58,7 +59,11 @@
protocol = "otlp_grpc"
else:
# Phoenix Cloud endpoints (app.phoenix.arize.com) include the space in the URL
if "app.phoenix.arize.com" in collector_endpoint:
collector_hostname = urlparse(collector_endpoint).hostname
if collector_hostname is not None and (
collector_hostname == "app.phoenix.arize.com"
or collector_hostname.endswith(".app.phoenix.arize.com")
):
endpoint = collector_endpoint
protocol = "otlp_http"
# For other HTTP endpoints, ensure they have the correct path
@@ -83,11 +88,16 @@
otlp_auth_headers = None
if api_key is not None:
otlp_auth_headers = f"Authorization=Bearer {api_key}"
elif "app.phoenix.arize.com" in endpoint:
# Phoenix Cloud requires an API key
raise ValueError(
"PHOENIX_API_KEY must be set when using Phoenix Cloud (app.phoenix.arize.com)."
)
else:
endpoint_hostname = urlparse(endpoint).hostname
if endpoint_hostname is not None and (
endpoint_hostname == "app.phoenix.arize.com"
or endpoint_hostname.endswith(".app.phoenix.arize.com")
):
# Phoenix Cloud requires an API key
raise ValueError(
"PHOENIX_API_KEY must be set when using Phoenix Cloud (app.phoenix.arize.com)."
)

project_name = os.environ.get("PHOENIX_PROJECT_NAME", "litellm-project")

Copilot is powered by AI and may make mistakes. Always verify output.
protocol = "otlp_grpc"
else:
# Phoenix Cloud endpoints (app.phoenix.arize.com) include the space in the URL
if "app.phoenix.arize.com" in collector_endpoint:

Check failure

Code scanning / CodeQL

Incomplete URL substring sanitization High

The string
app.phoenix.arize.com
may be at an arbitrary position in the sanitized URL.

Copilot Autofix

AI 8 days ago

To fix this issue, we need to parse collector_endpoint as a URL, extract its hostname, and then check if it matches or ends with the expected domain (e.g., "app.phoenix.arize.com"). We should do this instead of using the string containment (in) check. We'll use Python's standard urllib.parse library for this purpose, as it provides a safe way to extract the hostname. The change applies to line 61 in litellm/integrations/arize/arize_phoenix.py and any other "app.phoenix.arize.com" in ... checks, specifically line 86 for the value in endpoint. We'll need to import urlparse at the top. Only these spots are in scope for the change.

Suggested changeset 1
litellm/integrations/arize/arize_phoenix.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/integrations/arize/arize_phoenix.py b/litellm/integrations/arize/arize_phoenix.py
--- a/litellm/integrations/arize/arize_phoenix.py
+++ b/litellm/integrations/arize/arize_phoenix.py
@@ -1,6 +1,6 @@
 import os
 from typing import TYPE_CHECKING, Any, Union
-
+from urllib.parse import urlparse
 from litellm._logging import verbose_logger
 from litellm.integrations.arize import _utils
 from litellm.integrations.arize._utils import ArizeOTELAttributes
@@ -58,7 +58,7 @@
                 protocol = "otlp_grpc"
             else:
                 # Phoenix Cloud endpoints (app.phoenix.arize.com) include the space in the URL
-                if "app.phoenix.arize.com" in collector_endpoint:
+                if urlparse(collector_endpoint).hostname == "app.phoenix.arize.com":
                     endpoint = collector_endpoint
                     protocol = "otlp_http"
                 # For other HTTP endpoints, ensure they have the correct path
@@ -83,7 +83,7 @@
         otlp_auth_headers = None
         if api_key is not None:
             otlp_auth_headers = f"Authorization=Bearer {api_key}"
-        elif "app.phoenix.arize.com" in endpoint:
+        elif urlparse(endpoint).hostname == "app.phoenix.arize.com":
             # Phoenix Cloud requires an API key
             raise ValueError(
                 "PHOENIX_API_KEY must be set when using Phoenix Cloud (app.phoenix.arize.com)."
EOF
@@ -1,6 +1,6 @@
import os
from typing import TYPE_CHECKING, Any, Union

from urllib.parse import urlparse
from litellm._logging import verbose_logger
from litellm.integrations.arize import _utils
from litellm.integrations.arize._utils import ArizeOTELAttributes
@@ -58,7 +58,7 @@
protocol = "otlp_grpc"
else:
# Phoenix Cloud endpoints (app.phoenix.arize.com) include the space in the URL
if "app.phoenix.arize.com" in collector_endpoint:
if urlparse(collector_endpoint).hostname == "app.phoenix.arize.com":
endpoint = collector_endpoint
protocol = "otlp_http"
# For other HTTP endpoints, ensure they have the correct path
@@ -83,7 +83,7 @@
otlp_auth_headers = None
if api_key is not None:
otlp_auth_headers = f"Authorization=Bearer {api_key}"
elif "app.phoenix.arize.com" in endpoint:
elif urlparse(endpoint).hostname == "app.phoenix.arize.com":
# Phoenix Cloud requires an API key
raise ValueError(
"PHOENIX_API_KEY must be set when using Phoenix Cloud (app.phoenix.arize.com)."
Copilot is powered by AI and may make mistakes. Always verify output.
api_base=api_base, endpoint="skills", skill_id=skill_id
)

verbose_logger.debug("Delete skill request - URL: %s", url)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.

Copilot Autofix

AI 8 days ago

To fix the problem, we should prevent logging of API URLs that may contain sensitive or secret information, especially those derived from environment variables or secret managers. Concretely, in transform_delete_skill_request (and similar places), where the full URL including api_base is logged, we need to mask or avoid logging the actual value. A good approach is to redact the sensitive part:

  • Never log the api_base directly.
  • If logging is essential for debugging, log only the endpoint path or use a string like [REDACTED] in place of secrets.
  • Update the logger line on line 195 and line 166 to either remove the variable or mask it appropriately (e.g., "Delete skill request - URL: [REDACTED]").
  • Similarly update any other debug logs which output sensitive URLs, such as line 166 (Get skill request - URL: %s) if it uses the same mechanism.

No additional methods or imports are needed, just a change in the arguments to the logger.

Suggested changeset 1
litellm/llms/anthropic/skills/transformation.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/llms/anthropic/skills/transformation.py b/litellm/llms/anthropic/skills/transformation.py
--- a/litellm/llms/anthropic/skills/transformation.py
+++ b/litellm/llms/anthropic/skills/transformation.py
@@ -163,7 +163,7 @@
             api_base=api_base, endpoint="skills", skill_id=skill_id
         )
         
-        verbose_logger.debug("Get skill request - URL: %s", url)
+        verbose_logger.debug("Get skill request - URL: [REDACTED]")
         
         return url, headers
 
@@ -192,7 +192,7 @@
             api_base=api_base, endpoint="skills", skill_id=skill_id
         )
         
-        verbose_logger.debug("Delete skill request - URL: %s", url)
+        verbose_logger.debug("Delete skill request - URL: [REDACTED]")
         
         return url, headers
 
EOF
@@ -163,7 +163,7 @@
api_base=api_base, endpoint="skills", skill_id=skill_id
)

verbose_logger.debug("Get skill request - URL: %s", url)
verbose_logger.debug("Get skill request - URL: [REDACTED]")

return url, headers

@@ -192,7 +192,7 @@
api_base=api_base, endpoint="skills", skill_id=skill_id
)

verbose_logger.debug("Delete skill request - URL: %s", url)
verbose_logger.debug("Delete skill request - URL: [REDACTED]")

return url, headers

Copilot is powered by AI and may make mistakes. Always verify output.
api_base=api_base, endpoint="skills", skill_id=skill_id
)

verbose_logger.debug("Get skill request - URL: %s", url)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.
This expression logs
sensitive data (secret)
as clear text.

Copilot Autofix

AI 8 days ago

To fix the problem, avoid logging the entire request URL with potential secrets in logs. Instead, log only non-sensitive components (such as endpoint names, IDs, or operation types), or redact the sensitive api_base portion. Specifically, in transform_get_skill_request, on line 166, replace the logging of url with either a redacted version or log only the endpoint/skill ID. Other places in the code (shown here) do not log sensitive endpoints directly.

No additional dependencies are required; only update or replace the logging statement on line 166 to ensure no clear-text sensitive values are written to logs. For example, log "Get skill request" with just the endpoint and skill ID, and replace any logging of the full URL.


Suggested changeset 1
litellm/llms/anthropic/skills/transformation.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/llms/anthropic/skills/transformation.py b/litellm/llms/anthropic/skills/transformation.py
--- a/litellm/llms/anthropic/skills/transformation.py
+++ b/litellm/llms/anthropic/skills/transformation.py
@@ -163,7 +163,9 @@
             api_base=api_base, endpoint="skills", skill_id=skill_id
         )
         
-        verbose_logger.debug("Get skill request - URL: %s", url)
+        verbose_logger.debug(
+            "Get skill request for endpoint 'skills', skill_id: %s", skill_id
+        )
         
         return url, headers
 
EOF
@@ -163,7 +163,9 @@
api_base=api_base, endpoint="skills", skill_id=skill_id
)

verbose_logger.debug("Get skill request - URL: %s", url)
verbose_logger.debug(
"Get skill request for endpoint 'skills', skill_id: %s", skill_id
)

return url, headers

Copilot is powered by AI and may make mistakes. Always verify output.
@Sameerlite Sameerlite merged commit 1c317ac into litellm_vertex_ai_anthopic_cost_tracking Nov 26, 2025
48 of 72 checks passed
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
11 out of 12 committers have signed the CLA.

✅ Sameerlite
✅ krrishdholakia
✅ ishaan-jaff
✅ uc4w6c
✅ reflection
✅ CAFxX
✅ AlexsanderHamir
✅ otaviofbrito
✅ eagle-p
✅ choigawoon
✅ yuneng-jiang
❌ KeremTurgutlu
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.