-
-
Notifications
You must be signed in to change notification settings - Fork 4.9k
merge main #17145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
merge main #17145
Conversation
Fixes #16810 ## Problem When using completion() with models that have mode: "responses" (like o3-pro, gpt-5-codex), the response_format parameter with JSON schemas was being ignored or incorrectly handled, causing: - Large schemas (>512 chars) to fail with "metadata.schema_dict_json: string too long" error - Structured outputs to be silently dropped - Users' code to break unexpectedly ## Root Cause The completion -> responses bridge in litellm/completion_extras/litellm_responses_transformation/transformation.py was missing the conversion of response_format (Chat Completion format) to text.format (Responses API format). The inverse bridge (responses -> completion) already had this conversion implemented in commit 29f0ed2, but the completion -> responses direction was incomplete. ## Solution Added _transform_response_format_to_text_format() method that converts: - response_format with json_schema → text.format with json_schema - response_format with json_object → text.format with json_object - response_format with text → text.format with text Updated transform_request() to detect and convert response_format parameter before sending to litellm.responses(). ## Changes - Added _transform_response_format_to_text_format() method (lines 592-647) - Modified transform_request() to handle response_format (lines 199-203) - Added comprehensive tests to validate the conversion ## Testing - 5 new unit tests covering all conversion scenarios - Real API test with OpenAI confirming large schemas (>512 chars) work - No more metadata.schema_dict_json errors ## Impact Users can now use completion() with models that have mode: "responses" and: - Use large JSON schemas without hitting metadata 512 char limit - Get proper structured outputs - Have their existing code continue working
* add AWS fields for KeyManagementSettings * docs IAM roles * use aws iam auth on secret manager v2 * fix: load_aws_secret_manager * test_secret_manager_with_iam_role_settings
* feat: mcp prompts support * feat: mcp resources support
Updated pydantic version to 2.11.0 for compatibility.
…16898) * TestPromptRequest * add prompts/test endpoint for testing prompt * TestPromptTestEndpoint * feat: working v1 of this ui * workig prompt endpoints * add chat ui for prompts * add conversation panel * add init chat ui
Co-authored-by: Cursor Agent <[email protected]> Co-authored-by: ishaan <[email protected]>
* TestPromptRequest * add prompts/test endpoint for testing prompt * TestPromptTestEndpoint * feat: working v1 of this ui * workig prompt endpoints * add chat ui for prompts * add conversation panel * add init chat ui * allow clicking edit prompt * fix use get_base_prompt_id * add endpoints for viewing prompt versions * TestPromptVersioning * add getPromptVersions * add VersionHistorySidePanel * allow viewing version history * add version history
* fix images being dropped from tool results for bedrock * type fixes
* though signature tool call id * [stripe] refactor and tests * [stripe] remove md and move to factory * [stripe] remove redudant test * [stripe] ran black formatting * [stripe] add thought signature docs * [stripe] remove unused import
* Attempt CI/CD Fix * Adding test for coverage * Adding max depth to copilot and vertex * Fixing mypy lint and docker database * Fixing UI build issues * Update playwright test
…16929) * add _get_prompt_data_from_dotprompt_content * fix pre call hook for prompt template * fix: get_latest_version_prompt_id * fix get_latest_version_prompt_id * test_get_latest_version_prompt_id
#16932) * add _get_prompt_data_from_dotprompt_content * fix pre call hook for prompt template * fix: get_latest_version_prompt_id * fix get_latest_version_prompt_id * test_get_latest_version_prompt_id * fx info and delete lookup for prompts * refactor prompt table
…ish of showing version history (#16941) * add _get_prompt_data_from_dotprompt_content * fix pre call hook for prompt template * fix: get_latest_version_prompt_id * fix get_latest_version_prompt_id * test_get_latest_version_prompt_id * fx info and delete lookup for prompts * refactor prompt table * - rename to prompt studio * fix get_prompt_info * fix endpoints * add PromptCodeSnippets * prompt info view * add prompt info view * show correct version for prompts * fix version selector * fix endpoints and version * fix get_prompt_info * fix version display
Change model identifier from cerebras/openai/gpt-oss-120b to cerebras/gpt-oss-120b to match Cerebras API requirements. The Cerebras API only accepts 'gpt-oss-120b' as the model ID, not 'openai/gpt-oss-120b'. The previous name was causing "Model does not exist" errors when users tried to use it. Tested with real API calls to confirm: - cerebras/gpt-oss-120b → sends 'gpt-oss-120b' → ✅ works - cerebras/openai/gpt-oss-120b → sends 'openai/gpt-oss-120b' → ❌ fails Fixes #16924
* new model - add together_ai/zai-org/GLM-4.6 * together_ai/zai-org/GLM-4.6
…ody" (#16943) * add search_tool_name in litellm params * test_search_tool_name_in_all_litellm_params * bump config
* docs: fix mcp url format * fix: update Cursor MCP example to use url instead of server_url
Add gemini-3-pro-image-preview model configuration for Google's new image generation model (aka "Nano Banana Pro 🍌"). Model details: - Input: $2.00/1M tokens (text), $0.0011/image - Output: $12.00/1M tokens (text), $0.134/image (1K/2K) - Context: 65k input / 32k output tokens - Capabilities: structured outputs, web search, caching, thinking - No function calling support - Available on both Gemini API and Vertex AI Added variants: - gemini-3-pro-image-preview (base, uses Vertex AI) - gemini/gemini-3-pro-image-preview (Gemini API) - vertex_ai/gemini-3-pro-image-preview (Vertex AI) Source: https://ai.google.dev/gemini-api/docs/pricing Fixes: #16925
* feat: Add support for Grok 4.1 Fast models Add new xAI Grok 4.1 Fast models optimized for high-performance agentic tool calling: - xai/grok-4-1-fast (alias for grok-4-1-fast-reasoning) - xai/grok-4-1-fast-reasoning (with reasoning capabilities) - xai/grok-4-1-fast-reasoning-latest - xai/grok-4-1-fast-non-reasoning (without reasoning for faster responses) - xai/grok-4-1-fast-non-reasoning-latest Features: - Context window: 2,000,000 tokens - Pricing: $0.20/1M input, $0.50/1M output tokens - Cached tokens: $0.05/1M tokens - Supports: Function calling, Structured outputs, Vision, Audio input, Web search, Reasoning Fixes #16927 * docs: Add comprehensive Grok models documentation - Add 'Supported Models' section highlighting new Grok 4.1 Fast models - Include comparison guide for reasoning vs non-reasoning models - Add complete model family table (Grok 4.1, 4, 3, Code, 2) - Add features legend explaining capabilities - Remove pricing details (link to xAI docs instead for current rates) - Improve documentation clarity and consistency Related to #16927 * docs: Minor corrections to xai.md
…sponse (#16875) This fix addresses the same issue that was resolved for OpenAI video in PR #16708. The GeminiVideoConfig class was importing BaseVideoConfig only within TYPE_CHECKING, causing it to be 'Any' at runtime. This prevented the async_transform_video_content_response method from being available during video content downloads. Changes: - Moved BaseVideoConfig import from TYPE_CHECKING to top-level imports - Added test_gemini_video_config_has_async_transform() to verify the fix - Ensures GeminiVideoConfig properly inherits BaseVideoConfig at runtime Fixes video generation errors for Gemini Veo models: 'GeminiVideoConfig' object has no attribute 'async_transform_video_content_response'
* add DOCKER_MODEL_RUNNER * add DockerModelRunnerChatConfig Transorm * add docker_model_runner * add docker_model_runner * docs docker model runner * add DockerModelRunnerChatConfig * add docker_model_runner to providers * test_completion_hits_correct_url_and_body * fix sidebar * TestDockerModelRunnerIntegration * test_completion_with_custom_engine_and_host * docs docker model runner * docs fix
* include server_tool_use in streaming usage * add test
* fix transcription exception handling * reraise the exception
* init RAG api types * add RAG endpoints * init main.py for RAG ingest API * init RecursiveCharacterTextSplitter * add BaseRAGIngestion * fix OpenAIRAGIngestion * fix img handler * init OpenAIRAGIngestion * init BedrockRAGIngestion * init BedrockRAGIngestion * init rag tests * init BedrockVectorStoreOptions * implement BedrockRAGIngestion * add BaseRAGAPI * add endpoint for RAG ingest * add ingest RAG endpoints * add test doc * add parse_rag_ingest_request * update endpoints * docs add docs for new RAG API * fix qa check * fix linting * docs ficx * docs * add max depth checks * docs anthropic
…rmat-bridge-conversion fix: Support response_format parameter in completion -> responses bridge
- Automatically pass LiteLLM virtual key context as X-LiteLLM-* headers - Includes key_alias, user_id, team_id, org_id, and user_email - No configuration required - always enabled for application/user tracking - Excludes sensitive data (metadata, API tokens) for security - Add comprehensive tests (30 tests, all passing) - Update documentation with header details
This should allow postgres to perform a more efficient index scan instead of a sequential table scan. These two queries consistently show up in the longest-running ones in our instance, and are a major latency source for the usage page on the admin UI.
…meters (#17019) - Add model identifier to FLASH_IMAGE_PREVIEW_MODEL_IDENTIFIERS - Add imageSize parameter support (1K, 2K, 4K) with GeminiImageSize type - Add tests for imageSize parameter transformation - Update documentation with new model
[Feature] UI - Disable edit, delete, info, for dynamically generated spend tags
[Feature] UI - User Table Sort by All
[Feature] UI - Org Admin Team Permissions Fix
Fix videos lint errors
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
| f"MCP list_resource_templates - MCP servers from context: {mcp_servers}" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_resource_templates - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}" |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
sensitive data (password)
This expression logs
sensitive data (password)
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 8 days ago
To fix this problem, the code should avoid logging any part of the authentication header dictionary which may contain sensitive information, even its keys (since they may contain identifying data or can be used to enumerate the types of auth mechanisms present).
The best way to fix is to either remove the log statement at line 466 entirely, or replace it with a generic message that does not output any data from mcp_server_auth_headers. If visibility is required for debugging, log only that auth headers are present, or the count (number of headers), without listing names or values.
Edit only line 466 of litellm/proxy/_experimental/mcp_server/server.py to remove the risky log statement or make it generic, ensuring you do not log sensitive data.
No new imports or methods are needed.
-
Copy modified line R466
| @@ -463,7 +463,7 @@ | ||
| f"MCP list_resource_templates - MCP servers from context: {mcp_servers}" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_resource_templates - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}" | ||
| "MCP list_resource_templates - MCP server auth headers present: %s", bool(mcp_server_auth_headers) | ||
| ) | ||
|
|
||
| resource_templates = await _list_mcp_resource_templates( |
| f"MCP list_resource_templates - User API Key Auth from context: {user_api_key_auth}" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_resource_templates - MCP servers from context: {mcp_servers}" |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
sensitive data (password)
This expression logs
sensitive data (password)
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 8 days ago
The best way to fix the issue is to avoid logging the full contents of mcp_servers directly. Instead, log only non-sensitive, high-level information, such as the count of items, or specific non-sensitive attributes if required (e.g., server "aliases" but never credentials/keys/hostnames). This can be performed by changing the affected verbose_logger.debug call on line 463 in litellm/proxy/_experimental/mcp_server/server.py to redact sensitive details, such as logging only the type of value (if present), or the count/summary information, rather than the full data structure.
Specific steps:
- Edit the affected debug line (
verbose_logger.debug(...)) in the list_resource_templates endpoint. - Replace it with a log statement that does NOT include the full content of
mcp_servers, but instead logs, e.g., the number of servers (len(mcp_servers)if a list), or "present/absent" if None. - No changes to imports or additional methods are necessary.
- This change ensures sensitive details are never written to logs, following the project logging convention.
-
Copy modified line R463
| @@ -460,7 +460,7 @@ | ||
| f"MCP list_resource_templates - User API Key Auth from context: {user_api_key_auth}" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_resource_templates - MCP servers from context: {mcp_servers}" | ||
| f"MCP list_resource_templates - MCP servers present: {'Yes' if mcp_servers else 'No'}, number of servers: {len(mcp_servers) if mcp_servers else 0}" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_resource_templates - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}" |
| raw_headers, | ||
| ) = get_auth_context() | ||
| verbose_logger.debug( | ||
| f"MCP list_resource_templates - User API Key Auth from context: {user_api_key_auth}" |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
sensitive data (password)
This expression logs
sensitive data (password)
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 8 days ago
To fix clear-text logging of sensitive information, you should never log raw authentication objects or credentials. Instead:
- Identify the region to fix:
Inlitellm/proxy/_experimental/mcp_server/server.py, at line 460, the code logs the entireuser_api_key_authobject. - Best way:
Only log non-sensitive, high-level information. For example, ifuser_api_key_authcontains or has an identifier (such as a username, safe account id, or just that the key exists), log only that non-sensitive part—or omit the log altogether if it is not necessary. - Implement:
Change the logging statement at line 460 to either:- Not log the
user_api_key_authat all, or - Log a sanitized/minimal/safe detail (e.g., type, presence, or a masked identifier).
- This does not require new library imports since you already use logging and string formatting.
- If the class has something like
.user_idor.username, use that; otherwise log only that key is present.
- Not log the
-
Copy modified line R460
| @@ -457,7 +457,7 @@ | ||
| raw_headers, | ||
| ) = get_auth_context() | ||
| verbose_logger.debug( | ||
| f"MCP list_resource_templates - User API Key Auth from context: {user_api_key_auth}" | ||
| f"MCP list_resource_templates - User API Key Auth present: {user_api_key_auth is not None}" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_resource_templates - MCP servers from context: {mcp_servers}" |
| f"MCP list_resources - MCP servers from context: {mcp_servers}" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_resources - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}" |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
sensitive data (password)
This expression logs
sensitive data (password)
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 8 days ago
To fix this problem, we should avoid logging any part of the mcp_server_auth_headers dictionary that could contain sensitive information. The best approach is to remove this debug log line entirely, or to ensure only sanitized, non-sensitive, static data is logged instead (such as indicating presence of headers, how many keys, or listing only known-safe key names). Specifically, replace or remove line 428 in list_resources() to ensure no sensitive data—header keys or values—are ever written to logs.
Implementation:
- Remove or heavily sanitize line 428 of
list_resourcesinlitellm/proxy/_experimental/mcp_server/server.py. - If logging presence of headers is helpful, you may log the number of headers or a static string instead.
- No additional imports or dependencies are required.
-
Copy modified line R427 -
Copy modified line R429
| @@ -424,8 +424,9 @@ | ||
| verbose_logger.debug( | ||
| f"MCP list_resources - MCP servers from context: {mcp_servers}" | ||
| ) | ||
| # Avoid logging potentially sensitive auth header keys. | ||
| verbose_logger.debug( | ||
| f"MCP list_resources - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}" | ||
| f"MCP list_resources - MCP server auth headers present: {len(mcp_server_auth_headers) if mcp_server_auth_headers else 0}" | ||
| ) | ||
|
|
||
| resources = await _list_mcp_resources( |
| f"MCP list_resources - User API Key Auth from context: {user_api_key_auth}" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_resources - MCP servers from context: {mcp_servers}" |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
sensitive data (password)
This expression logs
sensitive data (password)
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 8 days ago
To address the clear-text logging of potentially sensitive data, we should avoid printing the full contents of mcp_servers directly. Instead, we should:
- Log only the count (number of elements) or a generic placeholder if
mcp_serversis present, rather than printing full values. - Optionally, if needed for diagnostics, ensure that only non-sensitive attributes (e.g., server aliases, not auth data) are logged.
- The fix is specific to line 425, where the log statement should be updated.
- No additional imports or methods are needed, only an edit to the log statement.
-
Copy modified line R425
| @@ -422,7 +422,7 @@ | ||
| f"MCP list_resources - User API Key Auth from context: {user_api_key_auth}" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_resources - MCP servers from context: {mcp_servers}" | ||
| f"MCP list_resources - MCP servers from context: ({len(mcp_servers) if mcp_servers else 0} servers)" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_resources - MCP server auth headers: {list(mcp_server_auth_headers.keys()) if mcp_server_auth_headers else None}" |
| raw_headers, | ||
| ) = get_auth_context() | ||
| verbose_logger.debug( | ||
| f"MCP list_prompts - User API Key Auth from context: {user_api_key_auth}" |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
sensitive data (password)
This expression logs
sensitive data (password)
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 8 days ago
The best way to fix this problem is to avoid logging sensitive details. Instead, log only non-sensitive metadata such as existence, type, or user id—never the actual key or credential values.
- In file
litellm/proxy/_experimental/mcp_server/server.py, in thelist_promptsasync function, change the log statement at line 342 to redact the sensitive details, e.g., log only whetheruser_api_key_authexists or (if it has auser_idor similar attribute) log that instead. - Do not log the string representation of the full object.
- No changes elsewhere or additional imports are needed.
-
Copy modified line R342
| @@ -339,7 +339,7 @@ | ||
| raw_headers, | ||
| ) = get_auth_context() | ||
| verbose_logger.debug( | ||
| f"MCP list_prompts - User API Key Auth from context: {user_api_key_auth}" | ||
| f"MCP list_prompts - User API Key Auth from context: {'present' if user_api_key_auth is not None else 'missing'}" | ||
| ) | ||
| verbose_logger.debug( | ||
| f"MCP list_prompts - MCP servers from context: {mcp_servers}" |
| f"Authorization={urllib.parse.quote(f'Bearer {api_key}')}" | ||
| if api_key is not None: | ||
| otlp_auth_headers = f"Authorization=Bearer {api_key}" | ||
| elif "app.phoenix.arize.com" in endpoint: |
Check failure
Code scanning / CodeQL
Incomplete URL substring sanitization High
app.phoenix.arize.com
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 8 days ago
To fix this, we should parse the endpoint URL using Python's urllib.parse.urlparse, extract its hostname, and check if it matches the trusted host (app.phoenix.arize.com) or its allowed subdomains (using explicit matching). The substring check ("app.phoenix.arize.com" in endpoint) should be replaced with code that parses endpoint and performs the check on the hostname field. The only lines to change are those that check for "app.phoenix.arize.com" in the endpoint string, which is line 61 and 86 in your code. We'll import urllib.parse.urlparse at the top, and replace both usages with parsed hostname checks.
-
Copy modified line R3 -
Copy modified lines R62-R66 -
Copy modified lines R91-R100
| @@ -1,5 +1,6 @@ | ||
| import os | ||
| from typing import TYPE_CHECKING, Any, Union | ||
| from urllib.parse import urlparse | ||
|
|
||
| from litellm._logging import verbose_logger | ||
| from litellm.integrations.arize import _utils | ||
| @@ -58,7 +59,11 @@ | ||
| protocol = "otlp_grpc" | ||
| else: | ||
| # Phoenix Cloud endpoints (app.phoenix.arize.com) include the space in the URL | ||
| if "app.phoenix.arize.com" in collector_endpoint: | ||
| collector_hostname = urlparse(collector_endpoint).hostname | ||
| if collector_hostname is not None and ( | ||
| collector_hostname == "app.phoenix.arize.com" | ||
| or collector_hostname.endswith(".app.phoenix.arize.com") | ||
| ): | ||
| endpoint = collector_endpoint | ||
| protocol = "otlp_http" | ||
| # For other HTTP endpoints, ensure they have the correct path | ||
| @@ -83,11 +88,16 @@ | ||
| otlp_auth_headers = None | ||
| if api_key is not None: | ||
| otlp_auth_headers = f"Authorization=Bearer {api_key}" | ||
| elif "app.phoenix.arize.com" in endpoint: | ||
| # Phoenix Cloud requires an API key | ||
| raise ValueError( | ||
| "PHOENIX_API_KEY must be set when using Phoenix Cloud (app.phoenix.arize.com)." | ||
| ) | ||
| else: | ||
| endpoint_hostname = urlparse(endpoint).hostname | ||
| if endpoint_hostname is not None and ( | ||
| endpoint_hostname == "app.phoenix.arize.com" | ||
| or endpoint_hostname.endswith(".app.phoenix.arize.com") | ||
| ): | ||
| # Phoenix Cloud requires an API key | ||
| raise ValueError( | ||
| "PHOENIX_API_KEY must be set when using Phoenix Cloud (app.phoenix.arize.com)." | ||
| ) | ||
|
|
||
| project_name = os.environ.get("PHOENIX_PROJECT_NAME", "litellm-project") | ||
|
|
| protocol = "otlp_grpc" | ||
| else: | ||
| # Phoenix Cloud endpoints (app.phoenix.arize.com) include the space in the URL | ||
| if "app.phoenix.arize.com" in collector_endpoint: |
Check failure
Code scanning / CodeQL
Incomplete URL substring sanitization High
app.phoenix.arize.com
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 8 days ago
To fix this issue, we need to parse collector_endpoint as a URL, extract its hostname, and then check if it matches or ends with the expected domain (e.g., "app.phoenix.arize.com"). We should do this instead of using the string containment (in) check. We'll use Python's standard urllib.parse library for this purpose, as it provides a safe way to extract the hostname. The change applies to line 61 in litellm/integrations/arize/arize_phoenix.py and any other "app.phoenix.arize.com" in ... checks, specifically line 86 for the value in endpoint. We'll need to import urlparse at the top. Only these spots are in scope for the change.
-
Copy modified line R3 -
Copy modified line R61 -
Copy modified line R86
| @@ -1,6 +1,6 @@ | ||
| import os | ||
| from typing import TYPE_CHECKING, Any, Union | ||
|
|
||
| from urllib.parse import urlparse | ||
| from litellm._logging import verbose_logger | ||
| from litellm.integrations.arize import _utils | ||
| from litellm.integrations.arize._utils import ArizeOTELAttributes | ||
| @@ -58,7 +58,7 @@ | ||
| protocol = "otlp_grpc" | ||
| else: | ||
| # Phoenix Cloud endpoints (app.phoenix.arize.com) include the space in the URL | ||
| if "app.phoenix.arize.com" in collector_endpoint: | ||
| if urlparse(collector_endpoint).hostname == "app.phoenix.arize.com": | ||
| endpoint = collector_endpoint | ||
| protocol = "otlp_http" | ||
| # For other HTTP endpoints, ensure they have the correct path | ||
| @@ -83,7 +83,7 @@ | ||
| otlp_auth_headers = None | ||
| if api_key is not None: | ||
| otlp_auth_headers = f"Authorization=Bearer {api_key}" | ||
| elif "app.phoenix.arize.com" in endpoint: | ||
| elif urlparse(endpoint).hostname == "app.phoenix.arize.com": | ||
| # Phoenix Cloud requires an API key | ||
| raise ValueError( | ||
| "PHOENIX_API_KEY must be set when using Phoenix Cloud (app.phoenix.arize.com)." |
| api_base=api_base, endpoint="skills", skill_id=skill_id | ||
| ) | ||
|
|
||
| verbose_logger.debug("Delete skill request - URL: %s", url) |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 8 days ago
To fix the problem, we should prevent logging of API URLs that may contain sensitive or secret information, especially those derived from environment variables or secret managers. Concretely, in transform_delete_skill_request (and similar places), where the full URL including api_base is logged, we need to mask or avoid logging the actual value. A good approach is to redact the sensitive part:
- Never log the
api_basedirectly. - If logging is essential for debugging, log only the endpoint path or use a string like
[REDACTED]in place of secrets. - Update the logger line on line 195 and line 166 to either remove the variable or mask it appropriately (e.g.,
"Delete skill request - URL: [REDACTED]"). - Similarly update any other debug logs which output sensitive URLs, such as line 166 (
Get skill request - URL: %s) if it uses the same mechanism.
No additional methods or imports are needed, just a change in the arguments to the logger.
-
Copy modified line R166 -
Copy modified line R195
| @@ -163,7 +163,7 @@ | ||
| api_base=api_base, endpoint="skills", skill_id=skill_id | ||
| ) | ||
|
|
||
| verbose_logger.debug("Get skill request - URL: %s", url) | ||
| verbose_logger.debug("Get skill request - URL: [REDACTED]") | ||
|
|
||
| return url, headers | ||
|
|
||
| @@ -192,7 +192,7 @@ | ||
| api_base=api_base, endpoint="skills", skill_id=skill_id | ||
| ) | ||
|
|
||
| verbose_logger.debug("Delete skill request - URL: %s", url) | ||
| verbose_logger.debug("Delete skill request - URL: [REDACTED]") | ||
|
|
||
| return url, headers | ||
|
|
| api_base=api_base, endpoint="skills", skill_id=skill_id | ||
| ) | ||
|
|
||
| verbose_logger.debug("Get skill request - URL: %s", url) |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
This expression logs
sensitive data (secret)
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 8 days ago
To fix the problem, avoid logging the entire request URL with potential secrets in logs. Instead, log only non-sensitive components (such as endpoint names, IDs, or operation types), or redact the sensitive api_base portion. Specifically, in transform_get_skill_request, on line 166, replace the logging of url with either a redacted version or log only the endpoint/skill ID. Other places in the code (shown here) do not log sensitive endpoints directly.
No additional dependencies are required; only update or replace the logging statement on line 166 to ensure no clear-text sensitive values are written to logs. For example, log "Get skill request" with just the endpoint and skill ID, and replace any logging of the full URL.
-
Copy modified lines R166-R168
| @@ -163,7 +163,9 @@ | ||
| api_base=api_base, endpoint="skills", skill_id=skill_id | ||
| ) | ||
|
|
||
| verbose_logger.debug("Get skill request - URL: %s", url) | ||
| verbose_logger.debug( | ||
| "Get skill request for endpoint 'skills', skill_id: %s", skill_id | ||
| ) | ||
|
|
||
| return url, headers | ||
|
|
1c317ac
into
litellm_vertex_ai_anthopic_cost_tracking
|
|
Title
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes