Merge pull request #17145 from BerriAI/main

merge main

Author: Sameerlite

.circleci/config.yml

@@ -1785,7 +1785,7 @@ jobs:
             - audio_coverage
   installing_litellm_on_python:
     docker:
-      - image: circleci/python:3.8
+      - image: cimg/python:3.11
         auth:
           username: ${DOCKERHUB_USERNAME}
           password: ${DOCKERHUB_PASSWORD}
@@ -3389,7 +3389,9 @@ jobs:
             nvm use 20
 
             cd ui/litellm-dashboard
-            npm ci || npm install
+            # Remove node_modules and package-lock to ensure clean install (fixes optional deps issue)
+            rm -rf node_modules package-lock.json
+            npm install
 
             # CI run, with both LCOV (Codecov) and HTML (artifact you can click)
             CI=true npm run test -- --run --coverage \

.github/workflows/test-litellm.yml

@@ -37,7 +37,7 @@ jobs:
     - name: Setup litellm-enterprise as local package
       run: |
         cd enterprise
-        python -m pip install -e .
+        poetry run pip install -e .
         cd ..
     - name: Run tests
       run: |

AGENTS.md

@@ -98,6 +98,25 @@ LiteLLM supports MCP for agent workflows:
 
 Use `poetry run python script.py` to run Python scripts in the project environment (for non-test files).
 
+## GITHUB TEMPLATES
+
+When opening issues or pull requests, follow these templates:
+
+### Bug Reports (`.github/ISSUE_TEMPLATE/bug_report.yml`)
+- Describe what happened vs. expected behavior
+- Include relevant log output
+- Specify LiteLLM version
+- Indicate if you're part of an ML Ops team (helps with prioritization)
+
+### Feature Requests (`.github/ISSUE_TEMPLATE/feature_request.yml`)
+- Clearly describe the feature
+- Explain motivation and use case with concrete examples
+
+### Pull Requests (`.github/pull_request_template.md`)
+- Add at least 1 test in `tests/litellm/`
+- Ensure `make test-unit` passes
+
+
 ## TESTING CONSIDERATIONS
 
 1. **Provider Tests**: Test against real provider APIs when possible

CLAUDE.md

@@ -28,6 +28,22 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 ### Running Scripts
 - `poetry run python script.py` - Run Python scripts (use for non-test files)
 
+### GitHub Issue & PR Templates
+When contributing to the project, use the appropriate templates:
+
+**Bug Reports** (`.github/ISSUE_TEMPLATE/bug_report.yml`):
+- Describe what happened vs. what you expected
+- Include relevant log output
+- Specify your LiteLLM version
+
+**Feature Requests** (`.github/ISSUE_TEMPLATE/feature_request.yml`):
+- Describe the feature clearly
+- Explain the motivation and use case
+
+**Pull Requests** (`.github/pull_request_template.md`):
+- Add at least 1 test in `tests/litellm/`
+- Ensure `make test-unit` passes
+
 ## Architecture Overview
 
 LiteLLM is a unified interface for 100+ LLM providers with two main components:

Dockerfile

@@ -48,7 +48,7 @@ FROM $LITELLM_RUNTIME_IMAGE AS runtime
 USER root
 
 # Install runtime dependencies
-RUN apk add --no-cache openssl tzdata
+RUN apk add --no-cache openssl tzdata nodejs npm
 
 # Upgrade pip to fix CVE-2025-8869
 RUN pip install --upgrade pip>=24.3.1

GEMINI.md

@@ -25,6 +25,25 @@ This file provides guidance to Gemini when working with code in this repository.
 - `poetry run pytest tests/path/to/test_file.py -v` - Run specific test file
 - `poetry run pytest tests/path/to/test_file.py::test_function -v` - Run specific test
 
+### Running Scripts
+- `poetry run python script.py` - Run Python scripts (use for non-test files)
+
+### GitHub Issue & PR Templates
+When contributing to the project, use the appropriate templates:
+
+**Bug Reports** (`.github/ISSUE_TEMPLATE/bug_report.yml`):
+- Describe what happened vs. what you expected
+- Include relevant log output
+- Specify your LiteLLM version
+
+**Feature Requests** (`.github/ISSUE_TEMPLATE/feature_request.yml`):
+- Describe the feature clearly
+- Explain the motivation and use case
+
+**Pull Requests** (`.github/pull_request_template.md`):
+- Add at least 1 test in `tests/litellm/`
+- Ensure `make test-unit` passes
+
 ## Architecture Overview
 
 LiteLLM is a unified interface for 100+ LLM providers with two main components:

README.md

@@ -11,7 +11,7 @@
         <p align="center">Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, Groq etc.]
         <br>
     </p>
-<h4 align="center"><a href="https://docs.litellm.ai/docs/simple_proxy" target="_blank">LiteLLM Proxy Server (LLM Gateway)</a> | <a href="https://docs.litellm.ai/docs/hosted" target="_blank"> Hosted Proxy (Preview)</a> | <a href="https://docs.litellm.ai/docs/enterprise"target="_blank">Enterprise Tier</a></h4>
+<h4 align="center"><a href="https://docs.litellm.ai/docs/simple_proxy" target="_blank">LiteLLM Proxy Server (LLM Gateway)</a> | <a href="https://docs.litellm.ai/docs/enterprise#hosted-litellm-proxy" target="_blank"> Hosted Proxy</a> | <a href="https://docs.litellm.ai/docs/enterprise"target="_blank">Enterprise Tier</a></h4>
 <h4 align="center">
     <a href="https://pypi.org/project/litellm/" target="_blank">
         <img src="https://img.shields.io/pypi/v/litellm.svg" alt="PyPI Version">
@@ -40,18 +40,14 @@ LiteLLM manages:
 LiteLLM Performance: **8ms P95 latency** at 1k RPS (See benchmarks [here](https://docs.litellm.ai/docs/benchmarks))
 
 [**Jump to LiteLLM Proxy (LLM Gateway) Docs**](https://github.com/BerriAI/litellm?tab=readme-ov-file#litellm-proxy-server-llm-gateway---docs) <br>
-[**Jump to Supported LLM Providers**](https://github.com/BerriAI/litellm?tab=readme-ov-file#supported-providers-docs)
+[**Jump to Supported LLM Providers**](https://docs.litellm.ai/docs/providers)
 
 🚨 **Stable Release:** Use docker images with the `-stable` tag. These have undergone 12 hour load tests, before being published. [More information about the release cycle here](https://docs.litellm.ai/docs/proxy/release_cycle)
 
 Support for more providers. Missing a provider or LLM Platform, raise a [feature request](https://github.com/BerriAI/litellm/issues/new?assignees=&labels=enhancement&projects=&template=feature_request.yml&title=%5BFeature%5D%3A+).
 
 # Usage ([**Docs**](https://docs.litellm.ai/docs/))
 
-> [!IMPORTANT]
-> LiteLLM v1.0.0 now requires `openai>=1.0.0`. Migration guide [here](https://docs.litellm.ai/docs/migration)
-> LiteLLM v1.40.14+ now requires `pydantic>=2.0.0`. No changes required.
-
 <a target="_blank" href="https://colab.research.google.com/github/BerriAI/litellm/blob/main/cookbook/liteLLM_Getting_Started.ipynb">
   <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
 </a>
@@ -114,6 +110,8 @@ print(response)
 }
 ```
 
+> **Note:** LiteLLM also supports the [Responses API](https://docs.litellm.ai/docs/response_api) (`litellm.responses()`)
+
 Call any model supported by a provider, with `model=<provider_name>/<model_name>`. There might be provider-specific details here, so refer to [provider docs for more information](https://docs.litellm.ai/docs/providers)
 
 ## Async ([Docs](https://docs.litellm.ai/docs/completion/stream#async-completion))
@@ -210,7 +208,7 @@ response = completion(model="openai/gpt-4o", messages=[{"role": "user", "content
 
 Track spend + Load Balance across multiple projects
 
-[Hosted Proxy (Preview)](https://docs.litellm.ai/docs/hosted)
+[Hosted Proxy](https://docs.litellm.ai/docs/enterprise#hosted-litellm-proxy)
 
 The proxy provides:
 

ci_cd/security_scans.sh

@@ -69,10 +69,13 @@ run_grype_scans() {
     # Allowlist of CVEs to be ignored in failure threshold/reporting
     # - CVE-2025-8869: Not applicable on Python >=3.13 (PEP 706 implemented); pip fallback unused; no OS-level fix
     # - GHSA-4xh5-x5gv-qwph: GitHub Security Advisory alias for CVE-2025-8869
+    # - GHSA-5j98-mcp5-4vw2: glob CLI command injection via -c/--cmd; glob CLI is not used in the litellm runtime image,
+    #   and the vulnerable versions are pulled in only via OS-level/node tooling outside of our application code
     ALLOWED_CVES=(
         "CVE-2025-8869"
         "GHSA-4xh5-x5gv-qwph"
         "CVE-2025-8291" # no fix available as of Oct 11, 2025
+        "GHSA-5j98-mcp5-4vw2"
     )
 
     # Build JSON array of allowlisted CVE IDs for jq

cookbook/LiteLLM_HuggingFace.ipynb

@@ -131,7 +131,7 @@
         "            {\n",
         "                \"type\": \"image_url\",\n",
         "                \"image_url\": {\n",
-        "                    \"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg\",\n",
+        "                    \"url\": \"https://awsmp-logos.s3.amazonaws.com/seller-xw5kijmvmzasy/c233c9ade2ccb5491072ae232c814942.png\",\n",
         "                },\n",
         "            },\n",
         "        ],\n",

cookbook/misc/RELEASE_NOTES_GENERATION_INSTRUCTIONS.md

@@ -43,6 +43,14 @@ hide_table_of_contents: false
 ## Key Highlights
 [3-5 bullet points of major features - prioritize MCP OAuth 2.0, scheduled key rotations, and major model updates]
 
+## New Providers and Endpoints
+
+### New Providers
+[Table with Provider, Supported Endpoints, Description columns]
+
+### New LLM API Endpoints
+[Optional table for new endpoint additions with Endpoint, Method, Description, Documentation columns]
+
 ## New Models / Updated Models
 #### New Model Support
 [Model pricing table]
@@ -53,9 +61,6 @@ hide_table_of_contents: false
 ### Bug Fixes
 [Provider-specific bug fixes organized by provider]
 
-#### New Provider Support
-[New provider integrations]
-
 ## LLM API Endpoints
 #### Features
 [API-specific features organized by API type]
@@ -70,16 +75,20 @@ hide_table_of_contents: false
 #### Bugs
 [Management-related bug fixes]
 
-## Logging / Guardrail / Prompt Management Integrations
-#### Features
-[Organized by integration provider with proper doc links]
+## AI Integrations
+
+### Logging
+[Logging integrations organized by provider with proper doc links, includes General subsection]
 
-#### Guardrails
+### Guardrails
 [Guardrail-specific features and fixes]
 
-#### Prompt Management
+### Prompt Management
 [Prompt management integrations like BitBucket]
 
+### Secret Managers
+[Secret manager integrations - AWS, HashiCorp Vault, CyberArk, etc.]
+
 ## Spend Tracking, Budgets and Rate Limiting
 [Cost tracking, service tier pricing, rate limiting improvements]
 
@@ -149,26 +158,34 @@ hide_table_of_contents: false
 - Admin settings updates
 - Management routes and endpoints
 
-**Logging / Guardrail / Prompt Management Integrations:**
+**AI Integrations:**
 - **Structure:**
-  - `#### Features` - organized by integration provider with proper doc links
-  - `#### Guardrails` - guardrail-specific features and fixes
-  - `#### Prompt Management` - prompt management integrations
-  - `#### New Integration` - major new integrations
-- **Integration Categories:**
+  - `### Logging` - organized by integration provider with proper doc links, includes **General** subsection
+  - `### Guardrails` - guardrail-specific features and fixes
+  - `### Prompt Management` - prompt management integrations
+  - `### Secret Managers` - secret manager integrations
+- **Logging Categories:**
   - **[DataDog](../../docs/proxy/logging#datadog)** - group all DataDog-related changes
   - **[Langfuse](../../docs/proxy/logging#langfuse)** - Langfuse-specific features
   - **[Prometheus](../../docs/proxy/logging#prometheus)** - monitoring improvements
   - **[PostHog](../../docs/observability/posthog)** - observability integration
   - **[SQS](../../docs/proxy/logging#sqs)** - SQS logging features
   - **[Opik](../../docs/proxy/logging#opik)** - Opik integration improvements
+  - **[Arize Phoenix](../../docs/observability/arize_phoenix)** - Arize Phoenix integration
+  - **General** - miscellaneous logging features like callback controls, sensitive data masking
   - Other logging providers with proper doc links
 - **Guardrail Categories:**
-  - LakeraAI, Presidio, Noma, and other guardrail providers
+  - LakeraAI, Presidio, Noma, Grayswan, IBM Guardrails, and other guardrail providers
 - **Prompt Management:**
   - BitBucket, GitHub, and other prompt management integrations
+  - Prompt versioning, testing, and UI features
+- **Secret Managers:**
+  - **[AWS Secrets Manager](../../docs/secret_managers)** - AWS secret manager features
+  - **[HashiCorp Vault](../../docs/secret_managers)** - Vault integrations
+  - **[CyberArk](../../docs/secret_managers)** - CyberArk integrations
+  - **General** - cross-secret-manager features
 - Use bullet points under each provider for multiple features
-- Separate logging features from guardrails and prompt management clearly
+- Separate logging, guardrails, prompt management, and secret managers clearly
 
 ### 4. Documentation Linking Strategy
 
@@ -232,6 +249,9 @@ From git diff analysis, create tables like:
 - **Cost breakdown in logging** → Spend Tracking section
 - **MCP configuration/OAuth** → MCP Gateway (NOT General Proxy Improvements)
 - **All documentation PRs** → Documentation Updates section for visibility
+- **Callback controls/logging features** → AI Integrations > Logging > General
+- **Secret manager features** → AI Integrations > Secret Managers
+- **Video generation tag-based routing** → LLM API Endpoints > Video Generation API
 
 ### 7. Writing Style Guidelines
 
@@ -370,10 +390,20 @@ This release has a known issue...
   - **Virtual Keys** - Key rotation and management
   - **Models + Endpoints** - Provider and endpoint management
 
-**Logging Section Expansion:**
-- Rename to "Logging / Guardrail / Prompt Management Integrations"
-- Add **Prompt Management** subsection for BitBucket, GitHub integrations
-- Keep guardrails separate from logging features
+**AI Integrations Section Expansion:**
+- Renamed from "Logging / Guardrail / Prompt Management Integrations" to "AI Integrations"
+- Structure with four main subsections:
+  - **Logging** - with **General** subsection for miscellaneous logging features
+  - **Guardrails** - separate from logging features
+  - **Prompt Management** - BitBucket, GitHub integrations, versioning features
+  - **Secret Managers** - AWS, HashiCorp Vault, CyberArk, etc.
+
+**New Providers and Endpoints Section:**
+- Add section after Key Highlights and before New Models / Updated Models
+- Include tables for:
+  - **New Providers** - Provider name, supported endpoints, description
+  - **New LLM API Endpoints** (optional) - Endpoint, method, description, documentation link
+- Only include major new provider integrations, not minor provider updates
 
 ## Example Command Workflow