LiteLLM v1.65.0 introduces significant enhancements including Model Context Protocol (MCP) tools, new models, and various performance improvements.
Model Context Protocol (MCP)​
This release introduces support for centrally adding MCP servers on LiteLLM. This allows you to add MCP server endpoints and your developers can list and call MCP tools through LiteLLM.
Read more about MCP here.
Expose and use MCP servers through LiteLLM
Custom Prompt Management​
This release allows you to connect LiteLLM to any prompt management service through our custom prompt management hooks. As proxy admin all you need to do is implement a get_chat_completion_prompt hook which accepts a prompt_id and prompt_variables and returns a formatted prompt.
Read more about custom prompt management here.
Connect LiteLLM to your prompt management service through custom hooks
Categorized Improvements and Fixes​
New Models / Updated Models​
- Support for Vertex AI gemini-2.0-flash-lite & Google AI Studio gemini-2.0-flash-lite PR
- Support for Vertex AI Fine-Tuned LLMs PR
- Nova Canvas image generation support PR
- OpenAI gpt-4o-transcribe support PR
- Added new Vertex AI text embedding model PR
LLM Translation​
- OpenAI Web Search Tool Call Support PR
- Vertex AI topLogprobs support PR
- Fixed Vertex AI multimodal embedding translation PR
- Support litellm.api_base for Vertex AI + Gemini across completion, embedding, image_generation PR
- Fixed Mistral chat transformation PR
Spend Tracking Improvements​
- Log 'api_base' on spend logs PR
- Support for Gemini audio token cost tracking PR
- Fixed OpenAI audio input token cost tracking PR
UI​
Model Management​
- Allowed team admins to add/update/delete models on UI PR
- Added render supports_web_search on model hub PR
Request Logs​
Usage Tab​
- Added Daily User Spend Aggregate view - allows UI Usage tab to work > 1m rows PR
- Connected UI to "LiteLLM_DailyUserSpend" spend table PR
Logging Integrations​
- Fixed StandardLoggingPayload for GCS Pub Sub Logging Integration PR
Performance / Reliability Improvements​
- LiteLLM Redis semantic caching implementation PR
- Gracefully handle exceptions when DB is having an outage PR
- Allow Pods to startup + passing /health/readiness when allow_requests_on_db_unavailable: True and DB is down PR
- Removed hard coded final usage chunk on Bedrock streaming usage PR
- Refactored Vertex AI passthrough routes - fixes unpredictable behaviour with auto-setting default_vertex_region on router model add PR
General Improvements​
- Support for exposing MCP tools on litellm proxy PR
- Support discovering Gemini, Anthropic, xAI models by calling their /v1/model endpoint PR
- Fixed route check for non-proxy admins on JWT auth PR
- Added baseline Prisma database migrations PR
- Get master key from environment, if not set PR
Documentation​
Security​
- Bumped next from 14.2.21 to 14.2.25 in UI dashboard PR