litellm-mcp

MCP server for the LiteLLM proxy. It is a development-and-operations surface for LiteLLM resources - creating, configuring, testing, invoking, observing, and cleaning up virtual keys, teams, users, orgs, customers, budgets, models, credentials, tags, guardrails, spend/usage, cache, health, proxy settings, the MCP gateway registry (backend servers, toolsets, access groups), prompts, and the platform areas (policies, evals, A2A agent registry, workflow runs, CloudZero export) - as risk-graded meta-tools an agent can drive.

Bulk inference and the OpenAI-compatible surface (chat/completions, embeddings, files, batches, assistants, vector stores, provider pass-throughs) stay out of scope: agents already have model access through their LLM client. What comes in is one-shot, dev-loop invocation that closes a loop through the MCP alone - test_prompt renders and runs a dotprompt, invoke_agent sends an A2A message/send. Both are graded as litellm_execute (they spend inference) and return bounded output, never a raw stream.

Built on the v2.5 MCP server family: risk-graded meta-tools dispatched by operation + params, strict Pydantic validation, per-op help and JSON schema introspection, list slimming with truncation metadata, and write-response verification.

Operations

• litellm_read (safe): lists, infos, spend/usage, health, settings reads, token/cost utils, MCP gateway registry reads, prompt registry reads (list/get/versions), agent daily activity.

• litellm_write (medium): create/update for keys, teams, users, orgs, customers, budgets, models, credentials, tags, guardrails, fallbacks, MCP servers/toolsets, access groups, policies, evals, agents, workflows, and prompts (create/update/patch).

• litellm_execute (medium): block/unblock toggles, key regenerate/reset, connection tests, targeted cache delete, applying a guardrail to text, and one-shot dev-loop invocation (test a prompt, invoke an agent) with bounded output.

• litellm_delete (high): irreversible deletes and cache flushall.

• litellm_admin (high): proxy-global settings, allowed IPs, global spend reset, bulk user update.

Root: litellm_version returns {"mcp": <package version>, "service": LiteLLMClient.check()}, the client's startup credential check - GET /health/readiness/details, which answers {status, db, cache, litellm_version, ...}. The public /health/readiness probe needs no token, so it cannot stand in for that check.

Related MCP server: litellm-mcp

Install

uvx --refresh \
  --extra-index-url https://nikitatsym.github.io/litellm-mcp/simple \
  litellm-mcp

Add the following to your MCP client configuration (Claude Desktop, Cursor, Claude Code, or any MCP-compatible client):

{
  "mcpServers": {
    "litellm": {
      "command": "uvx",
      "args": [
        "--refresh",
        "--extra-index-url",
        "https://nikitatsym.github.io/litellm-mcp/simple",
        "litellm-mcp"
      ],
      "env": {
        "LITELLM_URL": "https://litellm.example.com",
        "LITELLM_API_KEY": "sk-your-admin-key"
      }
    }
  }
}

Or use the interactive Setup Page to generate the config.

HTTP

litellm-mcp --http serves streamable HTTP at http://127.0.0.1:8000/mcp (--host, --port) instead of stdio, same environment variables. No authentication: put a gateway in front.

The package can also be imported: mcp, Settings, the client class, and client_var (a ContextVar the host sets per request) let one process serve several instances; such a host calls LiteLLMClient.check() on each client at startup, as main() does for its own.

Configuration

Both are read lazily: the server imports and lists ops without them. main() then calls LiteLLMClient.check() before serving, so a missing setting, an unreachable proxy, or a rejected key stops startup instead of surfacing on the first tool call. LITELLM_API_KEY is sent as Authorization: Bearer.

Minting an admin key

This MCP drives the proxy administration surface, so it needs an admin-scoped key, not a plain inference key. The master key works, but a dedicated virtual key with the proxy_admin role is easier to rotate and scope.

• Admin UI: Virtual Keys -> Create New Key, assign the proxy_admin role (or a role carrying admin permissions), and copy the key (shown once).

• API, calling with the master key:

curl -X POST "$LITELLM_URL/key/generate" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"user_role": "proxy_admin", "key_alias": "mcp-admin"}'

Using the tools

Each meta-tool takes operation (a PascalCase op name, or help / schema) plus params (a dict):

litellm_read(operation="help")
litellm_read(operation="help", params={"search": "spend"})
litellm_read(operation="schema", params={"op": "ListKeys"})
litellm_read(operation="ListKeys", params={"team_id": "..."})

litellm_write(operation="GenerateKey", params={"team_id": "..."})
litellm_execute(operation="BlockKey", params={"key": "sk-..."})
litellm_delete(operation="DeleteKeys", params={"keys": ["sk-..."]})

operation="help" lists the group's ops; add params={"search": "foo"} to filter by substring across names and docstrings (it also hints at matches in other groups). operation="schema" returns one op's full JSON Schema. Params are validated strictly via Pydantic: unknown keys, wrong types, and missing required fields return a contextual error result with field-level detail pointing at operation='schema'.

v2.5 dispatch model

• operation='help' renders every op's signature with typed params and a description bullet per field; the search param filters the listing.

• operation='schema' returns the full JSON Schema for one op (additionalProperties: false, descriptions embedded).

• Omitted-vs-null. Optional body params default to an internal UNSET sentinel. Omitting a param drops it from the request; passing an explicit null survives to the wire as JSON null - so a caller can clear a nullable field distinctly from leaving it untouched.

• List slimming. List ops return a slimmed row projection plus truncation metadata ({"total", "returned", "truncated"}), and secret-bearing fields (credentials, static headers, env vars) are dropped from list output. This keeps large responses within an agent's context budget.

• Write verification. Create/update ops presence-check that the fields they sent are echoed in the stored row the proxy returns; a silently dropped field raises with the full dotted path, so a partial write cannot pass unnoticed.

Upstream feature gating

Some endpoints depend on the LiteLLM edition or on extra provider config. Observed on the OSS ghcr.io/berriai/litellm:v1.93.0 image; the MCP does not special-case them - the upstream API context and body are returned in a contextual error result.

Enterprise-licensed (fail on the OSS image without LITELLM_LICENSE):

• GlobalSpendReport (GET /global/spend/report) - 400, "You must be a LiteLLM Enterprise user".

• RegenerateKey (POST /key/regenerate) - 500, "Regenerating Virtual Keys is an Enterprise feature".

Present but needs external provider credentials:

• evals (CreateEval / CreateEvalRun and the run/get/delete family) - the create body is accepted, then the run fails 500 "OPENAI_API_KEY is required for Evals API". Unusable without a real provider key on the proxy.

Working end to end on OSS (exercised by the integration smokes): keys, teams, users, budgets, models, tags, spend logs, the MCP gateway (servers, health, access groups), policies (create/attach/resolve/delete), A2A agents, workflow runs, and CloudZero settings.

Development

Requires uv. Enable the pre-commit hook once per clone (it runs the full gate on every commit):

git config core.hooksPath .githooks

dev.py is the task entry point:

uv run python dev.py check   # lint + mypy + codegen sync + tackbox + tests
uv run python dev.py lint    # ruff + mypy + codegen sync gate + tackbox
uv run python dev.py test    # unit tests only (no docker)
uv run python dev.py e2e     # integration smokes (needs the stack up)

Integration tests run against an ephemeral LiteLLM + Postgres stack. The npm scripts wrap the compose lifecycle:

npm run litellm:up      # compose up -d --wait (first run pulls + migrates)
uv run python dev.py e2e
npm run litellm:down    # tear down + remove volumes
npm run litellm:logs    # follow container logs

Codegen

The tool surface is generated, not hand-transcribed. codegen/inventory.py fixes the operation list; the judgment layer (param descriptions, docstring bodies, slim specs, verify skip sets, override list, and path/body collision dispositions) lives as plain data in codegen/annotations.py, slims.py, verify.py, overrides.py, bodyless_ok.py, and path_body.py. codegen/generate.py is a pure function of the committed OpenAPI snapshot (codegen/openapi-v1.93.0.json) plus that data, emitting the src/litellm_mcp/tools/_generated_*.py modules. Generated files are never hand-edited; ops that need bespoke logic are listed in codegen/overrides.py and implemented by hand in tools/overrides.py.

The sync gate (uv run python -m codegen.check, part of dev.py lint) regenerates into a temp dir and fails unless the result is byte-identical to the committed tree - so a hand-edit of a generated file, a stale data key, or a drifted snapshot all fail the build. To change the surface: edit the data (or the snapshot), regenerate, and commit the diff.

Generated parameters cannot shadow imported emitter symbols or the body and result locals. An opaque request body owns its body parameter.

License

MIT - see LICENSE.