Proxies completion requests to OpenAI-compatible downstream APIs, allowing AI agents to send prompts and receive completions from OpenAI models.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@mcp-llm-gateway generate a short poem about the ocean"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP-compatible LLM gateway that proxies completion requests to downstream OpenAI-compatible providers.
mcp-name: io.github.daedalus/mcp-llm-gateway
pip install mcp-llm-gatewayRelated MCP server: MCP Token Bridge
Set the following environment variables:
DOWNSTREAM_URL: Base URL for the OpenAI-compatible downstream API (required)
DEFAULT_MODEL: Default model to use for completions (required)
MODEL_LIST_URL: URL to fetch available models from (optional, defaults to models.dev)
API_KEY: Optional API key for downstream (passthrough)
TIMEOUT: Request timeout in seconds (optional, default: 60)
Run the MCP server with stdio transport:
mcp-llm-gatewayThe server exposes the following tools:
list_models(): List all available models from the remote endpoint
complete(prompt, model, max_tokens, temperature): Send a completion request to the downstream LLM provider
models://list: Returns the list of available models
config://info: Returns current gateway configuration
git clone https://github.com/daedalus/mcp-llm-gateway.git
cd mcp-llm-gateway
pip install -e ".[test]"
# run tests
pytest
# format
ruff format src/ tests/
# lint
ruff check src/ tests/
# type check
mypy src/Model: Dataclass representing an available LLM model
CompletionRequest: Dataclass for completion request payloads
GatewayConfig: Dataclass for gateway configuration
HTTPAdapter: HTTP client for downstream API communication
ModelListAdapter: Adapter for fetching model list from remote endpoints
ModelService: Service for managing model discovery and caching
CompletionService: Service for handling completion requests
ConfigService: Service for managing gateway configuration
Send a completion request to the downstream LLM provider.
Proxies the request to the configured OpenAI-compatible downstream endpoint.
Inputs: prompt (required), model, provider, max_tokens, +1 more. Returns: object
List all available models from the configured providers.
Fetches models from the configured providers, with caching. Can filter by provider ID.
Inputs: provider. Returns: result
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 2 tools
The two tools serve clearly distinct purposes: one for sending completion requests and one for listing models. There is no overlap or ambiguity between them.
The naming convention is inconsistent: 'complete' is a bare verb, while 'list_models' follows a verb_noun pattern. Consistency would improve predictability.
With only 2 tools, the surface is minimal for an LLM gateway. While it covers basic completion and model listing, it feels thin compared to typical gateways that offer more features.
The gateway lacks many expected operations such as streaming, token counting, embeddings, or health checks. This is a significant gap for a production-ready LLM gateway.
MCP server for OpenAI API (chat completions, image generation, embeddings) via AceDataCloud
MCP-Native LLM Orchestration Agent
Connect MCP clients to 2,000+ AI models without managing provider API keys.
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
