Allows sending prompts to Google Gemini's API as part of a fallback chain across multiple LLM providers.
Allows sending prompts to OpenAI's API (GPT models) as part of a fallback chain across multiple LLM providers.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@llm-fallback-mcp explain machine learning in one sentence"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP server that completes prompts with automatic provider fallback across OpenAI, Anthropic Claude, and Google Gemini. Built for production: rate-limit aware, retries on transient failures, transparent attempt log.
LLM APIs go down. They rate-limit you. One provider has an outage, the others usually don't. Production LLM apps need fallback. This server gives you the pattern as a single MCP tool.
Try OpenAI → Anthropic → Gemini in order (configurable)
Each provider retries once on 429 / 5xx / network errors with backoff
Return first success with a full per-provider attempt log
Skip providers without keys automatically
Zero SDK dependencies — calls each provider's REST API directly
Related MCP server: litellm-mcp
npm install -g llm-fallback-mcpOr npx:
npx llm-fallback-mcpAdd to claude_desktop_config.json (Windows: %APPDATA%\Claude\claude_desktop_config.json):
{
"mcpServers": {
"llm-fallback": {
"command": "npx",
"args": ["-y", "llm-fallback-mcp"],
"env": {
"OPENAI_API_KEY": "sk-...",
"ANTHROPIC_API_KEY": "sk-ant-...",
"GEMINI_API_KEY": "..."
}
}
}
}Only set keys for providers you want active. Missing keys = provider skipped.
completeArg | Type | Required | Default |
| string | yes | — |
| string[] | no |
|
| object | no | — |
| number | no |
|
| number | no |
|
health_checkReturns which providers are configured.
{
"text": "...",
"provider_used": "anthropic",
"model_used": "claude-haiku-4-5-20251001",
"attempts": [
{ "provider": "openai", "ok": false, "status": 429, "durationMs": 412 },
{ "provider": "anthropic", "ok": true, "durationMs": 1133 }
]
}git clone https://github.com/KhushalB25/llm-fallback-mcp.git
cd llm-fallback-mcp
npm install
npm run build
OPENAI_API_KEY=sk-... npm startMIT
Complete a prompt with automatic provider fallback. Tries OpenAI -> Anthropic Claude -> Google Gemini in order. Each provider gets one retry on rate limit / 5xx / network errors with backoff. Returns the response, the provider that succeeded, and a full per-provider attempt log. Requires at least one of OPENAI_API_KEY, ANTHROPIC_API_KEY, or GEMINI_API_KEY in the environment.
| Name | Required | Description | Default |
|---|---|---|---|
| chain | No | Optional provider order. Defaults to ['openai', 'anthropic', 'gemini']. | |
| prompt | Yes | The user prompt to send. | |
| max_tokens | No | Max output tokens. Default 1024. | |
| temperature | No | Sampling temperature 0-2. Default 0.5. | |
| model_overrides | No | Optional per-provider model id override. Defaults: openai=gpt-4o-mini, anthropic=claude-haiku-4-5-20251001, gemini=gemini-2.0-flash. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavioral traits: fallback order, retry policy (one retry on specific errors with backoff), return structure (response, provider, attempt log), and required environment variables.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff, key information front-loaded. Every sentence serves a clear purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (5 params, nested objects, no output schema), the description covers what the tool does, fallback, retries, return items, and prerequisites. It lacks explicit output schema but provides enough for agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with good individual parameter descriptions. The tool description adds value by explaining defaults and fallback behavior context, complementing the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it 'complete a prompt with automatic provider fallback' and lists the fallback order. It distinguishes itself from the only sibling tool 'health_check', which serves a very different purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (for completing prompts with fallback) and notes required environment variables. It does not explicitly contrast with 'health_check' but the distinction is evident.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return which providers are configured (have their API keys set in the environment) and their default models. Use this to confirm setup before calling 'complete'.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Clearly indicates a read-only operation returning configuration status. No mention of side effects, but implied non-destructive. Could be more explicit about no modifications, but adequate for a health check.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, then usage guidance. No wasted words; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and 0 parameters, the description fully explains what the tool returns (configured providers and default models) and when to use it. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 0 parameters, so baseline is 4. Description does not need to add parameter info; it correctly ignores parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Return which providers are configured and their default models', which is a specific verb-resource combination. It also distinguishes from sibling tool 'complete' by advising usage before calling it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to use this to confirm setup before calling 'complete', providing clear context. Does not state when not to use, but for a health check implied usage is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
The two tools have clearly distinct purposes: 'complete' executes the fallback completion logic, while 'health_check' verifies configuration. There is no ambiguity.
Both tool names use a consistent verb or verb_noun pattern ('complete', 'health_check') with snake_case, making them predictable and readable.
With only 2 tools, the server is tightly scoped to its core functionality: configuration check and completion execution. This count is ideal for the domain.
The tool surface covers all necessary operations for the fallback completion domain: confirming setup and performing the completion. No obvious gaps.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
An MCP server that provides an API to LLMs to manage their JumpCloud resources.
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
MCP server for building and testing AI agents with multi-model experimentation and insights.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/KhushalB25/llm-fallback-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
