Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Multi-Provider LLM MCP Server Review this Python code using Gemini and Claude for consensus."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An official Model Context Protocol (MCP) server that exposes a unified, secure routing interface to query multiple Large Language Model (LLM) providers. Built on top of TypeScript, it dynamically integrates with Google Gemini, OpenAI, Anthropic Claude, Cohere, Groq, Mistral AI, and OpenRouter using a modular Strategy Pattern.
This server enables agentic IDE clients (such as Claude Desktop, Cursor, Windsurf, or GitHub Copilot) to switch models dynamically based on task requirements, evaluate model outputs, and optimize API costs.
This MCP server goes beyond basic completions. It empowers your AI assistants with the following advanced capabilities:
Security and logic flaws can easily be missed by a single LLM. By invoking the query_llm tool across multiple providers (e.g. OpenAI, Anthropic, Gemini) on the same code snippet, your IDE agent can compare outputs:
How it works: The agent sends the same prompt to GPT-4o, Claude 3.5 Sonnet, and Gemini 2.5. If two out of three models flag a potential SQL injection or logic bug, the agent warns you before deploying.
Build automated feedback loops inside your workspace:
How it works: Model A (OpenAI) writes a function. Model B (Anthropic) generates unit test cases. Model C (Gemini) evaluates both the implementation and test coverage, grading the overall solution and suggesting improvements.
API rate limits, service-specific outages, or server lag can interrupt automated coding workflows:
How it works: If an API call to OpenAI fails or hits a rate limit, the routing service can dynamically catch the error and retry the query with Anthropic or Gemini, keeping your developer loop uninterrupted.
LLM costs scale with context size. Sending large files directly to high-reasoning models like Claude 3.5 Sonnet can quickly become expensive:
How it works: Send massive logs or document dumps to a fast, cost-efficient model (such as Groq/Llama-3 or Gemini Flash) first. Instruct it to compress the information into a concise summary, and then send that summary to Claude for final refactoring.
When working with sensitive projects, sending raw user data or company secrets directly to closed-source proprietary APIs poses security risks:
How it works: Route raw user messages to a local or open-source model (via Groq/Mistral) to redact secrets, keys, or PII (personally identifiable information) before forwarding the sanitized input to external proprietary models.
Testing interactive chatbot features requires realistic dialogues:
How it works: Prompt one model (e.g. Mistral) to act as a difficult customer and another (e.g. GPT-4o-mini) to act as the support representative. You can simulate multi-turn chats automatically to test error handling, response times, and chat flows.
Related MCP server: cloud-chat-assistant
The server is designed using the Strategy Pattern and a Central Registry to ensure clean separation of concerns and effortless extensibility:
src/
├── index.ts # Transport Bootstrapper
├── server.ts # MCP Tool Definition & Schema Validation
└── services/
├── types.ts # Common contracts and LLMProvider interface
├── llm.ts # Central Provider Registry
└── providers/ # Individual Provider Clients (Strategies)
├── gemini.ts
├── openai.ts
├── anthropic.ts
├── cohere.ts
├── groq.ts
├── mistral.ts
└── openrouter.tsEvery provider implements the LLMProvider contract:
export interface LLMProvider {
name: string;
defaultModel: string;
envKey: string;
query(params: QueryLLMParams): Promise<QueryLLMResult>;
}To support a new provider, you do not need to modify the server routing logic:
Create a new provider file under src/services/providers/your_provider.ts implementing LLMProvider.
Register it in src/services/llm.ts.
The server registers two primary tools with your MCP client:
query_llmQuery any supported LLM provider with a prompt.
provider (Required): "gemini" | "openai" | "anthropic" | "cohere" | "groq" | "mistral" | "openrouter".
prompt (Required): The input message.
apiKey (Optional): API Token. Falls back to environment variables if omitted.
model (Optional): Specific model name. Defaults to a sensible model if omitted.
systemPrompt (Optional): Guiding system context.
temperature (Optional): Controls randomness (0.0 to 2.0).
maxTokens (Optional): Max tokens to generate.
responseFormat (Optional): "text" (raw completion), "json_object" (strictly JSON), or "detailed" (returns markdown reporting execution time, provider metadata, and token usage).
chatHistory (Optional): Array of previous message history objects [{ role: "user" | "assistant", content: "..." }].
list_providersLists status of all LLM providers, including default models, required environment keys, and if keys are set.
Detailed templates for setting up this server in popular IDEs and clients are located in the sample_mcp_configs directory:
To publish this package publicly under your scope, run:
npm publish --access publicOnce published, users can configure their MCP client command to npx with args ["-y", "@abhishek-kumar-00019/llm-mcp-server"].
Check status of all LLM providers, showcasing default models, required environment keys, and if keys are set.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, but the description clearly indicates a read-only check operation with no destructive effects. It could explicitly state that calling this tool is safe and has no side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that conveys all necessary information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, but the description details what information is returned (default models, env keys, key status). It could add context about how to use the information, but it is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so description does not need to explain them. The schema coverage is 100%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool checks status of all LLM providers, listing default models, required environment keys, and key status. It distinguishes from sibling 'query_llm' which presumably performs queries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this is for checking provider status before using 'query_llm'. While it doesn't explicitly say when not to use, the context of sibling tool provides clear differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Query a supported LLM provider (Gemini, OpenAI, Anthropic, Cohere, Groq, Mistral, OpenRouter) with a prompt and retrieve the model response.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | Optional specific model ID (e.g., gpt-4o-mini, gemini-1.5-flash). Defaults to a recommended model if omitted. | |
| apiKey | No | Optional API Key for authentication. If omitted, falls back to environment variables (e.g. GEMINI_API_KEY, OPENAI_API_KEY). | |
| prompt | Yes | The user prompt or instruction message. | |
| provider | Yes | The LLM API provider to call. | |
| maxTokens | No | Optional maximum number of output tokens to generate. | |
| chatHistory | No | Optional array of message history objects for context. | |
| temperature | No | Optional sampling temperature (0.0 to 2.0). | |
| systemPrompt | No | Optional system instructions to direct behavior/formatting. | |
| responseFormat | No | Optional response structure: "text" (default raw response), "json_object" (forces valid JSON outputs), or "detailed" (returns markdown detailed report including token usage). |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral traits. It only states it queries and retrieves a response, omitting details about external API calls, potential costs, latency, or authentication fallback behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. It is appropriately front-loaded but could be slightly expanded without harming conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's 9 parameters, no output schema, and no annotations, the description is insufficient. It does not mention chat history, temperature, system prompt, or response format, leaving the agent without context for proper usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add extra meaning beyond the schema; it merely mentions a prompt without elaborating on optional parameters or defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool queries an LLM provider with a prompt and retrieves a response. It lists specific providers, distinguishing it from the sibling tool list_providers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention when not to use it or any prerequisites, such as needing an API key or choosing a provider.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Both tools serve entirely distinct purposes: querying a model vs. listing providers. No overlap or confusion possible.
Both tools use a consistent verb_noun pattern in snake_case: query_llm and list_providers. Perfectly uniform.
Only two tools, which is minimal but reasonable for a simple query-and-status server. Not overly sparse, but barely adequate.
Covers the two core needs: querying and checking provider availability. Missing advanced features like dynamic key management or model selection, but acceptable for a minimal interface.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Hosted MCP server for LLM cost estimation, model comparison, and budget-aware routing.
MCP server unifying ERPs, CRMs, APIs and knowledge base for Claude, ChatGPT and Gemini.
MCP server providing access to the Scorecard API to evaluate and optimize LLM systems.
Connect MCP clients to 2,000+ AI models without managing provider API keys.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/itachiuchihadev/llm-mcp-server'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
