How do I use local-llm-mcp?

1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@local-llm-mcp list available models" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

local-llm-mcp

by jgt87

Overview Schema Related Servers Score Discussions

JavaScript

local-llm-mcp

An MCP server that answers prompts with a local model via Ollama, synchronously.

Companion to codex-offload-mcp. That server exists for slow agentic work that must not block; this one is for fast, private, low-stakes calls where the answer is wanted in the same turn.

Tools

tool	does
`local_ask`	Prompt in, text out. Summaries, boilerplate, commit messages, extraction.
`local_classify`	Sort text into one of your labels. Replies outside the label set are rejected rather than guessed at, and the model can answer that none fit — though a small model will still label plainly unrelated text with confidence, so treat a result as triage, not a verdict.
`local_models`	List models Ollama has on disk, and the configured default.

No file access, no command execution, no memory between calls.

Related MCP server: Ollama MCP Server

Install

Prerequisites

Node.js 20+
Ollama running locally, with at least one model pulled. The server talks to it over HTTP and does not start it for you:

ollama pull qwen2.5-coder:7b
ollama list          # confirm the model is on disk

Build

git clone https://github.com/jgt87/local-llm-mcp.git
cd local-llm-mcp
npm install
npm run build

This produces dist/index.js. Note its absolute path — every step below needs it.

Add to VS Code

MCP support is built into current VS Code; if the Command Palette lists MCP: commands, you have it. Pick either route:

Guided. Command Palette (Ctrl+Shift+P) → MCP: Add Server → Command (stdio). Enter node as the command and the absolute path to dist/index.js as the argument, then name it local-llm.

By hand. Command Palette → MCP: Open User Configuration to open your user mcp.json (%APPDATA%\Code\User\mcp.json on Windows), and add the server:

{
  "servers": {
    "local-llm": {
      "type": "stdio",
      "command": "node",
      "args": ["C:/path/to/local-llm-mcp/dist/index.js"]
    }
  }
}

Use forward slashes on Windows, or escape backslashes as \\ — a raw C:\path is invalid JSON and the server will silently fail to start.

Non-default Ollama host or model? Add an env block alongside args:

      "env": { "LOCAL_LLM_MODEL": "llama3.2:3b" }

To scope it to one project instead of your whole profile, use MCP: Open Workspace Folder Configuration and put the same servers block in .vscode/mcp.json. That file can be committed, which gives everyone on the repo the same tools.

Verify. Open the Chat view, switch to Agent mode, click Configure Tools, and confirm local_ask, local_classify and local_models appear and are enabled. MCP: List Servers shows the server's status and its logs if it failed to start. If the tools load but every call errors, Ollama is not running — check ollama list.

Add to Claude Code

claude mcp add local-llm --scope user -- node /absolute/path/to/dist/index.js

Confirm with /mcp in a session, or claude mcp list from a shell.

After changing the code

A running server keeps serving the old dist/, so rebuild and restart it:

npm run build

VS Code — MCP: List Servers → select the server → Restart. (The experimental chat.mcp.autoStart setting can do this for you.)
Claude Code — restart the session; MCP servers connect at session start.

Configuration

env	default
`OLLAMA_HOST`	`http://127.0.0.1:11434`
`LOCAL_LLM_MODEL`	`qwen2.5-coder:7b`
`LOCAL_LLM_TIMEOUT_MS`	`120000`

Performance

Measured on a Ryzen AI 9 HX 370 (CPU inference, 61 GB RAM):

model	generation	prompt eval
llama3.2:3b	33.7 tok/s	~285 tok/s
qwen2.5-coder:7b	16.0 tok/s	~120 tok/s

Keep outputs short — maxTokens is the main latency lever. At 16 tok/s, 160 tokens is ~10 seconds.

Do not set OLLAMA_IGPU_ENABLE=1 on integrated-GPU hardware. The iGPU shares system memory with the CPU, so generation gets slower (26.2 vs 33.7 tok/s on a Radeon 890M) even though prompt ingest doubles.

Orchestration

There is no orchestrator. Nothing in this server decides what gets routed to the local model. There is no router, no classifier picking a backend, no fallback chain. The only thing steering the choice is the tool descriptions in src/index.ts, which the calling model reads at call time and judges against. Editing those descriptions is how you change routing behaviour; there is no config to tune.

Nothing is offloaded here — that is the point. These tools are synchronous: the prompt goes to Ollama over HTTP and the answer comes back in the same turn. There is no job id, no polling, no state on disk. A local 7B answers in seconds, and wrapping that in a job store would be pure overhead. The rule the two servers are built around: if a tool would need to be polled, it belongs in codex-offload, not here.

Deciding where work goes

Send it here	Send it to `codex-offload`	Keep it in the calling model
Seconds of work, answer needed now	Minutes of work, must not block	Needs the conversation
Verification cheaper than generation	Needs file access and repo context	Judgement, or the next decision hangs on it
Wrong answer is cheap to notice	Result checkable against a git diff	Exploratory — direction shifts as you learn
Privacy matters; nothing leaves the machine	Mechanical and self-contained	Wrong answer is expensive and hard to spot

Good fits: triage, classification, summarising long output, drafting boilerplate or commit messages, extracting fields from text. Bad fits: anything where a subtly wrong answer is expensive and hard to detect. This server has no file access, no repo context, and no memory between calls.

Output is validated, never trusted

local_classify checks the reply against your label set instead of taking it at face value: a reply naming two labels, or one outside the set, comes back as matched: false with the raw text rather than a guess. Substrings do not count, so informational never silently resolves to info.

This mirrors how the sibling server pairs Codex's self-report with a git diff — the delegate says what it did, and something independent checks it.

The limit is worth stating plainly: validation catches malformed and hedged replies, but cannot catch a confidently wrong one. The none escape hatch narrows that gap and does not close it — a small model will still hand back a plausible in-set label for text belonging to none of them. Treat a returned label as triage, not a verdict.

Licence

MIT

Available Tools

3 tools

local_askAsk the local modelA

Send a prompt to a local model via Ollama and get the answer back immediately. Runs on this machine, so nothing leaves it and there is no API cost. Use it for work where checking the answer is cheaper than producing it: summarising long output, drafting boilerplate or commit messages, extracting fields from text. It is a small model on CPU (~16 tok/s for a 7B), so keep outputs short — maxTokens is the main latency lever. It has no file access and cannot run anything. For work needing judgement, repo context, or edits on disk, do it yourself or use codex_start.

ParametersJSON Schema

Name	Required	Description
`model`	No	Model tag. Defaults to qwen2.5-coder:7b.
`prompt`	Yes	The full question or instruction. No conversation context is carried over.
`system`	No	Optional system prompt to set role or output format.
`maxTokens`	No	Cap on generated tokens. Roughly 16 tokens per second, so 160 is ~10s.
`temperature`	No	Defaults to 0 for repeatable output.

TDQS

A4.9/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden and does an excellent job: it discloses the local/offline nature (nothing leaves the machine), no API cost, no file access, cannot run anything, roughly 16 tok/s for a 7B model, and that no conversation context is carried over. This is rich behavioral context that goes well beyond any schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but efficiently structured: it opens with the core action, then capability boundaries, then usage examples, then constraints, then alternatives. Every sentence earns its place and there is zero filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is no output schema and no annotations, this description is remarkably complete: it covers purpose, constraints, performance characteristics, safety boundaries, and alternatives. For a parameter-rich tool (5 params) it fully compensates for the missing structured metadata.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds value by naming maxTokens as the main latency lever and relating it to the tok/s speed, which connects the parameter to a concrete performance behavior beyond mere syntax. It doesn't cover every parameter explicitly but the schema already handles most of that burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states exactly what the tool does: sends a prompt to a local model via Ollama and returns the answer immediately. It clearly identifies the resource (local model) and verb (send prompt / get answer), and differentiates it from siblings by noting it is the local-model option distinct from local_classify and local_models.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use guidance ('summarising long output, drafting boilerplate or commit messages, extracting fields from text') and explicit when-not-to-use with named alternatives ('For work needing judgement, repo context, or edits on disk, do it yourself or use codex_start'). Also gives practical guidance on keeping outputs short and mentions the CPU speed constraint.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

local_classifyClassify text with the local modelA

Put a piece of text into one of the labels you supply, using a local model. The reply is validated against your label set rather than trusted: if the model answers with something outside the list, or hedges between labels, this returns matched=false with the raw reply instead of guessing. By default the model may also answer that no label fits, which comes back as declined=true. That escape hatch helps but does not hold: a small model will still pick a confident in-set label for text that belongs to none of them, so a returned label is triage, not a verdict. Set allowNone=false only when a forced choice is genuinely wanted. Good for triage — log lines, error vs warning, which files look relevant, is this diff risky. Cheap and private; use it where a wrong answer is cheap for you to detect.

ParametersJSON Schema

Name	Required	Description
`model`	No	Model tag. Defaults to qwen2.5-coder:7b.
`labels`	Yes	Allowed labels. The answer is checked against these.
`content`	Yes	The text to classify.
`allowNone`	No	Default true: the model may reply that no label fits, returned as declined=true. Set false to force a choice, accepting that unrelated text will be mislabelled.
`instruction`	No	Optional extra guidance, e.g. what the labels mean.

TDQS

A4.8/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden — and it delivers richly. It discloses that output is validated against label set, returns matched=false with raw reply on hedging, can return declined=true, and honestly warns that a small model will still pick a confident in-set label for out-of-set text ('triage, not a verdict'). This is unusually candid behavioral disclosure beyond what any structured field could convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single focused paragraph that front-loads the core behavior then layers in caveats and use cases. Every sentence earns its place, but it's slightly dense and could be split into short sections for scannability. Efficient and non-redundant despite its length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a moderately complex tool (5 params, nuanced behavior around declined/matched and model error modes). With 100% schema coverage, no output schema, and no annotations, the description fully compensates: it explains validation behavior, edge cases (hedging, no-label), trust limitation, and appropriate use cases. The behavioral nuance of a model-based classifier is well covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaning beyond the schema by explaining the behavior behind allowNone (the declined escape hatch, forced-choice mislabelling risk) and clarifying that labels are the constraint the reply is validated against. It doesn't add detail on model or instruction params beyond the schema, but the schema already covers them. Marginal added value justifies a 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb+resource pair ('classify text into labels you supply') and clearly distinguishes from siblings local_ask and local_models by framing it as triage/classification rather than open-ended chat. It even names concrete uses (log lines, error vs warning, which files look relevant), grounding the purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use it ('Good for triage'), frames its limits ('triage, not a verdict'), and gives concrete when-to-use examples. It advises on the allowNone flag ('Set allowNone=false only when a forced choice is genuinely wanted'). It's missing an explicit 'not for X, use sibling instead' but the triage framing plus sibling names imply exclusion well.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

local_modelsList local modelsA

List the models Ollama has on disk, with sizes, plus which one this server uses by default. Call it when a request names a model you are not sure exists.

ParametersJSON Schema

Name	Required	Description	Default
No parameters

TDQS

A4.2/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses that the tool lists models on disk and indicates which is used by default, which is genuinely useful behavioral context. However, it doesn't mention performance, whether it hits the network, or the structure of the returned listing. For a read-only list operation, this is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with zero wasted words. It front-loads the core purpose and then adds a concrete usage trigger. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter read-only listing tool with no output schema, the description covers purpose, what the output contains (sizes, default), and when to use it. It could add a bit more about the returned data format or fields, but the description is largely sufficient for this simple tool type.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters and 100% schema coverage, so the baseline per the rubric is 4. The description adds meaning about what the output provides (sizes, default model), orients the agent on what information is returned, and there are no parameters requiring explanation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states what the tool does: list models on disk with sizes and the server's default model. It uses a specific verb ('List') with a clear resource ('local models') and distinct output details. It also distinguishes its purpose from siblings through the 'default' model note and the usage context about checking model existence.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives an explicit usage context: 'Call it when a request names a model you are not sure exists.' This tells the agent when to invoke it. It doesn't explicitly name alternatives or exclusions, but the sibling tools (local_ask, local_classify) serve clearly different purposes, so the available guidance is reasonably complete for this simple listing tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.4/5.0

Disambiguation5/5

Each tool has a clearly distinct purpose: listing models, asking a model a prompt, and classifying text into labels. There is no overlap between querying for answers and categorizing input, and the model-listing tool is unambiguously separate from both.

Naming Consistency5/5

All three tools follow a consistent local_ prefix with clear verb_verb-noun style (models, ask, classify). The pattern is uniform and predictable, making selection straightforward.

Tool Count3/5

At three tools, this is on the low end but each earns its place for a focused local-LLM utility server. The count is appropriate for the narrow scope of running local model inference, though it borders on thin.

Completeness4/5

The server covers the core lifecycle for local model interaction: discovery (local_models), free-form prompting (local_ask), and structured classification (local_classify). A minor gap is the lack of a tool to pull or manage models, but for the stated inference-focused purpose the surface is largely complete.

Maintenance

ActivitySlowing

ResponsivenessSyncing

Resources

GitHub Repository

Need Help?

Related Servers

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

mcp A
ai.llmse
Public MCP server for the LLM Search Engine
Search RAG Systems Developer Tools
102
mcp-aichat
io.github.AceDataCloud
MCP server for AI dialogue using various LLM models via AceDataCloud
AI & Machine Learning
2
supportsheep
io.github.pooriaarab
Remote MCP server for supportsheep: run AI interviews and manage support content for your blog.
Customer Support Content Management Systems Marketing
1
Remote Desktop Commander
app.desktopcommander
Hosted MCP server connecting claude.ai, ChatGPT and other AI apps to your own computer
Shell Access OS Automation
214

Related MCP Servers

mcp-ollama-python
AI & Machine Learning Developer Tools
pblagoje
A
license
Not graded
quality
A
maintenance
A Python MCP server that exposes local Ollama models as tools for AI assistants, enabling chat, generation, embeddings, and model management without cloud APIs.
Updated 12 days ago (2026-08-22 23:18 UTC)
5
MIT
Ollama MCP Server
AI & Machine Learning Developer Tools
m-mehdi14
A
license
B
quality
D
maintenance
A universal MCP server that integrates with local Ollama instances, enabling AI-powered chat, model management, and text generation from any MCP-compatible IDE or application.
Updated 7 months ago (2026-01-28 19:15 UTC)
6
226
3
MIT
mcp-coding
File Systems Command Line
cdemeo92
A
license
Not graded
quality
C
maintenance
Experimental MCP server for local LLM orchestration with filesystem tools (read, write, list, delete files) and a CLI agent that communicates via Ollama.
Updated 2 months ago (2026-06-24 05:58 UTC)
12
ISC
localllm-MCP
File Systems RAG Systems
tsxk1san
F
license
Not graded
quality
B
maintenance
MCP server that enables AI to read, search, and edit local files securely without external data exposure, using local LLMs via Ollama and integrating with Open WebUI or Claude Desktop.
Updated 9 days ago (2026-08-26 04:01 UTC)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/jgt87/local-llm-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server