mcp-local-file-llm

An MCP server that lets an orchestrating model (Claude Code) hand read-and-answer work to a local LLM without paying to move the source text through its own context.

Why this shape

The obvious design — an MCP tool that takes a text parameter — saves nothing. Tool-call arguments are assistant output tokens, billed at roughly five times the input rate, so passing a document into such a tool costs more than simply reading and answering it directly. The saving only appears when the bulk text never reaches the orchestrator at all.

So the tool takes a reference to the data (a file path, a line range, a shell command) rather than the data itself. The server reads it locally, sends it to the local model, and returns only the answer.

Related MCP server: Agent Helper

Tools

local_llm(prompt, data?, max_words?, max_tokens?, model?, json_schema?, background?)

• data.files — paths with optional line ranges: /a/b.py, /a/b.py:120-160, /a/b.py:120- (to end), /a/b.py:-80 (from start), /a/b.py:42 (one line)

• data.urls — URLs to fetch; HTML is reduced to text first

• data.command — a shell command whose stdout becomes the input

• data.text — literal text, for short framing only

• Omitting data entirely makes it a plain generation call

• json_schema constrains the answer to a JSON Schema, via ollama's native format field, so extraction returns structured data rather than prose

• background: true returns a job id immediately instead of waiting

local_find(prompt, files, max_files?, model?) — search many files, given as paths or globs, for whatever the prompt describes, and return only a ranked shortlist with a one-line reason per hit. Files are batched to roughly 50KB per request and judged against a JSON schema, one verdict per file. Use plain grep/rg when a literal string would do; this is for when the target is described rather than spelled.

local_result(job?, wait_ms?) — collect a background job. Omit job to list this session's jobs and their state. A finished job is returned once, then dropped.

local_status() — check the endpoint and list available models.

Inputs larger than one context window are split and processed map-reduce: the prompt runs against each chunk (a few chunks at a time, see LOCAL_LLM_CONCURRENCY), then a final pass merges the partial answers.

Background jobs

The point of background: true is to overlap local inference with the orchestrator's own work — fire off "read this page and tell me whether it mentions X" and keep searching elsewhere while it runs, then collect the answer with local_result. Jobs live in the server process, so they last as long as the session and do not survive a restart.

Note that the inference host has one GPU. Several background jobs in flight at once contend for it and each one gets slower; the overlap is with the orchestrator, not a way to get more throughput out of the box.

A note on concurrency

LOCAL_LLM_CONCURRENCY does much less than it looks like it should. Ollama sizes its parallel slots from whatever VRAM is free, and at num_ctx 65536 a single slot already claims what remains after a 19GB model is loaded, so large requests are served one at a time no matter what this is set to.

Measured on an RTX 4090 with qwen3-coder:30b, a 253KB five-chunk run:

Flat. Concurrency only helps when the individual requests are small enough for several slots to fit at once. The default of 2 is kept because it costs nothing and does help on smaller inputs.

Benchmarks against this backend are easy to get wrong: any other job on the same GPU distorts them badly. An earlier run of this same test appeared to show four-way concurrency taking over five minutes, which turned out to be an unrelated background job competing for the card. Serialise everything before measuring.

Routing

The tool descriptions carry explicit size thresholds, because the economics invert depending on how much data is involved:

Text that is already in the caller's context should never be passed back in: tool arguments are billed as output tokens, so returning it costs several times more than simply using it where it already is.

Usage tracking

Every call appends one JSON object to a usage log, so the question "is this actually worth having?" can be answered from data rather than argued about. Each record holds the tool, the model, a truncated prompt, the source paths, bytes and estimated tokens in and out, chunk count, duration, and any error.

npm run report                 # or: node report.mjs
node report.mjs --days 7       # last week only
node report.mjs --calls        # also list recent individual calls

The report gives per-tool call counts, how many tokens were kept out of the caller's context against how many came back, a compression ratio, and a cost comparison: what reading that text directly would have cost against what delegating actually cost. That figure is a floor — it counts the text once, and ignores the fact that text read directly also rides along in every later request of the session.

It also flags calls whose input was under 20KB. Those are below the threshold where delegation pays, so a high count there means the tool is being reached for too eagerly and the routing guidance needs tightening.

Set LOCAL_LLM_LOG=off to disable logging entirely, or LOCAL_LLM_LOG_PROMPTS=off to record only sizes and timings without any prompt text.

Configuration

Security note

Everything passed as data leaves this machine for the inference host, so paths that look like credentials (.ssh/id_*, .env, *.pem, *.key, .aws/credentials, secrets.yaml, and similar) are refused unless LOCAL_LLM_ALLOW_SECRETS=1. The data.command field runs a shell command, which is the same privilege the caller already has through its own shell tool, but it is worth knowing it is there. data.urls makes outbound requests from this machine to whatever URL it is given.