subforge-mcp

An MCP (Model Context Protocol) server that lets AI clients talk to every kind of model running locally in LM Studio — chat with LLMs, rank text with embedding models, get calibrated choices from decision models — and (with user consent) load a downloaded model into memory.

Install

Requires LM Studio running locally with its local server started (LM Studio → Developer → Start Server).

Claude Code:

claude mcp add subforge -- npx -y subforge-mcp

Claude Desktop / VS Code (mcp.json):

{
  "mcpServers": {
    "subforge": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "subforge-mcp"]
    }
  }
}

Related MCP server: LM Studio MCP Server

Configuration

MCP Tools

No tool implicitly loads a model: chat, delegate_task, rank_by_similarity and the decision tools refuse if the requested model isn't already loaded (or is the wrong type), and load_model is the only tool that can bring one into memory, gated behind an MCP elicitation prompt. unload_model is exempt from that gate by design — freeing memory is low-risk and reversible (load the model again any time), so it doesn't ask for consent.

Delegating tasks to a local model

delegate_task hands a small, mechanical, multi-step coding task (a variable rename, a boring find/replace, a boilerplate first draft) to a local LM Studio model, which runs its own read → edit → read → done loop directly against the filesystem. File contents and the tool back-and-forth never enter the calling conversation — only a short final summary does — which is what makes this worth using: rote, bounded edits that would otherwise burn context reading and re-writing file contents happen entirely on the local model's side.

Safety model. The local model gets exactly three tools — list_dir, read_file, edit_file — all resolved through safeResolvePath, which refuses any path that would resolve outside the caller-supplied dir. There is no shell/Bash tool in its loop, ever. edit_file is a search-and-replace (exactly one match of old_string required), not a whole-file overwrite, so a bad or ambiguous edit fails loudly and leaves the file untouched instead of silently clobbering it. It also cannot create new files.

Reliability caveat. Local models are much weaker than Claude — think "fast and free but low-reliability," not a peer. They're fine for boilerplate, renames, and simple mechanical transforms; they are not a good fit for anything requiring cross-file understanding, nuanced judgment, or high-stakes/hard-to-verify correctness. Always review the returned summary before trusting it.

Speed tip. For reasoning models (e.g. Qwen3) doing simple mechanical tasks, the "thinking" pass before each tool call adds latency with little quality benefit. The think: false input tries a best-effort chat_template_kwargs: { enable_thinking: false } passthrough on each request, but this isn't guaranteed to be honored — the reliable lever is disabling it directly in LM Studio's Jinja prompt template (add {%- set enable_thinking = false %} at the top of the template, editable under that model's settings in LM Studio).

Decision models

Decision models (System One / Jev-compatible, e.g. LiquidAI/d1-3B-GGUF) answer typed questions with probabilities instead of generated text. LM Studio serves them at POST /v1/systemone; this server wraps that endpoint in three tools.

decide sends one choice question: the state is Task: <task> plus Context: <context>, the options are the criteria. Describe options neutrally — their wording is the whole input.

// decide({ task, context, options: [{ id, description }, ...] })
{
  "ok": true,
  "model": "d1-3b",
  "choice": "postgres",
  "probability": 0.9138,
  "confidence": 0.8707,
  "ranking": [
    { "id": "postgres", "probability": 0.9138, "description": "..." },
    { "id": "redis", "probability": 0.0794, "description": "..." },
    { "id": "memory", "probability": 0.0067, "description": "..." },
  ],
}

Failures never throw. LM Studio down, model not loaded, wrong model type, timeout or a bad response all come back as ok: false with an error code (lm_studio_unreachable, model_not_loaded, no_decision_model_loaded, not_a_decision_model, timeout, invalid_request, ...) and probability: 0, so a caller gating on probability can never mistake a failure for a confident answer.

eval_decisions measures where that gate should sit. Each line of the .jsonl is either a decide case or a raw question case:

{"id": "sessions", "task": "store session tokens", "context": "Postgres in use", "options": [{"id": "postgres", "description": "..."}, {"id": "redis", "description": "..."}], "label": "postgres"}
{"id": "t1", "state": "Checkout is down!", "question": {"type": "noul", "instructions": "Is this urgent?"}, "label": true}

It reports overall accuracy, accuracy per probability band (0.5-0.6, 0.6-0.7, ...) and the lowest threshold whose auto-decisions reach target_accuracy (default 0.9) over at least min_support cases. Pass current_threshold to compare against the one in use. Probabilities are not calibrated for your data until you measure them like this.

Development

pnpm install
pnpm run build      # compile TypeScript to dist/
pnpm test           # run the test suite (node:test)
pnpm run test:live  # integration test against a real LM Studio with a decision model loaded
pnpm run typecheck  # type-check src/ + tests/
pnpm run lint        # oxlint (type-aware)

Project structure

subforge-mcp/
├── src/
│   ├── index.ts       # server + tool registrations
│   └── decisions.ts   # decision-model questions, eval parsing and metrics (pure)
└── tests/
    ├── helpers.ts      # fetch/extra mocks shared by tests
    └── *.test.ts

Releases

This project is trunk-based: every change lands on main via a squash-merged pull request titled as a Conventional Commit (feat: ..., fix: ..., feat!: ..., etc.). Merging to main triggers semantic-release, which determines the next version from commit history, updates CHANGELOG.md, tags a GitHub release, and publishes to npm.

License

MIT