Skip to main content
Glama

karpathy-wiki-mcp

A sandboxed MCP server that implements Andrej Karpathy's LLM Wiki pattern. Its purpose is to evaluate local models in LM Studio. Each agent session gets its own wiki folder. The model then ingests sources, cross-links pages, answers questions from the wiki and lints it. You grade the result by opening that folder in Obsidian.

This server is built for LM Studio. The setup, the eval workflow and the tuning of the tool text all assume LM Studio as the MCP host. The server itself is plain stdio MCP and never contacts LM Studio. Other MCP hosts should work, but they aren't tested.

The tools are written for 26–31B models such as Gemma 31B, Gemma 26B and Qwen:

  • arguments are flat;

  • replies are plain markdown and end with a Next: hint;

  • errors say how to fix the problem and give an example;

  • names are matched leniently, with "did you mean" suggestions;

  • every page write reports any links to pages that don't exist yet.

wiki_get_info returns a complete manual (src/guide.ts). It covers the method, the INGEST, QUERY and LINT recipes, a worked example, and the common mistakes.

Setup

This needs Node 20 or newer.

npm install
npm run build
npm test         # unit and in-memory MCP tests
npm run e2e      # starts the built server over stdio and runs a full ingest

Add the server to LM Studio (~/.lmstudio/mcp.json):

"karpathy-wiki": {
  "command": "/absolute/path/to/node",
  "args": ["/absolute/path/to/karpathy-wiki-mcp/dist/index.js"]
}

Options are set as environment variables or flags:

Env

Flag

Default

Meaning

WIKI_ROOT

--root

<repo>/wikis

Where sandboxes are created

WIKI_SEED_DIR

--seed

<repo>/seed

.md/.txt files copied into each new sandbox's raw/. Use none to start empty

Related MCP server: 50 First Tapes MCP Server

Running an evaluation

  1. Open a new LM Studio chat and use this system prompt, with <MODEL> replaced by the id of the model under test (for example qwen3-32b): You have access to a wiki MCP server. Your model id is <MODEL>. Call wiki_get_info before anything else.

    Always set the model id in the system prompt. A model doesn't know its own id, and the server doesn't ask LM Studio. Without the id, the model is told to ask you. wiki_start rejects a missing id and example values such as <your-model-id> or my-model, but it can't detect a wrong id. A model told it is gemma-3-31b will name the sandbox after Gemma.

  2. Paste a prompt from scenarios/, starting with 01-cold-start-ingest.md.

  3. Open wikis/<sandbox_id>/ as its own Obsidian vault, then go through that scenario's checklist. Opening the sandbox folder alone keeps [[links]] from resolving into other sandboxes.

After rebuilding and restarting the wiki MCP server, you can tell a continuing chat: "Call wiki_get_info again to refresh your tool instructions. For small corrections, use wiki_edit_page mode="patch" with edits containing old_text and new_text; omit content and section. Keep using our existing sandbox_id." The server also advertises patch mode in its connection instructions, tool description, the top of the manual, and wiki_status replies.

scenarios/README.md gives the expected pages and lists the deliberate conflicts in the seed data. Don't show that file to the model.

Sandbox layout

wikis/20261018-143005-qwen3-32b/      sandbox_id = folder name (no colons: Obsidian forbids them)
  SCHEMA.md   the manual (the schema layer)
  index.md    catalog, rebuilt by the server after every change
  log.md      append-only: "## [YYYY-MM-DD HH:mm] op | title"
  raw/        sources: seed files plus pasted text; read-only for the agent
  wiki/       pages; filename = title, YAML frontmatter written by the server

Tools

  • wiki_get_info

  • wiki_start(model)

  • wiki_status

  • wiki_list(folder)

  • wiki_read(name, offset)

  • wiki_search(query, scope)

  • wiki_create_page(title, type, summary, content, sources, tags)

  • wiki_edit_page(title, mode, content, section, edits, summary, type, sources, remove_sources, tags)

  • wiki_delete_page(title, reason)

  • wiki_add_source(title, content)

  • wiki_lint

  • wiki_log(operation, title, details)

Every tool except the first two takes sandbox_id.

For small corrections, wiki_edit_page supports mode="patch":

{
  "sandbox_id": "<your sandbox_id>",
  "title": "Observatory",
  "mode": "patch",
  "edits": [
    { "old_text": "The archive opens in 2027.", "new_text": "The archive opens in 2028." }
  ]
}

Omit content and section. Each nonempty old_text must match the page body exactly once, including whitespace; new_text is literal and may be empty to delete the match. A batch of 1–100 edits runs in order against the in-memory body. Missing or ambiguous matches reject the entire batch before writing the page, metadata, log, or index. Deleting the whole body is rejected. Metadata arguments still work alongside patches.

This reduces text the model must generate for a small correction. In an illustrative synthetic 407-word section, a one-sentence correction required 3,592 bytes of JSON arguments with replace_section, versus 169 with patch (95.3% smaller). These are payload bytes, not measured model tokens or latency; tokenizer, prompt overhead, and model behavior affect actual savings. Use append for new material and replace_section for substantial rewrites.

What the server does for the model

Be aware of these when you grade:

  • index.md is maintained for the model. The server rebuilds it from page frontmatter after every change, so the index can't fall out of sync with the pages. That means index upkeep is not part of the test.

  • log.md is written by both sides. The server logs every create, edit and delete. Ingest, query and lint entries come only from the model calling wiki_log, so their presence is part of the test. A retried call can produce a duplicate line.

  • "Ingested" is a proxy. A raw source counts as ingested once a source page cites it. That doesn't prove the model read or integrated the source.

  • Frontmatter comes from the server. The model passes type, summary, sources and tags. Source names that don't exist are rejected with a warning and not saved.

Trust model

LM Studio starts one server process that all chats share, and MCP doesn't tell the server which chat is calling. Sandboxes are therefore separate namespaces for one trusted local user, not security boundaries: an agent that knows another sandbox's id can use that sandbox.

What the server does enforce:

  • No path can leave WIKI_ROOT. Traversal and symlinks are refused.

  • Raw sources are never modified.

  • Writes are atomic.

  • Calls to the same sandbox run one at a time.

  • Only one server process runs per WIKI_ROOT, enforced by .server.lock.

Limits are 100 KB per file and 2,000 pages per sandbox.

License

MIT

Available Tools

12 tools
wiki_add_sourceAdd a raw sourceB

Save text the user pasted as a new read-only raw source (raw/.md). Then ingest it. Example: wiki_add_source sandbox_id="..." title="Interview Notes" content="...".

Inputs: title (required), content (required), sandbox_id (required).

wiki_create_pageCreate a wiki pageA

Create a NEW wiki page. The server writes the frontmatter and updates index.md. Fails if the page exists (then use wiki_edit_page). Example: wiki_create_page sandbox_id="..." title="Ada Lovelace" type="entity" summary="English mathematician." sources=["raw/Example Article"] content="## Overview\n...".

Inputs: type (required), title (required), content (required), summary (required), +3 more.

wiki_delete_pageDelete a wiki pageA
Destructive

Delete a wiki page (never a raw source). Reports pages whose links to it are now broken. Example: wiki_delete_page sandbox_id="..." title="Engine" reason="duplicate of Analytical Engine".

Inputs: title (required), reason (required), sandbox_id (required).

wiki_edit_pageEdit a wiki page (patch for small changes)A

SMALL EDITS: USE mode="patch" with edits=[{old_text:"exact existing text",new_text:"replacement"}]; omit content and section. Each old_text must match the body exactly once; edits run in order and all must succeed before saving. mode="append" adds content; "replace_section" replaces a section (give section); "replace_all" replaces the body. summary/type replace values; sources/tags are added (replace_all replaces sources); remove_sources deletes entries.

Inputs: mode (required), title (required), sandbox_id (required), tags, +7 more.

wiki_get_infoWiki manualA
Read-only

Read this FIRST for current tool instructions, including wiki_edit_page mode="patch" for small edits. Explains ingest/query/lint, page rules, a worked example, and every tool. No arguments.

Inputs: no named parameters.

wiki_lintLint the wikiA
Read-only

Health check: broken links, bad frontmatter, uningested sources, pages without sources, near-duplicates, orphans, stubs. Each issue has a fix. 30 per call; use offset for more.

Inputs: sandbox_id (required), offset.

wiki_listList pages or sourcesA
Read-only

List wiki pages (folder="wiki", default) or raw sources (folder="raw") with their summaries. 50 per call; use offset for more.

Inputs: sandbox_id (required), folder, offset.

wiki_logWrite a log entryA

Record what you did in log.md. Use after each ingest, after answering a question, and after a lint pass. Example: wiki_log sandbox_id="..." operation="ingest" title="Example Article" details="Created: A, B. Updated: C."

Inputs: title (required), operation (required), sandbox_id (required), details.

wiki_readRead a page or sourceA
Read-only

Read a wiki page by title, a raw source as "raw/", or "index", "log", "SCHEMA". Long files come in 8000-character parts; the reply says which offset to pass next. Example: wiki_read sandbox_id="..." name="raw/Example Article".

Inputs: name (required), sandbox_id (required), offset.

wiki_startStart a sandbox wikiA

Create a NEW sandbox wiki and get its sandbox_id. Only call this when the user wants a new wiki. If the user gave you a sandbox_id, use that instead (wiki_status). If you are not sure, ask the user first. Example: wiki_start model="", using your real model id. If nobody told you your model id, ask the user for it.

Inputs: model.

wiki_statusSandbox statusA
Read-only

Where things stand: page counts, which raw sources are not ingested yet, recent log entries, lint summary. Use at the start of a session and whenever you lose track.

Inputs: sandbox_id (required).

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4/5.0

    Scored across 12 tools

    Disambiguation4/5

    Most tools have crisp, distinct purposes: read vs. create vs. edit vs. delete vs. search vs. lint are clearly separated. The only mild overlap is among meta/read-ish tools (wiki_get_info, wiki_status, wiki_list, wiki_read), but descriptions distinguish them well (instructions vs. state overview vs. enumeration vs. content fetch).

    Naming Consistency5/5

    Every tool uses the same `wiki_` prefix followed by a snake_case verb or noun (wiki_read, wiki_create_page, wiki_delete_page, wiki_add_source). No mixed conventions or casing deviations.

    Tool Count5/5

    12 tools is well within the sweet spot and each tool maps to a distinct operation in the wiki lifecycle (bootstrap, source intake, page CRUD, search, lint, logging, status). Nothing feels redundant or padded.

    Completeness4/5

    The surface covers a full lifecycle: start sandbox, add raw sources, create/read/edit/delete pages, search, list, lint, log, and status. Minor gaps exist—no rename/move operation and raw sources cannot be deleted or edited, so some cleanup paths require workarounds.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables creation of persistent, compounding knowledge bases using Karpathy's LLM Wiki pattern with LLM-maintained markdown wikis. Supports automated ingestion, cross-referencing, synthesis, and linting of sources as an alternative to traditional RAG systems.
      63
      MIT
    • A
      license
      A
      quality
      C
      maintenance
      Enables an LLM to autonomously maintain a Markdown-based wiki with YAML frontmatter, using hybrid BM25 and local CPU vector search for fully offline retrieval without external APIs or databases.
      18
      31 npm
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to safely read, search, and write an Obsidian vault kept as an LLM wiki, with index-first access, Obsidian-aware linking, and code-enforced guards for blocked folders, secrets, and write restrictions.
      MIT