Skip to main content
Glama

karpathy-wiki-mcp

A sandboxed MCP server that implements Andrej Karpathy's LLM Wiki pattern. Its purpose is to evaluate local models in LM Studio. Each agent session gets its own wiki folder. The model then ingests sources, cross-links pages, answers questions from the wiki and lints it. You grade the result by opening that folder in Obsidian.

This server is built for LM Studio. The setup, the eval workflow and the tuning of the tool text all assume LM Studio as the MCP host. The server itself is plain stdio MCP and never contacts LM Studio. Other MCP hosts should work, but they aren't tested.

The tools are written for 26–31B models such as Gemma 31B, Gemma 26B and Qwen:

  • arguments are flat;

  • replies are plain markdown and end with a Next: hint;

  • errors say how to fix the problem and give an example;

  • names are matched leniently, with "did you mean" suggestions;

  • every page write reports any links to pages that don't exist yet.

wiki_get_info returns a complete manual (src/guide.ts). It covers the method, the INGEST, QUERY and LINT recipes, a worked example, and the common mistakes.

Setup

This needs Node 20 or newer.

npm install
npm run build
npm test         # unit and in-memory MCP tests
npm run e2e      # starts the built server over stdio and runs a full ingest

Add the server to LM Studio (~/.lmstudio/mcp.json):

"karpathy-wiki": {
  "command": "/absolute/path/to/node",
  "args": ["/absolute/path/to/karpathy-wiki-mcp/dist/index.js"]
}

Options are set as environment variables or flags:

Env

Flag

Default

Meaning

WIKI_ROOT

--root

<repo>/wikis

Where sandboxes are created

WIKI_SEED_DIR

--seed

<repo>/seed

.md/.txt files copied into each new sandbox's raw/. Use none to start empty

Related MCP server: 50 First Tapes MCP Server

Running an evaluation

  1. Open a new LM Studio chat and use this system prompt, with <MODEL> replaced by the id of the model under test (for example qwen3-32b): You have access to a wiki MCP server. Your model id is <MODEL>. Call wiki_get_info before anything else.

    Always set the model id in the system prompt. A model doesn't know its own id, and the server doesn't ask LM Studio. Without the id, the model is told to ask you. wiki_start rejects a missing id and example values such as <your-model-id> or my-model, but it can't detect a wrong id. A model told it is gemma-3-31b will name the sandbox after Gemma.

  2. Paste a prompt from scenarios/, starting with 01-cold-start-ingest.md.

  3. Open wikis/<sandbox_id>/ as its own Obsidian vault, then go through that scenario's checklist. Opening the sandbox folder alone keeps [[links]] from resolving into other sandboxes.

After rebuilding and restarting the wiki MCP server, you can tell a continuing chat: "Call wiki_get_info again to refresh your tool instructions. For small corrections, use wiki_edit_page mode="patch" with edits containing old_text and new_text; omit content and section. Keep using our existing sandbox_id." The server also advertises patch mode in its connection instructions, tool description, the top of the manual, and wiki_status replies.

scenarios/README.md gives the expected pages and lists the deliberate conflicts in the seed data. Don't show that file to the model.

Sandbox layout

wikis/20261018-143005-qwen3-32b/      sandbox_id = folder name (no colons: Obsidian forbids them)
  SCHEMA.md   the manual (the schema layer)
  index.md    catalog, rebuilt by the server after every change
  log.md      append-only: "## [YYYY-MM-DD HH:mm] op | title"
  raw/        sources: seed files plus pasted text; read-only for the agent
  wiki/       pages; filename = title, YAML frontmatter written by the server

Tools

  • wiki_get_info

  • wiki_start(model)

  • wiki_status

  • wiki_list(folder)

  • wiki_read(name, offset)

  • wiki_search(query, scope)

  • wiki_create_page(title, type, summary, content, sources, tags)

  • wiki_edit_page(title, mode, content, section, edits, summary, type, sources, remove_sources, tags)

  • wiki_delete_page(title, reason)

  • wiki_add_source(title, content)

  • wiki_lint

  • wiki_log(operation, title, details)

Every tool except the first two takes sandbox_id.

For small corrections, wiki_edit_page supports mode="patch":

{
  "sandbox_id": "<your sandbox_id>",
  "title": "Observatory",
  "mode": "patch",
  "edits": [
    { "old_text": "The archive opens in 2027.", "new_text": "The archive opens in 2028." }
  ]
}

Omit content and section. Each nonempty old_text must match the page body exactly once, including whitespace; new_text is literal and may be empty to delete the match. A batch of 1–100 edits runs in order against the in-memory body. Missing or ambiguous matches reject the entire batch before writing the page, metadata, log, or index. Deleting the whole body is rejected. Metadata arguments still work alongside patches.

This reduces text the model must generate for a small correction. In an illustrative synthetic 407-word section, a one-sentence correction required 3,592 bytes of JSON arguments with replace_section, versus 169 with patch (95.3% smaller). These are payload bytes, not measured model tokens or latency; tokenizer, prompt overhead, and model behavior affect actual savings. Use append for new material and replace_section for substantial rewrites.

What the server does for the model

Be aware of these when you grade:

  • index.md is maintained for the model. The server rebuilds it from page frontmatter after every change, so the index can't fall out of sync with the pages. That means index upkeep is not part of the test.

  • log.md is written by both sides. The server logs every create, edit and delete. Ingest, query and lint entries come only from the model calling wiki_log, so their presence is part of the test. A retried call can produce a duplicate line.

  • "Ingested" is a proxy. A raw source counts as ingested once a source page cites it. That doesn't prove the model read or integrated the source.

  • Frontmatter comes from the server. The model passes type, summary, sources and tags. Source names that don't exist are rejected with a warning and not saved.

Trust model

LM Studio starts one server process that all chats share, and MCP doesn't tell the server which chat is calling. Sandboxes are therefore separate namespaces for one trusted local user, not security boundaries: an agent that knows another sandbox's id can use that sandbox.

What the server does enforce:

  • No path can leave WIKI_ROOT. Traversal and symlinks are refused.

  • Raw sources are never modified.

  • Writes are atomic.

  • Calls to the same sandbox run one at a time.

  • Only one server process runs per WIKI_ROOT, enforced by .server.lock.

Limits are 100 KB per file and 2,000 pages per sandbox.

License

MIT

Available Tools

12 tools
wiki_add_sourceAdd a raw sourceB

Save text the user pasted as a new read-only raw source (raw/.md). Then ingest it. Example: wiki_add_source sandbox_id="..." title="Interview Notes" content="...".

NameRequiredDescriptionDefault
titleYesName for the source file.
contentYesThe full source text, unchanged. Do not summarize it.
sandbox_idYesYour sandbox id from wiki_start.

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full load: it usefully discloses that the source is read-only, gets written to raw/<title>.md, and that ingestion is triggered as a follow-on action — non-obvious side-effect information. It omits permissions, what ingestion produces, failure modes, and whether the source becomes immediately searchable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the action and consequence (save then ingest), followed by a compact example. Nothing is redundant, though the example borders on padding given 100% schema coverage.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-required-param mutation tool with no annotations and no output schema, the description covers the essential action and the ingestion side effect. It leaves gaps around what the agent should expect back and what state the source ends up in, which matters for chaining into wiki_read or wiki_search.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so all three parameters are already documented, including the 'do not summarize' constraint on content — baseline 3 applies. The example invocation adds concrete wiring of sandbox_id/title/content but no semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — save pasted text as a new read-only raw source — and names the target path (raw/<title>.md), which distinguishes it from page-creating siblings like wiki_create_page. It doesn't explicitly say 'not for wiki pages', so it falls short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Save text the user pasted' implies the trigger condition, and 'Then ingest it' hints at downstream effects. However, there is no explicit guidance on when to prefer wiki_create_page or wiki_edit_page instead, and no exclusions or prerequisites beyond the implied sandbox_id from wiki_start.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wiki_create_pageCreate a wiki pageA

Create a NEW wiki page. The server writes the frontmatter and updates index.md. Fails if the page exists (then use wiki_edit_page). Example: wiki_create_page sandbox_id="..." title="Ada Lovelace" type="entity" summary="English mathematician." sources=["raw/Example Article"] content="## Overview\n...".

NameRequiredDescriptionDefault
tagsNoOptional tags, e.g. ["person", "mathematics"].
typeYesOne of: source, entity, concept, comparison, synthesis, overview.
titleYesPage title = the subject's name. Others link to it as [[Title]].
contentYesMarkdown body with ## sections and [[links]]. No frontmatter, no # title.
sourcesNoRaw sources the facts come from, e.g. ["raw/Example Article"].
summaryYesOne sentence describing the page.
sandbox_idYesYour sandbox id from wiki_start.

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden well: it discloses side effects (the server writes frontmatter and updates index.md) and the precondition that the page must not already exist. It doesn't cover auth/permission needs or what the call returns, but the key mutation behavior is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action and its guard condition, then side effects, then a compact example. The example is the longest element but earns its place by showing a realistic invocation; minor verbosity from the inline content snippet.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter mutation tool with no annotations and no output schema, the description covers the essential behavior (frontmatter/index.md writes, existence precondition, sibling fallback). Return shape is unspecified, but nothing critical to correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter is already documented in the schema. The worked example reinforces how title/type/summary/sources/content combine, but adds little semantic meaning beyond what the schema provides. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Create a NEW wiki page") and emphasizes NEW, immediately distinguishing it from the sibling wiki_edit_page. An agent can pick between create and edit without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the failure condition ("Fails if the page exists") and names the alternative to use instead ("then use wiki_edit_page"). This is a clear when/when-not/alternative routing statement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wiki_delete_pageDelete a wiki pageA
Destructive

Delete a wiki page (never a raw source). Reports pages whose links to it are now broken. Example: wiki_delete_page sandbox_id="..." title="Engine" reason="duplicate of Analytical Engine".

NameRequiredDescriptionDefault
titleYesTitle of the page to delete.
reasonYesWhy (written to the log).
sandbox_idYesYour sandbox id from wiki_start.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, so the safety profile is covered. The description adds valuable behavioral context by disclosing the side-effect/return signal: it reports pages whose links to the deleted page are now broken. It could say more about permanence or authorization requirements, but adds real value beyond annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core action and constraint, followed by a concrete example. No wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description compensates by explaining the return information (broken-link report). It is largely complete for a destructive mutation tool, though it does not explicitly state irreversibility or permission requirements beyond destructiveHint.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters (title, reason, sandbox_id). The description adds only an example call, which illustrates usage but does not add semantic meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Delete a wiki page') and immediately scopes it against a plausible confusion with raw sources ('never a raw source'). This distinguishes it clearly from siblings like wiki_edit_page and wiki_create_page.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides a clear when-not constraint ('never a raw source') but does not name an alternative tool for that case or describe when deletion is preferred over editing/blanking. Still gives enough context to choose the tool correctly in most cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wiki_edit_pageEdit a wiki page (patch for small changes)A

SMALL EDITS: USE mode="patch" with edits=[{old_text:"exact existing text",new_text:"replacement"}]; omit content and section. Each old_text must match the body exactly once; edits run in order and all must succeed before saving. mode="append" adds content; "replace_section" replaces a section (give section); "replace_all" replaces the body. summary/type replace values; sources/tags are added (replace_all replaces sources); remove_sources deletes entries.

NameRequiredDescriptionDefault
modeYes"append", "replace_section", "replace_all" or "patch" (small edits).
tagsNoTags to add (optional).
typeNoNew page type (optional): source, entity, concept, comparison, synthesis, overview.
editsNoRequired for patch only: 1–100 edits, applied in order. Invalid edits save nothing.
titleYesTitle of the existing page.
contentNoMarkdown to add or replace with. Omit for patch; may be "" for metadata-only append.
sectionNoRequired for mode="replace_section": the heading text, e.g. "Early life".
sourcesNoRaw sources to add (optional).
summaryNoNew one-sentence summary (optional).
sandbox_idYesYour sandbox id from wiki_start.
remove_sourcesNoSources to remove from the list (optional), e.g. a deleted page.

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses exact-match requirements, ordered edits, all-or-nothing saving, and metadata mutation semantics such as sources/tags being added and remove_sources deleting entries. It does not discuss permissions or return behavior, but the critical mutation semantics are covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded and dense, leading with the patch workflow before summarizing other modes. It is appropriately sized, though the long semicolon-separated sentence compresses several distinct behaviors and could be slightly easier to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 11-parameter mutation tool with no annotations and no output schema, the description covers the essential mode behavior, patch atomicity, and metadata semantics. It leaves some lower-priority details to the schema, but an agent has enough context to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaningful mode-specific semantics beyond the schema: omit content/section for patch, provide section for replace_section, and how summary/type/sources/tags/remove_sources behave. This materially improves parameter interpretation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that this tool edits an existing wiki page and explains the different edit modes, especially patch for small edits. It distinguishes the operation from read/create/delete siblings by verb and scope, though it does not explicitly name or route against those siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage direction: use mode="patch" for small edits, omit content and section in that case, and give section for replace_section. It does not explicitly state when to choose other sibling tools such as wiki_create_page or wiki_delete_page, but the mode-level guidance is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wiki_get_infoWiki manualA
Read-only

Read this FIRST for current tool instructions, including wiki_edit_page mode="patch" for small edits. Explains ingest/query/lint, page rules, a worked example, and every tool. No arguments.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, so the safety profile is covered; the description adds that it takes no arguments and what topics the content covers, which is genuinely useful for deciding to open it. It stops short of describing output format or length, so it stops at 4.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences with the imperative cue front-loaded and the content inventory packed into one clause. Nothing is repeated from the title or schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-arg, no-output-schema documentation tool the description covers purpose, ordering, and content scope adequately. Only minor gaps remain, such as the return shape or size of the manual.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, so the baseline is high, and the description explicitly confirms 'No arguments', removing any ambiguity about invocation. No further parameter meaning is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (read the manual / current tool instructions) and enumerates the contents: ingest/query/lint workflows, page rules, a worked example, and every tool. The phrase 'Read this FIRST' plus the pointer to wiki_edit_page mode="patch" makes its role distinct from the sibling operation tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Read this FIRST' gives an unambiguous ordering cue relative to all siblings, and it routes a specific case (small edits) to wiki_edit_page mode="patch". There is no explicit when-not condition, but for a zero-argument manual tool that omission is low-cost.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wiki_lintLint the wikiA
Read-only

Health check: broken links, bad frontmatter, uningested sources, pages without sources, near-duplicates, orphans, stubs. Each issue has a fix. 30 per call; use offset for more.

NameRequiredDescriptionDefault
offsetNoStart position (default 0).
sandbox_idYesYour sandbox id from wiki_start.

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnlyHint=true, so the safety profile is known. The description adds meaningful behavior beyond that: it returns categorized issues, each issue carries a fix, and results are paginated at 30 per call with offset for continuation. This is useful operational context for a diagnostic tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core purpose, then the categories, then the pagination rule. No filler; every clause conveys distinct information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and a read-only annotation, the description carries the burden of explaining what comes back. It does this by listing issue categories, noting each has a fix, and specifying result pagination. Minor gaps remain (e.g., ordering or severity), but an agent has enough to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters (sandbox_id, offset) are already documented. The description adds the page-size constraint ('30 per call') and the purpose of offset ('use offset for more'), which is genuinely additive, but sandbox_id semantics are left entirely to the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the operation precisely ('Health check' / lint of the wiki) and enumerates the specific issue classes it detects: broken links, bad frontmatter, uningested sources, orphan pages, stubs, etc. This clearly separates it from sibling tools like wiki_read, wiki_search, and wiki_list, which retrieve content rather than audit it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by 'Health check' – an agent can infer this is the tool for auditing wiki integrity rather than reading content. However, there is no explicit statement of when to prefer this over wiki_status or wiki_list, nor any preconditions or when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wiki_listList pages or sourcesA
Read-only

List wiki pages (folder="wiki", default) or raw sources (folder="raw") with their summaries. 50 per call; use offset for more.

NameRequiredDescriptionDefault
folderNo"wiki" (default) or "raw".
offsetNoStart position for the next page of results (default 0).
sandbox_idYesYour sandbox id from wiki_start.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With readOnlyHint=true already covering safety, the description adds real behavior: fixed page size of 50, offset-based continuation, and that entries include summaries. It doesn't mention ordering or whether a total count is returned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, mode selection first, pagination second; nothing extraneous.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only listing tool with full schema coverage and annotations, this covers the essentials: modes, default, page size, and continuation. Only result ordering and truncation behavior are left unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so folder enum, its default, and offset semantics are already documented. The description restates the default but adds no syntax or constraint detail beyond the schema, which is the expected baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (list wiki pages / raw sources) and names what the results carry (summaries). It doesn't explicitly contrast with sibling list-like tools such as wiki_search or wiki_read, so an agent must infer that distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The default folder and the pagination rule ('50 per call; use offset for more') imply how to page through everything, but there is no guidance on when to choose this over wiki_search or wiki_read.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wiki_logWrite a log entryA

Record what you did in log.md. Use after each ingest, after answering a question, and after a lint pass. Example: wiki_log sandbox_id="..." operation="ingest" title="Example Article" details="Created: A, B. Updated: C."

NameRequiredDescriptionDefault
titleYesShort title: the source name, the question, or the topic.
detailsNoWhat you did: pages created/updated, the answer, issues fixed.
operationYes"ingest", "query", "lint" or "note".
sandbox_idYesYour sandbox id from wiki_start.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It implies an append-style log entry and gives an example, but it does not explicitly describe whether entries are appended versus overwritten, what happens if log.md is missing, or what a successful call returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with the purpose stated first, followed by usage triggers and a compact example. Every element earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple four-parameter logging tool with no output schema, the description is mostly complete: it names the log file, lists usage occasions, and shows an example call. A minor gap is that it does not explain append behavior or return behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already explains sandbox_id, operation, title, and details. The example in the description reinforces expected values, but it adds little semantic detail beyond the structured schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Record what you did in log.md.' It clearly distinguishes this tool from siblings such as wiki_read, wiki_search, and wiki_lint by making logging the explicit outcome.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives concrete triggers: 'after each ingest, after answering a question, and after a lint pass.' That is clear when-to-use guidance, though it does not explicitly say when not to use it or name a sibling as an alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wiki_readRead a page or sourceA
Read-only

Read a wiki page by title, a raw source as "raw/", or "index", "log", "SCHEMA". Long files come in 8000-character parts; the reply says which offset to pass next. Example: wiki_read sandbox_id="..." name="raw/Example Article".

NameRequiredDescriptionDefault
nameYesPage title, "raw/<source>", "index", "log" or "SCHEMA".
offsetNoCharacter position to continue from (default 0).
sandbox_idYesYour sandbox id from wiki_start.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already establishes that this is a safe read operation, and the description adds important behavior beyond the schema: long files are returned in 8000-character parts and the response indicates which offset to use next. It does not cover error behavior for missing pages, but the pagination disclosure is meaningful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core read operation, then adds pagination behavior, then a concrete example. It is short with no filler, and each sentence contributes useful invocation detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only tool with full schema descriptions and no output schema, the description covers the important operational details: accepted name formats, special targets, chunked reads, and offset continuation. Nothing critical for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the parameters are already documented. The description still adds value by clarifying that 'name' can be a page title, "raw/<source>", "index", "log", or "SCHEMA", and by explaining that the offset should come from the reply's next-offset guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb, 'Read', and names the exact resources: a wiki page by title, a raw source as "raw/<name>", or the special targets "index", "log", and "SCHEMA". This is concrete enough that an agent can distinguish it from siblings like wiki_search or wiki_list without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains the contexts in which this tool is used and gives the exact name formats for pages, raw sources, and special wiki files. It does not explicitly state when to prefer wiki_search, wiki_list, or wiki_get_info instead, so the routing guidance is clear but not fully comparative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wiki_startStart a sandbox wikiA

Create a NEW sandbox wiki and get its sandbox_id. Only call this when the user wants a new wiki. If the user gave you a sandbox_id, use that instead (wiki_status). If you are not sure, ask the user first. Example: wiki_start model="", using your real model id. If nobody told you your model id, ask the user for it.

NameRequiredDescriptionDefault
modelNoYour model id, as the user or system prompt gave it. Required; it becomes part of the sandbox_id. Never copy an example value.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does disclose key behavior: this is a creation/mutation operation, it returns a sandbox_id, and the model id becomes part of that id. It leaves gaps on sandbox lifetime, whether duplicates can be created, and any permission or rate-limit constraints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The routing constraint is front-loaded, which is good, but there is redundancy: 'ask the user' appears twice in different forms and the example repeats the model-id guidance already in the schema. Two or three sentences could be cut without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter creation tool with no annotations and no output schema, the description covers the return value, the parameter's downstream effect, and the disambiguation path. It does not say what a sandbox is for or how long it persists, which an agent may need to set user expectations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaning beyond the schema: the model value's consequence (it becomes part of the sandbox_id) and an explicit fallback ('if nobody told you your model id, ask the user for it'). It also warns against copying the example value, which guards a realistic failure mode.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Create a NEW sandbox wiki') plus the artifact returned ('get its sandbox_id'). It also distinguishes itself from the read-path sibling wiki_status, so an agent can route without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly gives the when ('only when the user wants a new wiki'), the when-not ('if the user gave you a sandbox_id'), the named alternative (wiki_status), and a fallback ('if you are not sure, ask the user first'). This is about as complete as usage routing gets.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

wiki_statusSandbox statusA
Read-only

Where things stand: page counts, which raw sources are not ingested yet, recent log entries, lint summary. Use at the start of a session and whenever you lose track.

NameRequiredDescriptionDefault
sandbox_idYesYour sandbox id from wiki_start.

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

readOnlyHint=true already covers the safety profile, so the description's job is lighter. It adds useful context by disclosing the four categories of information returned, which matters because there is no output schema. It doesn't say anything about freshness, cost, or size of the response.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with what the tool returns, then the usage cue. Nothing is wasted or buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must sketch return values, and it does so by listing the four status categories. For a one-parameter read-only summary tool, that's nearly complete; only the shape/format of the snapshot is left implicit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single sandbox_id parameter is fully documented in the schema (100% coverage), and the description adds no syntax or format detail beyond it. Baseline 3 applies when the schema carries parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description enumerates the concrete contents (page counts, un-ingested raw sources, recent log entries, lint summary), making the resource — an aggregated sandbox status snapshot — unambiguous. It lacks a crisp verb and doesn't name sibling tools it differs from (wiki_get_info, wiki_log, wiki_lint), so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Use at the start of a session and whenever you lose track" gives clear triggering conditions for invocation. It stops short of naming alternatives or exclusions — e.g., telling the agent to use wiki_log or wiki_lint instead when only that slice is needed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 12 tool updatesv0.1.0
    • First observedwiki_add_source
    • First observedwiki_create_page
    • First observedwiki_delete_page
    • First observedwiki_edit_page
    • First observedwiki_get_info
    • First observedwiki_lint
    • First observedwiki_list
    • First observedwiki_log
    • First observedwiki_read
    • First observedwiki_search
    • First observedwiki_start
    • First observedwiki_status

TDQS

A4/5.0

Scored across 12 tools

Disambiguation4/5

Most tools have crisp, distinct purposes: read vs. create vs. edit vs. delete vs. search vs. lint are clearly separated. The only mild overlap is among meta/read-ish tools (wiki_get_info, wiki_status, wiki_list, wiki_read), but descriptions distinguish them well (instructions vs. state overview vs. enumeration vs. content fetch).

Naming Consistency5/5

Every tool uses the same `wiki_` prefix followed by a snake_case verb or noun (wiki_read, wiki_create_page, wiki_delete_page, wiki_add_source). No mixed conventions or casing deviations.

Tool Count5/5

12 tools is well within the sweet spot and each tool maps to a distinct operation in the wiki lifecycle (bootstrap, source intake, page CRUD, search, lint, logging, status). Nothing feels redundant or padded.

Completeness4/5

The surface covers a full lifecycle: start sandbox, add raw sources, create/read/edit/delete pages, search, list, lint, log, and status. Minor gaps exist—no rename/move operation and raw sources cannot be deleted or edited, so some cleanup paths require workarounds.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    F
    maintenance
    Enables creation of persistent, compounding knowledge bases using Karpathy's LLM Wiki pattern with LLM-maintained markdown wikis. Supports automated ingestion, cross-referencing, synthesis, and linting of sources as an alternative to traditional RAG systems.
    61
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables an LLM to autonomously maintain a Markdown-based wiki with YAML frontmatter, using hybrid BM25 and local CPU vector search for fully offline retrieval without external APIs or databases.
    18
    12 npm
    MIT
  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables AI agents to safely read, search, and write an Obsidian vault kept as an LLM wiki, with index-first access, Obsidian-aware linking, and code-enforced guards for blocked folders, secrets, and write restrictions.
    7 npm
    MIT