corpus-mcp

An MCP server that lets an AI agent search, read, and cite the books in a Calibre library.

It indexes PDF, EPUB, and MOBI files into a SQLite full-text index. An agent can then find a passage, read it in context, and get a citation with a real locator: the printed page number for a PDF, or the chapter for a reflowable book.

It was built to supply book citations for Concept Clusters puzzle authoring. It complements a Zotero MCP server: Zotero handles papers and their attachments, and corpus-mcp handles the long-form books that live in Calibre.

Tools

The citekey, tag-proposal, and quote tools write to corpus-mcp's own data; corpus-mcp never writes to Calibre itself.

Related MCP server: Calibre MCP Server

Categories

Calibre tags (Calibre-Web's Categories) follow a controlled vocabulary that the taxonomy's owner supplies as a JSON file (format below). Here that owner is Concept Clusters; any project can supply its own. Term ids are the tags, nested with . such as social-sciences.psychology.social-psychology, so a renamed term leaves Calibre's tags valid. Tools and the review page show the titles ("Social Sciences › Psychology › Social Psychology") and accept either form. An agent takes books from tagging_worklist and calls propose_tags. A person approves, edits, or rejects each proposal at /admin/proposals, or sets a book's categories directly on its page. Then npm run tags:apply -- --apply runs tools/calibre-tags.sh on the Calibre-Web host. That script backs up metadata.db and adds the approved tags with calibredb, keeping each book's existing tags.

The vocabulary file (corpus-vocabulary/1) is a flat list of terms:

{
  "format": "corpus-vocabulary/1",
  "version": "any string",
  "generatedAt": "2026-10-25T12:00:00Z",
  "terms": [
    { "id": "social-sciences", "label": "Social Sciences", "assignable": false },
    { "id": "social-sciences.psychology", "label": "Psychology", "parent": "social-sciences" }
  ],
  "retired": [{ "id": "old.term", "replacedBy": ["social-sciences.psychology"] }]
}

• An id is its parent's id, ., and one segment of lowercase letters, digits, and single hyphens. A label is the term's own title, without . or ›.

• assignable: false marks a term that only groups others.

• A retired term with one replacement is re-tagged by the tags job; one that was split or dropped waits for a person.

• The server reads /var/lib/corpus-mcp/vocabulary.json (CORPUS_VOCABULARY) and reloads it when it changes; an invalid file is reported and the last good one stays in use. npm run deploy -- --vocabulary=<file> validates and installs one.

• vocabulary_report (and the categories page) lists retired terms still in Calibre, tags shaped like terms that the vocabulary lacks, and books tagged only at a term with narrower ones. corpus-mcp never edits the vocabulary.

Admin

An admin suite on the corpus server, on the model of Concept Clusters' own admin. Pages show the current state of each kind of object. History is kept on a separate Changes page. Anything an MCP agent can change, a person can change here too, not just approve.

An approved category set is a book's complete set of vocabulary tags. Applying it adds and removes vocabulary tags to match, and never touches other tags. Calibre changes always run as preview, then apply. The apply runs exactly the previewed plan, backs up metadata.db, and refreshes the index. They run on the Calibre-Web host as named commands of the sync key's forced command, never as a shell.

Every write needs a verified Cloudflare Access identity (CORPUS_ACCESS_TEAM_DOMAIN, CORPUS_ACCESS_AUD). Direct LAN access is read-only.

Quotes

Quotes are passages saved from the books, by agents (save_quote) or by people (select text in a book's reader). Either way, the passage is checked against the book's text. Quote marks, dashes, spacing, case, and a page break inside the passage may differ; what's stored is the book's exact wording. A passage that isn't in the book isn't saved, and the closest passages are returned instead.

Each quote records:

• an anchor: its pages, offsets, and the text around it

• free-form tags for themes and forms (overwork, parable) and links (cc:puzzle:<id>), kept to one spelling each

• a note

Quotes have no categories of their own: the book gives the subject, and searching by category matches the source book's categories.

When a book's file changes, its quotes are found again by their text, or marked orphaned; never deleted. Quotes live in data.db and are backed up nightly.

Shelves

Calibre-Web shelves are topic collections, such as Misinformation, kept in Calibre-Web's own app.db rather than in the Calibre library. The sync key's forced command, deploy/corpus-source.sh, allows exactly two things: read-only rsync of the library, and a shelves command that prints the public shelves and their book ids as JSON. app.db, which also holds password hashes, never leaves the Calibre-Web host. Each refresh replaces the index's shelves and book_shelves tables from that export. If the export fails, the previous membership is kept.

Citekeys

Every book gets a proposed citekey (author + year + first significant title word, as in nye2011future), built from its Calibre metadata by @jmajerus/citekey, a Better BibTeX–style rule packaged for reuse (other projects install it from a GitHub release). The preset is CORPUS_CITEKEY_PRESET: author-year-word (the default) or bbt-default.

To file a cited book in a reference manager, cite_passage returns a BibTeX @book entry built from Calibre's own metadata (bibtex) and the book's ISBN when it passes its check digit (isbn). The project that files the book assigns the final key; pin_citekey then stores it against the book's Calibre UUID. Any BibTeX-safe key is accepted and kept as given; one that doesn't follow this server's rule gets a warning. A pinned key never changes on its own and is accepted anywhere a book id is. A citekey identifier set in Calibre takes precedence over a pinned key.

Pinned keys live in data.db, which unlike the index is never rebuilt and is backed up nightly. corpus_status lists books whose metadata blocks a key: an unknown author, no year, or one author record naming several people.

When Calibre has no valid ISBN, cite_passage also returns isbn_candidates. These are the checksum-valid ISBNs printed in the book's first and last pages, each with its page, the printed line, and its binding (hardcover, paperback, or ebook) when one is printed beside it. Copyright pages often list several editions, and back pages often advertise other books, so these are offered for someone to confirm, never filed automatically.

Citations

cite_passage (and cite_quote, for a saved quote) returns:

{
  "csl": { "id": "nye2011future", "citation-key": "nye2011future", "type": "book", "title": "The Future of Power",
           "author": [{ "family": "Nye", "given": "Joseph S., Jr." }], "publisher": "PublicAffairs",
           "issued": { "date-parts": [[2011]] }, "ISBN": "9781586488925" },
  "locator": { "label": "page", "locator": "4-5" },
  "position": { "page": 19, "end_page": 20, "page_kind": "page", "page_label": "4", "end_page_label": "5", "section": "Part One › Chapter 1" },
  "citekey": "nye2011future", "citekey_source": "pinned",
  "bibtex": "@book{nye2011future, …}", "isbn": "9781586488925", "isbn_candidates": [],
  "warnings": []
}

• csl is a CSL JSON item. Its id is the citekey, or calibre:<uuid> when there is none. A Calibre author with a comma in the sort form ("Nye, Joseph S., Jr.") becomes family and given; one without becomes literal. issued is the year; empty fields are left out.

• locator is a CSL locator: printed page labels for PDFs (physical pages, with a warning, when a PDF has none), and null for EPUB and MOBI, whose locations are not pages. position is where the passage is in the index.

• bibtex is null when the citekey is blocked; isbn is null when Calibre's ISBN is missing or fails its check digit.

• Until consumers switch, the legacy citation (title, author, publisher, year, pages) and zotero_filing fields are also returned.

Locators

• PDF. Pages are physical pages. Each page also keeps its printed label from the PDF's page-label tree, so physical page 19 can be cited as page 4. Citations use the printed label.

• EPUB and MOBI. These reflow, so page numbers depend on layout. corpus-mcp lays every reflowable book out at one fixed size, which gives stable location numbers for reading. They are never presented as page numbers. Citations name the chapter, taken from the book's outline.

How it works

Calibre library ── rsync (read-only) ──▶ books/<library>/…
                                              │
                        corpus-mcp index      ▼  metadata.db → MuPDF → SQLite FTS5
                                           corpus.db
                                              │  read-only
                        corpus-mcp serve      ▼  Streamable HTTP /mcp, or stdio
                                              │
                                         MCP clients

• Catalogue. Each library's metadata.db is the source of truth for books, authors, publishers, ISBNs, and format files. The indexer reads an in-memory copy, so it never locks Calibre's database. It doesn't use the metadata.opf backups, which go stale.

• Extraction. The official mupdf WebAssembly package reads all three formats. It provides each page's text, printed page labels, and the outline. Running heads and folios are stripped from PDFs.

• Incremental indexing. A rescan skips files whose size and modification time are unchanged, and rehashes the rest. Only changed books are re-extracted. Identical files in more than one library are collapsed in search results.

• Exclusions. Only allowlisted libraries are indexed. Books carrying an excluded Calibre tag (by default private) are never indexed.

Quick start

Requires Node.js 22 or later. better-sqlite3 compiles natively, so you also need a C++ toolchain and Python.

npm install
npm run build

# Index a directory that holds one folder per Calibre library
CORPUS_BOOKS_ROOT=~/Calibre CORPUS_LIBRARIES=main CORPUS_DB=./corpus.db \
  node dist/cli.js index

# Serve over Streamable HTTP on :8000/mcp (or add --stdio)
CORPUS_BOOKS_ROOT=~/Calibre CORPUS_DB=./corpus.db node dist/cli.js serve

Register it with an MCP client, for example Claude Code:

claude mcp add --transport http corpus http://localhost:8000/mcp

Run the tests with npm test. The test fixtures (a PDF with page labels and an outline, an EPUB, and a Calibre metadata.db) are generated in code.

Configuration

The server also answers GET /health.

Deployment

deploy/ holds systemd units, the rsync pull script, and the setup scripts. tools/deploy.mjs provisions a Debian LXC, installs releases with a health check, and rolls back on failure. Both are written for the author's own Proxmox network (*.localdomain hosts), so adapt the hostnames before using them elsewhere. npm run deploy and npm run tags:apply read their settings from a gitignored .env (see .env.example). PLAN.md records the design decisions, the facts about the corpus, and the known issues.