Skip to main content
Glama

docshelf-mcp

Put your manuals on a shelf, hand the AI the index.

License: MIT Python 3.10+ MCP CI PyPI Glama

📖 Docs & landing page: https://ignatenkofi.github.io/docshelf-mcp/

     _                _          _  __
  __| | ___   ___ ___| |__   ___| |/ _|
 / _` |/ _ \ / __/ __| '_ \ / _ \ | |_
| (_| | (_) | (__\__ \ | | |  __/ |  _|
 \__,_|\___/ \___|___/_| |_|\___|_|_|
        MCP server for AI-friendly doc shelves

An MCP server that turns a folder of PDFs and Markdown into a chat-project-friendly document collection: AI agents see a single INDEX.md and pull individual sections by raw GitHub URL on demand — instead of choking on a 4 MB datasheet.


Why?

You have 30 hardware manuals, or 200 cooking recipes, or a stack of research PDFs.

You want Claude / ChatGPT / whatever to be able to answer questions across them — but:

  • ❌ You can't dump 80 MB of PDFs into a chat project. It won't fit, and you'd burn the context window even if it did.

  • ❌ You can manually copy-paste the relevant pages, but only after you remember which manual mentioned the thing you need.

  • ❌ Long files mean retrieval is wasteful — the model loads the whole RouterOS guide just to answer a question about VLANs.

docshelf-mcp solves it like this:

  1. You drop a PDF onto the shelf.

  2. The shelf converts it to Markdown, splits big files chapter-by-chapter, and regenerates a navigation INDEX.md.

  3. You commit and push to a public GitHub repo.

  4. Add only INDEX.md to your Claude project. When the model needs a section, it fetches it via raw.githubusercontent.com.

Result: a 5 KB index pointing at a 50 MB collection. The model reads exactly the chapter it needs.


Related MCP server: Obsidian MCP Server

📦 Install

From PyPI:

# uv (recommended)
uv pip install docshelf-mcp

# or plain pip
pip install docshelf-mcp

Or straight from main (always-latest, no PyPI required):

pip install "git+https://github.com/ignatenkofi/docshelf-mcp"

That gives you Markdown shelves. Input formats beyond Markdown are extras — pick the ones you ingest:

pip install "docshelf-mcp[pdf]"       # PDF via pymupdf4llm (the default engine)
pip install "docshelf-mcp[formats]"   # PDF + DOCX + HTML + EPUB; or [docx] / [html] / [epub]

The pdf extra pulls the PyMuPDF chain (~260 MB installed); a consumer that only reads and writes Markdown does not need it, which is why it is not core.

Breaking since 0.5.0: PDF conversion needs the [pdf] extra. Up to 0.4.1 a plain pip install docshelf-mcp converted PDFs because pymupdf4llm was a core dependency; it is now optional (#93). Upgrading from 0.4.x and still adding PDFs? Install docshelf-mcp[pdf] (or [formats]), otherwise add_document on a .pdf raises ConversionError telling you the same.

Optional high-quality PDF engine (pulls ~2 GB of PyTorch — only if you need it):

pip install "docshelf-mcp[high-quality]"

Python 3.14 (checked 2026-10-07): [high-quality] does not install from wheels — every marker-pdf 1.x requires pillow<11, and Pillow 10 has no 3.14 wheels, so pip would have to build it from source. Use Python 3.10–3.13 for it. [pdf] and [formats] install from wheels on 3.14 on Linux, Windows x64 and Apple-silicon macOS 14+, but not on Intel Macs or macOS 13: onnxruntime, which pymupdf4llm pulls in, has no 3.14 wheels there. To re-check from a checkout (exit 0 means wheels exist): uv pip compile pyproject.toml --extra high-quality --python-version 3.14 --only-binary :all:


📋 Project Prompt

Drop this into the Custom Instructions of any Claude project that consumes a docshelf-style INDEX.md:

This project uses the docshelf pattern. INDEX.md is the entry point. When answering: read INDEX → fetch ONLY the needed section file via its GitHub raw URL (use WebFetch / fetch / curl). Don't load full source files into context. For large manuals split into chapters, follow INDEX → chapter SUBINDEX → section file.

Medium (~150 words) and full (~400 words) versions, plus how-to snippets for Claude Code, Claude Desktop, and the Anthropic API, live in docs/PROJECT_PROMPT.md.


Quickstart (Python library)

from docshelf_mcp import Shelf

shelf = Shelf("~/Documents/my-homelab-docs").init(
    name="My HomeLab Docs",
    remote="https://github.com/me/my-homelab-docs",
    default_categories=["routers", "switches", "psu", "motherboards"],
)

shelf.add_document(
    "~/Downloads/MIKROTIK_RouterOS.pdf",
    category="routers",
    title="Mikrotik RouterOS — full manual",
    description="Official RouterOS reference, split by chapter.",
)
# → docs/routers/mikrotik-routeros-full-manual.md  +  docs/routers/.../001-..md, 002-..md, ...
# → INDEX.md is regenerated automatically.

Then in the shelf directory: git add . && git commit -m "docs: add RouterOS" && git push.

In your Claude project, attach only INDEX.md. Done.


Quickstart (MCP server)

1. Add to Claude Desktop

Edit ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%/Claude/claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "docshelf": {
      "command": "docshelf-mcp",
      "env": {
        "DOCSHELF_ROOT": "/Users/me/Documents/my-homelab-docs"
      }
    }
  }
}

Restart Claude Desktop. You now have eleven new tools available:

Tool

What it does

docshelf_init_shelf

Bootstrap a new shelf directory.

docshelf_add_document

Add a file (MD/PDF/DOCX/HTML/EPUB). Converts, splits, re-indexes.

docshelf_add_directory

Add every supported file (MD/PDF/DOCX/HTML/EPUB) in a folder in one call. Re-indexes once.

docshelf_read_document

Read a document/section's content over MCP (works on private shelves).

docshelf_remove_document

Remove a document, its sections, and metadata. Re-indexes.

docshelf_rename_document

Retitle / recategorize a document (moves file, sections, meta) — no re-conversion.

docshelf_rebuild_index

Regenerate INDEX.md from disk.

docshelf_doctor

Check shelf integrity; optionally auto-fix safe drift.

docshelf_search

Plain-text search across the shelf, with raw URLs.

docshelf_list_documents

List documents by category.

docshelf_convert_pdf

Standalone PDF → Markdown (no shelf).

The shelf files are also exposed as read-only MCP resources, so a client can browse and attach them natively — see MCP Resources below.

2. Add to Claude Code

claude mcp add docshelf -- docshelf-mcp

# Optional: pin a default shelf for this server
printf 'Path to your docshelf shelf: '; read -r shelf
shelf=${shelf/#\~/$HOME}                  # read does not expand a typed ~
shelf=$(cd "$shelf" && pwd) || exit 1     # absolute, and fails loudly if absent
claude mcp add docshelf --env DOCSHELF_ROOT="$shelf" -- docshelf-mcp

3. Test from the command line

# Sanity check — should print the server version then wait on stdin
docshelf-mcp

MCP Resources

Alongside the tools, every shelf file is exposed as a read-only MCP resource, so an MCP client (Claude Desktop, Claude Code, …) can browse and attach shelf content natively — no tool call required.

  • Scheme: docshelf:///<relative-path>, e.g. docshelf:///INDEX.md or docshelf:///docs/routers/mikrotik/003-firewall.md.

  • What's exposed: INDEX.md plus every document and every split section under docs/ — one resource each. A split document exposes both its whole-file parent and its individual section files.

  • Size cap: a resource read is capped at 1 MB (1,000,000 bytes). A larger file is truncated at a UTF-8 character boundary and ends with a notice pointing at the docshelf_read_document tool, which pages the rest.

  • Freshness: content is read from disk on every access, and the resource set is re-synced when the server starts and after each mutating tool call (add_document, add_directory, remove_document, rename_document, rebuild_index, init_shelf) — so newly added files appear and removed ones drop out. Reads are confined to the shelf root.

Resources are only registered for an initialized shelf (one that has a .docshelf.json); a non-shelf DOCSHELF_ROOT simply exposes none.


The shelf layout

my-shelf/
├── .docshelf.json        ← shelf metadata: name, remote, category order
├── INDEX.md              ← auto-generated navigation (your chat-project file)
├── .gitignore
└── docs/
    ├── routers/
    │   ├── .meta.json    ← per-document title/description overrides
    │   ├── mikrotik-routeros.md       (full document, lightly cleaned)
    │   └── mikrotik-routeros/         (auto-split sections)
    │       ├── SUBINDEX.md            (per-document navigation page)
    │       ├── 001-overview.md
    │       ├── 002-bridging.md
    │       └── 003-firewall.md
    └── switches/
        └── cudy-gs1010pe.md

Everything in docs/ is committed; everything is fetchable via raw URL once you push to GitHub.


How splitting works

A document is split when both conditions hold:

  1. UTF-8 size > 50 KB (configurable via .docshelf.json:split_threshold_bytes).

  2. The document has at least two ## (H2) headings.

The splitter:

  • Cleans PDF-extraction noise (collapses runaway blank lines, demotes CLI dumps mistaken for H1s).

  • Slices on H2 boundaries.

  • Names files NNN-<slug>.md so they sort naturally and survive title changes.

  • Wipes the previous split directory before regenerating — fully idempotent.

  • Writes a SUBINDEX.md navigation page into the split directory (title, description, per-section links) — regenerated on every rebuild_index.

In INDEX.md, split documents with up to 10 sections list every section inline; bigger splits get a single link to their SUBINDEX.md so the index stays small. Control this via .docshelf.json: "index_style": "auto" | "inline" | "subindex" and "subindex_threshold_sections": 10.

If you want to keep a document whole, pass split=False.


Examples

See the examples/ directory for three concrete use cases:

  • examples/homelab/ — original use case, hardware manuals for a home lab.

  • examples/recipes/ — a cookbook with one recipe per file.

  • examples/research-papers/ — academic PDFs with abstracts in .meta.json.

Each example shows the directory layout and the INDEX.md you'd end up with.


Optional: high-quality PDF conversion

The default engine (pymupdf4llm, installed by the pdf extra) is fast and good enough for ~95% of technical documents. For papers with complex tables, math, or scanned content, install the marker-pdf backend:

pip install "docshelf-mcp[high-quality]"

Then pass quality="high":

shelf.add_document("paper.pdf", category="research", title="...", quality="high")

⚠️ marker-pdf pulls in PyTorch (~2 GB) and is significantly slower (10–60 s per document on CPU). The library import is deferred — if you don't use quality="high", the dependency is never loaded. On Python 3.14 it does not install from wheels yet — see the 3.14 note under Install.


FAQ

Why GitHub raw URLs and not embeddings / RAG? Because it's dead simple, costs nothing to host, and the AI is already good at chasing links. You can layer embedding search on top later if you want — the on-disk shape is a normal git repo.

Does this work with private repos? Partly. The raw-URL trick needs a public repo — raw.githubusercontent.com won't serve private ones without auth. But docshelf_search and docshelf_read_document both work over MCP on private (or purely local, non-git) shelves: the model searches, then reads the exact section's content directly from the server, no raw URL required. You only lose the ability to hand a bare INDEX.md to a chat project and have it fetch by URL — with the MCP server attached, the full flow works either way. Make the doc repo public if you want the URL-fetch path too.

Do I have to use GitHub? No. Set provider in .docshelf.json (or at init_shelf): github (default), gitlab, gitea, custom, or none. The github provider also covers GitHub Enterprise Server: a self-hosted github.<company>.com remote gets the GHES raw form (https://<host>/<owner>/<repo>/raw/<branch>/<path>) automatically. custom takes a url_template with {owner}, {repo}, {branch}, {path} placeholders, so you can point at S3, Cloudflare R2, GitLab/Gitea raw, a GHES deployment on a fully custom domain, or any static host — the generated URLs are correct everywhere, no post-processing. none renders relative links in INDEX.md, which stay navigable offline / in a local checkout.

Does it edit the source PDFs? No. PDFs are converted on add_document and the source is left in place. The shelf only writes inside its own directory.

What about non-English documents? Slugify is Unicode-aware (NFKD-normalized, with \w under re.UNICODE). Cyrillic / CJK titles slug down to ASCII-ish forms; the body Markdown is preserved as-is.

Can I use it without MCP? Yes — from docshelf_mcp import Shelf and use the class directly. See docs/USAGE.md.


Limitations

  • Public GitHub only for the raw-URL trick (or whatever public static host you wire up).

  • Single repo per shelf. If you outgrow one repo, run multiple shelves and attach multiple INDEX.mds.

  • Heuristic splitting. The PDF→Markdown extract isn't always clean enough to split cleanly. For pathological cases (some 4+ MB datasheets), keep the file whole and rely on docshelf_search.

  • No automatic git commit. Tools regenerate INDEX.md on disk, but the caller (you, or an agent) is responsible for git add / commit / push. This is intentional — staying out of git's way keeps the tool safe to call from agents.


Demo — does it actually save tokens?

Measured on two real shelves (24 hardware manuals; a full novel split by chapter): answering a question the docshelf way costs ~3.7K tokens vs 1.22M to dump the collection — 99.7% fewer — and the biggest manual (RouterOS, ~1.05M tokens) doesn't even fit in a 200K context window, while a section fetch always does.

📊 Full write-up with the numbers, chart, and a reproducible benchmark: docs/demo.md (run benchmarks/token_savings.py on your own shelf).


Architecture

For a deeper dive, see docs/ARCHITECTURE.md — module layout, data flow, design rationale.


  • memshelf-mcp — the sibling project: the same index-and-fetch pattern applied to an AI agent's own working memory. Conversation episodes get offloaded to a private shelf with LLM-written digests; the agent keeps only INDEX.md in context and recalls sections on demand. Born as RFC-0001 in this repo; uses docshelf as its storage/index layer.


Contributing

Bug reports and PRs welcome. To set up a dev env:

git clone https://github.com/ignatenkofi/docshelf-mcp
cd docshelf-mcp
uv pip install -e ".[dev]"
ruff check src tests
pytest -v

License

MIT — see LICENSE.

Origin

docshelf-mcp started life as a 350-line Python script (homelab-encyclopedia.py) that managed a single homelab manuals repo. The split / index / clean logic is the same code, generalised to work for any category-organised document collection.

Available Tools

11 tools
docshelf_add_directoryA
Idempotent

Add every matching file in a directory, rebuilding INDEX.md once.

Scans source_dir (non-recursively) for patterns — every supported input type by default (Markdown, PDF, DOCX, HTML, EPUB) — adds each under category with a title derived from its filename, and regenerates INDEX.md a single time. A corrupt or unreadable file is reported in failed without aborting the batch.

Inputs: params (required). Returns: result

docshelf_add_documentA
Idempotent

Add a PDF or Markdown file to the shelf and refresh INDEX.md.

  • .pdf is converted to Markdown (pymupdf4llm by default; pass quality='high' to use marker-pdf).

  • Documents larger than 50 KB with multiple H2 headings are split into one file per section (turn this off with split=False).

  • If a different title/category slugifies onto a path an existing document already occupies, the call errors instead of overwriting it — pass overwrite=true to replace it. Re-adding the same title updates in place. The response reports overwritten.

  • The response warnings include suspicious section headings and an empty-conversion warning when the source yields little or no text (e.g. a scanned / image-only PDF — consider quality='high' / OCR).

  • INDEX.md is regenerated automatically. The caller still owns the git commit / push step.

Inputs: params (required). Returns: result

docshelf_convert_pdfA
Idempotent

Standalone PDF → Markdown conversion (no shelf, no INDEX update).

Use when you want the converted file but don't yet want to commit it to a shelf. Optionally splits the result by H2.

Inputs: params (required). Returns: result

docshelf_doctorA
Idempotent

Check the shelf for drift and optionally apply the safe fixes.

Reports stale .meta.json entries, orphaned split directories, split sections out of sync with their parent, a stale INDEX.md, duplicate titles, and empty categories. Read-only by default; pass fix=true to prune stale meta entries, delete orphaned split dirs, and rebuild the index (other findings stay report-only). Findings are sorted for stable diffing.

Inputs: params (required). Returns: result

docshelf_init_shelfA
Idempotent

Bootstrap a new document shelf at shelf_path.

Creates the directory layout (docs/, INDEX.md, .docshelf.json, .gitignore), pre-creates any default_categories, and stores the github_remote so generated INDEX entries link to raw GitHub URLs.

Idempotent — safe to call on an existing shelf to update metadata.

Inputs: params (required). Returns: result

docshelf_list_documentsA
Read-onlyIdempotent

List documents grouped by category.

Pass a category to filter; omit it to list everything.

Inputs: params (required). Returns: result

docshelf_read_documentA
Read-onlyIdempotent

Read a document or section file from inside the shelf's docs/.

Returns the file content directly over MCP — useful for private or purely-local shelves where the raw.githubusercontent.com fetch trick doesn't apply. Pass a relative_path from search / list_documents. Large files are truncated to max_bytes (default 100 KB) with truncated: true; page with the returned next_offset (slices snap to UTF-8 character boundaries, so it may differ from offset + max_bytes — using it avoids splitting a multibyte character) or read the individual split sections. Paths that escape docs/ are rejected.

Inputs: params (required). Returns: result

docshelf_rebuild_indexA
Idempotent

Regenerate INDEX.md from the current on-disk shelf state.

Useful after manual edits to docs/ or .docshelf.json.

Inputs: params (required). Returns: result

docshelf_remove_documentA
Destructive

Remove a document — its file, split sections, and metadata entry.

Accepts the filename, the slug, or the human title used at add time. INDEX.md is regenerated automatically. Pass dry_run=true to see what would be deleted without touching anything. The caller still owns the git commit / push step.

Inputs: params (required). Returns: result

docshelf_rename_documentA
Destructive

Retitle, recategorize, or re-describe a document — no re-conversion.

Moves the document .md, its split-section directory, and its .meta.json entry (changing the slug when the title changes, or the directory when the category changes), then regenerates INDEX.md. Give at least one of new_title / new_category / new_description. Refuses to clobber an existing target. Pass dry_run=true to preview. The caller still owns the git commit / push step.

Inputs: params (required). Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.4/5.0

    Scored across 11 tools

    Disambiguation5/5

    Each tool targets a distinct operation: init, add single, add batch, read, remove, rename, rebuild index, doctor check, search, list, and standalone conversion. While add_document and convert_pdf both handle PDF conversion, the former commits to the shelf while the latter does not, making their purposes clearly separable. No two tools appear to overlap in function.

    Naming Consistency5/5

    All tools follow a consistent `docshelf_<verb>_<noun>` snake_case pattern, with clear, descriptive verbs like add, remove, rename, search, list, convert. The only slightly unusual verb is 'doctor', but it is intuitive for a health-check tool. The prefix is uniformly applied across all 11 tools, creating a strong, predictable convention.

    Tool Count5/5

    11 tools is well within the ideal 3–15 range for a domain-specific server. Each tool serves a distinct need in the document shelf lifecycle, from initialization and document ingestion to search, maintenance, and standalone conversion. No tool feels superfluous, and the surface is neither too sparse nor overloaded.

    Completeness5/5

    The tool set covers the full lifecycle of a document shelf: create (init), add documents (both single and batch), read, remove, rename (update metadata), list, search, and rebuild the index. Additionally, a 'doctor' tool addresses drift repair and a standalone PDF converter supports pre-shelf workflows. There are no obvious gaps or dead ends for the stated purpose.

    Maintenance

    ActivitySlowing
    ResponsivenessLimited data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      An MCP server that enables AI agents to query specialized, domain-specific knowledge bases built using the LightRAG framework for enhanced retrieval-augmented generation. It allows for managing and searching knowledge graphs and vector embeddings to provide accurate, context-aware information during an AI assistant's reasoning process.
      34 PyPI
      59
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Zero-config knowledge base for AI coding agents. Loads your markdown docs into a searchable database and exposes them as MCP tools — search, read, and manage documentation without leaving your editor. Works instantly with SQLite (no setup), upgrades to PostgreSQL + pgvector for hybrid semantic search. 6 MCP tools, 3 resources, FTS5 keyword search, 176 tests.
      79 PyPI
      31
      MIT