<div align="center">

<h1 align="center">
  <img src="assets/logo.png" alt="logo" height="28" style="vertical-align: middle; margin-right: 8px;">
  MinerU Document Explorer
</h1>

<h4>Agent-native knowledge engine — search, deep-read, and build knowledge bases<br>from Markdown, PDF, DOCX, and PPTX.</h4>

<p>
  <a href="https://www.npmjs.com/package/mineru-document-explorer"><img src="https://img.shields.io/npm/v/mineru-document-explorer?style=flat-square&color=cb3837" alt="npm"/></a>
  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue?style=flat-square" alt="license"/></a>
  <a href="https://github.com/opendatalab/MinerU-Document-Explorer/actions"><img src="https://img.shields.io/github/actions/workflow/status/opendatalab/MinerU-Document-Explorer/ci.yml?style=flat-square&label=CI" alt="CI"/></a>
  <a href="https://github.com/opendatalab/MinerU-Document-Explorer"><img src="https://img.shields.io/github/stars/opendatalab/MinerU-Document-Explorer?style=flat-square" alt="stars"/></a>
</p>

<p>
  <a href="README-zh.md">中文文档</a> ·
  <a href="docs/mcp.md">MCP Setup</a> ·
  <a href="docs/cli.md">CLI Reference</a> ·
  <a href="demo/">Demo</a> ·
  <a href="CONTRIBUTING.md">Contributing</a>
</p>

</div>

---

## 🤔 Why MinerU Document Explorer?

MinerU Document Explorer equips your agent with three tool suites — **Retrieve, Deep Read, and Ingest** — closing the full knowledge loop:

![Overview of MinerU Document Explorer](assets/overview_en.png)

- **🔍 Retrieve** — Cross-collection search: BM25, vector, and hybrid with LLM reranking and query expansion
- **📖 Deep Read** — Navigate inside a single document without loading the whole file: table of contents, section reading, inline search, and element extraction
- **📝 Ingest** — Build and maintain a LLM wiki from raw documents, following the [Karpathy LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) pattern

Developed by the [MinerU](https://github.com/opendatalab/MinerU) team, building on [QMD](https://github.com/tobi/qmd) and [Karpathy's LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f).

### 💡 What can you do with it?

- **Research assistant** — Point it at a folder of PDFs and ask your AI agent to survey the literature, compare methods, or find specific results across papers
- **Project knowledge base** — Index your codebase, docs, and design specs; let the agent answer questions about architecture, find relevant code, or trace requirements
- **Study companion** — Upload textbooks or lecture notes; use `doc_toc` + `doc_read` to navigate, `doc_grep` to search, and build a wiki of key concepts
- **Enterprise document search** — Index contracts, reports, or manuals and search them with natural language queries enhanced by LLM reranking

> **See it in action**: The [demo/](demo/) folder contains a complete end-to-end example — an AI agent automatically reads ~10 arXiv papers on RAG, builds an interlinked wiki knowledge base, and writes a research survey. See the [Demo Guide](demo/README.md) for step-by-step instructions.

## 🚀 Quick Start

> Agent-Assisted Setup: If you're using an AI agent (Claude Code, Cursor, etc.), simply ask it to help you deploy MinerU Document Explorer and install skills — the agent can handle the entire setup process for you, including MCP configuration.

```
Follow the https://github.com/opendatalab/MinerU-Document-Explorer/blob/main/docs/quickstart.md to install MinerU Document Explorer and walk the user through configuration.
```

## 📖 Document Deep Reading

Navigate and search within a single document without reading the whole file:

```sh
# View document structure
qmd doc-toc papers/attention-is-all-you-need.pdf

# Read specific sections by address
qmd doc-read papers/attention-is-all-you-need.pdf "line:45-120"

# Search within one document
qmd doc-grep papers/attention-is-all-you-need.pdf "self-attention"
```

## 🔌 MCP Server — 15 Tools for AI Agents

Integrate with AI agents via [Model Context Protocol](https://modelcontextprotocol.io).

> **MCP Server vs CLI**: The MCP server runs as a **persistent process** — LLM models (embeddings, reranker, query expansion) are loaded once and stay in memory across requests. CLI commands like `qmd query` must reload all models on every invocation, adding ~5–15 s of startup overhead each time. **For agent workflows, always prefer the MCP server.**

Two transport modes:

| Mode | Command | Best for |
|:-----|:--------|:---------|
| **stdio** | `qmd mcp` | Claude Desktop, Claude Code — client spawns and manages the process |
| **HTTP daemon** | `qmd mcp --http --daemon` | Cursor, Windsurf, VS Code, multi-client setups — one shared persistent server |

```sh
# Start the HTTP daemon (recommended — models stay loaded across all requests)
qmd mcp --http --daemon             # default port 8181
qmd mcp --http --daemon --port 8080 # custom port

# Verify server is running
curl http://localhost:8181/health

# Stop the daemon
qmd mcp stop
```

### Client Configuration

<details>
<summary><b>Cursor</b> — add to <code>.cursor/mcp.json</code> (project) or <code>~/.cursor/mcp.json</code> (global)</summary>

**Option A — stdio** (Cursor manages the process lifecycle):

```json
{
  "mcpServers": {
    "qmd": {
      "command": "qmd",
      "args": ["mcp"]
    }
  }
}
```

**Option B — HTTP** (run `qmd mcp --http --daemon` first; models stay loaded, faster responses):

```json
{
  "mcpServers": {
    "qmd": {
      "url": "http://localhost:8181/mcp"
    }
  }
}
```

</details>

<details>
<summary><b>Claude Desktop</b> — add to <code>~/Library/Application Support/Claude/claude_desktop_config.json</code></summary>

```json
{
  "mcpServers": {
    "qmd": {
      "command": "qmd",
      "args": ["mcp"]
    }
  }
}
```

</details>

<details>
<summary><b>Claude Code</b> — add to <code>~/.claude/settings.json</code> or run <code>claude mcp add qmd -- qmd mcp</code></summary>

```json
{
  "mcpServers": {
    "qmd": {
      "command": "qmd",
      "args": ["mcp"]
    }
  }
}
```

</details>

<details>
<summary><b>Windsurf / VS Code / Other MCP Clients</b></summary>

For **stdio** transport, use `"command": "qmd"`, `"args": ["mcp"]` in your client's MCP configuration.

For **HTTP** transport, start `qmd mcp --http --daemon` and point your client to `http://localhost:8181/mcp`.

</details>

See [MCP setup guide](docs/mcp.md) for all 15 tools and HTTP transport details.

### Agent Skills

MinerU Document Explorer ships with a built-in [Agent Skill](skills/mineru-document-explorer/SKILL.md) that teaches AI agents how to use the full tool suite effectively — decision trees, usage patterns, and best practices for all 15 MCP tools.

```sh
# Install the skill (works with both npm and source installs)
qmd skill install              # local project (.agents/skills/)
qmd skill install --global     # global (~/.agents/skills/)

# Or from source repo
claude skill add ./skills/mineru-document-explorer/SKILL.md
```

## 📊 How It Compares

| | MinerU Doc Explorer | LlamaIndex | Obsidian | NotebookLM |
|:---|:---:|:---:|:---:|:---:|
| **Runs 100% locally** | ✅ | ⚠️ LLM APIs | ✅ | ❌ Cloud |
| **Agent integration (MCP)** | **15 tools** | Plugin | ❌ | ❌ |
| **Deep reading within docs** | ✅ | ❌ | ❌ | ✅ |
| **Wiki knowledge compilation** | ✅ | ❌ | Manual | ❌ |
| **Formats** | MD, PDF, DOCX, PPTX | Many | MD | PDF, URL |
| **Search pipeline** | BM25 + vec + rerank | Configurable | Basic | Proprietary |
| **Zero-config search** | ✅ `qmd search` | ❌ | Plugin | N/A |
| **Open source** | MIT | MIT | Partial | ❌ |

## ⚙️ Requirements

| Requirement | Notes |
|-------------|-------|
| **Node.js** >= 22 or **Bun** | Runtime |
| **Python** >= 3.10 | Document processing (`pymupdf`, `python-docx`, `python-pptx`) |
| **macOS** | `brew install sqlite` for extension support |

### 📄 Document Processing Setup

Python 3.10+ is required for document processing (PDF, DOCX, PPTX):

```sh
# Check Python version
python3 --version  # needs >= 3.10

# Install required Python packages
pip install pymupdf python-docx python-pptx

# Verify
python3 -c "import pymupdf; import docx; import pptx; print('OK')"
```

<details>
<summary><b>MinerU Cloud</b> — high-quality PDF extraction for scanned documents and complex layouts (optional)</summary>

```sh
pip install mineru-open-sdk
export MINERU_API_KEY="your-key"  # get from https://mineru.net
```

When `MINERU_API_KEY` is set, MinerU Cloud is automatically used as the primary PDF provider with PyMuPDF as fallback.

For advanced configuration (custom providers, local VLM models, GPT PageIndex), create `~/.config/qmd/doc-reading.json`:

```json
{
  "docReading": {
    "providers": {
      "fullText": { "pdf": ["mineru_cloud", "pymupdf"] }
    },
    "credentials": {
      "mineru": { "api_key": "your-api-key" }
    }
  }
}
```

</details>

### 🤖 LLM Models (auto-downloaded on first use)

| Model | Purpose | Size |
|:------|:--------|:-----|
| embeddinggemma-300M | Vector embeddings | ~300 MB |
| qwen3-reranker-0.6b | Re-ranking | ~640 MB |
| qmd-query-expansion-1.7B | Query expansion | ~1.1 GB |

> Models are only needed for `qmd embed`, `qmd vsearch`, and `qmd query`. `qmd search` runs BM25 retrieval.

## 📚 Documentation

| | |
|:---|:---|
| 🎯 [Demo Guide](demo/README.md) | End-to-end example: agent-driven RAG research survey |
| 📖 [CLI Reference](docs/cli.md) | All commands, options, output formats |
| 🔌 [MCP Server](docs/mcp.md) | Setup, 15 tools, HTTP transport |
| 📦 [SDK / Library](docs/sdk.md) | TypeScript API, types, examples |
| 🏗️ [Architecture](docs/architecture.md) | Search pipeline, scoring, data schema, chunking |
| 🤝 [Contributing](CONTRIBUTING.md) | Development setup, code style, how to contribute |

## ❤️ Acknowledgments

MinerU Document Explorer builds upon these foundational projects:

- **[QMD](https://github.com/tobi/qmd)** by [Tobi Lutke](https://github.com/tobi) — An on-device search engine and CLI toolkit for markdown documents
- **[LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)** by [Andrej Karpathy](https://github.com/karpathy) — the conceptual pattern for LLM-maintained knowledge bases
- **[MinerU](https://github.com/opendatalab/MinerU)** by [OpenDataLab](https://github.com/opendatalab) — high-quality document parsing and extraction

---

## 📝 Changelog

### v1 — 2026-04-07 (Current)

Rebuilt from an OpenClaw agent skill into a full agent-native knowledge engine: npm package (`npm install -g mineru-document-explorer`), `qmd` CLI, MCP server with 15 tools across three groups (Retrieval / Deep Reading / Knowledge Ingestion), multi-format support (MD, PDF, DOCX, PPTX), hybrid search (BM25 + vector + LLM reranking), and LLM Wiki knowledge base pattern.

### v0 — 2026-03-30 (Previous)

OpenClaw-native agent skill (`doc-search` CLI). Four capabilities: Logic Retrieval, Semantic Retrieval, Keyword Retrieval, Evidence Extraction. See the [v0 repository](https://github.com/opendatalab/MinerU-Document-Explorer/tree/v0).