Paper Extraction MCP

An MCP (Model Context Protocol) server for structured information extraction from academic PDF papers using LLM. It integrates seamlessly with Claude Desktop, allowing you to extract metadata and domain-specific content categories from research papers through natural language conversation.

Typhoon disaster governance is provided as a built-in example. The system is fully customizable for any research domain — see Adapting to Other Domains.

Features

• LLM-Powered Extraction — Uses OpenAI-compatible LLMs to extract structured data from full-text PDFs

• Customizable Schema — Define your own metadata fields and content categories via config.json

• Smart Chunking — Automatically splits long papers and merges results with deduplication

• Dual Output — JSON and CSV formats for downstream analysis

• MCP Protocol — Works directly inside Claude Desktop as a tool server

• Batch Processing — Extract from a single paper or all papers at once

Related MCP server: kivv

Quick Start

1. Install Dependencies

pip install -r requirements.txt

2. Configure API Key

Copy the template and fill in your API key:

cp config.json.template config.json

Edit config.json:

{
  "llm_config": {
    "api_key": "sk-your-api-key-here",
    "api_base": "https://api.openai.com/v1",
    "model": "gpt-4o"
  }
}

Any OpenAI-compatible API is supported (OpenAI, Azure OpenAI, local LLMs with OpenAI-compatible endpoints, third-party proxies, etc.).

3. Add PDF Papers

Place your PDF files in the papers/ directory.

4. Configure Claude Desktop

Edit the Claude Desktop config file:

• Windows: %APPDATA%\Claude\claude_desktop_config.json

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

Add:

{
  "mcpServers": {
    "paper-extraction": {
      "command": "python",
      "args": ["<full-path-to>/paper-extraction-MCP/server.py"],
      "cwd": "<full-path-to>/paper-extraction-MCP"
    }
  }
}

Replace <full-path-to> with your actual path. Then restart Claude Desktop.

5. Use It

In Claude Desktop, simply say:

List the PDF papers available for extraction.

Extract the paper "my_paper.pdf".

Extract all papers and show me a summary.

MCP Tools

Project Structure

paper-extraction-MCP/
├── server.py              # MCP server (entry point)
├── pdf_extractor.py       # Core extraction logic
├── config.json            # Your configuration (gitignored)
├── config.json.template   # Configuration template
├── requirements.txt       # Python dependencies
├── papers/                # Place PDF files here
├── outputs/
│   ├── json/              # JSON extraction results
│   └── csv/               # CSV extraction results
├── setup.bat              # Windows quick setup
├── setup.sh               # macOS/Linux quick setup
├── CLAUDE_SETUP.md        # Detailed Claude Desktop setup guide
├── LICENSE                # MIT License
└── README.md

How It Works

PDF file
  │
  ▼
pdfplumber (text extraction)
  │
  ▼
Full text ──► LLM API call ──► Structured JSON
                  │
          config.json defines:
          - extraction_prompt (fields & rules)
          - llm_config (model, temperature)
                  │
                  ▼
          JSON + CSV output

1. Text Extraction: pdfplumber extracts full text from each PDF page

2. LLM Extraction: The text is sent to an LLM with your extraction_prompt, which defines what fields to extract and how

3. Smart Chunking: If the text exceeds ~100K characters, it is automatically split into chunks, extracted separately, and merged with deduplication

4. Output: Results are saved as JSON and CSV

Built-in Example: Typhoon Disaster Governance

The default config.json is pre-configured for extracting information from typhoon disaster governance papers:

Metadata fields (5):

• DOI, Title, Journal, Author Affiliations, Publication Date

Content categories (7):

Adapting to Other Domains

The core of this tool is domain-agnostic. You only need to modify config.json — no code changes required. Here is a step-by-step guide:

Step 1: Define Your Categories

Decide what information you want to extract. For example:

Step 2: Write Your Extraction Prompt

Edit the extraction_prompt field in config.json. The prompt should:

1. Describe the assistant's role for your domain

2. List metadata fields (DOI, title, journal, etc. — usually the same across domains)

3. Define each content category with clear descriptions and examples

4. Set extraction rules (no hallucination, preserve original text, deduplication)

5. Specify the output JSON format with exact key names

Here is a template you can adapt:

{
  "extraction_prompt": [
    "You are an academic information extraction assistant specialized in [YOUR DOMAIN].",
    "",
    "From each paper, extract:",
    "1) Bibliographic metadata",
    "2) Domain-specific content, categorized as follows:",
    "",
    "METADATA FIELDS:",
    "- 论文DOI: Full DOI URL",
    "- 题目: Paper title",
    "- 期刊名称: Journal name",
    "- 作者机构: Author affiliations (semicolon-separated)",
    "- 发表日期: Publication date (Month Year)",
    "",
    "CONTENT CATEGORIES:",
    "- [Category1_Key]: [Description of what to extract]",
    "- [Category2_Key]: [Description of what to extract]",
    "- ... (add as many as needed)",
    "",
    "RULES:",
    "- Only extract content explicitly present in the paper",
    "- Preserve original text, do not summarize",
    "- Output as JSON with metadata as strings, categories as arrays of strings"
  ]
}

Step 3: Update the Field Mapping in pdf_extractor.py

If you change the Chinese key names in your extraction prompt (e.g., use "预防措施" instead of "检测预警措施"), update the _format_result() and _merge_chunk_results() methods in pdf_extractor.py to map your new keys to the internal field names.

For example, if your domain is public health:

# In _format_result():
final_result = {
    # ... metadata fields stay the same ...
    "prevention_measures": self._join_measures(result_data.get("预防措施", [])),
    "treatment_measures": self._join_measures(result_data.get("治疗措施", [])),
    "surveillance_measures": self._join_measures(result_data.get("监测措施", [])),
    # ... add your categories ...
}

Step 4: Update server.py Display (Optional)

If you want the MCP tool output to show your custom category names, update the call_tool() function in server.py where it formats the extraction result display.

Tips for Writing Good Extraction Prompts

1. Be specific: Provide concrete examples of what belongs in each category

2. Set boundaries: Clearly state what does NOT belong in each category to avoid overlap

3. Request detail: Ask for full paragraphs, not just keywords — this prevents information loss

4. Use the paper's language: Tell the LLM to preserve the original language (Chinese/English)

5. Test iteratively: Try your prompt on 2-3 papers, review the results, and refine

Configuration Reference

config.json

Recommended Models

Any OpenAI-compatible model works (DeepSeek, Qwen, local Ollama, etc.).

Cost Estimate

Using GPT-4o:

• Single paper (~10 pages): ~$0.04-0.07

• 100 papers: ~$4-7

Troubleshooting

See CLAUDE_SETUP.md for a detailed setup and troubleshooting guide.

License

MIT License

If this project helps your research, please give it a star!