Skip to main content
Glama
elpaca

ai-papers-mcp

by elpaca
OverviewSchemaSourceRelated ServersScoreDiscussionsAdmin
Python
Local

AI Papers Helper

Search, index, and read academic AI/ML papers — from the terminal or directly inside your AI coding assistant.

AI Papers Helper crawls top-tier AI/ML conference proceedings, indexes them in a local SQLite database with FTS5 search, queries arxiv, and converts paper PDFs into clean markdown for in-depth reading. It ships as both a CLI (papers) and an MCP server (ai-papers-mcp), so you can use it standalone or wire it into Claude Code / any MCP-compatible client.

✨ Features

  • 🌐 Conference crawling — Fetch papers from NeurIPS, ICML, ICLR, CVPR, ICCV, WACV, AAAI, IJCAI, MLSys, ACL, EMNLP with parallel workers and incremental crawl state.

  • 🔎 Local search — SQLite + FTS5 with BM25 ranking (title-weighted) and relevance/date ordering. Missing-abstract penalty keeps results meaningful.

  • 📡 arxiv search — Live arxiv search via web scraping (default) or the Atom API backend, with date filters (past 12 months, specific year, date range).

  • 📄 PDF → Markdown — Parse papers through the MinerU API with content-addressable caching, so each PDF is only parsed once.

  • 📑 TOC & grep — Extract a paper's table of contents and grep specific sections/equations from its full markdown.

  • 🤝 MCP server — Expose everything as 4 tools to Claude Code or any MCP client: search the local library, search arxiv, get a paper's TOC, and grep paper content.

  • 🧠 Smart resolution — Paper lookup falls back progressively: exact DB match → fuzzy match (SequenceMatcher, 0.8 threshold) → arxiv search.

Related MCP server: arXiv MCP Server

📦 Installation

Requires Python 3.12+.

# Clone
git clone https://github.com/<your-org>/ai_papers_helper.git
cd ai_papers_helper

# Install as a global tool (exposes `papers` and `ai-papers-mcp` on PATH)
uv tool install --force .

This exposes two console scripts:

Command

Description

papers

The CLI app

ai-papers-mcp

The MCP server

Environment variables

Variable

Default

Description

MINERU_API_KEY

—

Required for PDF→markdown parsing. Get one at mineru.net.

ARXIV_BACKEND

web

arxiv backend: web (scraping) or api (Atom API).

ARXIV_MIN_INTERVAL

10.0

Minimum seconds between arxiv API calls (rate limiting).

All data lives under ~/.ai_papers_helper/:

~/.ai_papers_helper/
├── papers.db            # SQLite database (FTS5 index)
├── crawl_state.json     # Incremental crawl state
├── cache/               # HTTP response cache
├── arxiv_cache/         # arxiv result cache (24h TTL)
└── papers/              # Parsed markdown (content-addressed by URL hash)

🚀 Quick start

1. Initialize the database

papers init

2. Crawl conference papers

# Crawl everything new (incremental — skips already-crawled years)
papers update

# Crawl a specific conference and year
papers update --conference acl,emnlp --year 2024

# Force re-crawl a specific source/year
papers update --force --conference cvpr --year 2023

# Tune parallelism
papers update --workers 16

3. Search

# Search the local library (default: titles only)
papers search-library "diffusion model"
papers sl "graph neural network" --order-by date

# Titles + full abstracts
papers sl "transformer attention" --full-abs

# Paginate
papers sl "reinforcement learning" --page 2
papers sl "reinforcement learning" --from 31

# Search arxiv
papers search-arxiv "mixture of experts"
papers sa "vision transformer" --sort-by date --date-filter-by past_12
papers sa "llm" --date-filter-by specific_year --date-year 2024
papers sa "diffusion" --date-filter-by date_range --date-from 2024-01 --date-to 2024-06

4. Read a paper

# Get the full markdown (creates a /tmp/<title>.md symlink to the cached file)
papers content "Attention Is All You Need"

# Show the table of contents
papers content "Attention Is All You Need" --toc

🧩 Skill (for AI coding agents)

The repo ships a ready-made Agent Skills skill at papers-skill/SKILL.md. It teaches AI coding agents (pi, Claude Code, etc.) how to drive the papers CLI: when to prefer the local library vs arxiv, the search -> content -> grep workflow, and the full option reference.

The agent loads the skill on-demand when a task matches, then runs papers itself via the shell - no server process required. This is the lightest-weight way to let an agent search and read papers.

Install the skill

Point your agent at the papers-skill directory. For pi:

# Global (available in every project)
ln -s "$(pwd)/papers-skill" ~/.pi/agent/skills/papers

# Or project-level
mkdir -p .pi/skills && ln -s "$(pwd)/papers-skill" .pi/skills/papers

Skill vs MCP: The skill is just instructions (the agent runs the CLI via shell); the MCP server below exposes typed tools. The skill needs nothing running, the MCP server gives more structured tool calls - pick whichever fits your agent.

🤖 Using the MCP server

The same functionality is exposed as an MCP server for use inside Claude Code or any MCP-compatible client.

4 tools

Tool

Description

search_library_papers

Search the local indexed database by keywords.

search_arxiv_papers

Search arxiv for the latest papers, with date filters.

get_paper_toc

Get a paper's table of contents. Call this first before reading.

grep_paper_content

grep patterns in a paper's full markdown (e.g. read a whole section).

🏗 Architecture

src/ai_papers_helper/
├── cli.py                 # Typer CLI: init, search-library, search-arxiv, update, content
├── mcp_server.py          # FastMCP server exposing 4 tools
├── config.py              # Paths, env vars, page-size constants
├── core/
│   ├── models.py          # Pydantic v2: Author, Paper
│   └── database.py        # SQLite + FTS5 singleton, BM25 ranking, auto-sync triggers
├── crawler/
│   ├── base.py            # BaseCrawler ABC + parallel detail-page fetching
│   ├── http.py            # Shared requests.Session w/ retry + file cache
│   ├── state.py           # CrawlState (per-source crawled years, JSON)
│   ├── cvf.py             # CVPR / ICCV / WACV
│   ├── aaai.py            # AAAI
│   ├── ijcai.py           # IJCAI
│   ├── icml.py            # ICML URL helpers
│   ├── acl_anthology.py   # ACL / EMNLP (ACL Anthology)
│   └── json_api.py        # Generic JSON API crawler (NeurIPS, ICML, ICLR, MLSys)
├── search/
│   ├── library_search.py  # Local FTS5 search, relevance/date ordering
│   └── arxiv_search.py    # arxiv dispatcher (web vs api backend)
├── retrieval/
│   ├── resolver.py        # Progressive lookup: exact → fuzzy → arxiv
│   ├── parser.py          # MinerU API client (async polling, content-addressed cache)
│   ├── content.py         # Markdown TOC extraction + section slicing
│   └── lookup.py          # End-to-end: title → paper → markdown
└── helper/                # arxiv web/api internals, pagination, rate limiting

How search works

  • Local library: FTS5 with porter unicode61 tokenizer. BM25 with title weight 10.0, abstract weight 1.0. Results with missing abstracts are penalized (* 0.9) so well-documented papers surface first.

  • Date ordering: BM25 rank is bucketed into relevance tiers; within a tier, newer papers come first — so you don't lose relevance entirely.

  • arxiv: Web scraping by default (no API key, gentler). Switch to the Atom API with ARXIV_BACKEND=api for query-syntax power (field prefixes, boolean operators).

How PDF reading works

  1. Resolve the paper by title (DB exact → fuzzy → arxiv).

  2. Resolve a PDF URL (paper.pdf_url, else arxiv lookup, backfilling the DB).

  3. Send to MinerU; poll until done; download & unzip the result.

  4. Cache under ~/.ai_papers_helper/papers/{sha256(url)[:16]}/full.md — content-addressed, so re-reads are instant.

🧪 Development

uv sync                 # install deps
pytest                  # run the full suite
pytest tests/test_database.py          # single file
pytest -k "fuzzy"                      # by name pattern

Tests use temp databases, mock network calls (patch.object(crawler, "_fetch_url", ...)), and HTML/JSON fixtures in tests/fixtures/. See CLAUDE.md for the full contributor guide.

Conventions

  • Python 3.12+, from __future__ import annotations in every file

  • No async; parallelism via ThreadPoolExecutor

  • Standard-library sqlite3 (no ORM), requests for HTTP

  • Logging via logging.getLogger(__name__)

📄 License

MIT

Available Tools

4 tools
get_paper_tocA
Read-onlyIdempotent
Get the table of contents of a paper. Always call this tool FIRST before reading a paper's content, then use grep_paper_content to read specific chapters or search for content.

Return the table of contents of the designated paper.

Args:
    title: Paper title (supports fuzzy matching, but provide as accurately as possible).
ParametersJSON Schema
NameRequiredDescriptionDefault
titleYes

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate read-only and idempotent behavior. The description adds useful context beyond annotations: fuzzy title matching and the sequential requirement to call before other content tools. Does not detail return format, but with annotations covering safety, this is a minor gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured: first sentence states the purpose, second provides usage order, and third explains the parameter. No unnecessary fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, read-only tool, the description covers purpose, usage sequence, and parameter semantics adequately. It does not describe the return format, but given the simple nature and existing annotations, the description is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema only defines title as a string. The description explains the meaning of the title parameter, notes that fuzzy matching is supported, and advises providing it as accurately as possible. This fully compensates for the 0% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves the table of contents of a paper. It uses a specific verb ('Get') and resource ('table of contents'), and distinguishes itself from sibling tools like grep_paper_content by positioning itself as the first step before content reading.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs to call this tool FIRST before reading a paper's content, then directs to grep_paper_content for reading chapters or searching. This provides clear when-to-use guidance and names the alternative tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

grep_paper_contentA
Read-onlyIdempotent
Grep patterns in the full converted markdown text of a paper. Pipes the paper's parsed markdown as stdin to the given grep command.

Returns the stdout of the grep command.

Args:
    title: Paper title (supports fuzzy matching, but provide as accurately as possible).
    grep_command: Full grep command to run. Paper markdown is piped as stdin. Examples:
        - 'grep -A 150 "3 METHODOLOGY"' to read a whole section
        - 'grep -i -C 3 "learning rate"' shows 3 lines of context around it
        - 'grep -A 10 "Equation 1"' to locate equations or figures
        Note: First call `get_paper_toc` to get the exact section names (e.g., "3 METHODOLOGY", "4.2 Baseline").
    max_chars: Max output characters (default 6000).
ParametersJSON Schema
NameRequiredDescriptionDefault
titleYes
max_charsNo
grep_commandYes

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this as a safe, read-only, idempotent operation, so the bar is lower. The description adds useful behavioral detail: it pipes markdown as stdin, returns stdout, enforces a max_chars limit, and explains fuzzy title matching. This enriches the annotation-only picture without contradicting it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured: a one-sentence purpose statement, a clear return line, and then parameter explanations with call-worthy examples. Every sentence earns its place without fluff, making it both concise and informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity and lack of output schema, the description covers all essential aspects: how it works (stdin piping), what it returns (stdout), parameters, usage tips, and output limits. The note to call get_paper_toc first completes the workflow guidance. It is complete for this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the full burden. It explains each parameter thoroughly: title with fuzzy matching, grep_command with multiple examples including a recommendation to use the TOC first, and max_chars with its default. This fully compensates for the empty schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool greps patterns in a paper's full markdown text by piping it to a user-supplied grep command. This specific verb+resource clearly distinguishes it from sibling tools like search_library_papers or get_paper_toc, which operate at a higher level.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear practical context with concrete examples (reading sections, finding context) and advises calling get_paper_toc first for exact section names. However, it does not explicitly name alternative tools or state when not to use this tool, so it falls short of the highest level of guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_arxiv_papersA
Read-onlyIdempotent

Search arxiv for the latest research papers.

Return paper list with title (and abstract). Submitt date is provided for each paper.

Args: query: Search keywords. mode: Result display mode: "title_only" (default), "abbr_abs" (abbreviated abstracts), or "full_abs" (full abstracts). sort_by: "relevance" or "date" (default is "relevance"). sort_order: "asc" or "desc" (default is "desc"). page: 1-based page number (mutually exclusive with start_from). start_from: 1-based paper index (not page index) to start from (mutually exclusive with page). date_filter_by: Date filter mode — all_dates (default), past_12 (show papers from the past 12 months), specific_year (show papers from a specific year), or date_range (show papers from a specific date range). date_year: Four-digit year (for specific_year). date_from: Start date for date_range, format YYYY[-MM[-DD]]. date_to: End date for date_range, format YYYY[-MM[-DD]].

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNotitle_only
pageNo
queryYes
date_toNo
sort_byNorelevance
date_fromNo
date_yearNo
sort_orderNodesc
start_fromNo
date_filter_byNoall_dates

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds valuable context: output includes title/abstract and submit date, and it explains parameter behaviors like the mutual exclusivity of 'page' and 'start_from'. This goes beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description front-loads the purpose in two sentences, then systematically lists parameters in a clear structure. Every sentence adds value, especially given the need to compensate for the absent schema descriptions.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 10-parameter search tool, the description covers output format, parameter semantics, and special constraints. It lacks examples or error behavior, but the rich parameter detail and annotations make it reasonably complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description's 'Args' section thoroughly documents all 10 parameters, including their meanings, defaults, formats, and mutual exclusivity. This fully compensates for the missing schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Search'), names the resource ('arxiv'), and clearly states the return value ('Return paper list with title (and abstract)'). The tool name itself distinguishes it from the sibling 'search_library_papers'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a clear context: search arxiv for latest papers. It implicitly differentiates from sibling tools by specifying 'arxiv', but does not explicitly mention alternatives or when-not-to-use. This fits 'clear context, no exclusions'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_library_papersA
Read-onlyIdempotent

Search the local paper database by keywords. Indexed conferences: none

Return paper list with title (and abstract). Source of each paper is also provided.

Args:
    query: Search keywords.
    mode: Result display mode: "title_only" (default), "abbr_abs" (abbreviated abstracts), or "full_abs" (full abstracts).
    order_by: "relevance" (default) or "date" (relevance-tiered, then newest first).
    page: 1-based page number (mutually exclusive with start_from).
    start_from: Paper index (not page index) to start from (mutually exclusive with page).
ParametersJSON Schema
NameRequiredDescriptionDefault
modeNotitle_only
pageNo
queryYes
order_byNorelevance
start_fromNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, non-destructive behavior; the description adds meaningful context beyond that by specifying the return list includes title/abstract and source, and by mentioning indexed-conference coverage. It could disclose pagination limits or result-count behavior, but the key traits are covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the purpose, followed by a coverage caveat, return summary, and a structured Args list. It is slightly padded by 'Indexed conferences: none,' but the sentence is informative and not redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a search tool without an output schema, the description explains the return shape, source inclusion, display modes, ordering, and pagination. It is not fully complete because page size and precise matching behavior (e.g., metadata vs full-text) are unspecified, but the main workflow is clear.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has zero descriptions, so the Args section carries the full explanatory burden and does so thoroughly: each parameter is named, with defaults, allowed values, and the page/start_from mutual-exclusivity constraint.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Search the local paper database by keywords.' It clarifies the local scope, distinguishing it from sibling search_arxiv_papers, and the return of paper lists with abstracts differentiates it from grep_paper_content/get_paper_toc.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this is for searching the local paper database, and 'Indexed conferences: none' provides a context cue, but it never explicitly states when to prefer this over sibling tools or gives exclusion criteria. No alternatives are named.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.0
    • First observedget_paper_toc
    • First observedgrep_paper_content
    • First observedsearch_arxiv_papers
    • First observedsearch_library_papers
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.4/5.0

Scored across 4 tools

Disambiguation4/5

The two search tools (search_library_papers and search_arxiv_papers) are clearly differentiated by their source (local vs arxiv), and the paper-reading tools (get_paper_toc and grep_paper_content) serve distinct purposes. Minor potential for confusion exists between the two search tools, but descriptions clarify the difference.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern using snake_case (search_library_papers, get_paper_toc, search_arxiv_papers, grep_paper_content). The verbs vary but the structure is uniform and predictable.

Tool Count5/5

Four tools is well-scoped for the server's purpose: two search tools cover different paper sources, and two tools support reading specific paper content. The count is neither sparse nor bloated.

Completeness4/5

The core workflow of finding papers and reading their content is covered. Minor gaps exist, such as lack of a tool to retrieve the full paper text directly (only grep-based section extraction) or list all papers in the local library, but these are workable limitations.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • ArXiv MCP Server

    huanongfish
    A
    license
    B
    quality
    D
    maintenance
    Enables AI assistants to search and access arXiv research papers through a simple Message Control Protocol interface, allowing for paper search, download, listing, and reading capabilities.
    4
    7
    Apache 2.0

  • arXiv MCP Server

    LiamConnell
    A
    license
    A
    quality
    D
    maintenance
    Enables searching, downloading, and managing academic papers from arXiv.org through natural language interactions. Provides tools for paper discovery, PDF downloads, and local paper collection management.
    4
    1
    MIT

  • arXiv MCP Server

    AnnaSuSu
    A
    license
    A
    quality
    D
    maintenance
    Enables LLMs to search, download, and read arXiv papers with automatic PDF text extraction and section filtering. Provides AI assistants direct access to scientific literature with local caching for fast re-access.
    3
    1
    MIT

  • paper-search-mcp

    xiaoxiaoxiaotao
    F
    license
    A
    quality
    D
    maintenance
    Enables agents to search papers across Semantic Scholar and arXiv, read and extract text from arXiv PDFs, align records across sources, and produce structured literature-analysis digests.
    10
    1
    -