academic-mcp

A local MCP service that is the whole backend of an academic research workflow: literature search, citation chains, per-project research memory, paper download and PDF→Markdown conversion. Any MCP client can drive it — the first one is the DSH academic agent preset, which contains no Python at all.

client (any MCP host)  ──JSON-RPC /mcp──▶  academic-mcp :8790
                                             ├─ Scopus + OpenAlex    (search, citation chains)
                                             ├─ project memory JSON  (notes, findings, progress)
                                             ├─ arXiv / Elsevier / browser → PDF
                                             └─ MinerU cloud API     (PDF → Markdown)

Everything that needs Python lives here. A client only sends tool calls.

Tools

convert_document is the one MinerU implementation for both the paper pipeline and general document reading.

Configuration

Settings come from the process environment, then <MCP_HOME>/.env (this directory), then ~/.shellrc. Copy .env.example to .env and fill it in. Nothing else is read — in particular, no client's directory is consulted.

The essentials:

Full list with defaults and tuning knobs: .env.example. ELSEVIER_INSTTOKEN is an optional subscription-institution token — leave it empty unless your library requires it.

Restart the service after editing (configuration is read once at startup):

systemctl --user restart academic-mcp      # or however you run it

Install

python -m venv .venv && . .venv/bin/activate
pip install -e .                 # or: pip install academic-mcp
cp .env.example .env             # then edit
academic-mcp                     # or: python -m academic_mcp

The browser fallback needs a one-time download and a display (Xvfb is fine):

camoufox fetch

academic-mcp.service and academic-mcp-xvfb.service in this directory are examples — they assume Linux with systemd --user. Adjust the paths before using them.

PDF → Markdown: MinerU only

There is deliberately no fallback converter. The previous backend tried local MinerU (GPU) and then PyMuPDF4LLM; both produced silently degraded text (lost formulas, broken tables, scrambled reading order) that a reading agent then summarised as authoritative. Failing loudly is better than fabricating structure.

Resolver order

Cheap and reliable first, expensive and fragile last:

1. arxiv-direct — the DOI is an arXiv ID (10.48550/...)

2. elsevier-api — 10.1016/ + ScienceDirect API key

3. direct-pdf — open-access publishers whose PDF needs no session cookies. Only entries verified to return application/pdf belong here. SciPost is deliberately excluded: it sits behind a proof-of-work gate, and dodging that with a non-browser User-Agent would evade a control someone installed on purpose — it goes through the browser instead.

4. camoufox-browser — headful Firefox, CloudFlare bypass, paywalls

5. arxiv-title-search — find the preprint by title (DDGS + verification)

The last one is a fallback for content, not access: when every publisher path is paywalled, an arXiv preprint of the same paper is still worth reading. It runs last so the version-of-record wins when reachable. Candidates are verified (title similarity + author) before use — fetching the wrong paper is worse than failing.

Display for the browser

Headful Firefox is required for CloudFlare Turnstile (headless is detected and challenged harder). The resolver picks, in order: an existing Xvfb on :99–:96, the ambient DISPLAY, a self-started Xvfb, otherwise headless.

Layout

academic-mcp/
├── academic-mcp.py          # script entrypoint (kept for compatibility)
├── academic_mcp/
│   ├── __main__.py          # python -m academic_mcp
│   ├── config.py            # env + .env settings
│   ├── server.py            # MCP tools (incl. contract / health)
│   ├── pipeline.py          # cache → resolve → convert
│   ├── mineru.py            # MinerU cloud API (the only converter)
│   ├── storage.py           # DOI→key, cache validity
│   ├── httpclient.py        # pooled client, retries, proxy policy
│   ├── validate.py          # session-scoped DOI authorization
│   ├── markdown.py          # outline + page→line map
│   ├── resolvers/           # arxiv, elsevier, publisher, camoufox
│   └── agent/               # search, snowball, memory + MCP wrappers
├── scripts/                 # manual publisher / network diagnostics
└── tests/                   # offline smoke tests

Development

pip install -e ".[dev]"
ruff check .
pytest -q          # offline: no network, no browser, no MinerU quota

The tests pin the parts clients depend on as a contract: the tool surface, the argument names, the cwd → session_id rule and the memory round-trip.

Security notes

• The service binds to 127.0.0.1 by default and keeps its keys in .env (never returned over the wire; health masks them).

• validate_doi only authorises a DOI that appeared in this session's search results, so a stray DOI cannot trigger a download.

• The browser fallback drives a real Firefox for CloudFlare challenges and paywalled publishers. Use it in line with those sites' terms.

License

GPL-3.0-or-later — see LICENSE.

Copyright (C) 2026 cxxiao.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. It is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose.