academic-mcp

A local MCP service that is the whole backend of an academic research workflow: literature search, citation chains, per-project research memory, paper download and PDF→Markdown conversion. Any MCP client can drive it — the first one is the DSH academic agent preset, which contains no Python at all.

client (any MCP host)  ──JSON-RPC /mcp──▶  academic-mcp :8790
                                             ├─ Scopus + OpenAlex    (search, citation chains)
                                             ├─ project memory JSON  (notes, findings, progress)
                                             ├─ arXiv / Elsevier / browser → PDF
                                             └─ MinerU cloud API     (PDF → Markdown)

Everything that needs Python lives here. A client only sends tool calls.

Tools

convert_document is the one MinerU implementation for both the paper pipeline and general document reading.

Related MCP server: openresearch-mcp

Configuration

Settings come from the process environment, then <MCP_HOME>/.env (this directory), then ~/.shellrc. Copy .env.example to .env and fill it in. Nothing else is read — in particular, no client's directory is consulted.

The essentials:

Full list with defaults and tuning knobs: .env.example. There are two proxy settings and one Elsevier credential: DOWNLOAD_PROXY (publisher/API access, and the rescue route in failover mode) and GFW_PROXY (arXiv/search engines/OpenAlex). ELSEVIER_API_KEY alone is enough for metadata; an institutional insttoken is not supported.

Restart the service after editing (configuration is read once at startup):

systemctl --user restart academic-mcp      # or however you run it

Install

python -m venv .venv && . .venv/bin/activate
pip install -e .                 # or: pip install academic-mcp
cp .env.example .env             # then edit
academic-mcp                     # or: python -m academic_mcp

The browser fallback needs a one-time download and a display (Xvfb is fine):

camoufox fetch

academic-mcp.service and academic-mcp-xvfb.service in this directory are examples — they assume Linux with systemd --user. Adjust the paths before using them.

PDF → Markdown: MinerU only

There is deliberately no fallback converter. The previous backend tried local MinerU (GPU) and then PyMuPDF4LLM; both produced silently degraded text (lost formulas, broken tables, scrambled reading order) that a reading agent then summarised as authoritative. Failing loudly is better than fabricating structure.

Resolver order

Cheap and reliable first, expensive and fragile last:

1. arxiv-direct — the DOI is an arXiv ID (10.48550/...)

2. elsevier-api — 10.1016/ + ScienceDirect API key

3. direct-pdf — open-access publishers whose PDF needs no session cookies. Only entries verified to return application/pdf belong here. SciPost is deliberately excluded: it sits behind a proof-of-work gate, and dodging that with a non-browser User-Agent would evade a control someone installed on purpose — it goes through the browser instead.

4. camoufox-browser — headful Firefox, CloudFlare bypass, paywalls. Walks the same routes as the HTTP client (DOWNLOAD_PROXY_MODE), one browser session per exit, because entitlements are per-IP: an exit that is a subscriber for one publisher is a stranger for the next, and the second route is what rescues the first.

5. arxiv-title-search — find the preprint by title (arXiv API + verification, with DDGS as a rescue for titles the API's phrase match cannot reach)

The last one is a fallback for content, not access: when every publisher path is paywalled, an arXiv preprint of the same paper is still worth reading. It runs last so the version-of-record wins when reachable. Candidates are verified (title similarity + author) before use — fetching the wrong paper is worse than failing. It asks arXiv's own API first and keeps web search as the rescue, because a resolver whose job is to work when the publisher refused us cannot depend on search engines that may be refusing us too.

Display for the browser

Headful Firefox is required for CloudFlare Turnstile (headless is detected and challenged harder). The resolver picks, in order: an existing Xvfb on :99–:96, the ambient DISPLAY, a self-started Xvfb, otherwise headless.

Layout

academic-mcp/
├── academic-mcp.py          # script entrypoint (kept for compatibility)
├── academic_mcp/
│   ├── __main__.py          # python -m academic_mcp
│   ├── config.py            # env + .env settings
│   ├── server.py            # MCP tools (incl. contract / health)
│   ├── pipeline.py          # cache → resolve → convert
│   ├── mineru.py            # MinerU cloud API (the only converter)
│   ├── storage.py           # DOI→key, cache validity
│   ├── httpclient.py        # pooled client, retries, proxy policy
│   ├── validate.py          # session-scoped DOI authorization
│   ├── markdown.py          # outline + page→line map
│   ├── resolvers/           # arxiv, elsevier, publisher, camoufox
│   └── agent/               # search, snowball, memory + MCP wrappers
├── scripts/                 # manual publisher / network diagnostics
└── tests/                   # offline smoke tests

Development

pip install -e ".[dev]"
ruff check .
pytest -q          # offline: no network, no browser, no MinerU quota

The tests pin the parts clients depend on as a contract: the tool surface, the argument names, the cwd → session_id rule and the memory round-trip.

Security notes

• The service binds to 127.0.0.1 by default and keeps its keys in .env (never returned over the wire; health masks them).

• validate_doi only authorises a DOI that appeared in this session's search results, so a stray DOI cannot trigger a download.

• The browser fallback drives a real Firefox for CloudFlare challenges and paywalled publishers. Use it in line with those sites' terms.

License

GPL-3.0-or-later — see LICENSE.

Copyright (C) 2026 cxxiao.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. It is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose.