research-digest

A personal library of arXiv papers that your AI assistant can read.

It fetches papers in the categories you care about, ranks them against your topics, and exposes the result to any MCP client. Claude Code can then answer "what have I read about evaluation harnesses" from your own library instead of from the open web.

Nothing leaves your machine. There is no account, no API key, and no server.

Install

git clone https://github.com/dneish2/research-digest-mcp
cd research-digest-mcp
pip install -e .

That gives you search, saved papers, trends and the web interface with no third-party dependencies at all.

Similarity search ("find papers like this one") needs numpy and scikit-learn:

pip install -e ".[embeddings]"

Related MCP server: mcp-arxiv

Build your library

research-digest fetch      # pull recent papers from arXiv
research-digest embed      # optional: build vectors for similarity search
research-digest status     # see what you have

fetch is safe to run daily. It only adds papers you have not seen. Run it a few times over a week and the trends view starts to mean something.

Use it

research-digest search agentic evaluation
research-digest web                       # browser interface on localhost

Connect it to Claude Code

claude mcp add research-digest -- research-digest mcp

Then ask Claude things like "search my library for retrieval papers" or "what have I saved about multi-agent systems".

For a client that wants JSON configuration instead:

{
  "mcpServers": {
    "research-digest": {
      "command": "research-digest",
      "args": ["mcp"]
    }
  }
}

Check it responds:

echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | research-digest mcp

The tools

How ranking works

No model decides the order. The score is arithmetic you can check:

The 0.20 for common words is the part that matters. A word like "learning" appears in nearly every paper in this field, so matching it tells you almost nothing. Without that discount the ranking fills up with noise.

Every result carries its own derivation, in the MCP response and in the web interface. The "How scoring works" tab lets you edit a title, an abstract and your topic list, and watch the arithmetic change.

Configuration

Settings live in settings.json inside your data directory:

research-digest config                          # show current settings
research-digest config --add-category cs.CV     # track another arXiv category
research-digest config --add-topic "world model"

Data lives in ~/.research-digest by default. Point RESEARCH_DIGEST_HOME somewhere else if you prefer:

export RESEARCH_DIGEST_HOME=~/notes/papers          # macOS, Linux
$env:RESEARCH_DIGEST_HOME = "$env:APPDATA\research-digest"   # Windows

The directory holds archive.json, saved.json, settings.json and, if you built them, embeddings.db.

A note on the embeddings

The default encoder is TF-IDF followed by SVD. It is fitted to your corpus, which means the 384 columns it produces are derived from the particular set of papers it saw. Vectors from two different fits are not comparable, even when both are 384 wide.

So research-digest embed always rebuilds the whole store in one pass, and the store records which fit produced every row. If it ever finds more than one, it refuses to compare them and tells you to re-embed rather than returning confident nonsense.

If you would rather have a fixed basis that survives incremental updates, install sentence-transformers and run research-digest embed --engine minilm. That downloads about 90 MB the first time.

Development

pip install -e ".[dev]"
python -m unittest discover -s tests -v

The MCP server is plain JSON-RPC over stdin and stdout, about 300 lines, with no SDK. One JSON object per line each way, which makes it easy to drive from a shell script when something looks wrong.

License

MIT