arXiv Research-Concept Companion

An AI/ML study-companion agent (KSE Agentic Lab assignment). It reads a concept-summary note from your Obsidian vault, finds related arXiv papers, scores each on topical relevance and age-adjusted citation impact, finds the well-established papers a surviving candidate builds on, and writes the findings back into the vault.

• Existing MCP server (Part A): Obsidian Local REST API MCP.

• Custom MCP server (Part B): custom_server/ — FastMCP app, 3 tools over the public arXiv and OpenAlex APIs (no auth).

• Agent: agent/ — a PydanticAI Agent (OpenRouter-backed) holding both MCP connections as toolsets, orchestrated by a LangGraph state machine.

Prerequisites

• Python 3.12+, uv.

• An OpenRouter API key.

• Obsidian with the Local REST API community plugin installed and running, and an MCP server that speaks to it (any Obsidian Local REST API MCP implementation — the launch command is configurable, see below).

Related MCP server: arxiv-mcp

Install

uv sync
cp .env.example .env

Fill in .env:

Running

Two independent processes, sharing one uv project:

# process 1 — the custom MCP server (arXiv + OpenAlex)
uv run python -m custom_server.server

# process 2 — the agent (connects to both MCP servers), driven by a free-text prompt
uv run python -m agent.graph "Find papers related to my 'Transformers Concept Note'"

agent/graph.py spawns custom_server/server.py itself as a stdio subprocess, so process 2 does not need process 1 already running — the two commands above just demonstrate that each is independently startable.

The prompt is not a literal note title — the agent's first step (parse_prompt) uses an LLM call to identify which Obsidian note the prompt refers to. If it can't identify one, the run halts immediately and prints "Not enough information: no Obsidian note or page was named in the prompt." without touching Obsidian. If the note it finds doesn't yield enough concept keywords (fewer than min_keywords, default 2), the run halts after reading it and prints a similar "not enough information" message instead of searching arXiv.

Offline / replay mode

The custom server calls three live network APIs (arXiv, OpenAlex, OpenRouter). Setting CUSTOM_SERVER_OFFLINE=1 serves its tools from recorded fixtures in custom_server/fixtures/ instead — no network access or OPENROUTER_API_KEY required. Useful for a demo/defence without reliable network, or for fast iteration.

CUSTOM_SERVER_OFFLINE=1 uv run python -m custom_server.server

What's covered: search_arxiv_papers (one recorded search feed, served for any query — see limitation below), and score_paper_relevance / find_foundational_citations for two recorded papers, GPT-3 (2005.14165) and ResNet (1512.03385).

Known limitations:

• search_arxiv_papers is query-agnostic in offline mode — it always returns the same recorded feed regardless of the query text.

• score_paper_relevance and find_foundational_citations only recognize the two recorded papers above. An unrecorded arxiv_id raises PaperNotFoundError (the same error a real OpenAlex miss would produce); an unrecorded paper title passed to score_paper_relevance raises FixtureNotFoundError — distinguishable, not a silent wrong answer.

To regenerate or extend the fixtures: uv run python -m custom_server.fixtures.record re-fetches the recorded arXiv/OpenAlex responses (both public, unauthenticated APIs) and overwrites the JSON/XML files in custom_server/fixtures/. To add a new paper, add its two httpx.get calls to record.py and a matching entry to relevance_scores.json (hand-authored — not real OpenRouter output, since recording its raw chat-completion response isn't worth the wire-format fragility; the structured {relevance, novelty, rationale} fields are replayed directly through a PydanticAI FunctionModel).

Tests

uv run pytest custom_server/tests agent/tests

All network calls (arXiv, OpenAlex, OpenRouter) are mocked; no live traffic during tests.

Tool contracts (Part C)

search_arxiv_papers (custom)

score_paper_relevance (custom)

find_foundational_citations (custom)

Obsidian Local REST API MCP (existing, Part A)

Used via the PydanticAI agent's natural-language tool calls (not a fixed wrapper function) for two operations in the flow:

Design rationale

• Why Obsidian: the assignment needs an existing MCP server the agent both reads from and writes to. A student's own concept notes are a natural "what do I already know" input, and writing survivors back closes the loop visibly in the vault.

• Why arXiv + OpenAlex instead of a login-walled site: the originally considered KSE schedule/Moodle sources both require personal login, which the assignment's public-API rule rules out. arXiv and OpenAlex are public, unauthenticated, and directly support the "relevance + impact" domain.

• Why relevance is LLM-judged, not embeddings: OpenRouter has no embeddings endpoint (verified against its live model catalog), so score_paper_relevance uses a PydanticAI structured-output call instead of vector similarity — reusing the one model credential the project already needs.

• Why find_foundational_citations isn't "search again with OpenAlex": it takes one specific paper's reference list and ranks it by citation impact, the same kind of controlled indicator-comparison the assignment's own examples use — distinct responsibility and processing from the keyword-driven search_arxiv_papers.

• Filtering is plain Python, not a 4th tool: the relevance-threshold + impact_pass filter in agent/graph.py's filter_candidates_node is deterministic post-processing over already-scored data, not new domain logic — a tool would just be indirection around an if.

• Trade-offs / limitations: the custom server's offline/replay mode (see "Offline / replay mode" above) covers two recorded papers and a query-agnostic arXiv search — not a general record/replay of arbitrary queries. agent/'s own Obsidian and OpenRouter calls are unaffected by it and still require live access. Impact/relevance thresholds are .env values, not runtime-tunable per request.

Deferred (flagged, not dropped)

• Exposing the hardcoded thresholds as richer runtime config beyond .env.

Demo / defence checklist

• uv run python -m custom_server.server starts standalone; a raw MCP client's list_tools shows all 3 tools.

• uv run pytest custom_server/tests agent/tests — all green, network mocked.

• CUSTOM_SERVER_OFFLINE=1 uv run python -m custom_server.server starts and serves all 3 tool calls with no live network or API keys required (see "Offline / replay mode").

• Seed a demo vault note with a concept summary (e.g. "attention mechanisms"), titled e.g. "Transformers Concept Note".

• uv run python -m agent.graph "Find papers related to my 'Transformers Concept Note'" — full live run: resolves the note reference, reads the note, searches arXiv, scores candidates, filters, finds foundational citations, writes "<note> — Related Papers" back to the vault.

• Show both MCP connections feeding the final output: the write-back note cites both arXiv/OpenAlex data (custom server) and the original concept note content (Obsidian).

• Insufficient-information demo: run with a prompt that names no note (e.g. "What's a transformer?") — show the agent halts and prints "Not enough information..." without calling Obsidian. Then run against a note with near-empty content — show it halts after reading the note, before calling arXiv.

• Failure demo, Obsidian: stop the Local REST API plugin (or use a bad OBSIDIAN_API_KEY / a nonexistent note title) — show the agent surfaces a distinguishable error, not a silent empty result.

• Failure demo, custom server: call search_arxiv_papers with an invalid category, or find_foundational_citations with an arXiv ID absent from OpenAlex — show ValueError / PaperNotFoundError respectively, distinct from a valid empty result.