# arXiv Research-Concept Companion

An AI/ML study-companion agent (KSE Agentic Lab assignment). It reads a
concept-summary note from your Obsidian vault, finds related arXiv papers,
scores each on topical relevance and age-adjusted citation impact, finds the
well-established papers a surviving candidate builds on, and writes the
findings back into the vault.

- **Existing MCP server (Part A):** Obsidian Local REST API MCP.
- **Custom MCP server (Part B):** `custom_server/` — FastMCP app, 3 tools over
  the public arXiv and OpenAlex APIs (no auth).
- **Agent:** `agent/` — a PydanticAI `Agent` (OpenRouter-backed) holding both
  MCP connections as toolsets, orchestrated by a LangGraph state machine.

## Prerequisites

- Python 3.12+, [`uv`](https://docs.astral.sh/uv/).
- An OpenRouter API key.
- Obsidian with the [Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api)
  community plugin installed and running, and an MCP server that speaks to
  it (any Obsidian Local REST API MCP implementation — the launch command is
  configurable, see below).

## Install

```bash
uv sync
cp .env.example .env
```

Fill in `.env`:

| Variable | Meaning |
|---|---|
| `OPENROUTER_API_KEY` | OpenRouter key — used by the agent and by `score_paper_relevance`. |
| `OPENROUTER_MODEL` | Model slug, e.g. `openai/gpt-4o-mini`. |
| `OBSIDIAN_API_KEY` / `OBSIDIAN_BASE_URL` | Local REST API plugin credentials. |
| `OBSIDIAN_MCP_COMMAND` | Space-separated argv to launch your Obsidian MCP server, e.g. `npx -y <obsidian-mcp-package>`. |
| `RELEVANCE_PASS_THRESHOLD` | Minimum relevance score (0–1) to survive the filter. Default `0.5`. |
| `CITATIONS_PER_YEAR_THRESHOLD` | Minimum citations/year to pass the impact check. Default `5`. |
| `NEW_PAPER_AGE_EXEMPT_YEARS` | Papers younger than this are exempt from the impact check. Default `1`. |

## Running

Two independent processes, sharing one `uv` project:

```bash
# process 1 — the custom MCP server (arXiv + OpenAlex)
uv run python -m custom_server.server

# process 2 — the agent (connects to both MCP servers), driven by a free-text prompt
uv run python -m agent.graph "Find papers related to my 'Transformers Concept Note'"
```

`agent/graph.py` spawns `custom_server/server.py` itself as a stdio
subprocess, so process 2 does not need process 1 already running — the two
commands above just demonstrate that each is independently startable.

The prompt is not a literal note title — the agent's first step
(`parse_prompt`) uses an LLM call to identify which Obsidian note the prompt
refers to. If it can't identify one, the run halts immediately and prints
"Not enough information: no Obsidian note or page was named in the prompt."
without touching Obsidian. If the note it finds doesn't yield enough concept
keywords (fewer than `min_keywords`, default 2), the run halts after reading
it and prints a similar "not enough information" message instead of
searching arXiv.

## Offline / replay mode

The custom server calls three live network APIs (arXiv, OpenAlex, OpenRouter).
Setting `CUSTOM_SERVER_OFFLINE=1` serves its tools from recorded fixtures in
`custom_server/fixtures/` instead — no network access or `OPENROUTER_API_KEY`
required. Useful for a demo/defence without reliable network, or for fast
iteration.

```bash
CUSTOM_SERVER_OFFLINE=1 uv run python -m custom_server.server
```

What's covered: `search_arxiv_papers` (one recorded search feed, served for
any query — see limitation below), and `score_paper_relevance` /
`find_foundational_citations` for two recorded papers, GPT-3
(`2005.14165`) and ResNet (`1512.03385`).

Known limitations:
- `search_arxiv_papers` is query-agnostic in offline mode — it always
  returns the same recorded feed regardless of the query text.
- `score_paper_relevance` and `find_foundational_citations` only recognize
  the two recorded papers above. An unrecorded arxiv_id raises
  `PaperNotFoundError` (the same error a real OpenAlex miss would produce);
  an unrecorded paper title passed to `score_paper_relevance` raises
  `FixtureNotFoundError` — distinguishable, not a silent wrong answer.

To regenerate or extend the fixtures: `uv run python -m
custom_server.fixtures.record` re-fetches the recorded arXiv/OpenAlex
responses (both public, unauthenticated APIs) and overwrites the JSON/XML
files in `custom_server/fixtures/`. To add a new paper, add its two `httpx.get`
calls to `record.py` and a matching entry to `relevance_scores.json`
(hand-authored — not real OpenRouter output, since recording its raw
chat-completion response isn't worth the wire-format fragility; the
structured `{relevance, novelty, rationale}` fields are replayed directly
through a PydanticAI `FunctionModel`).

## Tests

```bash
uv run pytest custom_server/tests agent/tests
```

All network calls (arXiv, OpenAlex, OpenRouter) are mocked; no live traffic
during tests.

## Tool contracts (Part C)

### `search_arxiv_papers` (custom)

| | |
|---|---|
| **Purpose** | Primary data-source tool: search arXiv for candidate papers on a topic. |
| **Model-facing description** | "Search arXiv for papers on a topic, optionally restricted to categories and a minimum submission date. Use this to find candidate papers before evaluating them individually with score_paper_relevance. A valid query that matches nothing returns an empty list — that is a normal result, not an error." |
| **Input** | `query: str`, `categories: list[str] = [cs.LG, cs.AI, cs.CL, stat.ML]`, `since_date: str \| None` (`YYYY-MM-DD`), `max_results: int = 10` (1–50) |
| **Output** | `list[{arxiv_id, title, abstract, authors: list[str], published_date, categories: list[str]}]` |
| **Error conditions** | `ValueError` on an invalid category code, a malformed `since_date`, or `max_results` out of `[1, 50]` — raised before any network call. Upstream HTTP failure raises via `raise_for_status()`. Zero matches is a valid empty list, not an error. |
| **Side effects** | None — read-only HTTP GET to `export.arxiv.org`. |
| **Example** | `search_arxiv_papers(query="transformer attention", max_results=5)` → 5 candidate papers with abstracts. |

### `score_paper_relevance` (custom)

| | |
|---|---|
| **Purpose** | Evaluative tool: judge one candidate's topical fit and whether its citation record clears an age-adjusted bar. |
| **Model-facing description** | "Score how relevant and novel a paper is to a concept summary, and check whether its citation impact clears a minimum bar (citations per year, exempting papers younger than one year). Use this on each candidate from search_arxiv_papers to decide whether it belongs in a reading list. Raises if the paper has no OpenAlex record, or if the underlying relevance-scoring model call fails." |
| **Input** | `concept_summary: str`, `paper: {arxiv_id, title, abstract}` |
| **Output** | `{relevance: float, novelty: float, citation_count: int, publication_year: int, citations_per_year: float, impact_pass: bool, rationale: str}` |
| **Error conditions** | `PaperNotFoundError` (from `custom_server.openalex`) if OpenAlex has no record for the paper's arXiv DOI — distinct from a found-but-uncited paper, which is a valid `citation_count: 0`. `UnexpectedModelBehavior` if the OpenRouter call's structured output fails schema validation after retries. |
| **Side effects** | Read-only: one OpenAlex GET, one OpenRouter chat-completion call. |
| **Example** | `score_paper_relevance(concept_summary="attention mechanisms in NLP", paper={...})` → `{relevance: 0.92, novelty: 0.6, citation_count: 84331, impact_pass: True, ...}` |

### `find_foundational_citations` (custom)

| | |
|---|---|
| **Purpose** | Citation-graph analysis: given one paper, rank its own references by citation count to surface the well-established work it builds on. Distinct from `search_arxiv_papers` — it analyzes a specific paper's reference list, not a keyword search. |
| **Model-facing description** | "Given one paper's arXiv ID, return its most-cited references — the well-established prior work it builds on. Use this after selecting a paper to read, to surface the background literature behind it. A paper with no recorded references returns an empty list — that is a normal result, not an error." |
| **Input** | `arxiv_id: str`, `max_results: int = 3` (1–3) |
| **Output** | `list[{openalex_id, title, cited_by_count, publication_year}]`, sorted by `cited_by_count` descending, top `max_results` |
| **Error conditions** | `ValueError` if `max_results` outside `[1, 3]`. `PaperNotFoundError` if OpenAlex has no record for the arXiv ID. A paper with zero references returns `[]` — valid, not an error. |
| **Side effects** | Read-only: one OpenAlex paper lookup + one or more batched OpenAlex works lookups (chunked at 50 IDs per request). |
| **Example** | `find_foundational_citations(arxiv_id="2005.14165", max_results=3)` → the 3 most-cited papers GPT-3 references. |

### Obsidian Local REST API MCP (existing, Part A)

Used via the PydanticAI agent's natural-language tool calls (not a fixed
wrapper function) for two operations in the flow:

| | |
|---|---|
| **Reference resolution** | Before any Obsidian call, `parse_prompt` asks the PydanticAI agent (plain LLM reasoning, not an MCP call) to identify the note title implied by the user's free-text prompt. If none is identifiable, the flow halts with an "insufficient information" status and never calls Obsidian. |
| **Read** | The agent is prompted to read the note titled `note_title` (from `parse_prompt`) and return its plain-text content — feeds `concept_text`, the input to keyword extraction and relevance scoring. |
| **Write** | The agent is prompted to create/overwrite a note titled `"{note_title} — Related Papers"` with the markdown produced by `compose_note_content` — the observable effect that closes the loop between both MCP servers. |
| **Error conditions** | Stopped plugin, invalid API key, or a missing note surface as a distinguishable tool-call failure from the MCP server, not a silent empty result. |

## Design rationale

- **Why Obsidian:** the assignment needs an existing MCP server the agent
  both reads from and writes to. A student's own concept notes are a natural
  "what do I already know" input, and writing survivors back closes the loop
  visibly in the vault.
- **Why arXiv + OpenAlex instead of a login-walled site:** the originally
  considered KSE schedule/Moodle sources both require personal login, which
  the assignment's public-API rule rules out. arXiv and OpenAlex are public,
  unauthenticated, and directly support the "relevance + impact" domain.
- **Why relevance is LLM-judged, not embeddings:** OpenRouter has no
  embeddings endpoint (verified against its live model catalog), so
  `score_paper_relevance` uses a PydanticAI structured-output call instead
  of vector similarity — reusing the one model credential the project
  already needs.
- **Why `find_foundational_citations` isn't "search again with OpenAlex":**
  it takes one specific paper's reference list and ranks it by citation
  impact, the same kind of controlled indicator-comparison the assignment's
  own examples use — distinct responsibility and processing from the
  keyword-driven `search_arxiv_papers`.
- **Filtering is plain Python, not a 4th tool:** the relevance-threshold +
  `impact_pass` filter in `agent/graph.py`'s `filter_candidates_node` is
  deterministic post-processing over already-scored data, not new domain
  logic — a tool would just be indirection around an `if`.
- **Trade-offs / limitations:** the custom server's offline/replay mode (see
  "Offline / replay mode" above) covers two recorded papers and a
  query-agnostic arXiv search — not a general record/replay of arbitrary
  queries. `agent/`'s own Obsidian and OpenRouter calls are unaffected by it
  and still require live access. Impact/relevance thresholds are `.env`
  values, not runtime-tunable per request.

## Deferred (flagged, not dropped)

- Exposing the hardcoded thresholds as richer runtime config beyond `.env`.

## Demo / defence checklist

- [ ] `uv run python -m custom_server.server` starts standalone; a raw MCP
      client's `list_tools` shows all 3 tools.
- [ ] `uv run pytest custom_server/tests agent/tests` — all green, network
      mocked.
- [ ] `CUSTOM_SERVER_OFFLINE=1 uv run python -m custom_server.server` starts
      and serves all 3 tool calls with no live network or API keys required
      (see "Offline / replay mode").
- [ ] Seed a demo vault note with a concept summary (e.g. "attention
      mechanisms"), titled e.g. "Transformers Concept Note".
- [ ] `uv run python -m agent.graph "Find papers related to my 'Transformers
      Concept Note'"` — full live run: resolves the note reference, reads
      the note, searches arXiv, scores candidates, filters, finds
      foundational citations, writes `"<note> — Related Papers"` back to
      the vault.
- [ ] Show both MCP connections feeding the final output: the write-back
      note cites both arXiv/OpenAlex data (custom server) and the original
      concept note content (Obsidian).
- [ ] Insufficient-information demo: run with a prompt that names no note
      (e.g. `"What's a transformer?"`) — show the agent halts and prints
      "Not enough information..." without calling Obsidian. Then run against
      a note with near-empty content — show it halts after reading the note,
      before calling arXiv.
- [ ] Failure demo, Obsidian: stop the Local REST API plugin (or use a bad
      `OBSIDIAN_API_KEY` / a nonexistent note title) — show the agent
      surfaces a distinguishable error, not a silent empty result.
- [ ] Failure demo, custom server: call `search_arxiv_papers` with an
      invalid category, or `find_foundational_citations` with an arXiv ID
      absent from OpenAlex — show `ValueError` / `PaperNotFoundError`
      respectively, distinct from a valid empty result.