Skip to main content
Glama
tbaraniuk

arxiv-agent-mcp

by tbaraniuk

arXiv Research-Concept Companion

一个 AI/ML 学习伴侣代理(KSE Agentic Lab 作业)。它从你的 Obsidian 库中读取概念摘要笔记,查找相关的 arXiv 论文,根据主题相关性和按年龄调整的引用影响力对每篇论文进行评分,找出幸存候选者所依赖的成熟论文,并将结果写回库中。

  • 现有 MCP 服务器(A 部分): Obsidian Local REST API MCP。

  • 自定义 MCP 服务器(B 部分): custom_server/ — FastMCP 应用,通过公共 arXiv 和 OpenAlex API 提供 3 个工具(无需认证)。

  • 代理: agent/ — 一个 PydanticAI Agent(基于 OpenRouter),将两个 MCP 连接作为工具集持有,由 LangGraph 状态机编排。

先决条件

  • Python 3.12+,uv。

  • 一个 OpenRouter API 密钥。

  • 安装了 Local REST API 社区插件并正在运行的 Obsidian,以及一个与之通信的 MCP 服务器(任何 Obsidian Local REST API MCP 实现——启动命令可配置,见下文)。

Related MCP server: arxiv-mcp

安装

uv sync
cp .env.example .env

填写 .env:

变量

含义

OPENROUTER_API_KEY

OpenRouter 密钥——由代理和 score_paper_relevance 使用。

OPENROUTER_MODEL

模型标识,例如 openai/gpt-4o-mini。

OBSIDIAN_API_KEY / OBSIDIAN_BASE_URL

Local REST API 插件凭据。

OBSIDIAN_MCP_COMMAND

用于启动你的 Obsidian MCP 服务器的空格分隔参数,例如 npx -y <obsidian-mcp-package>。

RELEVANCE_PASS_THRESHOLD

通过筛选的最低相关性分数(0–1)。默认 0.5。

CITATIONS_PER_YEAR_THRESHOLD

通过影响力检查的最低每年引用次数。默认 5。

NEW_PAPER_AGE_EXEMPT_YEARS

比这更年轻的论文可免于影响力检查。默认 1。

运行

两个独立进程,共享一个 uv 项目:

# process 1 — the custom MCP server (arXiv + OpenAlex)
uv run python -m custom_server.server

# process 2 — the agent (connects to both MCP servers), driven by a free-text prompt
uv run python -m agent.graph "Find papers related to my 'Transformers Concept Note'"

agent/graph.py 自行将 custom_server/server.py 作为 stdio 子进程启动,因此进程 2 不需要进程 1 已经运行——上面的两个命令只是演示每个进程都可以独立启动。

提示不是字面上的笔记标题——代理的第一步(parse_prompt)使用 LLM 调用来识别提示所指的 Obsidian 笔记。如果无法识别,运行会立即停止并打印“Not enough information: no Obsidian note or page was named in the prompt.”,而不会触碰 Obsidian。如果找到的笔记没有产生足够的概念关键词(少于 min_keywords,默认 2),运行会在读取后停止,并打印类似的“not enough information”消息,而不是搜索 arXiv。

离线 / 重放模式

自定义服务器调用三个实时网络 API(arXiv、OpenAlex、OpenRouter)。设置 CUSTOM_SERVER_OFFLINE=1 后,其工具将从 custom_server/fixtures/ 中的录制夹具提供——无需网络访问或 OPENROUTER_API_KEY。适用于没有可靠网络的演示/答辩,或快速迭代。

CUSTOM_SERVER_OFFLINE=1 uv run python -m custom_server.server

覆盖范围:search_arxiv_papers(一个录制的搜索源,为任何查询提供——见下面的限制),以及 score_paper_relevance / find_foundational_citations 对两篇录制的论文:GPT-3(2005.14165)和 ResNet(1512.03385)。

已知限制:

  • search_arxiv_papers 在离线模式下与查询无关——无论查询文本如何,它总是返回相同的录制源。

  • score_paper_relevance 和 find_foundational_citations 只识别上面两篇录制的论文。未录制的 arxiv_id 会引发 PaperNotFoundError(与真实 OpenAlex 未命中产生的错误相同);传递给 score_paper_relevance 的未录制论文标题会引发 FixtureNotFoundError——可区分,而不是静默的错误答案。

要重新生成或扩展夹具:uv run python -m custom_server.fixtures.record 会重新获取录制的 arXiv/OpenAlex 响应(两者都是公共、无需认证的 API),并覆盖 custom_server/fixtures/ 中的 JSON/XML 文件。要添加新论文,请将其两个 httpx.get 调用添加到 record.py,并在 relevance_scores.json 中添加匹配条目(手工编写——不是真实的 OpenRouter 输出,因为录制其原始聊天完成响应不值得线格式的脆弱性;结构化的 {relevance, novelty, rationale} 字段直接通过 PydanticAI FunctionModel 重放)。

测试

uv run pytest custom_server/tests agent/tests

所有网络调用(arXiv、OpenAlex、OpenRouter)都被模拟;测试期间没有实时流量。

工具契约(C 部分)

search_arxiv_papers(自定义)

目的

主要数据源工具:搜索 arXiv 以查找某个主题的候选论文。

面向模型的描述

“Search arXiv for papers on a topic, optionally restricted to categories and a minimum submission date. Use this to find candidate papers before evaluating them individually with score_paper_relevance. A valid query that matches nothing returns an empty list — that is a normal result, not an error.”

输入

query: str,categories: list[str] = [cs.LG, cs.AI, cs.CL, stat.ML],since_date: str | None(YYYY-MM-DD),max_results: int = 10(1–50)

输出

list[{arxiv_id, title, abstract, authors: list[str], published_date, categories: list[str]}]

错误条件

无效类别代码、格式错误的 since_date 或 max_results 超出 [1, 50] 时引发 ValueError——在任何网络调用之前引发。上游 HTTP 失败通过 raise_for_status() 引发。零匹配是有效的空列表,不是错误。

副作用

无——只读 HTTP GET 到 export.arxiv.org。

示例

search_arxiv_papers(query="transformer attention", max_results=5) → 5 篇带摘要的候选论文。

score_paper_relevance(自定义)

目的

评估工具:判断一个候选者的主题契合度,并检查其引用记录是否通过年龄调整的基准。

面向模型的描述

“Score how relevant and novel a paper is to a concept summary, and check whether its citation impact clears a minimum bar (citations per year, exempting papers younger than one year). Use this on each candidate from search_arxiv_papers to decide whether it belongs in a reading list. Raises if the paper has no OpenAlex record, or if the underlying relevance-scoring model call fails.”

输入

concept_summary: str,paper: {arxiv_id, title, abstract}

输出

{relevance: float, novelty: float, citation_count: int, publication_year: int, citations_per_year: float, impact_pass: bool, rationale: str}

错误条件

如果 OpenAlex 没有该论文的 arXiv DOI 记录,则从 custom_server.openalex 引发 PaperNotFoundError——与找到但未被引用的论文不同,后者是有效的 citation_count: 0。如果 OpenRouter 调用的结构化输出在重试后未通过模式验证,则引发 UnexpectedModelBehavior。

副作用

只读:一次 OpenAlex GET,一次 OpenRouter 聊天完成调用。

示例

score_paper_relevance(concept_summary="attention mechanisms in NLP", paper={...}) → {relevance: 0.92, novelty: 0.6, citation_count: 84331, impact_pass: True, ...}

find_foundational_citations(自定义)

用途

引文图谱分析:给定一篇论文,按其自身引文数量对其参考文献进行排序,以揭示其所依托的成熟工作。与 search_arxiv_papers 不同——它分析的是特定论文的参考文献列表,而非关键词搜索。

面向模型的描述

"给定一篇论文的 arXiv ID,返回其被引次数最多的参考文献——即它所依托的成熟先前工作。在选定要阅读的论文之后使用此工具,以揭示其背后的背景文献。没有记录到参考文献的论文会返回空列表——这是正常结果,而非错误。"

输入

arxiv_id: str,max_results: int = 3(1–3)

输出

list[{openalex_id, title, cited_by_count, publication_year}],按 cited_by_count 降序排序,取前 max_results 条

错误条件

若 max_results 超出 [1, 3] 范围则抛出 ValueError。若 OpenAlex 中没有该 arXiv ID 的记录则抛出 PaperNotFoundError。零参考文献的论文返回 []——有效结果,而非错误。

副作用

只读:一次 OpenAlex 论文查询 + 一次或多次批量 OpenAlex works 查询(每次请求按 50 个 ID 分块)。

示例

find_foundational_citations(arxiv_id="2005.14165", max_results=3) → GPT-3 所引用的被引次数最多的 3 篇论文。

Obsidian Local REST API MCP(现有,Part A)

通过 PydanticAI 代理的自然语言工具调用(而非固定的包装函数)在流程中用于两种操作:

引用解析

在任何 Obsidian 调用之前,parse_prompt 要求 PydanticAI 代理(纯 LLM 推理,而非 MCP 调用)识别用户自由文本提示所隐含的笔记标题。若无法识别,流程将以"信息不足"状态中止,且绝不调用 Obsidian。

读取

提示代理读取标题为 note_title(来自 parse_prompt)的笔记并返回其纯文本内容——为 concept_text 提供输入,即关键词提取和相关性评分的数据来源。

写入

提示代理创建/覆盖标题为 "{note_title} — Related Papers" 的笔记,内容为 compose_note_content 生成的 markdown——这是闭合两个 MCP 服务器之间循环的可观察效果。

错误条件

插件已停止、API 密钥无效或笔记缺失,会以 MCP 服务器可区分的工具调用失败形式呈现,而非静默的空结果。

设计理由

  • 为何选择 Obsidian: 作业需要一个现有的 MCP 服务器,代理既能从中读取也能向其写入。学生自己的概念笔记是自然的"我已知道什么"输入,而将幸存论文写回可在 vault 中直观地闭合循环。

  • 为何选择 arXiv + OpenAlex 而非需要登录的网站: 最初考虑的 KSE 课程表/Moodle 数据源均需个人登录,这被作业的公共 API 规则所排除。arXiv 和 OpenAlex 是公开的、无需认证的,且直接支持"相关性 + 影响力"这一领域。

  • 为何相关性由 LLM 判定而非嵌入: OpenRouter 没有嵌入端点(已对照其实时模型目录验证),因此 score_paper_relevance 使用 PydanticAI 结构化输出调用而非向量相似度——复用项目本已需要的同一个模型凭据。

  • 为何 find_foundational_citations 不是"再用 OpenAlex 搜索一次": 它取一篇特定论文的参考文献列表并按引文影响力排序,与作业自身示例所用的受控指标比较属于同一类型——与关键词驱动的 search_arxiv_papers 具有不同的职责和处理逻辑。

  • 过滤是纯 Python,而非第 4 个工具: agent/graph.py 中 filter_candidates_node 的相关性阈值 + impact_pass 过滤是对已评分数据的确定性后处理,而非新的领域逻辑——工具只是对 if 的间接包装。

  • 权衡 / 局限: 自定义服务器的离线/回放模式(见上文"离线 / 回放模式")覆盖两篇已记录的论文和一次与查询无关的 arXiv 搜索——并非对任意查询的通用记录/回放。agent/ 自身的 Obsidian 和 OpenRouter 调用不受其影响,仍需要实时访问。影响力/相关性阈值是 .env 值,而非每次请求可运行时调整。

已推迟(已标记,未放弃)

  • 将硬编码阈值暴露为超越 .env 的更丰富的运行时配置。

演示 / 答辩检查清单

  • uv run python -m custom_server.server 可独立启动;原始 MCP 客户端的 list_tools 显示全部 3 个工具。

  • uv run pytest custom_server/tests agent/tests — 全部通过,网络已模拟。

  • CUSTOM_SERVER_OFFLINE=1 uv run python -m custom_server.server 可启动并在无实时网络或 API 密钥的情况下提供全部 3 个工具调用(见"离线 / 回放模式")。

  • 在演示 vault 笔记中植入概念摘要(例如"attention mechanisms"),标题例如"Transformers Concept Note"。

  • uv run python -m agent.graph "Find papers related to my 'Transformers Concept Note'" — 完整实时运行:解析笔记引用、读取笔记、搜索 arXiv、对候选论文评分、过滤、查找基础引文、将 "<note> — Related Papers" 写回 vault。

  • 展示两个 MCP 连接如何共同支撑最终输出:写回笔记同时引用 arXiv/OpenAlex 数据(自定义服务器)和原始概念笔记内容(Obsidian)。

  • 信息不足演示:使用未指明任何笔记的提示运行(例如 "What's a transformer?")— 展示代理中止并打印"Not enough information..."而不调用 Obsidian。然后针对内容近乎为空的笔记运行 — 展示其在读取笔记之后、调用 arXiv 之前中止。

  • 失败演示,Obsidian:停止 Local REST API 插件(或使用错误的 OBSIDIAN_API_KEY / 不存在的笔记标题)— 展示代理呈现可区分的错误,而非静默的空结果。

  • 失败演示,自定义服务器:使用无效类别调用 search_arxiv_papers,或使用 OpenAlex 中不存在的 arXiv ID 调用 find_foundational_citations — 分别展示 ValueError / PaperNotFoundError,与有效的空结果相区分。

Available Tools

3 tools
find_foundational_citationsA

Given one paper's arXiv ID, return its most-cited references — the well-established prior work it builds on.

Use this after selecting a paper to read, to surface the background literature behind it. A paper with no recorded references returns an empty list — that is a normal result, not an error.

ParametersJSON Schema
NameRequiredDescriptionDefault
arxiv_idYes
max_resultsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden and does disclose one important behavioral fact: an empty list is normal, not an error. It also implies a read-only operation. Yet it does not explain truncation behavior related to max_results or what the output format contains, though an output schema is present.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences with meaningful information: what the tool returns, when to use it, and an important normalization about empty results. No filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description tells the agent what input is expected, when to invoke it, and how to interpret an empty result. Since an output schema exists, return-value details are covered. The main remaining gap is clarifying max_results, but the schema's default and name largely bridge that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It clarifies what arxiv_id represents (the paper's identifier), but it does not explain max_results, which limits the number of references returned. The name max_results is somewhat self-explanatory, and its default of 3 helps, but the description could have explicitly connected it to the 'most-cited' behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (return references) and resource (most-cited references for a given arXiv ID). It also conveys the purpose (surface well-established prior work), but it does not explicitly distinguish itself from sibling tools such as search_arxiv_papers or score_paper_relevance.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says to use the tool after selecting a paper to read, which gives a clear context. However, it does not mention when not to use it or point to alternative sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

score_paper_relevanceA

Score how relevant and novel a paper is to a concept summary, and check whether its citation impact clears a minimum bar (citations per year, exempting papers younger than one year).

Use this on each candidate from search_arxiv_papers to decide whether it belongs in a reading list. Raises if the paper has no OpenAlex record, or if the underlying relevance-scoring model call fails.

ParametersJSON Schema
NameRequiredDescriptionDefault
paperYes
concept_summaryYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the behavioral burden. It discloses two error conditions (missing OpenAlex record and underlying model failure) and explains the citation-impact filtering rule, including the one-year exemption. This goes well beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every sentence carries necessary content: the scoring criterion, the usage context, and the failure behavior. There is no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter tool with an output schema and sibling context, the description covers its need: why to call it, what to pass, how it decides, and what to expect when it errors.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Since schema description coverage is 0%, the description has to convey parameter meaning. It explains concept_summary as the reference for scoring and clarifies that paper is a candidate from search_arxiv_papers. It does not detail paper field internals, but those are inherited from that tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource: score a paper's relevance and novelty against a concept summary, and check its citation impact. This clearly distinguishes it from search_arxiv_papers (retrieval) and find_foundational_citations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states the intended use: apply this tool to each candidate from search_arxiv_papers to decide reading-list membership. It does not explicitly name alternatives or exclusions, but the context is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_arxiv_papersA

Search arXiv for papers on a topic, optionally restricted to categories and a minimum submission date.

Use this to find candidate papers before evaluating them individually with score_paper_relevance. A valid query that matches nothing returns an empty list — that is a normal result, not an error.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYes
categoriesNo
since_dateNo
max_resultsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the behavioral burden. It usefully discloses that an empty result is a normal outcome, not an error. However, it adds little about other behaviors such as result ordering, matching semantics, rate limits, or external API behavior, beyond what is essentially implied by a 'Search' operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, each earning their place: the first states the operation and optional filters, the second gives pipeline guidance, and the third normalizes an empty result. It is front-loaded and free of filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that an output schema exists, return values are covered. The missing context is input format guidance for categories and since_date and the behavior of max_results. The description is adequate for a basic search call but not fully self-sufficient for an agent invoking an external API it has no prior knowledge about.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, so the description must compensate. It maps 'query' to topic, 'categories' to category restrictions, and 'since_date' to a minimum submission date. However, it does not explain expected formats for categories or since_date, and 'max_results' is never addressed, even though its name and default make its role partially inferable.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Search arXiv for papers on a topic.' It also names the optional scope (categories, minimum submission date) and differentiates from the sibling by framing this as the discovery step before score_paper_relevance.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs when to use it: 'Use this to find candidate papers before evaluating them individually with score_paper_relevance.' This gives a clear pipeline context. It does not explicitly mention when not to use it or how it relates to the other sibling, find_foundational_citations, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 3 tool updatesv0.1.0
    • First observedfind_foundational_citations
    • First observedscore_paper_relevance
    • First observedsearch_arxiv_papers

TDQS

A4.2/5.0

Scored across 3 tools

Disambiguation5/5

The three tools have clearly distinct roles: searching arXiv, scoring a candidate paper, and tracing foundational references. There is no meaningful overlap between discovery, evaluation, and citation expansion.

Naming Consistency5/5

All tool names follow a clean verb_noun snake_case pattern: search_arxiv_papers, score_paper_relevance, find_foundational_citations. The naming style is predictable and signals the action clearly.

Tool Count5/5

Three tools is a well-scoped size for a focused arXiv triage workflow. Each tool covers a necessary step without redundant or filler tools.

Completeness4/5

The set covers the core pipeline: find candidate papers, evaluate relevance and citation impact, then discover foundational references. Minor omissions like retrieving full paper metadata or listing available arXiv categories could be useful but do not create dead ends.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    This MCP server enables users to search for scientific papers on arXiv and retrieve detailed metadata for specific papers. It provides tools to perform search queries and fetch in-depth information using paper IDs.
    3
    Apache 2.0
  • F
    license
    A
    quality
    D
    maintenance
    A streamlined MCP server that connects AI assistants to arXiv's vast collection of academic papers, enabling search, retrieval, and analysis of research papers.
    7
    1
    -
  • F
    license
    Not graded
    quality
    D
    maintenance
    An advanced scholarly research MCP server that enables AI assistants to discover, fetch, process, and manage academic papers across multiple sources like arXiv, PubMed, and Semantic Scholar, with capabilities for summarization, citation analysis, and concept relationship extraction.
    2
    -