학술 논문을 위한 구조화된 방법 추출 및 재현성 휴리스틱을 제공하는 가볍고 온디맨드 방식의 MCP 서버입니다. Worldwide AI Science Fellowship 빌드 챌린지를 위해 제작되었습니다.
⚠️ 상태: 알파(0.1.x). 도구 표면과 출력 형태는 마이너 버전 간에 변경될 수 있습니다. 프로덕션에서는 정확한 버전으로 고정하세요. 버그 보고는 GitHub Issues를 통해 언제나 환영합니다.
$ uvx --from methods-mcp methods-mcp --version
methods-mcp 0.1.6
# In a Claude Code session:
> /mcp add methods-mcp methods-mcp
> Run methods_repro_review on https://arxiv.org/abs/2509.06917
→ tool: methods_repro_review({"input_str":"https://arxiv.org/abs/2509.06917"})
# Returns a MethodsReproReview object. Read `narrative` first — it explains
# everything else in plain English, so no tool-learning is required:
{
"status": "ok",
"narrative":
"Resolved the paper: 'Paper2Agent' by Miao et al. (arxiv 2509.06917, "
"2025-09-08). Extracted 11 methods steps at moderate self-reported "
"confidence (0.72) — the procedure is clearly described but hyperparameters "
"and software versions are absent. Detected the associated code repository "
"https://github.com/jmiao24/Paper2Agent from an inline link in the paper "
"text (detection confidence 0.94). The repo scored 0.90/1.00 on the "
"reproducibility heuristic — verdict: likely reproducible. Present signals: "
"substantive README, dependencies file, notebooks, figure-plotting script, "
"recent activity, permissive license. Missing: data/fixtures directory. "
"Suggested entrypoint: `python make_figures.py`.",
"metadata": { ... }, # PaperMetadata
"methods": { ... }, # MethodsStructured (null if extraction failed)
"code_repo": { ... }, # CodeRepo (null only if input unresolvable)
"repro_assessment": { ... }, # ReproAssessment (null if no repo detected)
"errors": [] # [{step, error_type, message, hint}] on partial
}methods-mcp는 작고 명확하게 범위가 정해진 Model Context Protocol 서버입니다. 모든 AI 에이전트(Claude Code, Claude Desktop, Agent SDK 스크립트 등)에게 학술 논문 URL을 다음으로 변환하는 여덟 가지 도구를 제공합니다:
정식 메타데이터,
최선의 전체 텍스트 + 섹션 분할,
Pydantic으로 검증된 구조화된 방법 객체 (단계 / 시약 / 장비 / 분석),
논문과 연관된 코드 저장소 (최선의 탐색),
해당 저장소에 대한 실행 불필요 재현성 판정, 그리고
다중 모드 요약.
핵심 차별점: Paper2Agent(Stanford)와 같은 대형 파이프라인은 논문을 에이전트 준비 도구로 변환하는 데 30분에서 수 시간이 걸립니다. methods-mcp는 에이전트가 호출 가능한 온디맨드 보완 도구로, 모든 도구가 몇 초 안에 결과를 반환하며 클론이나 실행이 필요 없습니다.
Related MCP server: paperstack
uv add methods-mcp
# or, install globally:
uv tool install methods-mcp
# or, classic pip:
pip install methods-mcp최상의 성능을 위해 둘 다 설정하세요:
변수 | 필수 여부 | 없을 때의 영향 |
|
| 해당 도구는 |
|
| GitHub 인증되지 않은 속도 제한(IP당 시간당 60회 요청)에 걸립니다. 각 저장소 평가는 약 3회 호출이므로 시간당 약 15–20개 저장소 후 한도에 도달합니다. 토큰 사용 시: 시간당 5,000회 요청(사실상 무제한). |
export ANTHROPIC_API_KEY=sk-ant-...
export GITHUB_TOKEN=ghp_... # optional but recommended두 키 모두 로그에 기록되거나 저장되지 않습니다. 각각 api.anthropic.com과 api.github.com으로만 전송됩니다. SECURITY.md를 참조하세요.
/mcp add methods-mcp methods-mcp그런 다음 Claude Code 채팅에서:
https://arxiv.org/abs/2509.06917을 가져와
methods_repro_review를 실행하세요. 논문이 하는 일, 방법 단계, 저장소가 얼마나 재현 가능해 보이는지 요약하세요.
from claude_agent_sdk import ClaudeAgentOptions, ClaudeSDKClient
options = ClaudeAgentOptions(
mcp_servers={
"methods-mcp": {
"type": "stdio",
"command": "methods-mcp",
"args": [],
}
},
allowed_tools=["mcp__methods-mcp__methods_repro_review"],
)
async with ClaudeSDKClient(options=options) as client:
await client.query(
"Run methods_repro_review on https://arxiv.org/abs/2509.06917 "
"and tell me whether the repo looks reproducible."
)
async for msg in client.receive_response():
print(msg)도구 | 기능 |
| 서버 활성 상태 + 구성 확인. |
| URL / arXiv ID / DOI를 정식 메타데이터로 변환. arXiv 입력은 arXiv export API를 통해 제목/저자/초록을 가져옵니다. |
| 전체 텍스트 + 섹션 분할. arXiv 논문의 경우 기본적으로 ar5iv HTML(저렴하고 구조화됨)을 사용하며, 그 외에는 PDF 폴백을 사용합니다. |
| LLM 기반, Pydantic으로 검증된 구조화된 방법 추출. |
| 논문 텍스트 → 초록 → Papers With Code를 통해 논문의 코드 저장소를 탐색합니다. |
| GitHub REST API를 통한 휴리스틱, 클론 없는 재현성 평가. 가중 신호(README, 의존성, 픽스처, 노트북, 그림 스크립트, 최근 유지보수, 라이선스) → |
| 세 가지 깊이의 LLM 요약. |
| 복합 — 메타데이터 + 방법 + 저장소 + 재현성을 한 번에 제공. |
모든 도구는 Pydantic v2 모델(검증됨, JSON 직렬화 가능)을 반환합니다. 전체 타입 표면은 src/methods_mcp/schemas.py를 참조하세요.
extract_methods는 Anthropic 도구 사용을 통해 모델이 MethodsStructured Pydantic 스키마의 인스턴스를 출력하도록 강제합니다. 검증 실패 시 검증 오류가 포함된 수리 메시지를 한 번 보내고 다시 시도한 후 오류를 발생시킵니다.
assess_repo_reproducibility는 클론하거나 실행하지 않습니다. 공개적으로 읽을 수 있는 GitHub 메타데이터와 재귀적 트리 목록을 기반으로 저장소를 점수화합니다. 이는 실제로 논문을 재실행하려는 배치 도구에 대한 의도적인 차별점입니다.
fetch_paper_text는 arXiv 논문의 경우 PDF 파싱보다 ar5iv HTML을 선호합니다. arXiv가 아닌 입력에는 pypdf로 폴백합니다.
기본 모델은 claude-sonnet-4-6입니다. METHODS_MCP_MODEL 환경 변수 또는 호출별 model= 인자로 재정의할 수 있습니다.
methods_repro_review는 자기 설명적 응답을 반환합니다. 모든 호출은 최상위 status("ok" / "partial" / "empty")와 검색된 모든 내용을 평이한 영어로 요약하는 narrative 문자열을 설정합니다. 여기에는 모든 숫자 점수가 문맥과 함께 포함됩니다. narrative + status만 읽는 독자는 하위 객체 구조를 배울 필요 없이 전체 그림을 얻을 수 있습니다. 하위 객체는 사용할 수 없을 때 null일 수 있습니다(예: 감지된 저장소가 없는 논문의 repro_assessment: null — "저장소 없음"은 실패가 아니므로 status는 "ok"로 유지됩니다). 실패한 하위 단계는 {step, error_type, message, hint} 형태의 구조화된 항목을 errors에 추가합니다. 여기서 hint는 인식된 패턴(API 키 누락, 속도 제한, 404, 시간 초과 등)에 대한 실행 가능한 평이한 영어 제안이며, 그 외에는 null입니다.
도구 출력에는 비슷해 보이지만 의미가 매우 다른 세 가지 숫자 필드가 있습니다. 이는 논문을 파고들 가치가 있는지 결정하는 에이전트를 위한 분류 신호이며, 정확성에 대한 보정된 주장이 아닙니다.
필드 | 범위 | 계산 방법 | 읽는 방법 |
| 0–1 | LLM 자체 보고. 추출 모델이 시스템 프롬프트의 지침에 따라 설정합니다: 논문이 명시적인 시약/용량/장비를 제공하는 경우에만 | *"구체적인 절차가 있는 습식 실험 논문인가, 아니면 빈약한 시스템 논문인가?"*에 대한 소프트 신호. 플래그로 유용하며 신뢰 비율로 취급하지 마세요. |
| 0–1 |
| 저장소가 어떻게 발견되었는지, 얼마나 결정적으로 발견되었는지 알려줍니다. 높은 점수 + |
| 0–1 | 8개의 이진 신호의 가중 합으로, 모두 GitHub REST API에서 계산됩니다(클론 없음, 실행 없음): | 세 점수 중 유일하게 완전히 결정론적인 점수입니다. 여전히 휴리스틱이지 증명이 아닙니다 — 높은 점수는 저장소가 재현을 위해 잘 구조화되어 보인다는 뜻입니다. 실제 검증은 Paper2Agent를 참조하세요. |
판정 버킷(repro_assessment.verdict)은 overall_score에 대한 임계값입니다:
Verdict | Score | Meaning |
| ≥ 0.70 | 재현에 유리한 신호가 대부분 존재. 실행을 시도해 볼 가치가 있음. |
| ≥ 0.45 | 일부 인프라는 있으나 공백이 있을 가능성. 누락된 부분을 채워야 할 것으로 예상. |
| ≥ 0.20 | 신호가 최소한. 스캐폴딩 없이 재실행할 수 없는 코드 덤프일 가능성. |
| < 0.20 또는 저장소에 접근 불가 | 판단할 정보가 부족. 어느 쪽으로도 결론을 내리지 말 것. |
출력에서 볼 수 있는 열거형 값:
code_repo.detection_method: paper-text | abstract-link | papers-with-code | metadata | none
metadata.source: arxiv | biorxiv | doi | url | unknown
이 서버가 설치 및 실행 시 실제로 수행하는 작업:
네트워크 호출 대상은 다음뿐: export.arxiv.org, ar5iv.labs.arxiv.org, arxiv.org (PDF), api.github.com, paperswithcode.com, api.anthropic.com. 텔레메트리, 분석, 전화 집 없음.
환경 변수에서 ANTHROPIC_API_KEY(LLM 도구에 필요)와 선택적으로 GITHUB_TOKEN을 읽음. 이 값은 각각 Anthropic / GitHub에만 전송됨. 절대 로깅되지 않으며 디스크에 저장되지 않음.
파일 시스템에 아무것도 쓰지 않음. 캐시 디렉터리, 다운로드된 PDF, 임시 파일 없음.
사용자 제공 코드를 실행하지 않음. eval, exec, subprocess, pickle.loads 또는 셸 호출 없음. 재현성 도구는 의도적으로 저장소를 클론하거나 실행하지 않음 — GitHub REST API에서만 점수를 산출함.
알아야 할 제한 사항:
악의적인 논문은 오해를 불러일으키는 구조화된 출력을 생성할 수 있음. extract_methods 도구는 논문 텍스트를 Claude에 전송함. 프롬프트 인젝션 콘텐츠가 포함된 논문은 잘못된(그러나 스키마는 유효한) 구조화된 메서드를 생성할 수 있음. 출력을 연구 보조 자료로 취급하고, 절대적인 진실로 취급하지 말 것.
재현성 판정은 휴리스틱이지 증명이 아님. 높은 점수는 저장소가 재현에 적합한 구조로 보인다는 뜻일 뿐, 코드를 실행하면 논문이 재현된다는 것을 보장하지 않음. 전체 검증은 Paper2Agent를 참조.
로컬 stdio 사용을 위한 도구임. HTTP/SSE 전송은 개발 편의를 위해 제공되며 신뢰할 수 있는 네트워크에서만 노출해야 함(httpx가 제공하는 것 이상의 SSRF 보호는 없음).
문제 보고:
보안 문제: flynnlachendro@hotmail.co.uk로 이메일을 보내 주세요(SECURITY.md도 참조). 기능 버그: GitHub 이슈를 열어 주세요.
paper-mcp와 함께 사용더 넓은 논문 검색 / 인용 그래프 도구를 원한다면 동일한 Claude Code 세션에서 paper-mcp (Bhvaik)를 함께 실행하세요. paper-mcp는 제목 키 검색, 전문 가져오기, 인용 및 참고문헌을 처리하고, methods-mcp는 그 위에 구조화된 메서드 + 재현성 계층을 추가합니다. 두 도구는 의도적으로 함께 구성되도록 설계되었습니다.
git clone https://github.com/FlynnLachendro/methods-mcp
cd methods-mcp
uv sync --extra dev --extra agent
uv run pytest # 49 tests, offline (respx-mocked httpx + unittest.mock for Anthropic)
uv run ruff format .
uv run ruff check . --fix
uv run mypy src
uv run methods-mcp --helpMIT — LICENSE 참조.
Worldwide AI Science Fellowship 첫 번째 코호트를 위해 제작됨. 자유로운 브리핑을 제공한 Michael Raspuzzi에게 감사드립니다.
다음을 기반으로 구축됨:
FastMCP 3.x — MCP 서버 스캐폴드.
Claude Agent SDK — 데모의 에이전트 루프.
ar5iv.labs.arxiv.org — arXiv 논문의 깔끔한 HTML.
Anthropic Claude — 구조화된 추출을 담당하는 LLM.
Heuristic, no-clone reproducibility assessment of a GitHub repo.
Returns a verdict (likely-reproducible / partial / unlikely / insufficient-info) plus weighted signals: README quality, dependency files, fixture data, notebooks, figure-generating scripts, recent maintenance, license. No code is downloaded or executed. Set GITHUB_TOKEN to raise the API rate limit.
| Name | Required | Description | Default |
|---|---|---|---|
| paper_id | No | ||
| repo_url | Yes |
| Name | Required | Description |
|---|---|---|
| notes | Yes | |
| signals | No | |
| verdict | Yes | |
| paper_id | No | |
| repo_url | Yes | |
| overall_score | Yes | |
| recommended_entrypoint | No | E.g. 'python make_figures.py' if a likely figure-generating script was found. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are entirely absent, so the description carries the full disclosure burden — and it meets it. It explicitly states the operation is non-destructive ('No code is downloaded or executed'), that this is a heuristic scoring exercise, that weighted signals feed the verdict, and even discloses the GitHub API rate-limit dependency plus the GITHUB_TOKEN mitigation. This is comprehensive behavior disclosure with no annotation support needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is tight and dense with zero filler: first sentence states purpose and mode, second summarizes the verdict taxonomy, third enumerates the signals, fourth discloses side-effect-free behavior, fifth gives the auth hint. Every sentence earns its place and the most decision-relevant facts (heuristic, no-clone) are front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The overall assessment logic, safety profile, and auth requirement are well covered, and an output schema exists so the return structure doesn't need full restatement. However, completeness is undermined by the unexplained paper_id parameter and the lack of any guidance on when this tool applies versus methods_repro_review. For a moderately complex heuristic tool this is adequate but leaves discernible gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate — and largely fails to. repo_url's purpose is inferable from context but is never actually explained. paper_id (an accepted parameter, optional with null default) is completely unexplained: an agent cannot tell whether it links the assessment to a paper in the system, serves as a label, or affects scoring. No parameter receive explicit definitional help.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening line is specific: 'Heuristic, no-clone reproducibility assessment of a GitHub repo' — a clear verb (assess) plus a concrete resource (GitHub repo reproducibility), with the heuristic/no-clone qualifiers delimiting its scope. It states the return verdicts and the signal dimensions, so an agent understands what this produces. It doesn't explicitly distinguish itself from the closest sibling, methods_repro_review, but the 'no-clone' and 'heuristic vs deep review' framing sufficiently separates it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description never states when to choose this tool over alternatives. It enumerates what it does — heuristic signals, no code execution — but gives no explicit 'use when you need a quick pass' or 'for a full methods review use methods_repro_review instead.' Given rich sibling context (methods_repro_review, find_code_repo, extract_methods), the absence of routing or exclusion guidance is a real gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract a structured methods object from a paper.
Pulls full text, isolates the methods-relevant section(s), then asks Claude (default: claude-sonnet-4-6) to fill out a Pydantic schema with steps, reagents, equipment, and analyses. Validation-enforced — invalid responses trigger one repair attempt before raising.
Requires ANTHROPIC_API_KEY in the server's environment.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | ||
| input_str | Yes |
| Name | Required | Description |
|---|---|---|
| notes | No | Free-form caveats from the extractor (e.g. 'methods section is sparse'). |
| steps | No | |
| summary | Yes | 2-3 sentence overview of the methods. |
| analyses | No | |
| paper_id | Yes | |
| reagents | No | |
| equipment | No | |
| confidence | Yes | LLM self-reported confidence in the structured extraction. |
| extraction_model | Yes | Model identifier used for extraction. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since annotations are absent, the description carries the full burden and does substantial work: it discloses that the tool fetches full text, isolates methods sections, calls an external Claude model, validates against a Pydantic schema, performs one repair attempt, and requires ANTHROPIC_API_KEY. These are behavioral traits an agent needs to anticipate dependencies and side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description front-loads the core purpose, then moves through process and prerequisites in three compact paragraphs. No filler or repetition of the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the pipeline, validation/repair behavior, and auth prerequisite, and an output schema exists so return semantics needn't be restated. However, it leaves the required input parameter semantically ambiguous, which is a meaningful gap for invoking the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description only partially compensates: it mentions the default model (claude-sonnet-4-6) but never defines the required input_str (e.g., paper ID vs plain text) or the effect of passing a custom model. An agent cannot confidently populate parameters from this definition alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Extract') and resource ('structured methods object') plus source ('a paper'), and names output components (steps, reagents, equipment, analyses). It is clearly distinguishable from sibling tools like summarize_paper and fetch_paper_text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description makes the general use case clear (extracting structured methods), but it never explicitly says when to prefer this over siblings such as fetch_paper_text or methods_repro_review, and it gives no exclusion criteria. Usage is implied from the purpose rather than explicitly guided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return best-effort full text + section split for a paper.
prefer: one of "auto" (default), "html" (force ar5iv HTML — arXiv only), or "pdf". Sections are normalised to lowercase canonical names: abstract, introduction, methods, experiments, results, discussion, conclusion, limitations, related_work.
| Name | Required | Description | Default |
|---|---|---|---|
| prefer | No | auto | |
| input_str | Yes |
| Name | Required | Description |
|---|---|---|
| text | Yes | |
| paper_id | Yes | |
| sections | No | Section name (lowercased) -> text. Empty if extraction couldn't segment. |
| char_count | Yes | |
| source_url | Yes | |
| extraction_method | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral burden. It discloses 'best-effort' semantics (implying possible partial/failed text retrieval), the arXiv-only constraint for the html preference, and the section normalization behavior. However, it does not disclose what happens when html is requested for non-arXiv papers (fallback behavior) or how failures surface to the caller.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Roughly 60 words with the core purpose front-loaded in the first sentence. Each subsequent sentence explains the prefer values and the canonical section names without waste. It is appropriately tight, though slightly terse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values are partially covered there. The description discloses section-normalization output behavior. But the undefined 'input_str' format is a critical gap: an agent cannot reliably invoke the tool without knowing what identifier to pass. The failure/fallback semantics of 'best-effort' and non-arXiv html requests are also unspecified. For a 2-param tool at 0% schema coverage, the description needed to compensate and did not.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must define both parameters. 'prefer' is explained thoroughly (auto/html/pdf values, default, and the html constraint), but 'input_str' — the single required parameter — is never explained. An agent has no idea whether it expects an arXiv ID, DOI, or URL. This is the most critical parameter for calling the tool and is left undefined.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Return best-effort full text + section split for a paper.' This clearly distinguishes it from siblings like get_paper_metadata (metadata only) and summarize_paper (summary), making it easy for an agent to know this tool retrieves the raw full text with section boundaries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The purpose implicitly signals when to use this tool (need full text + sections, not metadata or analysis), but there is no explicit when-to-use or when-not-to-use guidance, and no alternative tools are named. The agent must infer the selection logic from the name and description alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Discover the code repo (if any) associated with a paper.
Strategy: scan the paper's full text and abstract for github.com URLs; fall back to Papers With Code lookup for arXiv papers.
| Name | Required | Description | Default |
|---|---|---|---|
| input_str | Yes |
| Name | Required | Description |
|---|---|---|
| notes | No | |
| paper_id | No | |
| repo_url | No | |
| confidence | Yes | |
| detection_method | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the algorithm: scanning for GitHub URLs and using a Papers With Code fallback. This is meaningful behavioral transparency beyond just 'finds a repo.' It also hints at a potential outcome ('if any'), implying it might not always find a repo. However, it does not mention limitations like other repo hosts or what happens when a paper is not on arXiv. Still, the strategy is a strong disclosure of internal behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with zero fluff. The first sentence states the purpose clearly and the second provides the strategy. It is front-loaded with the core action, and every word contributes value. The structure is efficient and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is relatively simple with one parameter and an output schema, so the return format is presumably covered by the output schema. However, the input parameter is not described, which is a critical gap. The description explains the method but leaves the agent unsure what to pass as input_str. Given the lack of annotations, the description must carry more weight, and this omission makes it incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one required parameter, 'input_str', with no description and 0% schema description coverage. The description refers to 'a paper' but does not explain what input_str should contain (e.g., a paper ID, URL, or title). It does not clarify the expected format or how the tool will use it. The description fails to compensate for the lack of schema detail, leaving the agent uncertain about the parameter's meaning. This is a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Discover the code repo (if any) associated with a paper.' It uses a specific verb ('discover') and resource ('code repo'), and the scope is clear. This distinguishes it from siblings like get_paper_metadata or assess_repo_reproducibility, which handle metadata or reproducibility assessment. The strategy also clarifies the exact function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it: whenever you need to locate a paper's code repository. It provides a concrete strategy (scanning full text/abstract for GitHub URLs, falling back to Papers With Code for arXiv papers), which gives context on its operation. It does not explicitly exclude scenarios or name alternative tools, but the purpose is clear enough that an agent would know it is the right tool for finding code repos. Lacks explicit 'when not to use' guidance, so a 4 is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Resolve an arXiv ID/URL, bioRxiv URL, DOI, or generic URL to canonical metadata.
For arXiv inputs this hits the arXiv export API to fetch title/authors/abstract. Other sources return the resolved URL with empty bibliographic fields (cheap by design).
| Name | Required | Description | Default |
|---|---|---|---|
| input_str | Yes |
| Name | Required | Description |
|---|---|---|
| title | Yes | |
| venue | No | |
| source | Yes | |
| authors | No | |
| pdf_url | No | |
| abstract | No | |
| html_url | No | ar5iv HTML URL for arXiv papers. |
| paper_id | Yes | Canonical identifier (e.g. arXiv ID, DOI). |
| raw_input | Yes | The original URL/ID the caller supplied. |
| primary_url | Yes | |
| publication_date | No | ISO 8601 date when known. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that arXiv inputs trigger an API call to fetch metadata, while others return a resolved URL with empty bibliographic fields (termed 'cheap by design'). However, it does not mention side effects, error conditions, or rate limits, and does not explicitly declare it as read-only. It adds useful context but is not fully transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no fluff. The core purpose is front-loaded, followed by the key behavioral distinction for arXiv versus other sources. Every sentence earns its place, and it is appropriately concise for a tool with one parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no annotations) and the existence of an output schema, the description covers essential behavior: resolving inputs and returning specific metadata for arXiv, while handling other sources efficiently. It could be more complete by mentioning error handling or URL format requirements, but these are minor gaps. Overall, it provides enough context for an agent to call the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does by explaining that input_str accepts arXiv IDs/URLs, bioRxiv URLs, DOIs, or generic URLs, adding meaning beyond the schema's simple 'string' type. It does not specify exact formats or validation rules, but the accepted types are clearly stated, which is sufficient for a single simple parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Resolve') and a clear resource ('arXiv ID/URL, bioRxiv URL, DOI, or generic URL') with a defined output ('canonical metadata'). It also differentiates behavior by input type, which helps distinguish it from sibling tools like fetch_paper_text that likely handle full text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The intended use is implied by the purpose ('resolve... to canonical metadata'), but there is no explicit mention of when to use this tool versus alternatives or when not to use it. It does not compare with siblings or provide selection criteria, such as 'use fetch_paper_text for full text'. Guidance is only implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Return a small health report — confirms the server is alive and configured.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| server | No | |
| version | Yes | |
| transports | Yes | |
| anthropic_configured | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool returns a report and indicates a read-only behavior ('confirms' and 'report'). It does not mention side effects or permissions, but for a health check, these are arguably non-issues. The description goes beyond trivial naming by explaining the report's content.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one concise, front-loaded sentence that states the action and its purpose without any fluff. Every word contributes value. It is appropriately sized for a tool this trivial.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with an output schema present, the description is fully sufficient. It tells the agent what the tool returns and what it confirms. No additional details (e.g., exact format) are needed because the output schema handles that. The description is complete for its purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema fully covers parameter semantics (coverage is 100%). The baseline for 0-parameter tools is 4, and the description adds no unnecessary parameter information. It correctly focuses on the output and purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Return') and resource ('health report') and specifies the purpose: 'confirms the server is alive and configured.' This clearly distinguishes it from the sibling tools, all of which are paper-related and involve data retrieval or analysis.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for checking server status but does not explicitly state when to use it versus alternatives or provide exclusions. There is no mention of 'use this when...' or 'instead of...'. Given the simplicity and obvious distinctiveness from paper-processing siblings, the implicit context is acceptable but not explicitly stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Resolve a paper + extract methods + find its code repo + assess reproducibility — in one call.
Returns a MethodsReproReview object. Read narrative first — it's a
plain-English summary of everything retrieved, with every numeric score
echoed in context (e.g. "moderate confidence (0.72)"). Then:
status: "ok" (all sub-steps succeeded), "partial" (some failed — see
errors), or "empty" (couldn't even resolve the input paper).
metadata, methods, code_repo, repro_assessment: the structured
sub-results. Any step that failed contributes null for its key;
null means unavailable, not zero.
errors: list of {step, error_type, message, hint} — one entry per
failed sub-step, with an actionable hint where the failure pattern is
recognised (e.g. missing ANTHROPIC_API_KEY, rate-limit, 404, timeout).
For the meaning of numeric scores and verdict buckets, see the "Scores & verdicts explained" section of the README.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | ||
| input_str | Yes |
| Name | Required | Description |
|---|---|---|
| errors | No | |
| status | Yes | Top-level status of a composite-tool call. `ok` — every sub-step succeeded. `partial` — at least one sub-step failed (see `errors`) but we have usable results. `empty` — we couldn't even resolve the paper input; no sub-steps ran. |
| methods | No | |
| metadata | No | |
| code_repo | No | |
| narrative | Yes | |
| repro_assessment | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavior, and it does so well. It explains the possible status values (ok/partial/empty), the semantics of null keys (unavailable, not zero), and the structure of errors including hints. It also tells the agent to read 'narrative' first and mentions failure patterns like missing ANTHROPIC_API_KEY, rate-limit, and timeouts. It does not explicitly state read-only semantics or potential side effects, but the error hints imply network calls; overall it is transparent for a composite read tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is reasonably concise for the complexity, with the core purpose front-loaded in the first sentence and the return structure laid out in a clear bullet list. It avoids redundancy and uses formatting to separate fields. It references the README for score meanings, which is a minor downside but acceptable given the length. Overall, it is well-structured and not bloated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the complexity (composite of four steps), the description omits critical details needed to call the tool: the input format for input_str and the meaning of model. It also defers score/verdict details to the README, which is a gap because the agent receives no inline explanation. With no annotations and no parameter descriptions, the description is not self-sufficient for correct invocation, leaving the agent with unanswered questions about required inputs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain both parameters. It only hints that input_str is a paper identifier ('Resolve a paper') but never specifies the format (DOI, arXiv ID, URL, or free text). The model parameter is completely unmentioned, leaving the agent clueless about its purpose or allowed values. This is a significant gap that forces the agent to guess or consult external docs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb and resource: it resolves a paper, extracts methods, finds a code repo, and assesses reproducibility in one call. It enumerates the sub-steps explicitly, which distinguishes it from the single-purpose sibling tools (e.g., extract_methods, find_code_repo) and makes its combined scope immediately obvious. The phrase 'in one call' reinforces its composite nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is a convenience for getting all four results at once, but it does not explicitly state when to use it versus calling individual siblings, nor does it mention any exclusions or prerequisites. It says 'in one call' but does not say 'use this when you need all steps' or 'if you only need X, use Y'. This leaves the agent to infer the use case without explicit routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Generate an LLM summary of a paper in one of three modes.
abstract = 2-3 sentences close to the authors' framing. tldr = one-line takeaway. exec = executive summary (what / found / why-it-matters).
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | tldr | |
| model | No | ||
| input_str | Yes |
| Name | Required | Description |
|---|---|---|
| mode | Yes | |
| summary | Yes | |
| paper_id | Yes | |
| key_findings | No | |
| extraction_model | Yes | |
| methodology_oneliner | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It reveals that summaries are LLM-generated and specifies the output format for each mode, which is useful. However, it does not mention read-only status, side effects, required permissions, or any potential costs or delays. This is a moderate level of transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise and well-structured. It opens with a clear purpose sentence, then lists each mode with its output specification using a bullet-like format. Every sentence contributes value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool appears simple (3 params, 1 required), and the description explains the output for each mode. However, the required parameter 'input_str' is not defined, and the 'model' parameter is left unexplained. No mention of error handling or typical usage scenarios. The description is adequate for a simple tool but has noticeable gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does explain the 'mode' enum values thoroughly, adding meaning beyond the schema. However, it does not clarify 'input_str' (presumably the paper text) or 'model' (which model to use). This partial compensation earns a middle score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('Generate') and resource ('a paper'), and defines three distinct modes of output. This distinguishes it from sibling tools like extract_methods or fetch_paper_text, which have different functions. The mode definitions add precision.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the three mode options in detail, which helps an agent decide which mode to use. However, it does not explicitly state when to use this tool versus alternatives such as extract_methods or methods_repro_review. The usage context is implied by the name and purpose, but not explicitly guided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.7assess_repo_reproducibilityextract_methodsfetch_paper_textfind_code_repoget_paper_metadatahealthmethods_repro_reviewsummarize_paperScored across 8 tools
Each tool has a clearly distinct purpose: health check, metadata resolution, text fetching, methods extraction, code repo discovery, reproducibility assessment, summarization, and a combined orchestration tool. The composite tool is explicitly described as a one-call pipeline, so no confusion with individual steps.
Most tools follow a verb_noun pattern (get, fetch, extract, find, assess, summarize), but 'health' is a bare noun and 'methods_repro_review' is a noun phrase without a verb. These deviations break the otherwise consistent snake_case convention.
8 tools is well within the ideal range, and each tool maps to a specific stage in the paper analysis pipeline. No redundancy or unnecessary duplication, and the count feels appropriately scoped for the server's reproducibility-focused purpose.
The tool surface covers the entire workflow from paper resolution to reproducibility assessment, including a composite endpoint. Minor gaps like search or fine-grained section access are absent but not essential to the stated domain, making the set largely complete.
Reliable PDF table extraction. Pass a URL, get structured JSON tables with citations.
Clean Markdown and AI-readability scoring for any URL. Built for AI agents.
Academic papers, reference verification and research data for AI agents.
Extract structured data points from research papers and other documents with an LLM.
