Skip to main content
Glama
co-scientist-plugin

co-scientist-plugin

co-scientist plugin

코딩 에이전트 내부에서 멀티 에이전트 과학 연구 엔진을 실행하세요.

Claude Code, Codex CLI, Cursor, VS Code, Copilot 및 Kiro가 AI co-scientist 엔진을 구동할 수 있게 해주는 Agent Plugins 1.0 패키지입니다: 가설을 생성하고, 검토하고, Elo 토너먼트에서 순위를 매기고, 승자를 진화시키고, 종합 결과를 다시 읽어옵니다.

tests Python 3.11 to 3.13 License: Apache 2.0 Agent Plugins 1.0

설치 · 에이전트가 얻는 것 · 설계 원칙 · ScientistOS와 함께 사용 · 기여하기


아이디어

AI co-scientist 엔진은 Gottweis 외, Nature, 2026의 오픈 재구현입니다. 연구 목표를 받아 여섯 개의 전문 에이전트를 실행합니다: 생성(Generation), 반성(Reflection), 순위(Ranking), 진화(Evolution), 근접성(Proximity), 메타 검토(Meta-review)이며, Supervisor가 내구성 있는 SQLite 작업 큐를 통해 일정을 관리합니다.

그 엔진에는 CLI가 있습니다. 이 패키지는 에이전트 지향 인터페이스를 제공하므로, 명령을 직접 입력하는 대신 Claude Code나 Codex가 엔진을 구동할 수 있습니다:

you    → "review hypothesis 3 in that Crohn's session and tell me what breaks"
agent  → reads the hypothesis, searches the literature, applies the rubric,
         records the review into the engine's database, reports the verdict

에이전트가 작성한 검토는 이제 엔진 자체의 Reflection 에이전트가 쓰는 것과 동일한 테이블의 행이 됩니다. 토너먼트에 집계됩니다. 메타 검토가 이를 확인할 수 있습니다.

Related MCP server: multivon-mcp

에이전트가 얻는 것

하나의 MCP 서버에 열다섯 개의 도구가 있습니다:

그룹

도구

수명주기

session_start · session_resume · session_pause · session_abort · session_list · session_status

읽기

hypotheses_list · hypothesis_get · overview_get

쓰기

hypothesis_record · review_record · match_record · feedback_add

진단

estimate_cost · doctor

도구가 처리할 수 없는 판단을 담은 일곱 개의 스킬:

스킬

기능

running-research-sessions

모호한 목표를 실제 목표로 다듬고, 실행 규모에 따라 게이트를 걸고, 시작하고 모니터링합니다

reading-research-overviews

다섯 가지 알려진 실패 모드를 확인하며 순위 결과를 정직하게 읽습니다

reviewing-hypotheses

점수 기준과 판정 결정 규칙이 있는 루브릭으로 검토한 후 기록합니다

ranking-hypotheses

3~5턴 토론을 진행하고, 정렬된 기준으로 결정하며, Elo 매치를 기록합니다

evolving-hypotheses

결합, 단순화, 실현 가능성, 틀에서 벗어나기 — 각각 고유한 참조 파일과 실패 모드가 있습니다

benchmarking-models

모델 간 Elo 벤치마크와, 과장 없이 결과를 읽는 방법

grounding-claims

인용이 주장하는 바를 실제로 말하는지, 주장별로 판정과 함께 확인합니다

설계 원칙

엔진은 모든 상태를 유지합니다: Elo, 작업 큐, 중복 제거, 임베딩, 토큰 예산, 종료 조건. 이 패키지는 그중 어떤 것도 재구현하지 않습니다.

에이전트가 생성하는 모든 것은 엔진 자체의 에이전트가 쓰는 것과 동일한 저장소를 통해 다시 들어갑니다. match_record는 엔진의 Elo 모듈과 elo_journal 멱등성을 재사용합니다. review_record는 Reflection이 하는 것과 정확히 동일하게 동일한 Review 모델을 쓰고 draft를 reviewed로 승격시킵니다. hypothesis_record는 부모가 주어졌는지 여부에서 created_by를 파생합니다.

스킬은 그 결과를 평이한 말로 명시합니다: 기록되지 않은 출력은 존재하지 않습니다. review_record를 호출하지 않고 채팅에서 검토를 설명한 에이전트는 시스템이 사용할 수 있는 것을 아무것도 생산하지 않은 것입니다.

이것이 "프롬프트를 스킬로 옮기고 끝내는" 흔한 실패를 막는 방법입니다. 품질의 원천은 프롬프트가 아닙니다. 토너먼트가 그 원천이며, 토너먼트에는 점수를 기록할 곳이 필요합니다.

ScientistOS와 함께 사용

이 플러그인은 후보 가설을 생성하고 순위를 매깁니다. 가설에서 무엇을 주장할 수 있는지는 결정하지 않으며, 그 차이는 중요합니다: 가설이 토너먼트에서 1위를 해도 문헌이 모순하는 메커니즘에 기반할 수 있습니다. reading-research-overviews 스킬은 그 실패 모드와 수동으로 확인해야 할 몇 가지 다른 것들을 나열합니다.

ScientistOS는 그 확인을 권고가 아닌 구조적으로 만듭니다. 타입이 있는 증거 그래프를 유지하고, 계보(lineage)가 뒷받침하지 않는 강도로 주장을 기록하는 것을 거부합니다.

this plugin   →  ranked hypotheses, reviews with claim / url / excerpt per source
                              ↓
ScientistOS   →  hypothesis and source nodes, evidence at its honest level,
                 an integrity gate before anything gets written up

둘 다 스킬 및 MCP 패키지이므로 나란히 설치할 수 있고, 둘 다 로드된 에이전트는 전체 경로를 얻습니다. 여기에는 ScientistOS가 필요하지 않으며, ScientistOS도 이 패키지에 의존하지 않습니다.

설치

API 키가 필요 없고, 결제에 대해 결정할 것도 없습니다. 이 글을 읽고 있다면 이미 Claude Code 또는 Codex CLI에 로그인되어 있을 것이며, 그것이 전부입니다: 연구 엔진은 코딩 에이전트가 사용하는 것과 동일한 구독을 통해 에이전트를 실행합니다.

python3.12 -m venv .venv
source .venv/bin/activate

pip install "co-scientist @ git+https://github.com/Kaimen-Inc/Co-Scientist.git"
pip install "co-scientist-plugin @ git+https://github.com/Ahmad-Jaradat-Space/co-scientist-plugin.git"

co-scientist init

첫 연결 시 서버는 작동하는 백엔드를 찾습니다. 찾지 못하면 설치되어 로그인된 에이전트 CLI를 채택하고, 그 선택을 ~/.co-scientist/config.toml에 기록한 후 계속 진행합니다. 기존 provider = 줄은 절대 건드리지 않으므로, 직접 구성한 설정은 그대로 유지됩니다. 전체 기능을 끄려면 COSCI_AUTO_BACKEND=0을 설정하세요.

doctor가 여전히 백엔드가 없다고 보고하면, 키를 찾는 대신 CLI에 로그인하세요 (claude 또는 codex login). 또한 로컬 해시 임베더를 보고할 텐데, 이는 예상되는 키 없는 상태입니다: 중복 제거는 반복된 표현은 잡지만 같은 내용을 다르게 말하는 두 가설은 잡지 못합니다.

모든 것이 사용자 머신에 유지됩니다: ./data 아래의 SQLite, 계정 없음, 가입할 서비스 없음, 그리고 CLI가 이미 만들고 있던 모델 호출 외의 트래픽 없음.

config/subscription.toml은 동일한 선택을 손으로 작성한 것이며, CLI가 해석하는 별칭에 맡기는 대신 모델 ID를 고정합니다. export COSCI_CONFIG=/path/to/config/subscription.toml로 엔진을 지정하세요.

종량제 API도 작동합니다: 첫 실행 전에 ~/.co-scientist/config.toml에 프로바이더와 키를 설정하면 자동 선택이 개입하지 않습니다. 그러면 estimate_cost가 예산에 따라 실행 비용을 산정하고, 스킬은 시간이 아닌 지출에 대한 진행 승인을 요청합니다. 엔진의 백엔드 가이드를 참조하세요.

Codex CLI

codex plugins install ./co-scientist-plugin

Claude Code

ln -s "$PWD/skills/"* ~/.claude/skills/
claude mcp add co-scientist -- co-scientist-mcp

그 외

Agent Plugins 1.0을 읽는 모든 클라이언트는 디렉토리를 그대로 로드합니다. SKILL.md와 MCP 구성을 별도로 읽는 모든 클라이언트는 skills/와 mcp.json을 각각 가져갈 수 있습니다.

구성

서버는 엔진의 구성을 읽습니다. 필요하다면 mcp.json의 env 블록에 다음을 설정하세요:

변수

효과

COSCI_CONFIG

오버레이할 추가 TOML 구성의 경로

COSCI_ALLOW_START

0은 session_start와 session_resume을 거부합니다

COSCI_START_TIMEOUT

생성된 실행이 등록될 때까지 기다리는 초, 기본값 120

COSCI_AUTO_BACKEND

0은 아무것도 구성되지 않았을 때 로그인된 CLI 선택을 끕니다

읽기 전용 설치

세션은 수백 번의 모델 호출을 만들고 대화가 끝난 후에도 계속 실행됩니다. 공유 머신이나 에이전트가 스스로 세션을 시작할 수 없어야 하는 곳에서는:

co-scientist-mcp --read-only

읽기는 계속 작동합니다. session_start와 session_resume은 설명과 함께 거부합니다.

Virtualenv 경로

mcp.json은 co-scientist-mcp를 호출하며, 콘솔 스크립트가 PATH에 있다고 가정합니다. 에이전트가 virtualenv를 활성화하지 않으면 절대 경로를 사용하세요:

{
  "command": "/path/to/.venv/bin/co-scientist-mcp",
  "args": []
}

개발

pip install -e ".[dev]"
ruff check coscientist_plugin tests
pytest -q
co-scientist-plugin-validate

모든 테스트는 일회용 데이터베이스에 대해 오프라인으로 실행됩니다. 키 없음, 네트워크 없음, 모델 호출 없음.

이 패키지를 지키는 세 가지 검사가 있습니다:

  • **co-scientist-plugin-validate**는 사양 적합성을 검사합니다: 허용된 열 개의 매니페스트 필드, 전송 유형, 스킬 frontmatter, 이름과 디렉토리 일치, 본문 길이, 그리고 모든 상대 링크가 해석되고 한 단계 깊이를 유지하는지.

  • **tests/test_plugin.py**는 검증기가 할 수 없는 것을 검사합니다: 어떤 스킬도 에이전트에게 존재하지 않는 MCP 도구나 엔진 명령을 호출하라고 지시하지 않는지, 그리고 기록 규율이 쓰기 스킬에 실제로 명시되어 있는지.

  • **tests/test_server.py**는 설계가 의존하는 불변식을 다룹니다: 제로섬 Elo, 라운드별 멱등성, 교차 세션 거부, 발췌문 없는 증거 거부, 그리고 순위가 매겨진 가설을 뒤로 끌어내리지 않는 상태 전이.

엔진과의 관계

이 저장소는 플러그인 레이어일 뿐입니다. 구동하는 엔진은 Kaimen-Inc/Co-Scientist에 있으며 동일한 라이선스 하의 별도 프로젝트입니다. NOTICE를 참조하세요.

두 프로젝트 모두 Google이나 논문 저자와 제휴하지 않습니다. 둘 중 하나가 생성하는 가설은 과학자의 출발점이지, 발견이 아닙니다.

라이선스

Apache-2.0. LICENSE 및 NOTICE를 참조하세요.

Available Tools

15 tools
doctorA

Check the install: which LLM backend is configured, whether its credentials resolve, whether embeddings are real or the hash fallback, and where the database lives. Run this first when anything fails.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It lists what is checked but does not disclose whether the tool has side effects, what the output looks like, or what 'hash fallback' implies. It gives partial transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, with a clear front-loaded action ('Check the install') and a structured list of what is checked. Every sentence earns its place with no waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no parameters and no output schema, the description covers the key context: what it checks and when to run it. The lack of return format details is a minor gap, but it's adequate for selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the empty schema provides full coverage. Baseline 4 is appropriate since there are no parameter semantics to explain.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states what the tool does: it checks the install configuration, specifying backend, credentials, embeddings, and database location. This is a distinct diagnostic function, differentiating it from sibling session/review tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides an explicit trigger: 'Run this first when anything fails.' While it doesn't mention exclusions or alternatives, the directive is clear and actionable for a diagnostic tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

estimate_costA

Pre-flight size of a session under the current config: calls and tokens per agent, and how long it may run. On a metered API backend it also prices it. Run this before session_start and show the user.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description must carry the behavioral disclosure burden. It explains scope ('under the current config'), the kind of estimate provided, and the conditional pricing behavior on metered backends. Still, it does not explicitly state whether the operation is read-only, what side effects it has, or how precise the estimate is.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler. The key action, output content, conditional behavior, and usage instruction are all front-loaded and clearly expressed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no params, no annotations, and no output schema, the description covers the essential inputs ('current config'), outputs (calls/tokens/duration/price), and timing ('before session_start'). It is slightly ambiguous what 'current config' refers to, but it is adequate for this tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is no schema coverage gap to compensate for. The description adds meaning by explaining that the estimate is based on 'the current config' and what is being measured.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies a specific verb-like action ('Pre-flight size of a session') and details exactly what it produces: calls, tokens per agent, duration, and price on metered backends. It also distinguishes itself from siblings by explicitly framing it as a pre-`session_start` step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit guidance on when to use the tool: 'Run this before session_start and show the user.' However, it does not mention when not to use it or name alternatives from the sibling list, so it stops short of full exclusion guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

feedback_addA

Inject researcher feedback into a session. A running session picks it up on its next planning step; a paused one picks it up on resume. kind='pin' or 'rejection' with a target also moves that hypothesis's state.

NameRequiredDescriptionDefault
kindNo
textYesThe feedback, free text.
targetNoHypothesis id this is about.
session_idYesSession id, e.g. ses_01J...

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It transparently states the timing of feedback delivery (next planning step or on resume) and the special effect of kind='pin' or 'rejection' with a target on hypothesis state. This goes beyond the schema. However, it does not mention potential failure modes or prerequisites like session existence, leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no wasted words. It front-loads the primary purpose and immediately provides the timing and state-change nuance. Every clause adds value, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (session state dependency, hypothesis mutation) and the lack of annotations or output schema, the description covers the key behavioral aspects. It could be more complete by specifying that the session must exist or what happens on invalid input, but it is adequate for a tool with this scope. The absence of return value documentation is acceptable since no output schema is provided.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 75% (three of four parameters have descriptions; kind only has an enum). The description adds semantic value by explaining the interaction between kind and target for moving hypothesis state, which is not present in the schema. This compensates for the missing description of kind, justifying a score above the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool injects researcher feedback into a session, with a specific verb and resource. It differentiates from siblings by focusing on feedback injection rather than session control (like session_pause) or hypothesis recording (like hypothesis_record). The mention of how it affects hypothesis state further clarifies its unique role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear contextual usage: it explains when the feedback takes effect (running vs paused sessions). It does not explicitly name alternative tools or state when not to use it, but the behavioral context is sufficient for an agent to infer the appropriate use case. Since sibling tools like review_record or match_record are not referenced for exclusion, this is not a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hypotheses_listA

List a session's hypotheses ranked by Elo, with title, summary, state, matches played and dedup cluster. Bodies are omitted; use hypothesis_get.

NameRequiredDescriptionDefault
limitNoMax rows. Default 20.
stateNoFilter by state: draft, reviewed, in_tournament, pinned, rejected, quarantined, retired.
offsetNoRows to skip. Default 0.
session_idYesSession id, e.g. ses_01J...

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the ordering by Elo, the exact fields returned, and the deliberate omission of bodies. It does not explicitly mention pagination behavior or read-only status, but these are inferable from the 'List' verb and the limit/offset parameters in the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that states the action, scope, ordering, and fields returned. The following note about omitted bodies and the alternative tool adds essential guidance without waste. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list tool with four parameters and no output schema, the description is nearly complete: it specifies the return field set, ordering rule, and key omission. It could have explicitly described pagination behavior, but limit/offset parameters in the schema cover that. Overall, it provides sufficient context for an AI agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description does not add further meaning to individual parameters like session_id, limit, state, or offset beyond what the schema already provides. It only restates the overall purpose rather than detailing parameter nuances.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists a session's hypotheses ranked by Elo, enumerating specific fields (title, summary, state, matches played, dedup cluster). It distinguishes itself from hypothesis_get by noting bodies are omitted, and from session_list by focusing on hypotheses within a session.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an explicit alternative: 'Bodies are omitted; use hypothesis_get.' This communicates a clear when-not scenario (when bodies are needed) and points to the correct sibling tool. The context implies this is the go-to for summary listings of hypotheses per session.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hypothesis_getA

One hypothesis in full: statement, detailed text, citations, Elo history, every review recorded against it, and its tournament matches. Read this before reviewing or ranking it.

NameRequiredDescriptionDefault
hypothesis_idYesHypothesis id, e.g. hyp_9f2c...
include_matchesNoInclude the match list. Default true.

TDQS

A3.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full burden of behavioral disclosure. It does not mention any side effects, permissions, rate limits, or potential large data transfer. The only behavioral hint is that it returns a comprehensive object, and the instruction to read it before reviewing implies it is read-only, but this is not explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, consisting of two short sentences. It is front-loaded with the core purpose ('One hypothesis in full') and lists the included content without unnecessary elaboration. There is no fluff or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description provides a good overview of what the tool returns, enumerating the fields included. It also gives usage context. However, it does not mention any limitations or optional behaviors (such as how include_matches affects the response), though the schema handles that. Overall, it is fairly complete for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides clear descriptions for both parameters (hypothesis_id and include_matches). The tool description does not add any additional meaning or context about these parameters; it focuses solely on the output. Since schema coverage is 100%, the baseline is 3, and the description adds no extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool returns a comprehensive single hypothesis object including all relevant details. It explicitly mentions the content (statement, detailed text, citations, Elo history, reviews, tournament matches) and provides a clear use case ('Read this before reviewing or ranking it'). This distinguishes it from sibling tools that likely list or retrieve summaries.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a direct instruction to read this before reviewing or ranking, indicating when to use the tool. However, it does not explicitly mention alternative tools or specify when not to use it, though the context of 'full hypothesis' implies it is the comprehensive retrieval option.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hypothesis_recordA

Add a hypothesis to a session, in the same table Generation and Evolution write to. It enters as a draft, gets reviewed, and then competes in the tournament like any other. Set parent_ids when it derives from existing hypotheses, which is what makes it an Evolution result rather than a fresh idea. Recording is what makes a hypothesis real: an idea only described in chat is never reviewed, ranked, or seen by the meta-review.

NameRequiredDescriptionDefault
titleYesOne line naming the mechanism.
summaryYesAbout three sentences. This is the text that gets embedded for dedup and clustering, so make it the mechanism, not a teaser.
strategyYesHow it was produced. literature or debate for a fresh idea; combine, simplify, feasibility or out_of_box for an evolution.
citationsNoSources behind it. Cite only what you read.
full_textYesThe hypothesis in full, markdown, written for a domain expert: mechanism, predictions, and the experiment that would test it.
parent_idsNoHypotheses this derives from. Omit for a fresh idea.
session_idYesSession id, e.g. ses_01J...

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the behavior (enters as draft, gets reviewed, competes in tournament) and the significance of recording (makes it real). It also mentions that it writes to the same table as Generation and Evolution, adding context. Minor omissions like permission requirements or error handling are not critical for this tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured paragraph that front-loads the purpose, then explains lifecycle and key parameter usage. Every sentence adds value, no fluff. It is concise yet sufficiently detailed, earning a high score though not as terse as the calibration example.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 7 parameters (5 required) and no output schema, the description provides enough context for the agent to invoke correctly. It explains the tool's role in the hypothesis workflow, the meaning of key fields, and the distinction between fresh vs evolved hypotheses. The rich schema descriptions complement the text, making the overall documentation complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although the input schema already provides descriptions for all parameters (100% coverage), the tool description adds nuanced guidance beyond the schema: it explains the meaning of parent_ids (derivation vs fresh idea) and strategy (how produced, linking to evolution), and emphasizes that summary is used for dedup/clustering. This enriches parameter understanding significantly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Add a hypothesis to a session' and explains its lifecycle (draft, reviewed, tournament). It also distinguishes from siblings like hypotheses_list and hypothesis_get (read tools) and review_record/match_record (review/match tools). The verb-resource pairing is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear guidance on when to use parent_ids ('Set parent_ids when it derives from existing hypotheses') and explains the consequence of not recording ('an idea only described in chat is never reviewed'). It implies this is the proper way to register a hypothesis versus just chatting, and mentions the strategy enum values for fresh vs evolution. It lacks explicit 'when not to use' but offers enough context for correct usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

match_recordA

Record the outcome of a head-to-head comparison between two hypotheses and apply the Elo update. Compare them on the paper's criteria (correctness, novelty, testability, and how well each survives its reviews) and give the rationale. Idempotent per (pair, round_id): recording the same pair twice under the same round moves no Elo.

NameRequiredDescriptionDefault
modeNopairwise = single comparison. debate = multi-turn scientific debate. Default pairwise.
hyp_aYesFirst hypothesis id.
hyp_bYesSecond hypothesis id.
winnerYesWhich one wins. There are no draws.
round_idNoRound label for idempotency. Default 'manual'. Change it to rematch a pair deliberately.
rationaleYesWhy it won, in the terms the criteria are stated in.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses key side effects: 'apply the Elo update' and idempotency per (pair, round_id). Since no annotations are provided, the description carries the full burden, and it does so well by noting repeat recordings move no Elo. It doesn't mention return format, but that's not a critical gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the main purpose, followed by criteria guidance and idempotency note. Every sentence earns its place with no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no annotations and no output schema, the description covers purpose, comparison criteria, rationale expectations, and idempotency behavior. The mode parameter is documented in the schema, so not repeating it is fine. This is sufficient for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by specifying what 'rationale' should contain (criteria-based reasoning) and clarifying that round_id controls idempotency/rematching. This extra context lifts it to 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'Record the outcome of a head-to-head comparison between two hypotheses and apply the Elo update.' This clearly distinguishes the tool from sibling record tools like hypothesis_record and review_record, which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The context is clear: use this when you need to record a comparison between two hypotheses and update their Elo ratings. It doesn't explicitly name alternatives or state when not to use it, but the head-to-head comparison context is distinct enough. The rationale guidance ('in the terms the criteria are stated in') also adds practical usage direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

overview_getA

The final research overview for a session: the meta-review synthesis of the ranked hypotheses. Only exists once the session has finalized.

NameRequiredDescriptionDefault
session_idYesSession id, e.g. ses_01J...

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It adds one useful trait (the existence condition), but does not explain what happens if the session hasn't finalized (e.g., error vs. empty response), nor does it confirm read-only semantics or describe the output structure. Some transparency is present, but significant gaps remain.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with two concise clauses. It delivers the essential purpose and a key condition without any fluff. Every word earns its place, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description gives a reasonable high-level idea of the return content ('meta-review synthesis'), but it omits details like the expected structure or format of the overview, and does not clarify behavior when the precondition is unmet. It is adequate but not comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents the single parameter (session_id) with an example. The description adds no additional meaning to the parameter. Since schema coverage is 100%, the baseline is 3; the description neither enhances nor detracts from parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'The final research overview for a session: the meta-review synthesis of the ranked hypotheses.' This is specific and distinguishes it from siblings like hypotheses_list (which lists individual hypotheses) and hypothesis_get (fetches a single hypothesis). The resource is explicit, and the verb 'get' is implied by the name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear usage condition: 'Only exists once the session has finalized.' This tells the agent when the tool is appropriate to call. It doesn't explicitly name alternatives or exclusions, but the condition effectively implies it should be used only after finalization. This is sufficient guidance for a simple retrieval tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

review_recordA

Record a review against a hypothesis, in the same table and format the Reflection agent writes. Every factual claim in evidence needs a url and an excerpt you actually read. A 'full' review on a draft hypothesis also promotes it to reviewed. Do not report a review to the user without recording it first, or the tournament and meta-review will never see it.

NameRequiredDescriptionDefault
bodyYesThe review itself, markdown.
kindYesfull = novelty, correctness, testability. verification = assumption decomposition. observation = does it explain a published observation. simulation = step-through of the mechanism.
scoresNo
verdictNo
evidenceNoSources backing the review. Cite only what you read.
iterationNoDisambiguates repeat reviews of the same kind. Omit and the server picks the next free slot.
assumptionsNoDeep-verification decomposition, one row per assumption.
hypothesis_idYesHypothesis id, e.g. hyp_9f2c...

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that evidence requires a url and excerpt, that a 'full' review promotes a draft to reviewed, and the consequence of not recording first. These are behavioral traits beyond a simple 'record' action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with purpose, then critical constraints. No fluff, each sentence adds essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 8 params, nested objects, and no output schema, the description covers the key edge cases: evidence provenance, promotion logic, and ordering. It gives enough context for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 75%, so baseline is 3. The description adds value by explaining evidence requirements (url+excerpt you actually read) and the effect of 'full' kind. This goes beyond the schema's per-field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States exactly what it does: 'Record a review against a hypothesis'. Specifies verb+resource, and adds context about format and side-effect (promotes to reviewed). Clearly distinct from siblings like 'hypothesis_record' which record hypotheses, not reviews.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context: use it when you have a review to record, and gives a critical ordering constraint ('Do not report a review to the user without recording it first'). Does not explicitly name alternatives, but the purpose is self-evident and the constraint is practical.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_abortA

Abort a session for good. The main loop exits at its next check.

NameRequiredDescriptionDefault
session_idYesSession id, e.g. ses_01J...

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden. It adds 'for good' indicating irreversibility and 'at its next check' showing deferred execution, which are useful behavioral details beyond the name alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with no wasted words. The action is front-loaded, and the second sentence adds a key behavioral detail without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description explains the purpose and the timing of the effect. It does not describe return values, but that is not critical given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for session_id, and the description does not add any parameter-specific meaning. Baseline 3 applies because the schema already provides adequate documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Abort a session for good', which is a specific verb + resource. The phrase 'for good' distinguishes this from session_pause or session_resume, clarifying a permanent termination.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description conveys that this is for permanent termination and notes the main loop exits at a check, providing context for when to use it. It does not explicitly exclude temporary pauses, but the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_listA

List sessions newest first, with status, hypothesis count, top Elo and budget spent. Use this to find a session id the user refers to by topic.

NameRequiredDescriptionDefault
limitNoMax rows. Default 20.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the burden. It discloses that sessions are listed newest first and includes specific fields, but it does not explicitly state it is read-only or mention any side effects, pagination behavior, or error cases. Basic safety profile is implied by 'List' but not explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and details, then a practical usage hint. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with one parameter and no output schema, the description covers the returned fields, ordering, and a use case. It lacks explicit mention of pagination or result limits beyond the limit parameter, but overall it is sufficiently complete for the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers 100% of the single parameter (limit) with a clear description ('Max rows. Default 20.'). The tool description adds no additional parameter semantics, so it meets but does not exceed the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists sessions with specific fields (status, hypothesis count, top Elo, budget spent) and sorting (newest first). It is distinct from siblings like session_status or session_pause, and the usage hint further clarifies its purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use it: 'to find a session id the user refers to by topic.' It does not mention alternative tools for exclusion, but the use case is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_pauseA

Pause a running session. Workers drain and the loop sleeps until resumed.

NameRequiredDescriptionDefault
session_idYesSession id, e.g. ses_01J...

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the behavioral disclosure burden and does so well by explaining what happens: workers drain and the loop sleeps until resumed. It implies reversibility and non-destructiveness, though it does not state error behavior for already-paused sessions or permissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short, front-loaded sentences. The first states the core purpose, and the second adds essential behavioral detail with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with no annotations and no output schema, the description provides enough context: it names the action, the precondition, and the resulting behavior. It does not explain return values, but the tool's simplicity and clear behavior make the description reasonably complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already fully documents the only parameter, session_id, with type, requirement, and example format. The description adds no further parameter-specific meaning, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool pauses a running session, using a specific verb and resource. It distinguishes itself from sibling tools like session_resume and session_abort by describing a reversible pause with workers draining and the loop sleeping until resumed.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Pause a running session' provides clear context for when to use the tool, and 'until resumed' implies the resume counterpart without explicitly naming session_resume. It lacks explicit when-not-to-use guidance or alternative tool names, but the intended usage is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_resumeA

Resume a paused or interrupted session. The engine reclaims expired task leases and continues where it stopped. Returns immediately; poll session_status.

NameRequiredDescriptionDefault
session_idYesSession id, e.g. ses_01J...

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that the engine reclaims expired task leases and continues where it stopped, and that it returns immediately, requiring polling of session_status. This gives insight into behavior, though it lacks details on error handling or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise—two sentences—and directly conveys the essential information without unnecessary filler. It is well-structured and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema, the description explains the immediate return and the need to poll session_status. It also explains the lease reclaiming behavior. It could be more complete about success/failure conditions, but for this simple tool it is adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'session_id' is well-described in the schema with an example, so the schema coverage is high. The tool description does not add extra information, but it is not needed given the clear schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool resumes a paused or interrupted session. It is specific and distinguishes this tool from siblings like session_start, session_pause, and session_abort.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description indicates when to use it (for resuming sessions) but does not explicitly mention alternatives or when not to use it. It does provide a hint that it returns immediately and to poll session_status, which implies a usage pattern.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_startA

Start a new research session from a natural-language goal. The engine runs the full Generation, Reflection, Elo tournament, Evolution and Meta-review loop in a detached process; this returns as soon as the session id exists. Poll session_status for progress. It runs for minutes to hours, so agree the scale with the user first: call estimate_cost.

NameRequiredDescriptionDefault
goalYesThe research goal. Write it as a scientist would: the system, the phenomenon, and the kind of answer wanted.
n_initialNoParallel initial Generation calls. Default 3.
budget_usdNoAdvanced, and rarely needed. On a subscription backend this caps an equivalent-cost gauge, not a bill. Leave it unset unless the user asks about spend.
concurrencyNoOverride worker concurrency.
preferencesNoOptional extra constraints or preferences, free text.
wall_clock_secondsNoOverride the wall-clock cap.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It states the detached process, early return after session id exists, and duration of minutes to hours. This is valuable context. It does not cover failure modes or edge cases, but for a session start, the disclosed behavior is sufficient to set expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded. It opens with the core purpose, then explains the asynchronous behavior, and finishes with practical instructions. Every sentence contributes value; no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity—a long-running asynchronous process with multiple internal stages—the description adequately covers the key context: detached execution, early return, polling, and cost estimation. It does not describe the output or error handling, but since there is no output schema and the tool is a starter, the provided context is sufficient for typical usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (all 6 parameters have descriptions), so the schema already provides parameter semantics. The tool description does not add extra details about parameters beyond what the schema offers. Per the baseline rule for high coverage, a score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb and resource: 'Start a new research session' from a natural-language goal. It distinguishes from siblings like session_status and session_resume by focusing on the initiation action and describing the long-running process. The purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly directs the agent to poll session_status for progress and to call estimate_cost to agree on scale before starting. It also implicitly distinguishes from session_resume by focusing on new sessions. These explicit references to sibling tools provide strong usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_statusA

Progress for one session: status, task counts by state, hypothesis counts by state, matches played, Elo spread, budget spent, and whether the final overview exists yet. This is the polling tool.

NameRequiredDescriptionDefault
session_idYesSession id, e.g. ses_01J...

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden of behavioral transparency. It indicates that the tool returns status information (status, counts, Elo spread, budget, existence of final overview) but does not disclose whether it is read-only (presumably yes), any side effects, or whether it blocks. It provides useful detail on the returned data but could be more explicit about non-mutating behavior, especially given it is a polling tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that efficiently enumerates all key metrics and ends with a clear purpose statement. Every part contributes value, and it is front-loaded with the core purpose. No fluff, perfect length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity (single parameter, no output schema) and the rich list of returned metrics in the description, the description is fairly complete for an agent to understand what to expect. It lacks details on return format or error handling, but for a polling tool, this is sufficient. The provided info covers the essential data points, so a 4 is warranted.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100% (session_id is described as 'Session id, e.g. ses_01J...'). The description adds no additional parameter semantics beyond that. Since the schema already covers the parameter, the baseline is 3, and the description does not add extra meaning, so this score is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: reporting progress for a single session, with specific metrics listed. It distinguishes itself from siblings like session_list (which lists sessions) and overview_get (which gets the final overview) by focusing on session status polling. It could be more explicit about being a read-only status check, but the purpose is clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states 'This is the polling tool,' which is a clear usage guideline for when to use this tool (for polling status). It does not explicitly say when not to use it or name alternatives, but it implies a monitoring context rather than a control context, distinguishing it from session_pause/abort/start. The context is somewhat clear, but exclusions are not spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 15 tool updatesv0.1.0
    • First observeddoctor
    • First observedestimate_cost
    • First observedfeedback_add
    • First observedhypotheses_list
    • First observedhypothesis_get
    • First observedhypothesis_record
    • First observedmatch_record
    • First observedoverview_get
    • First observedreview_record
    • First observedsession_abort
    • First observedsession_list
    • First observedsession_pause
    • First observedsession_resume
    • First observedsession_start
    • First observedsession_status

TDQS

A4.1/5.0

Scored across 15 tools

Disambiguation4/5

Tools are mostly distinct by resource and action (sessions vs hypotheses vs reviews vs matches). Minor overlap between session_pause/session_abort and hypothesis_record/review_record, but descriptions are detailed enough to avoid misselection.

Naming Consistency4/5

Most tools follow a noun_verb pattern (session_list, hypotheses_list, hypothesis_get, review_record), but session_resume, session_start, session_pause, session_abort are verb-first. doctor breaks the pattern entirely. Consistent within each resource group, mixed across groups.

Tool Count5/5

15 tools for a complex scientific research orchestration server. Each tool maps to a distinct lifecycle step: session management (7), hypothesis operations (4), review/match/feedback (3), overview (1), cost estimation (1), diagnostics (1). Well-scoped for the complexity.

Completeness4/5

Covers the full session lifecycle (start, poll, pause, resume, abort), hypothesis CRUD (list, get, record), review and tournament operations. Missing explicit update/delete for hypotheses or sessions, but the domain (adding to a persistent tournament) doesn't require them. Overview_get exists only after finalization; no explicit wait/finalize tool but polling covers it.

Maintenance

ActivityInactive
ResponsivenessNo recent community issues

Related MCP Connectors

Related MCP Servers