Skip to main content
Glama
Ahmad-Jaradat-Space

co-scientist-plugin

Плагин co-scientist

Запускайте мультиагентный научный движок прямо из вашего агента.

Пакет Agent Plugins 1.0, который позволяет Claude Code, Codex CLI, Cursor, VS Code, Copilot и Kiro управлять движком AI co-scientist: генерировать гипотезы, рецензировать их, проводить турнир по системе Эло, развивать лучшие и получать итоговую сводку.

tests Python 3.11 to 3.13 License: Apache 2.0 Agent Plugins 1.0

Установка · Что получает агент · Принцип проектирования · Совместимость с ScientistOS · Вклад


Идея

Движок AI co-scientist — это открытая реализация подхода из статьи Gottweis et al., Nature, 2026. Он берёт исследовательскую цель и прогоняет её через шесть специализированных агентов: генерацию, рефлексию, ранжирование, эволюцию, оценку близости и мета-рецензирование, управляемых супервизором поверх долговечной очереди задач в SQLite.

У движка есть CLI. Этот пакет даёт ему агент-ориентированный интерфейс, чтобы управлять им мог Claude Code или Codex, а не вы, набирая команды:

you    → "review hypothesis 3 in that Crohn's session and tell me what breaks"
agent  → reads the hypothesis, searches the literature, applies the rubric,
         records the review into the engine's database, reports the verdict

Написанный им обзор теперь — строка в той же таблице, куда пишет агент рефлексии самого движка. Он учитывается в турнире. Мета-рецензент его увидит.

Related MCP server: multivon-mcp

Что получает агент

Один MCP-сервер с пятнадцатью инструментами:

Группа

Инструменты

Жизненный цикл

session_start · session_resume · session_pause · session_abort · session_list · session_status

Чтение

hypotheses_list · hypothesis_get · overview_get

Запись

hypothesis_record · review_record · match_record · feedback_add

Диагностика

estimate_cost · doctor

Семь навыков, несущих суждения, которые инструменты дать не могут:

Навык

Что он делает

running-research-sessions

Оттачивает расплывчатую цель до настоящей, контролирует масштаб запуска, запускает и отслеживает его

reading-research-overviews

Вдумчиво читает ранжированные результаты, проверяя пять известных режимов отказа

reviewing-hypotheses

Рецензирует по рубрике с якорями оценок и правилом вынесения вердикта, затем записывает результат

ranking-hypotheses

Проводит турнир из трёх-пяти раундов, решает по упорядоченным критериям, фиксирует матч по Эло

evolving-hypotheses

Комбинирует, упрощает, проверяет на реализуемость и нестандартность, каждая — со своим эталонным файлом и режимом отказа

benchmarking-models

Сравнивает модели в турнире по Эло и учит, как читать результат, не делая необоснованных выводов

grounding-claims

Проверяет, что цитаты действительно подтверждают заявленные утверждения, по каждому — вердикт

Принцип проектирования

Движок хранит всё состояние: Эло, очередь задач, дедупликацию, эмбеддинги, бюджет токенов, завершение. Этот пакет ничего из этого не переimplement'ирует.

Всё, что создаёт агент, возвращается в те же репозитории, куда пишут собственные агенты движка. match_record переиспользует модуль Эло движка и его идемпотентность через elo_journal. review_record пишет ту же модель Review и переводит статус из draft в reviewed точно так же, как это делает рефлексия. hypothesis_record выводит created_by из того, были ли указаны родительские гипотезы.

Навыки формулируют следствие простыми словами: то, что не записано, не существует. Агент, который описал рецензию в чате, не вызвав review_record, не произвёл ничего, что система могла бы использовать.

Это предотвращает типичный сбой: «перенести промпты в навыки и считать задачу выполненной». Промпты — не там, где рождается качество. Качество рождается в турнире, а турниру нужно место, где ведётся счёт.

Совместимость с ScientistOS

Этот плагин генерирует и ранжирует гипотезы-кандидаты. Он не решает, что вам можно утверждать на их основе, — и в этом разница: гипотеза может выиграть турнир и при этом опираться на механизм, которому противоречит литература. Навык reading-research-overviews перечисляет этот и другие частые режимы отказа, которые стоит проверить вручную.

ScientistOS превращает эту проверку из рекомендации в обязательное требование. Он хранит структурированный граф доказательств и отказывается фиксировать утверждение, если его обоснованность не подтверждается происхождением данных.

this plugin   →  ranked hypotheses, reviews with claim / url / excerpt per source
                              ↓
ScientistOS   →  hypothesis and source nodes, evidence at its honest level,
                 an integrity gate before anything gets written up

Оба проекта устанавливаются рядом: это независимые навык и MCP-пакет, и агенту с обоими доступна полная картина. При этом ScientistOS не является обязательной зависимостью — плагин работает и без него.

Установка

Никаких ключей API и никаких платежей. Если вы уже вошли в Claude Code или Codex CLI, этого достаточно: исследовательский движок проводит свои агентские сессии через ту же подписку, что и ваш кодинг-агент.

python3.12 -m venv .venv
source .venv/bin/activate

pip install "co-scientist @ git+https://github.com/Kaimen-Inc/Co-Scientist.git"
pip install "co-scientist-plugin @ git+https://github.com/Ahmad-Jaradat-Space/co-scientist-plugin.git"

co-scientist init

При первом подключении сервер ищет рабочий бэкенд. Не найдя, он берёт тот CLI, который установлен и выполнил вход, записывает выбор в ~/.co-scientist/config.toml и продолжает работу. Строка provider = никогда не перезаписывается — ваша ручная настройка останется нетронутой. Чтобы полностью отключить автовыбор, установите COSCI_AUTO_BACKEND=0.

Если doctor по-прежнему сообщает об отсутствии бэкенда, войдите в CLI вручную (claude или codex login) — ключ не понадобится. doctor также проверит локальный эмбеддер; его отсутствие — нормальное состояние без ключей: дедупликация ловит повторные формулировки, но не различит две гипотезы, которые говорят одно и то же разными словами.

Все данные остаются на вашей машине: SQLite в каталоге ./data, никаких аккаунтов, никаких сервисов, никакого трафика, кроме вызовов моделей, которые ваш CLI и так уже совершает.

config/subscription.toml — тот же выбор, но сделанный вручную: идентификаторы моделей зафиксированы явно, а не предоставлены алиасу, который разрешает ваш CLI. Укажите движку путь к нему через export COSCI_CONFIG=/path/to/config/subscription.toml.

Подойдёт и обычный API с оплатой за использование: укажите провайдера и ключ в ~/.co-scientist/config.toml до первого запуска, и автовыбор не вмешается. estimate_cost затем оценит расходы на сессию относительно вашего бюджета, а навыки запросят подтверждение на траты, а не просто продолжат. Подробнее — в руководстве по бэкендам движка.

Codex CLI

codex plugins install ./co-scientist-plugin

Claude Code

ln -s "$PWD/skills/"* ~/.claude/skills/
claude mcp add co-scientist -- co-scientist-mcp

Любой другой клиент

Любой клиент, поддерживающий Agent Plugins 1.0, загрузит каталог как есть. Любой клиент, читающий SKILL.md и MCP-конфиг по отдельности, может взять skills/ и mcp.json самостоятельно.

Конфигурация

Сервер читает конфиг движка. При необходимости задайте эти переменные в блоке env вашего mcp.json:

Переменная

Эффект

COSCI_CONFIG

Путь к дополнительному TOML-конфигу для наложения

COSCI_ALLOW_START

0 запрещает session_start и session_resume

COSCI_START_TIMEOUT

Секунд ожидания регистрации запущенного прогона, по умолчанию 120

COSCI_AUTO_BACKEND

0 отключает автовыбор вошедшего CLI, если ничего не настроено

Установки только для чтения

Сессия совершает сотни вызовов моделей и продолжает работать после завершения разговора. На общей машине — или везде, где агенту нельзя позволить запускать сессии по своей инициативе:

co-scientist-mcp --read-only

Чтение продолжает работать; session_start и session_resume отказывают с объяснением причины.

Пути к виртуальному окружению

mcp.json вызывает co-scientist-mcp, предполагая, что консольный скрипт находится в PATH. Если ваш агент не активирует виртуальное окружение, используйте абсолютный путь:

{
  "command": "/path/to/.venv/bin/co-scientist-mcp",
  "args": []
}

Разработка

pip install -e ".[dev]"
ruff check coscientist_plugin tests
pytest -q
co-scientist-plugin-validate

Каждый тест работает офлайн, на одноразовой базе данных. Никаких ключей, сети или вызовов моделей.

Пакет защищают три уровня проверок:

  • co-scientist-plugin-validate проверяет соответствие спецификации: десять разрешённых полей манифеста, типы транспорта, frontmatter навыков, согласование имени и каталога, длину тела и то, что все относительные ссылки разрешаются и остаются на один уровень вглубь.

  • tests/test_plugin.py проверяет то, что валидатор не может: ни один навык не велит агенту вызывать несуществующий MCP-инструмент или команду движка, а дисциплина записи действительно прописана в навыках, которые пишут.

  • tests/test_server.py покрывает инварианты, на которых стоит дизайн: нулевая сумма Эло, идемпотентность раунда, отказ при пересечении сессий, отказ на свидетельство без выдержки и переходы состояний, которые никогда не откатывают ранжированную гипотезу назад.

Отношение к движку

Этот репозиторий — только слой плагина. Движок, которым он управляет, живёт в Kaimen-Inc/Co-Scientist и является отдельным проектом под той же лицензией. См. NOTICE.

Ни один из проектов не аффилирован с Google или с авторами статьи. Гипотезы, которые они порождают, — отправная точка для учёного, а не научный результат.

Лицензия

Apache-2.0. См. LICENSE и NOTICE.

Available Tools

15 tools
doctorA

Check the install: which LLM backend is configured, whether its credentials resolve, whether embeddings are real or the hash fallback, and where the database lives. Run this first when anything fails.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It lists what is checked but does not disclose whether the tool has side effects, what the output looks like, or what 'hash fallback' implies. It gives partial transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, with a clear front-loaded action ('Check the install') and a structured list of what is checked. Every sentence earns its place with no waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no parameters and no output schema, the description covers the key context: what it checks and when to run it. The lack of return format details is a minor gap, but it's adequate for selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the empty schema provides full coverage. Baseline 4 is appropriate since there are no parameter semantics to explain.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states what the tool does: it checks the install configuration, specifying backend, credentials, embeddings, and database location. This is a distinct diagnostic function, differentiating it from sibling session/review tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides an explicit trigger: 'Run this first when anything fails.' While it doesn't mention exclusions or alternatives, the directive is clear and actionable for a diagnostic tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

estimate_costA

Pre-flight size of a session under the current config: calls and tokens per agent, and how long it may run. On a metered API backend it also prices it. Run this before session_start and show the user.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description must carry the behavioral disclosure burden. It explains scope ('under the current config'), the kind of estimate provided, and the conditional pricing behavior on metered backends. Still, it does not explicitly state whether the operation is read-only, what side effects it has, or how precise the estimate is.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler. The key action, output content, conditional behavior, and usage instruction are all front-loaded and clearly expressed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no params, no annotations, and no output schema, the description covers the essential inputs ('current config'), outputs (calls/tokens/duration/price), and timing ('before session_start'). It is slightly ambiguous what 'current config' refers to, but it is adequate for this tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is no schema coverage gap to compensate for. The description adds meaning by explaining that the estimate is based on 'the current config' and what is being measured.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies a specific verb-like action ('Pre-flight size of a session') and details exactly what it produces: calls, tokens per agent, duration, and price on metered backends. It also distinguishes itself from siblings by explicitly framing it as a pre-`session_start` step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit guidance on when to use the tool: 'Run this before session_start and show the user.' However, it does not mention when not to use it or name alternatives from the sibling list, so it stops short of full exclusion guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

feedback_addA

Inject researcher feedback into a session. A running session picks it up on its next planning step; a paused one picks it up on resume. kind='pin' or 'rejection' with a target also moves that hypothesis's state.

NameRequiredDescriptionDefault
kindNo
textYesThe feedback, free text.
targetNoHypothesis id this is about.
session_idYesSession id, e.g. ses_01J...

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It transparently states the timing of feedback delivery (next planning step or on resume) and the special effect of kind='pin' or 'rejection' with a target on hypothesis state. This goes beyond the schema. However, it does not mention potential failure modes or prerequisites like session existence, leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences with no wasted words. It front-loads the primary purpose and immediately provides the timing and state-change nuance. Every clause adds value, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (session state dependency, hypothesis mutation) and the lack of annotations or output schema, the description covers the key behavioral aspects. It could be more complete by specifying that the session must exist or what happens on invalid input, but it is adequate for a tool with this scope. The absence of return value documentation is acceptable since no output schema is provided.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 75% (three of four parameters have descriptions; kind only has an enum). The description adds semantic value by explaining the interaction between kind and target for moving hypothesis state, which is not present in the schema. This compensates for the missing description of kind, justifying a score above the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool injects researcher feedback into a session, with a specific verb and resource. It differentiates from siblings by focusing on feedback injection rather than session control (like session_pause) or hypothesis recording (like hypothesis_record). The mention of how it affects hypothesis state further clarifies its unique role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear contextual usage: it explains when the feedback takes effect (running vs paused sessions). It does not explicitly name alternative tools or state when not to use it, but the behavioral context is sufficient for an agent to infer the appropriate use case. Since sibling tools like review_record or match_record are not referenced for exclusion, this is not a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hypotheses_listA

List a session's hypotheses ranked by Elo, with title, summary, state, matches played and dedup cluster. Bodies are omitted; use hypothesis_get.

NameRequiredDescriptionDefault
limitNoMax rows. Default 20.
stateNoFilter by state: draft, reviewed, in_tournament, pinned, rejected, quarantined, retired.
offsetNoRows to skip. Default 0.
session_idYesSession id, e.g. ses_01J...

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the ordering by Elo, the exact fields returned, and the deliberate omission of bodies. It does not explicitly mention pagination behavior or read-only status, but these are inferable from the 'List' verb and the limit/offset parameters in the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that states the action, scope, ordering, and fields returned. The following note about omitted bodies and the alternative tool adds essential guidance without waste. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a list tool with four parameters and no output schema, the description is nearly complete: it specifies the return field set, ordering rule, and key omission. It could have explicitly described pagination behavior, but limit/offset parameters in the schema cover that. Overall, it provides sufficient context for an AI agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description does not add further meaning to individual parameters like session_id, limit, state, or offset beyond what the schema already provides. It only restates the overall purpose rather than detailing parameter nuances.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists a session's hypotheses ranked by Elo, enumerating specific fields (title, summary, state, matches played, dedup cluster). It distinguishes itself from hypothesis_get by noting bodies are omitted, and from session_list by focusing on hypotheses within a session.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an explicit alternative: 'Bodies are omitted; use hypothesis_get.' This communicates a clear when-not scenario (when bodies are needed) and points to the correct sibling tool. The context implies this is the go-to for summary listings of hypotheses per session.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hypothesis_getA

One hypothesis in full: statement, detailed text, citations, Elo history, every review recorded against it, and its tournament matches. Read this before reviewing or ranking it.

NameRequiredDescriptionDefault
hypothesis_idYesHypothesis id, e.g. hyp_9f2c...
include_matchesNoInclude the match list. Default true.

TDQS

A3.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full burden of behavioral disclosure. It does not mention any side effects, permissions, rate limits, or potential large data transfer. The only behavioral hint is that it returns a comprehensive object, and the instruction to read it before reviewing implies it is read-only, but this is not explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, consisting of two short sentences. It is front-loaded with the core purpose ('One hypothesis in full') and lists the included content without unnecessary elaboration. There is no fluff or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description provides a good overview of what the tool returns, enumerating the fields included. It also gives usage context. However, it does not mention any limitations or optional behaviors (such as how include_matches affects the response), though the schema handles that. Overall, it is fairly complete for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides clear descriptions for both parameters (hypothesis_id and include_matches). The tool description does not add any additional meaning or context about these parameters; it focuses solely on the output. Since schema coverage is 100%, the baseline is 3, and the description adds no extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool returns a comprehensive single hypothesis object including all relevant details. It explicitly mentions the content (statement, detailed text, citations, Elo history, reviews, tournament matches) and provides a clear use case ('Read this before reviewing or ranking it'). This distinguishes it from sibling tools that likely list or retrieve summaries.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a direct instruction to read this before reviewing or ranking, indicating when to use the tool. However, it does not explicitly mention alternative tools or specify when not to use it, though the context of 'full hypothesis' implies it is the comprehensive retrieval option.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hypothesis_recordA

Add a hypothesis to a session, in the same table Generation and Evolution write to. It enters as a draft, gets reviewed, and then competes in the tournament like any other. Set parent_ids when it derives from existing hypotheses, which is what makes it an Evolution result rather than a fresh idea. Recording is what makes a hypothesis real: an idea only described in chat is never reviewed, ranked, or seen by the meta-review.

NameRequiredDescriptionDefault
titleYesOne line naming the mechanism.
summaryYesAbout three sentences. This is the text that gets embedded for dedup and clustering, so make it the mechanism, not a teaser.
strategyYesHow it was produced. literature or debate for a fresh idea; combine, simplify, feasibility or out_of_box for an evolution.
citationsNoSources behind it. Cite only what you read.
full_textYesThe hypothesis in full, markdown, written for a domain expert: mechanism, predictions, and the experiment that would test it.
parent_idsNoHypotheses this derives from. Omit for a fresh idea.
session_idYesSession id, e.g. ses_01J...

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the behavior (enters as draft, gets reviewed, competes in tournament) and the significance of recording (makes it real). It also mentions that it writes to the same table as Generation and Evolution, adding context. Minor omissions like permission requirements or error handling are not critical for this tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured paragraph that front-loads the purpose, then explains lifecycle and key parameter usage. Every sentence adds value, no fluff. It is concise yet sufficiently detailed, earning a high score though not as terse as the calibration example.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 7 parameters (5 required) and no output schema, the description provides enough context for the agent to invoke correctly. It explains the tool's role in the hypothesis workflow, the meaning of key fields, and the distinction between fresh vs evolved hypotheses. The rich schema descriptions complement the text, making the overall documentation complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although the input schema already provides descriptions for all parameters (100% coverage), the tool description adds nuanced guidance beyond the schema: it explains the meaning of parent_ids (derivation vs fresh idea) and strategy (how produced, linking to evolution), and emphasizes that summary is used for dedup/clustering. This enriches parameter understanding significantly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Add a hypothesis to a session' and explains its lifecycle (draft, reviewed, tournament). It also distinguishes from siblings like hypotheses_list and hypothesis_get (read tools) and review_record/match_record (review/match tools). The verb-resource pairing is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear guidance on when to use parent_ids ('Set parent_ids when it derives from existing hypotheses') and explains the consequence of not recording ('an idea only described in chat is never reviewed'). It implies this is the proper way to register a hypothesis versus just chatting, and mentions the strategy enum values for fresh vs evolution. It lacks explicit 'when not to use' but offers enough context for correct usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

match_recordA

Record the outcome of a head-to-head comparison between two hypotheses and apply the Elo update. Compare them on the paper's criteria (correctness, novelty, testability, and how well each survives its reviews) and give the rationale. Idempotent per (pair, round_id): recording the same pair twice under the same round moves no Elo.

NameRequiredDescriptionDefault
modeNopairwise = single comparison. debate = multi-turn scientific debate. Default pairwise.
hyp_aYesFirst hypothesis id.
hyp_bYesSecond hypothesis id.
winnerYesWhich one wins. There are no draws.
round_idNoRound label for idempotency. Default 'manual'. Change it to rematch a pair deliberately.
rationaleYesWhy it won, in the terms the criteria are stated in.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses key side effects: 'apply the Elo update' and idempotency per (pair, round_id). Since no annotations are provided, the description carries the full burden, and it does so well by noting repeat recordings move no Elo. It doesn't mention return format, but that's not a critical gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the main purpose, followed by criteria guidance and idempotency note. Every sentence earns its place with no redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no annotations and no output schema, the description covers purpose, comparison criteria, rationale expectations, and idempotency behavior. The mode parameter is documented in the schema, so not repeating it is fine. This is sufficient for an agent to invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds value beyond the schema by specifying what 'rationale' should contain (criteria-based reasoning) and clarifying that round_id controls idempotency/rematching. This extra context lifts it to 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'Record the outcome of a head-to-head comparison between two hypotheses and apply the Elo update.' This clearly distinguishes the tool from sibling record tools like hypothesis_record and review_record, which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The context is clear: use this when you need to record a comparison between two hypotheses and update their Elo ratings. It doesn't explicitly name alternatives or state when not to use it, but the head-to-head comparison context is distinct enough. The rationale guidance ('in the terms the criteria are stated in') also adds practical usage direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

overview_getA

The final research overview for a session: the meta-review synthesis of the ranked hypotheses. Only exists once the session has finalized.

NameRequiredDescriptionDefault
session_idYesSession id, e.g. ses_01J...

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It adds one useful trait (the existence condition), but does not explain what happens if the session hasn't finalized (e.g., error vs. empty response), nor does it confirm read-only semantics or describe the output structure. Some transparency is present, but significant gaps remain.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with two concise clauses. It delivers the essential purpose and a key condition without any fluff. Every word earns its place, making it highly efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description gives a reasonable high-level idea of the return content ('meta-review synthesis'), but it omits details like the expected structure or format of the overview, and does not clarify behavior when the precondition is unmet. It is adequate but not comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents the single parameter (session_id) with an example. The description adds no additional meaning to the parameter. Since schema coverage is 100%, the baseline is 3; the description neither enhances nor detracts from parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'The final research overview for a session: the meta-review synthesis of the ranked hypotheses.' This is specific and distinguishes it from siblings like hypotheses_list (which lists individual hypotheses) and hypothesis_get (fetches a single hypothesis). The resource is explicit, and the verb 'get' is implied by the name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear usage condition: 'Only exists once the session has finalized.' This tells the agent when the tool is appropriate to call. It doesn't explicitly name alternatives or exclusions, but the condition effectively implies it should be used only after finalization. This is sufficient guidance for a simple retrieval tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

review_recordA

Record a review against a hypothesis, in the same table and format the Reflection agent writes. Every factual claim in evidence needs a url and an excerpt you actually read. A 'full' review on a draft hypothesis also promotes it to reviewed. Do not report a review to the user without recording it first, or the tournament and meta-review will never see it.

NameRequiredDescriptionDefault
bodyYesThe review itself, markdown.
kindYesfull = novelty, correctness, testability. verification = assumption decomposition. observation = does it explain a published observation. simulation = step-through of the mechanism.
scoresNo
verdictNo
evidenceNoSources backing the review. Cite only what you read.
iterationNoDisambiguates repeat reviews of the same kind. Omit and the server picks the next free slot.
assumptionsNoDeep-verification decomposition, one row per assumption.
hypothesis_idYesHypothesis id, e.g. hyp_9f2c...

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that evidence requires a url and excerpt, that a 'full' review promotes a draft to reviewed, and the consequence of not recording first. These are behavioral traits beyond a simple 'record' action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with purpose, then critical constraints. No fluff, each sentence adds essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 8 params, nested objects, and no output schema, the description covers the key edge cases: evidence provenance, promotion logic, and ordering. It gives enough context for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 75%, so baseline is 3. The description adds value by explaining evidence requirements (url+excerpt you actually read) and the effect of 'full' kind. This goes beyond the schema's per-field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States exactly what it does: 'Record a review against a hypothesis'. Specifies verb+resource, and adds context about format and side-effect (promotes to reviewed). Clearly distinct from siblings like 'hypothesis_record' which record hypotheses, not reviews.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context: use it when you have a review to record, and gives a critical ordering constraint ('Do not report a review to the user without recording it first'). Does not explicitly name alternatives, but the purpose is self-evident and the constraint is practical.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_abortA

Abort a session for good. The main loop exits at its next check.

NameRequiredDescriptionDefault
session_idYesSession id, e.g. ses_01J...

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden. It adds 'for good' indicating irreversibility and 'at its next check' showing deferred execution, which are useful behavioral details beyond the name alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with no wasted words. The action is front-loaded, and the second sentence adds a key behavioral detail without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description explains the purpose and the timing of the effect. It does not describe return values, but that is not critical given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for session_id, and the description does not add any parameter-specific meaning. Baseline 3 applies because the schema already provides adequate documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Abort a session for good', which is a specific verb + resource. The phrase 'for good' distinguishes this from session_pause or session_resume, clarifying a permanent termination.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description conveys that this is for permanent termination and notes the main loop exits at a check, providing context for when to use it. It does not explicitly exclude temporary pauses, but the context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_listA

List sessions newest first, with status, hypothesis count, top Elo and budget spent. Use this to find a session id the user refers to by topic.

NameRequiredDescriptionDefault
limitNoMax rows. Default 20.

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the burden. It discloses that sessions are listed newest first and includes specific fields, but it does not explicitly state it is read-only or mention any side effects, pagination behavior, or error cases. Basic safety profile is implied by 'List' but not explicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and details, then a practical usage hint. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with one parameter and no output schema, the description covers the returned fields, ordering, and a use case. It lacks explicit mention of pagination or result limits beyond the limit parameter, but overall it is sufficiently complete for the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers 100% of the single parameter (limit) with a clear description ('Max rows. Default 20.'). The tool description adds no additional parameter semantics, so it meets but does not exceed the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists sessions with specific fields (status, hypothesis count, top Elo, budget spent) and sorting (newest first). It is distinct from siblings like session_status or session_pause, and the usage hint further clarifies its purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use it: 'to find a session id the user refers to by topic.' It does not mention alternative tools for exclusion, but the use case is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_pauseA

Pause a running session. Workers drain and the loop sleeps until resumed.

NameRequiredDescriptionDefault
session_idYesSession id, e.g. ses_01J...

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the behavioral disclosure burden and does so well by explaining what happens: workers drain and the loop sleeps until resumed. It implies reversibility and non-destructiveness, though it does not state error behavior for already-paused sessions or permissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short, front-loaded sentences. The first states the core purpose, and the second adds essential behavioral detail with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with no annotations and no output schema, the description provides enough context: it names the action, the precondition, and the resulting behavior. It does not explain return values, but the tool's simplicity and clear behavior make the description reasonably complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already fully documents the only parameter, session_id, with type, requirement, and example format. The description adds no further parameter-specific meaning, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool pauses a running session, using a specific verb and resource. It distinguishes itself from sibling tools like session_resume and session_abort by describing a reversible pause with workers draining and the loop sleeping until resumed.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Pause a running session' provides clear context for when to use the tool, and 'until resumed' implies the resume counterpart without explicitly naming session_resume. It lacks explicit when-not-to-use guidance or alternative tool names, but the intended usage is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_resumeA

Resume a paused or interrupted session. The engine reclaims expired task leases and continues where it stopped. Returns immediately; poll session_status.

NameRequiredDescriptionDefault
session_idYesSession id, e.g. ses_01J...

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that the engine reclaims expired task leases and continues where it stopped, and that it returns immediately, requiring polling of session_status. This gives insight into behavior, though it lacks details on error handling or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise—two sentences—and directly conveys the essential information without unnecessary filler. It is well-structured and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema, the description explains the immediate return and the need to poll session_status. It also explains the lease reclaiming behavior. It could be more complete about success/failure conditions, but for this simple tool it is adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter 'session_id' is well-described in the schema with an example, so the schema coverage is high. The tool description does not add extra information, but it is not needed given the clear schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that the tool resumes a paused or interrupted session. It is specific and distinguishes this tool from siblings like session_start, session_pause, and session_abort.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description indicates when to use it (for resuming sessions) but does not explicitly mention alternatives or when not to use it. It does provide a hint that it returns immediately and to poll session_status, which implies a usage pattern.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_startA

Start a new research session from a natural-language goal. The engine runs the full Generation, Reflection, Elo tournament, Evolution and Meta-review loop in a detached process; this returns as soon as the session id exists. Poll session_status for progress. It runs for minutes to hours, so agree the scale with the user first: call estimate_cost.

NameRequiredDescriptionDefault
goalYesThe research goal. Write it as a scientist would: the system, the phenomenon, and the kind of answer wanted.
n_initialNoParallel initial Generation calls. Default 3.
budget_usdNoAdvanced, and rarely needed. On a subscription backend this caps an equivalent-cost gauge, not a bill. Leave it unset unless the user asks about spend.
concurrencyNoOverride worker concurrency.
preferencesNoOptional extra constraints or preferences, free text.
wall_clock_secondsNoOverride the wall-clock cap.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It states the detached process, early return after session id exists, and duration of minutes to hours. This is valuable context. It does not cover failure modes or edge cases, but for a session start, the disclosed behavior is sufficient to set expectations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded. It opens with the core purpose, then explains the asynchronous behavior, and finishes with practical instructions. Every sentence contributes value; no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity—a long-running asynchronous process with multiple internal stages—the description adequately covers the key context: detached execution, early return, polling, and cost estimation. It does not describe the output or error handling, but since there is no output schema and the tool is a starter, the provided context is sufficient for typical usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (all 6 parameters have descriptions), so the schema already provides parameter semantics. The tool description does not add extra details about parameters beyond what the schema offers. Per the baseline rule for high coverage, a score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb and resource: 'Start a new research session' from a natural-language goal. It distinguishes from siblings like session_status and session_resume by focusing on the initiation action and describing the long-running process. The purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly directs the agent to poll session_status for progress and to call estimate_cost to agree on scale before starting. It also implicitly distinguishes from session_resume by focusing on new sessions. These explicit references to sibling tools provide strong usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

session_statusA

Progress for one session: status, task counts by state, hypothesis counts by state, matches played, Elo spread, budget spent, and whether the final overview exists yet. This is the polling tool.

NameRequiredDescriptionDefault
session_idYesSession id, e.g. ses_01J...

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden of behavioral transparency. It indicates that the tool returns status information (status, counts, Elo spread, budget, existence of final overview) but does not disclose whether it is read-only (presumably yes), any side effects, or whether it blocks. It provides useful detail on the returned data but could be more explicit about non-mutating behavior, especially given it is a polling tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that efficiently enumerates all key metrics and ends with a clear purpose statement. Every part contributes value, and it is front-loaded with the core purpose. No fluff, perfect length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity (single parameter, no output schema) and the rich list of returned metrics in the description, the description is fairly complete for an agent to understand what to expect. It lacks details on return format or error handling, but for a polling tool, this is sufficient. The provided info covers the essential data points, so a 4 is warranted.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100% (session_id is described as 'Session id, e.g. ses_01J...'). The description adds no additional parameter semantics beyond that. Since the schema already covers the parameter, the baseline is 3, and the description does not add extra meaning, so this score is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: reporting progress for a single session, with specific metrics listed. It distinguishes itself from siblings like session_list (which lists sessions) and overview_get (which gets the final overview) by focusing on session status polling. It could be more explicit about being a read-only status check, but the purpose is clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly states 'This is the polling tool,' which is a clear usage guideline for when to use this tool (for polling status). It does not explicitly say when not to use it or name alternatives, but it implies a monitoring context rather than a control context, distinguishing it from session_pause/abort/start. The context is somewhat clear, but exclusions are not spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 15 tool updatesv0.1.0
    • First observeddoctor
    • First observedestimate_cost
    • First observedfeedback_add
    • First observedhypotheses_list
    • First observedhypothesis_get
    • First observedhypothesis_record
    • First observedmatch_record
    • First observedoverview_get
    • First observedreview_record
    • First observedsession_abort
    • First observedsession_list
    • First observedsession_pause
    • First observedsession_resume
    • First observedsession_start
    • First observedsession_status

TDQS

A4.1/5.0

Scored across 15 tools

Disambiguation4/5

Tools are mostly distinct by resource and action (sessions vs hypotheses vs reviews vs matches). Minor overlap between session_pause/session_abort and hypothesis_record/review_record, but descriptions are detailed enough to avoid misselection.

Naming Consistency4/5

Most tools follow a noun_verb pattern (session_list, hypotheses_list, hypothesis_get, review_record), but session_resume, session_start, session_pause, session_abort are verb-first. doctor breaks the pattern entirely. Consistent within each resource group, mixed across groups.

Tool Count5/5

15 tools for a complex scientific research orchestration server. Each tool maps to a distinct lifecycle step: session management (7), hypothesis operations (4), review/match/feedback (3), overview (1), cost estimation (1), diagnostics (1). Well-scoped for the complexity.

Completeness4/5

Covers the full session lifecycle (start, poll, pause, resume, abort), hypothesis CRUD (list, get, record), review and tournament operations. Missing explicit update/delete for hypotheses or sessions, but the domain (adding to a persistent tournament) doesn't require them. Overview_get exists only after finalization; no explicit wait/finalize tool but polling covers it.

Maintenance

ActivityStale
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers