Skip to main content
Glama

Server Details

Assign Linear, Jira or GitHub issues to an AI coding agent team that plans, builds, reviews, tests.

Ownership verified
Status
Healthy
Uptime
100.0% over 22 days
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
Repository
letsship/mcp
GitHub Stars
0

TDQS

A3.6/5.0

Scored across 74 tools

Disambiguation3/5

Most tools target distinct resources and actions, and their descriptions carefully explain edge cases. However, the large set has several overlapping areas: create_mission vs start_mission, get_mission vs inspect_mission, get_agent_run vs watch_agent_run, and multiple sandbox-template operations require close reading to distinguish.

Naming Consistency4/5

The set overwhelmingly follows a verb_noun snake_case convention such as list_missions, create_mission, and update_scheduled_task. The only notable deviations are healthz and whoami, plus minor verb variation like save/update/set across similar operations.

Tool Count2/5

74 tools is far beyond a manageable MCP surface for agent selection and context. While SHIP is a broad platform, the tool count is heavy enough that discovery and correct selection will be costly.

Completeness4/5

Coverage is broad across missions, agent runs, sandboxes, scheduled tasks, MCP bindings, billing, setup, notices, and preview environments. Minor gaps remain, such as project creation/deletion and likely environment creation, but core workflows are well covered.

Available Tools

74 tools
activate_sandbox_template_versionRoll a sandbox back (or forward) to a versionA
Idempotent
Inspect

Make a version the one missions use from their next dispatch. It is rebuilt only if its snapshot is gone; the response says which.

NameRequiredDescriptionDefault
versionIdYesVersion id (stv_...)
templateIdYesSandbox id (sbt_...)

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare write, idempotent, and non-destructive traits. The description adds a valuable behavioral nuance: the version is rebuilt only if its snapshot is gone, and the response indicates whether that happened. This goes beyond structured data, though it does not cover other aspects like permissions or error cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action and followed by a key conditional detail. Every sentence earns its place with no redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no output schema, the description covers the main action and an important conditional behavior (rebuild + response indicator). It is largely complete, though it doesn't address error handling or permission requirements.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters fully. The description adds no additional meaning about versionId or templateId beyond what the schema provides, making the baseline score of 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Make a version the one missions use') and resource (sandbox template version), making the purpose clear and distinguishable from read-only siblings like list_sandbox_template_versions. However, it does not explicitly name or contrast with related write tools such as save_sandbox_template or build_sandbox_template.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context ('from their next dispatch') but provides no explicit when-to-use, when-not-to-use, or alternative recommendations. It lacks guidance on prerequisites or how this tool relates to version creation or listing tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

answer_qa_checkpointAnswer the QA checkpoint of a missionA
DestructiveIdempotent
Inspect

A mission waits at the QA checkpoint (get_mission shows pauseSource 'qa-unverifiable') when QA could not check some acceptance criteria, so no code change can move it. A person checks those criteria another way, then answers. 'accept' counts QA as passed, 'reject' ends the mission as failed, and 'rerun' runs QA again on the same commit. Send only the decision the person made. Answers 409 when the mission does not wait at the QA checkpoint. Requires the missions:write scope.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412
decisionYesaccept (QA counts as passed), reject (the mission fails) or rerun (QA runs again)

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds substantial context beyond the annotations: each decision's downstream effect ('accept' passes QA, 'reject' fails the mission, 'rerun' re-runs QA on the same commit), the human-in-the-loop precondition, a 409 failure mode, and the required missions:write scope. This is consistent with destructiveHint=true, since 'reject' terminates the mission.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the eligibility condition, then the decision semantics, then the error and auth constraints. Four dense sentences with no duplication or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a state-mutating tool with only two required params and no output schema, the description covers precondition, detection method, per-value effects, error behavior, and authorization — everything an agent needs to decide and invoke correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the enum values are already documented there, so the baseline is 3. The description still earns credit by spelling out the semantic consequence of each decision and by adding a detail absent from the schema — that 'rerun' evaluates the same commit.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb and resource ('answer the QA checkpoint of a mission') and immediately delimits the situation it applies to (pauseSource 'qa-unverifiable'), which separates it from siblings like approve_mission_ceiling, decline_mission, and set_mission_disposition.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit trigger condition (mission waits at the QA checkpoint, detectable via get_mission pauseSource) and an explicit when-not (409 when the mission is not at the checkpoint). It stops short of naming the sibling tools an agent should reach for instead in those other cases, which keeps it off a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

approve_mission_ceilingLet a mission continue past its credit limitAInspect

A mission that reached its credit limit pauses until a member lets it continue. This raises the mission's limit by one step and runs the stage that stopped again, so it spends credits: ask the person first. Returns the new limit. Answers 409 when the mission is not at its limit. Requires the missions:run scope.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412
projectIdYesProject id the mission belongs to (get_mission's summary.projectId)

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations: it discloses that the limit increases by exactly one step, that the interrupted stage re-runs, that credits are spent, that human consent is expected, the returned value, and the 409 failure condition. Annotations only supply the generic write/non-idempotent profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but front-loaded: context, then the action's side effects, then the warning, then the return/error/scope facts. Every sentence earns its place, though the second sentence packs behavior, cost, and a warning into one long clause.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description still covers the return value, the error case, the authorization scope, and the irreversible credit spend. Nothing an agent needs to invoke this safely is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so issueId and projectId are fully documented in the schema (including the GitHub key format and where projectId comes from). The description adds no parameter-level meaning beyond that, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource with mechanism: raises the mission's credit ceiling by one step and re-runs the stopped stage. An agent can distinguish this from resume_mission or request_more_credits without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear operating condition ('ask the person first' since it spends credits) and a negative condition (409 when the mission is not at its limit), plus the missions:run scope requirement. It stops short of naming sibling alternatives like request_more_credits for agents that need more than one step.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

archive_missionArchive a missionA
Idempotent
Inspect

Hide a finished mission from list_missions and the board. Nothing else changes: unarchive_mission shows it again. Requires the missions:write scope and the config:write capability (an owner or admin).

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, idempotentHint=true, and destructiveHint=false, yet the description adds real value on top: the exact blast radius ('Nothing else changes'), the reversibility path (unarchive_mission), and the auth requirements (scope plus owner/admin capability). That is precisely the mutation context annotations cannot express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the effect, then the constraint, then the permissions. Every sentence carries distinct information and nothing is repeated.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, reversible mutation with no output schema, the description covers effect scope, reversibility, and authorization, which is everything an agent needs to invoke it correctly. Nothing material is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter is fully documented in the schema (Linear key or GitHub Issues id, e.g. SHIP-412). The description adds no format or validation detail beyond it, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Hide a finished mission') and scopes the effect precisely to list_missions and the board, which distinguishes it from siblings such as delete_sandbox_template or destroy_environment. An agent can immediately tell this is a soft, listing-level hide rather than a deletion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives conditions for calling it (finished mission, missions:write scope, config:write capability, owner/admin) and names unarchive_mission as the reversal path. It does not explicitly contrast with a destructive delete-style alternative, so it stops just short of full when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

bind_mcp_serverBind an MCP serverA
DestructiveIdempotent
Inspect

Create or update the binding for a server the repository declares. roles is the full set of pipeline roles that may use it (reviewer and qa only ever receive read-only tools, and cannot use a server that runs inside the sandbox). transport: 'remote' (the gateway proxies to the server's URL), 'isolate' (the gateway runs the server's npm package in isolation, for a command that needs a secret) or 'local' (the command runs inside the sandbox, no credential). Omit transport and remoteUrl on an update to keep the server the binding is pinned to, with its credential (a roles, tools or credential edit); set them only to bind or re-point the server. Requires the config:write capability.

NameRequiredDescriptionDefault
rolesYesEvery role that may use the server ([] disables it)
projectIdYesProject id, e.g. from list_projects
remoteUrlNoA fixed https URL: a hosted endpoint for a command server, or an override for a URL server
transportNoHow the gateway reaches the server: 'remote' proxies to remoteUrl, 'isolate' runs the server's npm package away from the sandbox so a secret stays out of it, and 'local' runs the command inside the sandbox with no credential. Omit it on an update to keep the server the binding is pinned to.
credentialNoWrite-only credential the gateway attaches; agents never receive it. Omit to keep the stored one. OAuth credentials are not set here: bind as remote with kind 'none', then start_mcp_server_oauth.
serverNameYesServer name as the repository's MCP file declares it (see list_mcp_servers)
approvedToolsNoThe approved subset of the tool snapshot (refresh_mcp_server_tools first). Omit to keep it.

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds real context beyond the annotations: credential is write-only and never seen by agents, reviewer/qa only ever receive read-only tools and cannot use sandbox-internal servers, and omitting fields preserves existing state while setting them re-points. It stops short of saying what happens to the previously stored credential on re-point or how failures (bad transport/credential combo) surface, which is why it isn't a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense paragraph with no filler; the binding/update distinction is front-loaded and the transport and field-omission rules follow in a logical order. Every clause carries actionable information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter mutation tool with no output schema, the description covers purpose, prerequisites, transport selection, and update semantics — enough to call it correctly. Gaps are minor: it does not touch approvedTools or the refresh_mcp_server_tools prerequisite, though the schema covers those.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, but the description adds semantics the schema lacks: the role-scoping rule for reviewer/qa, the intent behind each transport mode, and the keep-vs-replace behavior of transport/remoteUrl on update. It says nothing about approvedTools (left entirely to the schema).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb pairing ('Create or update') and a precise resource ('the binding for a server the repository declares'), which cleanly separates it from list_mcp_bindings and unbind_mcp_server. An agent can identify it as the write-side counterpart to the read siblings without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit conditional guidance: omit transport/remoteUrl on an update to keep the pinned server and credential, set them only to bind or re-point. It also enumerates the three transport choices with the situation each serves (secret needed, no credential, proxied URL) and names the config:write capability prerequisite.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

build_sandbox_templateBuild a sandbox nowBInspect

Build the sandbox's active version, or return the build already running for it.

NameRequiredDescriptionDefault
templateIdYesSandbox id (sbt_...)

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false and destructiveHint=false, so safety is covered. The description usefully discloses that a call returns an in-flight build rather than always starting a new one, which is real behavioral context. However that 'reuse existing build' behavior sits in mild tension with idempotentHint=false, and nothing about auth requirements or build lifecycle/status is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with zero filler, and the primary action (build the active version) is front-loaded ahead of the fallback clause. Nothing is wasted, though there is little structure to speak of.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter mutation tool with no output schema, the description covers the action but not the outcome: it never says what a caller receives (build id, status, log reference) despite get_sandbox_build_log existing as a sibling. Adequate but leaves a visible gap around return semantics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With a single parameter at 100% schema coverage (templateId documented as 'Sandbox id (sbt_...)'), the schema fully carries parameter meaning. The description adds nothing about the identifier's format or source, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Build) and resource (the sandbox's active version), and adds a second clause about returning an already-running build. It is clearly distinguishable in intent, though it never names the closely related siblings like activate_sandbox_template_version or save_sandbox_template that an agent might confuse it with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisite (e.g., does building require the version to be saved first?), and no condition pointing to alternatives such as activate_sandbox_template_version. The only temporal hint is that an existing running build is reused, which is inferred rather than framed as a decision rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

clear_agents_overrideClear a mission's agents-overrideA
DestructiveIdempotent
Inspect

Delete the per-issue agents/baseBranch override — the mission resolves from ship.yml again.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-353 or WEB-42
projectIdYesProject id the mission belongs to

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, idempotentHint=true, and readOnlyHint=false, so the safety profile is covered. The description adds genuinely useful context beyond that: exactly what is destroyed (the per-issue agents/baseBranch override) and the post-condition state (mission resolves from ship.yml again).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with zero filler; the destructive action leads and the consequence follows. Nothing to trim.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Two required params, no output schema, and annotations that cover safety, so the description's remaining job is explaining the effect — which it does. It could be slightly stronger by noting the override must already exist or how to inspect it via get_agents_override first.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so issueId and projectId are fully documented in the schema, including the Linear/GitHub id format. The description adds no parameter detail beyond the word 'per-issue', so the baseline 3 for schema-complete tools is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Delete') and resource ('per-issue agents/baseBranch override') and adds the resulting effect ('the mission resolves from ship.yml again'). An agent can distinguish it from set_agents_override by the delete semantics, though the sibling tools are never named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: the description says the override is removed and the mission falls back to ship.yml, which conveys the intent of reverting to config-driven behavior. There is no explicit when-to-use, when-not-to-use, or pointer to get_agents_override/set_agents_override as alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_missionCreate a missionAInspect

Create the tracker issue AND start the run in one call, for work that has no issue yet. Pass entryStartAt='builder' with an entryPlan to hand over a plan you already wrote; pass entryStartAt='reviewer'/'qa' with entryPrNumber to hand over an existing build — in which case title may be omitted and the pull request's own title names the mission. This tool calls the SHIP API, documented at https://letsship.ai/docs/api. Requires the missions:run token scope.

NameRequiredDescriptionDefault
titleNoIssue title. Omit ONLY with entryPrNumber — the pull request's own title is then used
agentsNoPer-role harness/model override for THIS mission only; overrides ship.yml without changing it
labelsNoTracker labels to apply
entryPlanNoHand-off plan markdown, for entryStartAt='builder'
projectIdYesProject id the mission belongs to
descriptionNoIssue body; acceptance criteria written here are parsed as they would be on any issue
entryStartAtNoPipeline stage to enter at; omit for the default full run starting at the planner
entryPrNumberNoExisting open PR number, required for entryStartAt='reviewer' or 'qa'

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare it as a non-read-only, non-idempotent, open-world write, so the description's job is to add context beyond that — and it does, disclosing the required missions:run token scope and pointing to the SHIP API docs. It stops short of saying what is destroyed on failure or whether retries duplicate issues, which matters for a non-idempotent creation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The compound action and the 'no issue yet' scope are front-loaded, and each sentence carries information. The middle sentence is long and packs two conditional hand-off modes into one breath, which slightly hurts scannability but wastes no words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter tool with nested agent overrides and no output schema, the description covers the non-obvious parts: the two entry modes, the title-omission rule, auth scope, and API docs. It omits nothing critical, though it doesn't mention the agents/labels/description fields (all adequately schema-documented).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter, including the entryPlan/entryPrNumber coupling. The description restates those relationships (entryStartAt pairing, title omission with entryPrNumber) rather than adding new semantics, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific compound action (create the tracker issue AND start the run in one call) and scopes it with 'for work that has no issue yet', which implicitly but clearly distinguishes it from sibling tools like start_mission or list_missions. An agent can tell what resource is produced without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit branching guidance: use entryStartAt='builder' with an entryPlan to hand over a plan, or entryStartAt='reviewer'/'qa' with entryPrNumber to hand over an existing build. It does not name the alternative tool (e.g. start_mission) for work that already has an issue, so the when-not is only implied by 'for work that has no issue yet'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_scheduled_taskCreate a scheduled taskAInspect

Create a recurring agent mission. Firing it — on schedule, or immediately via run_scheduled_task — creates an ordinary tracker issue assigned to the agent, which drives a normal pipeline run: a scheduled task is a stored mission request, not a new execution mode. This tool calls the SHIP API, documented at https://letsship.ai/docs/api. Requires the config:write capability.

NameRequiredDescriptionDefault
nameYesScheduled task name, shown in the console and used in the issue title
timeYes24-hour wall-clock time in `timezone`, e.g. '09:00'.
labelsNoTracker label names put on every issue this scheduled task files (default none). The label must ALREADY exist in the project's tracker team — the platform attaches labels but cannot create them, so a name the workspace does not have is dropped.
promptYesBecomes the created issue's description verbatim on every fire
enabledNoWhether the schedule is armed (default true)
weekdayNoDay of week — required when interval is 'weekly'.
intervalYesFire cadence. 'weekly' requires weekday; 'monthly' requires dayOfMonth.
timezoneYesIANA timezone id, e.g. 'Europe/Amsterdam'.
projectIdYesProject id the scheduled task belongs to
dayOfMonthNoDay of month, 1-28 — required when interval is 'monthly' (capped at 28 so every month has the day).
templateKeyNoAttribution only, when authored from a template — see list_scheduled_task_templates
skipIfPreviousOpenNoSkip a scheduled fire while the previous fire's mission is still open (default true — the guardrail is opt-out, not opt-in)

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only cover the safety profile (readOnlyHint=false, destructiveHint=false, idempotentHint=false), so the description adds genuine value: the auth requirement (config:write), the SHIP API dependency with a docs link, and the key behavioral fact that firing files an ordinary tracker issue rather than spawning a separate execution mode. It does not address the non-idempotent hint (duplicate tasks on retry), which is a notable omission.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each load-bearing, with the core purpose front-loaded before the mechanics and the API/capability footnotes. The middle sentence is dense but the parenthetical aside about execution mode is worth the words; minor tightening is possible.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parameter mutation with no output schema, the description covers the conceptual model, auth requirement, and downstream side effect, which is what an agent needs to invoke it correctly. It omits retry/duplicate behavior and does not point at get_scheduled_task or list_scheduled_tasks for verification.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with 12 well-documented parameters, including conditional requirements (weekday for weekly, dayOfMonth for monthly) and the label-must-already-exist caveat. The description adds no parameter-level detail beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource ("Create a recurring agent mission") and immediately clarifies the conceptual model: a stored mission request, not a new execution mode. It also distinguishes itself from run_scheduled_task by explaining that firing is what produces the tracker issue and pipeline run.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It routes the agent between this tool and run_scheduled_task (immediate firing) and states the required config:write capability, which is real usage guidance. It stops short of explicitly naming when not to use this versus update_scheduled_task or create_mission, so it is clear but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

decline_missionDecline a mission's pull requestA
DestructiveIdempotent
Inspect

Close the mission's open pull request on GitHub without merging it; the mission becomes declined, and no call here reopens it. Requires the missions:write scope.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true, so the safety profile is covered. The description adds real value beyond that: the required 'missions:write' scope and the permanent nature of the decline ('no call here reopens it'), which the annotations do not convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence front-loads the action and effect, then adds the irreversibility constraint and auth requirement with zero wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter mutation with no output schema, the description covers the action, the resulting state, permanence, and required scope. Only an explicit routing hint toward merge_mission is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single issueId parameter is fully documented in the schema with an example. The description adds no additional meaning about the parameter, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Close the mission's open pull request on GitHub without merging it') and names the resulting state change ('the mission becomes declined'). It clearly distinguishes itself from the merge sibling by emphasizing the non-merging close.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'without merging it' implicitly contrasts with merge_mission, and the irreversibility note hints at when caution is needed, but it never explicitly states when to choose this tool over merge_mission. Usage is implied rather than spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_sandbox_registryRemove a private registry credentialA
DestructiveIdempotent
Inspect

Delete the stored credential of a private registry (an id from list_sandbox_registries). A sandbox build that installs from that host then gets no credential. Requires the connectors:write scope and the config:write capability.

NameRequiredDescriptionDefault
registryIdYesRegistry credential id, from list_sandbox_registries

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true, and the description goes beyond them by naming the required connectors:write scope and config:write capability plus the consequence that installs from that host lose their credential. It stops short of stating whether the removal is reversible or recoverable via save_sandbox_registry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, with the mutation and its identifier source front-loaded before the downstream consequence. Every clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive delete with no output schema, the description covers identity, parameter source, required auth scopes, and the effect on sandbox builds. Nothing material to correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter's schema description already says 'Registry credential id, from list_sandbox_registries'. The description repeats that provenance without adding format or validation detail, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Delete) plus resource (stored credential of a private registry) and clarifies the identifier source with '(an id from list_sandbox_registries)'. This distinguishes it cleanly from save_sandbox_registry, list_sandbox_registries, and delete_sandbox_template.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Points the agent at list_sandbox_registries as the way to obtain the required id, and explains the context of use via the downstream effect on sandbox builds. There is no explicit when-not-use or named alternative for removing only part of a credential set, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_sandbox_templateDelete a sandboxA
DestructiveIdempotent
Inspect

Delete a sandbox with all its versions and builds. Answers 409 while an agent or a project's default sandbox uses it: point those at another sandbox first (update_project with defaultSandboxTemplateId). Missions that run in it now finish. Requires the agents:write scope and the config:write capability.

NameRequiredDescriptionDefault
templateIdYesSandbox id (sbt_...)

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructive=true and idempotent=true, but the description adds substantial context the annotations cannot: the 409 conflict semantics, the exact prerequisite ordering to clear the conflict, the effect on in-flight missions (they finish), and the required scopes/capabilities (agents:write, config:write). This is exactly the beyond-annotation disclosure the dimension rewards.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the destructive scope, then the conflict/remediation, then permissions. Every sentence carries operational information an agent needs; no filler or restatement of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive tool with no output schema, the description covers the full decision surface: what is destroyed, when the call will fail, how to unblock it, side effects on running missions, and authorization requirements. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single templateId parameter is documented in the schema as 'Sandbox id (sbt_...)'. The description adds no format or targeting semantics beyond what the schema already provides, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource (delete a sandbox template) and scopes it precisely: all versions and builds go with it. That distinguishes it from the near-named sibling delete_sandbox_registry and from archive-style tools without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly defines the blocking condition (409 while an agent or a project's default sandbox uses it) and the remediation path (repoint via update_project with defaultSandboxTemplateId). It also states when the operation is safe to proceed despite live missions. Nothing about when to use or avoid it is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_scheduled_taskDelete a scheduled taskA
DestructiveIdempotent
Inspect

Delete a scheduled task and its run history, which no call can bring back. Missions it already started continue. Setting enabled to false stops new runs and keeps the task and its history. Requires the scheduled-tasks:write scope and the config:write capability.

NameRequiredDescriptionDefault
scheduledTaskIdYesScheduled task id to delete

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already flag destructiveHint=true, but the description adds critical context: deletion also removes run history irreversibly, already-started missions continue, and the safer alternative preserves the task. It also discloses required scope and capability.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences are front-loaded with purpose and irreversible impact, followed by side effects, alternative behavior, and permissions. Every sentence adds necessary decision-making information without waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single-parameter schema, destructive annotations, and no output schema, the description provides everything needed: consequences, run history loss, in-progress mission behavior, the safer alternative, and required scopes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single required parameter, so the schema already documents 'scheduledTaskId'. The description adds no additional parameter semantics, which is acceptable but not value-adding here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: delete a scheduled task, and clarifies the scope includes its run history. It also distinguishes itself from the non-destructive alternative of setting enabled to false.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly describes the intended use and names the alternative for stopping new runs without deleting history. It also states the permission requirements, giving the agent clear preconditions before invoking.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

destroy_environmentDestroy a preview environmentA
DestructiveIdempotent
Inspect

Request teardown of a preview environment (its deployed Worker(s) + every provisioned datastore). Enqueues to the orchestrator; requires project write capability.

NameRequiredDescriptionDefault
environmentIdYesEnvironment id to tear down

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, idempotentHint=true, and openWorldHint=true, so the safety profile is covered. The description adds genuinely new behavioral context: teardown is enqueued to an orchestrator (asynchronous, not immediate) and it destroys both Workers and all provisioned datastores, plus the write-capability requirement.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly packed sentences with zero filler; the destructive scope is front-loaded before the async/auth caveats.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter mutation with no output schema, the description covers scope, asynchronous execution, and the capability requirement. It could note whether the teardown is reversible or what status the caller receives, but nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter (environmentId), and its schema description ('Environment id to tear down') already carries the meaning at 100% coverage. The description adds no syntax or format detail beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (teardown), resource (preview environment), and the exact scope of destruction (deployed Worker(s) plus every provisioned datastore). An agent can distinguish this from read siblings like get_environment/list_environments without inspecting schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives a prerequisite ('requires project write capability') but never states when to use this versus alternatives or when-not to use it (e.g., production vs preview environments). Usage is implied by the name and scope rather than spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

dismiss_noticeDismiss a workspace noticeA
Idempotent
Inspect

Close a notice for the person this token acts for, as the console's close button does. Only a notice with dismissible: true can be closed (a low balance). It stays closed while its id lasts: a low balance that a top-up ends and a later spend brings back has a new id and shows again. Critical notices cannot be closed.

NameRequiredDescriptionDefault
noticeIdYesThe notice's id from get_notices, e.g. credits_low:<grant>

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds substantial behavior beyond the annotations: per-token subject scoping, the persistence rule tied to notice id lifetime, and the concrete case where a topped-up-then-depleted balance returns as a new id and reappears. The annotations only supply safety/idempotency hints; this description explains the actual dismissal semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four tight sentences, front-loaded with the action and subject before the preconditions and id-lifecycle caveat. The low-balance example is the only elaboration and it earns its place by making the id-reuse rule concrete.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter mutation with no output schema, the description covers the precondition, the scope, the idempotency-persistent behavior, and the critical-notice carve-out. Nothing an agent needs to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a format example ('credits_low:<grant>'), so the baseline is 3. The description goes further by explaining what the notice id means operationally — dismissal is bound to that id, and id churn is why a notice can reappear — which adds real meaning about the parameter's role.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (close/dismiss) and resource (a workspace notice) with an explicit analogy to the console's close button, and scopes it to the token's subject. This cleanly distinguishes it from the read-side sibling get_notices.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit preconditions ('Only a notice with dismissible: true can be closed') and explicit exclusions ('Critical notices cannot be closed'), so an agent knows when not to call it. It does not, however, name the sibling (get_notices) that supplies notice ids in the description text itself; that reference lives only in the schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_agent_runGet agent run detailA
Read-onlyIdempotent
Inspect

Fetch one agent run: its run record, metric series (CPU/mem/disk over time), and the agent session summaries (role, outcome, cost, sessionId). sessions lists every session of the run's role on the mission; runSessionId names this run's own, so read its log and cost from that one.

NameRequiredDescriptionDefault
sandboxIdYesAgent run id: the run's sandbox id, from list_agent_runs

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description goes beyond them usefully by describing the shape of the returned data and by flagging a non-obvious trap: 'sessions' contains every session of the run's role on the mission, so the agent must use runSessionId for this run's own log and cost. That is genuine disambiguation an annotation cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler, with the return payload front-loaded before the session-id caveat. The second sentence is somewhat dense ('sessions lists every session of the run's role on the mission; runSessionId names this run's own') and requires careful parsing, keeping it just short of ideal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must carry the return-value burden, and it does: it names the run record, the metric series, and the session summary fields (role, outcome, cost, sessionId), plus the session-vs-runSessionId distinction. Error or missing-run behavior is not covered, but for a read-only getter that is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the single sandboxId parameter is already documented as the run's sandbox id sourced from list_agent_runs. The description adds no format or syntax detail beyond what the schema provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Fetch) and resource (one agent run) and enumerates the payload it returns: run record, CPU/mem/disk metric series, and session summaries. It implicitly distinguishes itself from list_agent_runs, get_agent_run_logs, and watch_agent_run by describing detail retrieval rather than listing, logging, or streaming, but it never names a sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: the id comes from list_agent_runs (noted in the schema, not the description), and the tool is obviously the detail-fetcher for a known run. The description does supply one practical routing hint (read the log and cost from runSessionId rather than any other session), but there is no explicit when-to-use/when-not guidance or named alternative such as get_agent_run_logs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_agent_run_logsGet agent run logsA
Read-onlyIdempotent
Inspect

Fetch a session's log content from one agent run. part=session is the agent stream-json (reasoning + tool calls); part=system is the container/runner stdout (clone, dependency install, validation gate, git, callback); part=prompt is the assembled input prompt. The role and sessionId are fields of the agent run's own record. A log longer than 40,000 characters returns its end, from a whole line, with a note.

NameRequiredDescriptionDefault
partNoWhich log stream to readsession
roleYesAgent role: planner | builder | reviewer | qa
sandboxIdYesAgent run id: the run's sandbox id
sessionIdYesSession id from get_agent_run's sessions list

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description adds genuinely non-obvious behavior beyond that: logs over 40,000 characters return the tail only, aligned to a whole line, with a note. That truncation contract is exactly the kind of operational detail an agent needs and cannot get from annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four front-loaded sentences: purpose first, then stream semantics, then parameter provenance, then the truncation caveat. No filler sentences, though the parenthetical list of system-log stages is dense enough to slow reading slightly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates by explaining the payload of each part plus the truncation rule. What is missing is error behavior (e.g., log not yet available for a running or stopped run) and guidance on pairing with watch_agent_run, but otherwise an agent has enough to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description goes beyond the schema's one-line 'Which log stream to read' by explaining what each enum value actually contains (reasoning+tool calls vs clone/dependency-install/validation/git stdout vs assembled prompt), which is real added meaning for the only parameter that requires judgment.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource+scope: 'Fetch a session's log content from one agent run.' It then defines the three log streams (session=stream-json reasoning+tool calls, system=container/runner stdout, prompt=assembled input prompt), which cleanly separates it from get_agent_run (metadata), list_agent_runs, and get_sandbox_build_log. An agent can identify this tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It tells the agent that role and sessionId come from the agent run's own record, which implies a prerequisite call to get_agent_run, but it never names an alternative tool or states when to prefer this over list_agent_runs / get_agent_run / get_sandbox_build_log. Usage is implied rather than explicit, so it lands at the minimum-viable level.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_agents_overrideRead a mission's agents-overrideA
Read-onlyIdempotent
Inspect

Read the per-issue agents/baseBranch override record for one mission, or null when unset.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-353 or WEB-42
projectIdYesProject id the mission belongs to

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly, idempotent, and non-destructive behavior, so the bar is lower. The description adds a genuine behavioral detail beyond the annotations: the return is null when the override is unset, which tells the agent how to interpret an empty result.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with the operation and scope front-loaded and the null-case tucked in at the end. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly carries the return-value burden by naming what is returned and the unset case. For a simple two-parameter getter this is nearly complete; only the permissions or failure modes are unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both issueId and projectId documented including format examples (SHIP-353). The description adds only the framing that the record is per-issue and per-mission, which the schema already conveys; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb (Read) plus a precisely named resource (the per-issue agents/baseBranch override record for one mission), scoped to a single mission. It does not name the closely related sibling tools (set_agents_override, clear_agents_override), so an agent must infer the read-vs-write split from the verb alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the usage (fetch the override before acting on it) but never states when to call this versus set_agents_override, clear_agents_override, or get_mission. No prerequisites or exclusions are given; guidance is only inferable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_artifactGet artifactA
Read-onlyIdempotent
Inspect

Fetch one artifact's metadata + a long-lived signed serving url (open it to view the screenshot/video).

NameRequiredDescriptionDefault
idYesArtifact id (from list_artifacts)

Output Schema

NameRequiredDescription
idYes
urlYes
kindYes
roleYes
captionYes
issueIdYes
createdAtYes
criterionYes
expiresAtYes
projectIdYes
sandboxIdYes
sessionIdYes
sizeBytesYes
contentTypeYes
traceViewerUrlYes
criterionPassedYes

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds genuine context beyond that: the return includes a long-lived signed serving URL and instructions for consuming it (open to view the screenshot/video), which tells the agent what it can do with the result.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the primary action and its output, with a parenthetical that earns its place by explaining how to use the URL. No filler or restatement of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read tool with full annotation coverage and an output schema, the description covers purpose, output content, and the practical use of the returned URL. Nothing needed to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single id parameter is already documented as coming from list_artifacts, so the baseline is 3. The description adds no syntax, format, or sourcing detail beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ("Fetch") and resource ("one artifact's metadata") plus the signed serving URL, and the singular "one artifact" implicitly contrasts with the list_artifacts sibling. It does not name the sibling explicitly, so the differentiation is inferable rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: an agent understands this is the single-item lookup that pairs with list_artifacts, and the schema note "id (from list_artifacts)" reinforces that. There is no explicit when-to-use/when-not guidance or named alternative, so this is minimum-viable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_billingGet workspace creditsA
Read-onlyIdempotent
Inspect

Read the workspace's credit balance by source and expiry, whether a new mission would start now, the per-mission credit ceiling, the project allowance and recent usage. Check it before dispatching work.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered and the bar is lower. The description adds genuine context beyond that: it is a composite read combining balance-by-source, start-eligibility, ceiling and allowance, and it discloses the decision it supports (dispatch). Auth needs and rate limits remain unstated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action and the returned facets, closing with the actionable trigger. No filler, no restatement of the title, and every clause carries distinct information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no parameters, the description carries the full burden of describing what comes back, and it does so thoroughly across five value categories. What is missing is minor: no indication of freshness/refresh behavior or any cost of invoking it, though for a zero-arg read-only call that is a small gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, which is the baseline-4 case; there is nothing schema-side to document. No parameter information is needed or missing, so the description is not penalized here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Read') and resource ('the workspace's credit balance') and enumerates the distinct facets returned: balance by source and expiry, mission-start viability, per-mission ceiling, project allowance, recent usage. That enumeration implicitly separates it from siblings like list_billing_ledger (raw ledger entries) and list_credit_offers (purchasable offers), so an agent can tell them apart without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Check it before dispatching work' gives a clear triggering context for calling the tool. It stops short of naming an alternative (e.g., list_billing_ledger or list_credit_offers) or stating when not to call it, so it is clear context without exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_environmentGet preview environment detailA
Read-onlyIdempotent
Inspect

Fetch one preview environment's registry row (status, worker name, timestamps).

NameRequiredDescriptionDefault
environmentIdYesEnvironment id (from list_environments)

Output Schema

NameRequiredDescription
environmentYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds the return-shape hint (registry row with status, worker name, timestamps), but since an output schema exists this is largely redundant and no error/not-found behavior is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with the resource scope front-loaded and the returned fields parenthetically appended. No filler and nothing to trim.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema, rich annotations, and a fully documented single parameter, the description only needs to establish purpose and scope, which it does. Minor omission of not-found/error behavior keeps it short of a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single environmentId parameter is fully documented in the schema, including its origin ('from list_environments'). The description adds nothing about the id format or source beyond what the schema already says, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Fetch') and resource ('one preview environment's registry row') and even enumerates the returned fields (status, worker name, timestamps). This cleanly separates it from the sibling list_environments, though it does not name that sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The singular scope implies this is the detail-fetch counterpart to list_environments, and the schema points at list_environments as the id source. However, there is no explicit when-to-use statement or exclusion criteria beyond that implication.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_missionGet mission detailA
Read-onlyIdempotent
Inspect

Fetch one mission. view='summary' (the default, small enough to poll) returns its summary (activity, stage, review/check/test state, PR, paused and pauseSource, cost), its agent runs (sandboxId, role, status, harness, model, cost) and proofRunSandboxId (pass it to list_artifacts as sandboxId for the proof). view='full' adds its stages, events, stats and the plan/review/QA outcomes. pauseSource 'checkpoint:' means a checkpoint holds it: resume_mission approves that stage. pauseSource 'qa-unverifiable' means QA could not check some acceptance criteria: answer_qa_checkpoint answers it. For a mission that needs attention, summary.escalation says whether an error or a platform judgement stopped it and summary.escalationCause holds the whole error. Requires the missions:read scope.

NameRequiredDescriptionDefault
viewNosummary (default) for the state, runs and proof run; full for every stage, event and outcomesummary
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412
orgSlugNoNarrow an issue id to one org
projectIdNoNarrow an issue id that two projects share

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnly/idempotent annotations: it discloses the required missions:read scope, the payload cost differential between views, the meaning of each pauseSource value, and where escalation details live (summary.escalation / escalationCause). It also chains behavior by telling the caller to pass proofRunSandboxId to list_artifacts.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the single-sentence purpose, then dense but purposeful detail. A few clauses (the enumeration of summary fields) are long but each carries information an agent would otherwise have to guess.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the full burden of return-value explanation and does so thoroughly for both views, including nested field semantics and escalation reporting. An agent can call this correctly and interpret the response without further documentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning: it enumerates what each view actually returns (summary fields vs full additions) and explains the downstream use of proofRunSandboxId. orgSlug and projectId narrowing are left to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Fetch one mission") and immediately qualifies scope via the view modes. It does not, however, distinguish itself from the sibling inspect_mission, which plausibly reads the same resource, so an agent cannot fully disambiguate from the description alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly frames the choice between view='summary' ("small enough to poll") and view='full' (adds stages, events, stats, outcomes), and routes two specific states to alternatives: resume_mission for checkpoint pauses and answer_qa_checkpoint for qa-unverifiable. It gives no guidance on when to prefer this over list_missions or inspect_mission.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_noticesGet workspace noticesA
Read-onlyIdempotent
Inspect

Read what needs attention in the workspace, critical first: a tracker connection its vendor no longer accepts (missions cannot start or report back until someone reconnects it), or credits running out. Each notice has a stable id and kind, a message and, when the caller can act, an action URL to hand to a person. A notice the caller dismissed is left out. Check it before dispatching work.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnly/idempotent/non-destructive), and the description still adds real substance: notices are returned critical-first, carry a stable id/kind/message/action URL, and dismissed notices are filtered out. It even explains the operational consequence of a disconnected tracker (missions cannot start or report back), which helps the agent prioritize escalation to a human.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the purpose and severity ordering. The return-shape sentence and the dismissal-exclusion sentence each carry distinct information, and the closing line lands the call-to-action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no input schema, no output schema and annotations already covering the safety profile, the description supplies everything an agent needs: what the notices represent, ordering, returned fields, exclusion rule, and when to check. Nothing material is missing despite the lack of a documented response schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline is 4. The description correctly implies no caller-side filtering, noting that dismissal state (not a parameter) governs exclusion from results.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb (read) + resource (workspace notices) with scope and ordering (critical first) made explicit. The two example notice kinds (rejected tracker connection, credits running out) make it unmistakable against siblings like get_billing, get_setup_progress or dismiss_notice.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear proactive directive ('Check it before dispatching work'), which is exactly the trigger context an agent needs. It does not explicitly name alternatives (e.g. go to get_billing instead for billing detail), so 5 is not warranted.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_projectGet projectA
Read-onlyIdempotent
Inspect

Fetch a single project by its UUID. Returns 404 if the project is not in the caller's org.

NameRequiredDescriptionDefault
projectIdYesProject UUID

Output Schema

NameRequiredDescription
idYes
nameYes
slugYes
orgIdYes
statusYes
scmRepoYes
trackerYes
scmOwnerYes
autoMergeYes
createdAtYes
issueCodeYes
trackerTeamIdYes
llmConnectorIdYes
scmConnectorIdYes
trackerProjectIdYes
trackerConnectorIdYes
missingConnectorKindsNo
defaultSandboxTemplateIdYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive behavior, and closed-world scope. The description adds useful behavioral context by noting the 404 response when the project is outside the caller's org, though it does not cover rate limits or permission details beyond org scoping.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the core action and followed by the key failure condition. No wasted language.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-by-ID tool with rich annotations, complete input schema, and an output schema, the description covers the essential behavior including the notable 404 org-scoping condition. Nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter is documented as 'Project UUID.' The description reinforces that the lookup is by UUID but adds no syntax or format details beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Fetch a single project by its UUID.' This clearly distinguishes it from list_projects and update_project without needing to name siblings explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when to use it: retrieving one project by UUID. It does not explicitly name alternatives such as list_projects or state when not to use it, but the single-resource intent is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_sandbox_build_logRead a sandbox build's logA
Read-onlyIdempotent
Inspect

A page of a build's log after sequence number after. Call again with after = nextAfter until done is true.

NameRequiredDescriptionDefault
afterNoReturn chunks after this sequence number (default 0)
buildIdYesBuild id (tbd_...)
templateIdYesSandbox id (sbt_...)

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly, idempotent, non-destructive, so the safety profile is covered. The description goes further by disclosing pagination semantics and two response fields (nextAfter, done) despite there being no output schema, which is genuinely additive context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the resource, then the continuation contract. No filler and nothing buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of explaining the return shape and does so for the key pagination fields. It stops short of describing log content or chunk format, but the pagination contract is the operationally critical part and it is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so templateId, buildId, and after are all documented in the schema itself; that establishes a baseline of 3. The description restates the role of `after` and links it to nextAfter, adding only marginal meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies the resource precisely (a page of a build's log) and frames it as a paginated read. It does not, however, differentiate from the sibling get_agent_run_logs, leaving the agent to infer which log source applies.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit driving loop: call again with after = nextAfter until done is true. That is real usage guidance for a paginated tool, but there is no when-to-use / when-not-to-use guidance relative to the alternative log-retrieval tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_sandbox_capacityRead how many agents are runningA
Read-onlyIdempotent
Inspect

How many agents the org's missions are running at once against its limit (null while the org is unclassified), which missions wait for an agent slot and why, and whether every slot on the platform is taken. Read from the platform's once-a-minute capacity check.

NameRequiredDescriptionDefault
orgIdNoRequired only for a caller in several orgs

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds genuinely new behavioral context: the value is null while the org is unclassified, and the data is sourced from a once-a-minute capacity check, which signals staleness/freshness to the caller. It does not discuss auth or permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler, and the returned fields are packed efficiently. The first sentence is a long triple-clause construction that is somewhat dense but every clause carries information, so it stays close to optimal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of explaining the return value, and it does so by naming the three result components. Combined with annotations that cover the safety profile, the definition is nearly complete; only permission/authorization context is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single orgId parameter, and the schema already explains it is required only for a caller in several orgs. The description alludes to 'the org's missions' but adds no syntax or format detail beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource (org's agent capacity against its limit) and enumerates the three things it returns: current usage vs limit, queued missions and why, and whether the platform is fully saturated. It is far more specific than a tautology, but it never contrasts itself with siblings like get_mission or get_agents_override, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use, when-not-to-use, or alternative-tool guidance. The only contextual clue is the parenthetical about the once-a-minute capacity check, which implies freshness rather than routing. An agent must infer on its own when this tool is preferable to inspecting a mission directly.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_scheduled_taskGet scheduled taskA
Read-onlyIdempotent
Inspect

Fetch one scheduled task by id: its project, name, prompt, triggers (schedule), labels, template key, skip-if-previous-open setting, enabled state, and next/last run times. An id outside the caller's accessible projects returns not found. For what each fire produced, use list_scheduled_task_runs.

NameRequiredDescriptionDefault
scheduledTaskIdYesScheduled task id, e.g. from list_scheduled_tasks

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only/idempotent/non-destructive/non-open-world, so the bar is lower. The description still adds genuine context: 'An id outside the caller's accessible projects returns not found' discloses access-scoping behavior not present in the annotations or schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the verb and the field enumeration, followed by the one relevant caveat and the sibling pointer. No wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return-shape burden and does so by listing the returned fields (project, name, prompt, triggers, labels, template key, skip setting, enabled, run times). Access behavior and the alternative tool are also covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single parameter is already documented (including its source 'from list_scheduled_tasks'). The description adds no additional syntax or format meaning beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Fetch one scheduled task by id') plus the resource, then enumerates the returned fields, so the agent knows exactly what it retrieves. It distinguishes itself from siblings by pointing to list_scheduled_task_runs for run-level data.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes the agent to list_scheduled_task_runs when the goal is run output, which is clear alternative guidance. It does not explicitly contrast against list_scheduled_tasks or state full when-not conditions, keeping it short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_setup_progressGet setup progressA
Read-onlyIdempotent
Inspect

Read what the workspace has set up with SHIP so far, as the console's setup checklist shows it: up to 13 milestones in three groups, taken in order (get started, make it yours, make it autonomous). activeGroup names the group to work on now, and each group carries its own count. Each milestone says whether it is reached and, when a console page takes it and the caller's role can, the page that does it. next names the first milestone not reached yet. Use it to suggest the next step to a person. If they say they will not finish setup, ask why and record it with report_setup_blocker.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
nextYes
orgIdYes
groupsYes
guidesYes
completeYes
totalCountYes
activeGroupYes
firstMissionYes
reachedCountYes
activeGroupProgressYes

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive, closed-world), yet the description adds real behavioral context the annotations don't: the ordered group model, the role-dependent gating of the per-milestone console page, and that `activeGroup`/`next` encode what to work on. Some of this restates the output schema's fields, which keeps it from being fully additive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and readable in one pass, with no filler sentences. It is dense in the middle with return-field detail that the output schema already carries, so a little of it could be trimmed without losing agent-relevant meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With annotations covering safety, an output schema defining the return shape, and no input parameters, the description supplies everything an agent needs: what the checklist contains, how groups and milestones are ordered, what `next` means, and which sibling to call when setup stalls.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; the baseline for a no-parameter tool applies. The prose describes response semantics instead, which is a reasonable substitute but not parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource: 'Read what the workspace has set up with SHIP so far, as the console's setup checklist shows it.' It further scopes the payload (13 ordered milestones across three named groups) so an agent can immediately distinguish this read-only progress query from mutating siblings like mark_setup_step or report_setup_blocker.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit use case ('Use it to suggest the next step to a person') and routes the adjacent case to a named sibling: if the person won't finish setup, ask why and record it with report_setup_blocker. That is a concrete when-to-use plus alternative-selection rule, not inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

healthzAPI health probeA
Read-onlyIdempotent
Inspect

Returns the upstream API status, environment, and version.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the tool as read-only, idempotent, non-destructive, and not open-world. The description adds value by specifying the returned content (status, environment, version), which is useful context beyond the structured safety hints. However, it does not disclose any further behavioral traits such as authentication requirements or rate limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that efficiently communicates the tool's purpose without any wasted words. It is appropriately sized for such a simple probe.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (0 parameters, no output schema) and the annotations covering its safety profile, the description provides enough context by stating what it returns. It could be slightly more complete by mentioning that it takes no arguments or by noting typical usage, but it is largely sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool accepts zero parameters, and the schema description coverage is 100% (vacuously). Per the rubric, a zero-parameter tool receives a baseline of 4, as there are no parameter semantics to clarify.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Returns') and resource ('upstream API status, environment, and version'), making the tool's function immediately clear. No sibling tool performs a similar health/status check, so it is easily distinguishable from the rest of the toolset.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description offers no explicit guidance on when to use this tool versus alternatives, nor does it state any prerequisites or exclusions. While the name 'healthz' implies a health-check use case, the description itself provides no such direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

inspect_missionInspect missionA
Read-onlyIdempotent
Inspect

Fetch the typed platform-inspection report for a mission, by its issue id. Returns per-section KV/D1/R2 state: task state, retry reason, dispatch error, latest review + QA verdicts, run index, per-attempt provenance, running sandboxes, and session-log heads. Requires the inspect:read scope.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-199 or WEB-42
orgSlugNoNarrow an ambiguous issue code to one org
sectionsNoComma-separated section filter (default all): taskState,retryReason,provenance,…
projectIdNoScope to a specific project id

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive), and the description adds material context beyond them: the required 'inspect:read' scope, the fact that the report is 'typed' and partitioned per-section across KV/D1/R2 stores, and the specific fields returned. It stops short of noting rate limits or access-failure behavior, but the auth requirement is a real addition.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the verb and resource, followed by the payload inventory and the auth requirement. The long section list is dense but every element earns its place by telling the agent what it can inspect.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly carries the burden of describing the return payload, enumerating sections down to session-log heads, and it states the required scope. The remaining gap is that it never clarifies the relationship to get_mission, so an agent with both tools in context must guess which to call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the schema documenting issueId syntax (SHIP-199, WEB-42), orgSlug disambiguation, the sections filter, and projectId scoping. The description adds only the 'by its issue id' framing, which the schema already supplies, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Fetch the typed platform-inspection report for a mission, by its issue id') and enumerates the report's contents, so the agent knows this is a read of a diagnostic report keyed by issue id. It does not explicitly distinguish itself from the sibling get_mission, which fetches the mission itself, leaving the boundary implied rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use or when-not, and the obvious alternative (get_mission) is never named. Usage is only implied by the content list — task state, retry reason, dispatch error, provenance — which signals a diagnostic/troubleshooting purpose an agent could infer.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_agent_runsList agent runsA
Read-onlyIdempotent
Inspect

List the agent runs, live and finished, in the caller's accessible projects, newest first. Each run is one agent in its own sandbox. Returns id (the run's sandbox id), issueId, role, harness, status, instance type, and latest CPU/memory/disk metrics. When more runs match, nextCursor is set: pass it back as cursor, with the same filters, for the next page.

NameRequiredDescriptionDefault
roleNoOnly the runs of this agent role
limitNoMax results (default 50). Narrow with projectId, issueId, role or status before raising it.
cursorNonextCursor from the previous page; omit for the newest runs
statusNoFilter by how the run stands: running, completed (finished), stopped (ended on purpose before it finished), failed, or stale (closed without a report)
issueIdNoOnly the runs of this mission, by its issue id, e.g. SHIP-412
projectIdNoFilter to one project id

Output Schema

NameRequiredDescription
dataYes
nextCursorYes

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this a read-only, idempotent, non-destructive operation, so safety is covered. The description adds genuinely useful context beyond that: the result is auth-scoped to the caller's accessible projects, ordering is newest-first, and pagination is driven by a nextCursor that must be replayed with the same filters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with no filler: scope and ordering first, then the unit of a run, then the return shape, then pagination. Every sentence carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, the return-value enumeration is mildly redundant, and the remaining gap is routing guidance versus sibling get/list tools. Otherwise the description covers scope, ordering, filtering intent, and the pagination contract needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3, and the description goes slightly beyond by explaining the cursor chaining contract (pass nextCursor back as cursor with the same filters) rather than just restating the field. It doesn't otherwise add syntax for role/status enums or the projectId/issueId identifiers, which are already well documented in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (list agent runs), plus scope (caller's accessible projects) and ordering (newest first), which distinguishes it from get_agent_run and list_scheduled_task_runs. The extra sentence defining a run as one agent in its own sandbox removes any ambiguity about the unit of listing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: the description covers what is listed and how pagination works, but never says when to prefer this over get_agent_run for a single run or how it relates to list_scheduled_task_runs. An agent can infer the listing role from the name alone, so this is adequate but not prescriptive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_artifactsList artifactsA
Read-onlyIdempotent
Inspect

List proof-of-work artifacts (QA screenshots/video/traces) for the caller's accessible projects. Each row carries kind, content type, caption, the acceptance criterion it proves, the sandboxId of the agent run that produced it, and a long-lived signed serving url. Filter by issueId to get one ticket's evidence: the artifacts of every run, superseded QA attempts included, up to limit; a note says when more exist. For the mission's proof of work specifically, read proofRunSandboxId from GET /v1/missions/{id} and pass it as sandboxId.

NameRequiredDescriptionDefault
limitNoMax results (default 50). Narrow with issueId or sandboxId before raising it.
issueIdNoFilter to one issue (e.g. SHIP-133)
projectIdNoFilter to one project id
sandboxIdNoFilter to ONE agent run's artifacts (e.g. the mission's proofRunSandboxId)

Output Schema

NameRequiredDescription
dataYes
nextCursorYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare a safe, idempotent read, so the description's value is in behavioral detail: results include superseded QA attempts, a note indicates when more results exist, and serving urls are long-lived and signed. Some of the row-field enumeration overlaps with the existing output schema, which lowers the marginal value slightly.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three dense sentences, front-loaded with the resource and scope before the filtering workflow. No filler, though the row-field enumeration in sentence two is the least essential part given the output schema exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a four-parameter, all-optional list tool with an output schema and full annotation coverage, the description covers what the agent needs: scope, filters, superseded coverage, pagination signal, and the mission proof-of-work recipe. Minor gap is not pointing to get_artifact for single-item retrieval.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds real meaning: issueId returns every run's artifacts including superseded attempts up to limit, and sandboxId is tied to a specific agent run (the mission's proofRunSandboxId). That exceeds what the schema descriptions alone convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb and resource ('List proof-of-work artifacts') and scopes it to 'QA screenshots/video/traces' for the caller's accessible projects. An agent can distinguish it from get_artifact or list_agent_runs without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete routing guidance: use issueId to get one ticket's evidence, and for a mission's proof of work read proofRunSandboxId from GET /v1/missions/{id} and pass it as sandboxId. It stops short of naming get_artifact as the alternative for fetching a single artifact, so it is clear context rather than full when/when-not coverage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_billing_ledgerList workspace credit ledgerA
Read-onlyIdempotent
Inspect

List where the workspace's credits went, newest first: one entry per mission (its start, each further 3 agent-hours, and messages charged before messages became free), per grant, per adjustment and per expiry. Pass nextCursor back as cursor for the next page. Owners and admins only.

NameRequiredDescriptionDefault
limitNoEntries per page, 1-100 (default 50).
cursorNonextCursor from the previous page; omit for the newest entries.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive, non-open-world behavior, so the description is not responsible for the safety profile. It adds useful context beyond annotations: the reverse-chronological order, what each ledger entry represents, how to page, and the owner/admin restriction. It does not cover rate limits or return fields, but it meaningfully enriches the behavioral picture.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences with no wasted words, front-loading the core purpose before pagination and permission details. Every sentence earns its place by adding distinct information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description bears some burden for explaining return content, and it does so by naming entry types and ordering. It covers pagination and access requirements but does not describe the fields within each entry (amount, timestamp, etc.), leaving a small gap for a ledger tool. Overall it is complete enough for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters fully. The description reinforces cursor usage ('Pass nextCursor back as cursor for the next page') and implies newest-first ordering, but adds no syntax or meaning beyond what the schema provides. The baseline of 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'List where the workspace's credits went.' It further distinguishes itself from siblings like get_billing by specifying a paginated ledger with one entry per mission, grant, adjustment, and expiry. An agent can tell exactly what this tool returns and how it differs from a billing summary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage context: newest-first ordering, entry composition, pagination via nextCursor, and an explicit permissions restriction ('Owners and admins only'). It does not name alternative tools or state when not to use this one, but the operating context is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_credit_offersList credit packs, the monthly plans and EnterpriseB
Read-onlyIdempotent
Inspect

List what SHIP sells: the credit packs (credits, the price in US dollars before tax, which checkout adds for the buyer's location, and the price per credit), the monthly plans under team.plans (Starter and Scale: each its credits a month, monthly price, price per credit, which credits added on top also cost, and agents at once), the range of credits a workspace on a plan can add at once, Enterprise under enterprise (the minimum yearly credit commitment, the highest and lowest price per credit, and the agents at once at the minimum; contracts are sold by SHIP, not through checkout), what Free and the packs allow (limits), plus the trial size. The same catalog the pricing page shows.

NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnly, idempotent, non-destructive, non-open-world behavior, so the safety profile is covered. The description adds genuinely useful context beyond that: that checkout applies location-based tax, that Enterprise contracts are not purchasable through checkout, and precisely what each catalog section contains — valuable given no output schema exists.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The entire definition is a single sprawling sentence with deeply nested parentheticals that reads like a transcribed response schema. It is dense and poorly front-loaded, forcing the reader to parse clauses rather than grasp the tool's role at a glance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no parameters, no output schema, and annotations covering safety, the description's job is largely to convey what comes back — which it does exhaustively (packs, team.plans, enterprise, limits, trial). It is complete enough to call and interpret, short of explicitly stating its relationship to the checkout tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is no parameter meaning to convey; the 4 baseline applies. The description does not need to compensate for any schema gap here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (list) and resource (SHIP's catalog of credit packs, monthly plans, Enterprise tiers, limits, trial). The scope is unambiguous, though it does little to differentiate itself from sibling list_* tools and reads more like a response inventory than a purpose statement.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit when-to-use, when-not-to-use, or named alternatives. The mention that 'contracts are sold by SHIP, not through checkout' faintly gestures at start_credit_checkout, but the agent is left to infer that this is the read-only catalog-lookup counterpart to the checkout/request_more_credits tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_environmentsList preview environmentsA
Read-onlyIdempotent
Inspect

List per-mission preview environments for the caller's accessible projects. Returns id, issueId, projectId, prNumber, workerName, and lifecycle status.

NameRequiredDescriptionDefault
limitNoMax results (default 50). Narrow with issueId, projectId or status before raising it.
statusNoFilter by lifecycle status
issueIdNoFilter to one issue id
projectIdNoFilter to one project id

Output Schema

NameRequiredDescription
dataYes
nextCursorYes

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, openWorldHint=false and destructiveHint=false, so the safety profile is fully covered by structured data. The description adds one genuinely behavioral fact beyond annotations — results are scoped to the caller's accessible projects (authorization surface) — but no pagination, ordering, or empty-result behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste, with the scope constraint front-loaded ahead of the returned fields. Nothing here needs trimming or reordering.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema exists so return values need not be explained, and annotations carry the safety profile; the description supplies the scope qualifier. The one gap is that the returned-field sentence partially duplicates the output schema rather than using the space for filtering/ordering guidance.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all four filters (limit, status, issueId, projectId) including the status enum are fully documented in the schema. The description adds no parameter meaning beyond that, which is the expected baseline when the schema does the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('List') and resource ('per-mission preview environments') plus scope ('caller's accessible projects'). It is clearly distinguishable from the singular get_environment and the mutating destroy_environment siblings, though it never names them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: listing is the obvious read operation, and the schema's limit description carries the real guidance ('Narrow with issueId, projectId or status before raising it'). The description itself offers no when-to-use vs alternatives statement or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_bindingsList the org's MCP bindingsA
Read-onlyIdempotent
Inspect

Every MCP server binding across the projects you can see in your org: project, server, transport, status, enabled roles, how it authenticates, and the NAMES of the credential fields set (never values). Use it to find where a credential is used; use list_mcp_servers for one project's full detail.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, openWorldHint=false and destructiveHint=false, so safety is covered. The description still adds real context beyond them: the security-relevant guarantee that credential fields are returned as NAMES and 'never values', plus the visibility scope limiting results to projects the caller can see. It stops short of covering pagination or result size.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with scope, then the field inventory, then the disambiguation. The parenthetical '(never values)' is a compact way of carrying a critical guarantee. No filler sentences.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description does the work of describing the return surface (project, server, transport, status, roles, auth, credential field names) and the visibility boundary. Combined with the sibling disambiguation and the annotations' safety profile, an agent has everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is no parameter semantics to document and the baseline is 4. The description instead spends its space on the shape of the returned records, which is the useful equivalent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('list ... MCP server binding') plus its scope ('across the projects you can see in your org'), then enumerates exactly what each row contains: project, server, transport, status, enabled roles, auth method, and credential field NAMES. It explicitly separates itself from list_mcp_servers, so an agent can route without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an affirmative use case ('Use it to find where a credential is used') and names the alternative with its distinguishing condition ('use list_mcp_servers for one project's full detail'). Nothing about when to pick this over its sibling is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_mcp_serversList a project's MCP serversA
Read-onlyIdempotent
Inspect

The MCP servers a project's repository declares in its MCP file (.mcp.json by default), each with its binding: enabled roles, transport, status, approved tools, per-role effective tool counts (null when the server runs inside the sandbox and the role gets every tool it exposes), and the NAMES of the credential fields set (never values). Also reports whether the file could be read (configStatus).

NameRequiredDescriptionDefault
projectIdYesProject id, e.g. from list_projects

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive and closed-world, so safety is covered. The description still adds real behavioral context: per-role effective tool counts are null when the server runs inside the sandbox and the role gets every exposed tool, credential fields are returned as NAMES and never values, and configStatus reports whether the file could be read.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One long but well-ordered sentence that front-loads the resource and then enumerates the returned binding fields; the null-case and never-values notes are embedded where they matter. Slightly dense, but nothing is wasted and no boilerplate is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must characterize the return value, and it does so field-by-field (enabled roles, transport, status, approved tools, per-role tool counts, credential field names, configStatus). Safety is covered by annotations and the single parameter by the schema, so nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single parameter at 100% schema description coverage ('Project id, e.g. from list_projects'), so the schema carries the semantics. The description adds nothing about projectId (no format, no source beyond the schema), which is the expected baseline when coverage is complete.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('The MCP servers a project's repository declares in its MCP file (.mcp.json by default)') and enumerates the binding detail returned, so the agent knows exactly what this retrieves. It does not differentiate itself from the sibling list_mcp_bindings, but the scope (config-file declared servers for one project) is otherwise unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use, when-not, or alternative is named. With list_mcp_bindings, bind_mcp_server, unbind_mcp_server and refresh_mcp_server_tools in the sibling set, an agent gets no help deciding between reading declared config versus listing bindings. Usage is only faintly implied by the word 'declares'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_missionsList missionsA
Read-onlyIdempotent
Inspect

List missions in the caller's accessible projects, newest activity first, with total for paging. Each has issueId, title, activity (what it does now), pipelineProgress, review/check/test state, PR, cost and paused. Archived missions are left out. Requires the missions:read scope.

NameRequiredDescriptionDefault
limitNoPage size (default 25). Narrow with projectId, activity or search before raising it.
offsetNoRows to skip, for the next page
searchNoMatch the issue id, title, owner or repository
activityNoFilter by what the mission does now
projectIdNoFilter to one project id

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive and closed-world, so the safety profile is covered. The description adds real value beyond that: sort order by newest activity, that a total is returned for paging, that archived missions are filtered out, and the required missions:read scope.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tightly packed sentences, front-loaded with the resource and scope before exclusions and auth. The enumeration of returned fields is somewhat long but earns its place since there is no output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully enumerates key returned fields (issueId, title, activity, pipelineProgress, review/check/test state, PR, cost, paused), and it covers ordering, exclusions, paging total and required scope. Only the precise pagination mechanics are left vague.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the schema already explains limit, offset, search, activity and projectId in detail. The description only alludes to paging ('total for paging') and filters, adding little beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List missions'), the scope ('caller's accessible projects'), the ordering ('newest activity first'), and paging behavior ('with total for paging'). This clearly distinguishes it from get_mission and inspect_mission without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description notes the exclusion of archived missions and the required missions:read scope, plus the schema tells the caller to narrow with projectId/activity/search before raising limit. However, it never states when to prefer this over get_mission or inspect_mission, so selection guidance is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_orgsList every org (cross-tenant)A
Read-onlyIdempotent
Inspect

List every organization on the platform with its owner email, member count, and pending invite status. Requires platform-admin (cross_org) access. Cursor-paginated (default page size 50) — feed the response's nextCursor back as the cursor input to reach the next page; a null nextCursor means there are no more pages.

NameRequiredDescriptionDefault
limitNoPage size, 1-100 (API default 50)
cursorNoOpaque page cursor, echoed back as nextCursor by a prior call

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive), and the description adds substantive context beyond them: the required cross_org privilege level, the pagination mechanism, the default page size of 50, and the termination condition. This is unusually rich behavioral disclosure for a read-only list tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler, front-loaded with purpose and access requirement before the pagination mechanics. Every clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by naming the fields returned and fully describing the cursor lifecycle. Nothing needed to page through results correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the cursor and limit parameters are already documented, setting a baseline of 3. The description adds value beyond the schema by explaining the round-trip contract - nextCursor from the response is fed back as the cursor input - and that a null nextCursor signals the last page.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) and resource (every organization on the platform), and immediately scopes it as cross-tenant with the returned fields (owner email, member count, pending invite status). No sibling tool lists orgs, so an agent can tell this is the only cross-tenant org enumeration available.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides a clear precondition for use - platform-admin (cross_org) access - which lets an agent decide whether it is eligible to call it. It does not name an alternative or state when NOT to use it, but no sibling tool overlaps with org listing, so the missing exclusions are low-impact.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_projectsList projectsA
Read-onlyIdempotent
Inspect

List the projects accessible to the calling principal. Returns id, slug, name, status, createdAt for each. An empty list means the workspace is not set up yet: no mission can run until the person connects GitHub and a model key and creates a project in the SHIP console. Call get_setup_progress and give the person the console page of its next step.

NameRequiredDescriptionDefault
limitNoMax results (default 20)
cursorNoOpaque pagination cursor from a previous call

Output Schema

NameRequiredDescription
dataYes
nextCursorYes

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world, non-destructive behavior, so the description does not need to restate safety. It adds genuine behavioral context the annotations lack: the exact fields returned and, importantly, the semantics of an empty result as a not-yet-configured workspace with a hard consequence ('no mission can run').

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with purpose, then return shape, then the remediation path for the empty case. Minor redundancy in listing return fields when an output schema already exists, but nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Output schema and annotations carry the return shape and safety profile, so the description correctly focuses on the non-obvious empty-state interpretation and the follow-up call. Pagination behavior for the cursor is left entirely to the schema, a small remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters (limit with default 20 and max 100, and the opaque cursor) are fully documented in the schema. The description adds nothing about pagination mechanics or how to iterate with the cursor, so the baseline of 3 is warranted.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List the projects accessible to the calling principal'), names the scope of accessibility, and enumerates the returned fields. It is trivially distinguishable from the singular get_project sibling and from the other list_* tools, which target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit conditional branch: an empty result means the workspace is not set up, and the agent should call get_setup_progress and surface the console page of its next step. It does not explicitly say when to prefer get_project over this tool, but the list-vs-get distinction is self-evident.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_sandbox_registriesList private registriesA
Read-onlyIdempotent
Inspect

The org's private package registries sandboxes install from: host, auth scheme, username. Never the secret.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, and non-destructive, so the safety bar is low, yet the description adds genuinely new behavioral information: the response includes auth scheme and username but 'Never the secret.' That credential-exposure disclosure is exactly the kind of thing annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single clause with no filler, and the key constraint — 'Never the secret.' — is placed at the end for emphasis. The telegraphic, fragment-like phrasing ('registries sandboxes install from') costs a little readability but wastes no space.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no parameters, no output schema, and full annotation coverage, the description is nearly sufficient: it names the resource and the returned fields. Explaining the return shape stands in for the missing output schema, though it doesn't say how many registries or how ordering works.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing to disambiguate; the schema coverage is 100% by definition. Baseline 4 applies, and the description correctly spends no words on nonexistent inputs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific resource — the org's private package registries that sandboxes install from — and enumerates the fields returned (host, auth scheme, username). This is far more than a restated title, though it never names the sibling write tools (save_sandbox_registry, delete_sandbox_registry) to sharpen the boundary.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the read-only listing nature and the readOnlyHint annotation rather than stated. There is no explicit when-to-use or when-not, and no pointer to save_sandbox_registry/delete_sandbox_registry for the mutating counterparts. Adequate but leaves routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_sandbox_startersList sandbox startersA
Read-onlyIdempotent
Inspect

Ready-made sandbox definitions (Python, Java, Go, Rust, Ruby) to create a sandbox from.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false and openWorldHint=false, so the safety and side-effect profile is fully covered. The description adds only that the definitions are 'ready-made' and enumerated by language, but says nothing about return shape, pagination, or whether starters can be customized - modest added value given the annotations carry the bulk.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single compact sentence with no filler and the resource front-loaded; the language enumeration is informative rather than padding. It is a touch terse - an explicit verb ('List') would make it self-contained instead of relying on the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter list tool with no output schema, the description conveys the basic content but not the return structure - the agent cannot tell whether it gets names, versions, or full build definitions. Given the absence of an output schema, one clause about what a starter record contains would close the remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so per the rubric the baseline is 4. There is nothing for the description to clarify beyond confirming there is no filtering or input, which it implicitly does by describing a fixed set of starters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource - ready-made sandbox definitions - and even enumerates the available languages (Python, Java, Go, Rust, Ruby), which tells an agent concretely what it will get. The verb 'list' comes from the name rather than the description, and it does not explicitly distinguish itself from the crowded family of sibling list_* tools (e.g. list_sandbox_templates).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The trailing phrase 'to create a sandbox from' implies the tool is a precursor to sandbox creation, which is useful implied usage context. However, there is no explicit when-to-use guidance and no mention of how it relates to sibling tools such as list_sandbox_templates or build_sandbox_template, leaving the agent to infer the choice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_sandbox_templatesList custom sandboxesA
Read-onlyIdempotent
Inspect

List the org's sandboxes (definitions agents start in), with each one's active version, size and latest build status.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered structurally. The description adds genuine value by disclosing what each entry includes (active version, size, latest build status), but says nothing about ordering, pagination, or whether inactive/archived templates appear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single well-formed sentence that front-loads the core action and packs the return contents into a compact clause. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter read tool with no output schema, the description is nearly sufficient: it names the resource and previews the returned fields, and annotations carry the safety profile. The only real omission is any hint of ordering or result-size behavior, which is minor here.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline is 4. The description correctly implies an unfiltered org-wide listing, which is consistent with there being no filter inputs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List the org's sandboxes') and clarifies the concept with a parenthetical ('definitions agents start in') that disambiguates it from a runtime sandbox instance. However, it never differentiates itself from close siblings like list_sandbox_starters, list_sandbox_registries, or list_sandbox_template_versions, which an agent must pick between.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance and no mention of alternatives, despite the sibling list containing at least four other sandbox-related list tools. The agent is left to infer the selection criteria from names alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_sandbox_template_versionsList a sandbox's versionsA
Read-onlyIdempotent
Inspect

Every saved version of a sandbox, newest first, with the active one marked and each one's latest build.

NameRequiredDescriptionDefault
templateIdYesSandbox id (sbt_...)

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive), so the description correctly focuses on output behavior: ordering (newest first), the active-version marker, and inclusion of each version's latest build. That extra content detail is genuinely useful beyond what the annotations state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the ordering and marking details are packed efficiently. Nothing wasteful, though it is terse enough to leave some gaps.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter read tool with no output schema, the description conveys what comes back (versions, order, active flag, builds). Nothing critical is missing, though it could note that no filtering/pagination applies.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the sole templateId parameter is documented as a sandbox id (sbt_...). The description adds no further parameter meaning, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (list) and resource (a sandbox's saved versions), which cleanly separates it from sibling list_sandbox_templates. It does not explicitly name that sibling, but the resource boundary is unambiguous from the wording.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use or when-not-to-use guidance, and no mention of the alternative activate_sandbox_template_version or list_sandbox_templates. Usage is only implied by the resource being listed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_scheduled_task_runsList a scheduled task's run historyA
Read-onlyIdempotent
Inspect

Run history for one scheduled task, each row's status (dispatching/dispatched/skipped/failed) and, once its issue exists, the joined mission progress/cost. A 'dispatched' status means the issue was created and the mission started — never that the work succeeded; read the joined mission state for that. Newest first; when older runs exist, pass nextCursor back as cursor for the next page.

NameRequiredDescriptionDefault
limitNoRuns per page, 1-200 (default 50)
cursorNonextCursor from the previous page; omit for the newest runs
scheduledTaskIdYesScheduled task id

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, and the description goes well beyond them with the highest-value disclosure here: the status enum interpretation and the explicit warning that 'dispatched' means the mission started, never that work succeeded. That prevents a serious misread of the data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, each front-loading its key fact (what rows contain, the status caveat, ordering/pagination). No filler, and the critical caveat is placed where an agent will read it.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return-value burden and does so: row contents, status semantics, joined mission data, ordering, and pagination are all covered. An agent can call and correctly interpret this without further context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so scheduledTaskId, limit, and cursor are already documented; the description largely restates the cursor round-trip that the schema describes. Baseline 3 is appropriate since the schema does the parameter work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Run history for one scheduled task') and immediately enumerates the row contents (status, joined mission progress/cost). This clearly separates it from sibling get_scheduled_task (task config) and list_scheduled_tasks (the task list itself).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explains ordering ('newest first') and the exact pagination workflow ('pass nextCursor back as cursor for the next page'), which is the operative how-to-use guidance. It does not explicitly contrast with get_scheduled_task or list_scheduled_tasks, so it falls short of 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_scheduled_tasksList scheduled tasksA
Read-onlyIdempotent
Inspect

List recurring agent missions (scheduled tasks) across the caller's accessible projects — schedule, enabled state, and next/last fire time for each. Pass projectId for the tasks of one project.

NameRequiredDescriptionDefault
projectIdNoOnly the scheduled tasks of this project

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the safe read-only, idempotent, non-destructive, closed-world profile, so the description only needs to add context. It does: it discloses the per-item return fields (schedule, enabled state, next/last fire time) and the authorization scope (only accessible projects). Pagination or result-count behavior is not mentioned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, with the scope and returned fields front-loaded before the parameter usage note. Every clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by enumerating the notable returned fields, and it defines global vs project-scoped behavior for a tool with one optional parameter. It stops short of mentioning pagination, ordering, or volume limits, which is the only remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single parameter is fully documented in the schema, so the baseline is 3. The description earns above baseline by spelling out the behavioral difference between supplying projectId and omitting it (all accessible projects vs one project), which the schema's parameter text does not state.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb (list) and resource (recurring agent missions / scheduled tasks) plus the scope (caller's accessible projects), and pre-empts the mission-vs-scheduled-task terminology overlap that exists among siblings like list_missions and list_scheduled_task_runs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The closing sentence gives a clear conditional: omit projectId to span all accessible projects, pass it to narrow to one. That covers the practical when-to-use decision, but it names no alternative tools and states no exclusions, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_scheduled_task_templatesList scheduled task templatesA
Read-onlyIdempotent
Inspect

Read-only gallery of curated scheduled task templates — a prepared prompt plus a suggested cadence. A template is content, not a code path: pass its key as templateKey on create_scheduled_task for attribution, or copy its prompt/schedule into a hand-written scheduled task.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description still adds real conceptual context the annotations cannot: templates are pure content, not code paths, and their keys serve an attribution role. It does not describe return shape or pagination, but for a no-param read against fully-covered annotations this is solid added value.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with zero filler. The core identity ('read-only gallery of curated scheduled task templates') is front-loaded, and the second sentence immediately resolves the natural confusion about how templates relate to actual task execution.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple zero-parameter, read-only listing with no output schema, the description supplies enough: it says what templates contain, that keys are meaningful, and how they are consumed. The main residual gap is that it does not hint at the shape of the returned list beyond prompt/cadence/key, which matters slightly more given there is no output schema to lean on.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is no input semantics to clarify and the baseline is 4. The only field-like mention (templateKey) belongs to a sibling create tool, not to this one, so it neither helps nor distorts parameter understanding here.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb+resource (listing curated scheduled task templates) and defines what a template actually is — a prepared prompt plus a suggested cadence. It also differentiates the concept from related siblings by explaining that a template is content rather than an executable path, so an agent can place it relative to create_scheduled_task without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly explains the downstream context: pass a template's key as templateKey on create_scheduled_task, or copy its prompt/schedule into a hand-written task. This frames when the listing is useful (discovery/attribution before creating a task), but it never explicitly states an exclusion or says 'call this before creating a scheduled task' as a directive, so it stops just short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mark_setup_stepMark a setup stepA
Idempotent
Inspect

Mark a step of the setup checklist completed or skipped for the person this token acts for, as the console's Mark as complete and Skip controls do; open removes the mark. Give step for one milestone, or group to mark every open step of that group (the console's Skip all). A marked step counts as done on that person's checklist and its status says how; reached stays false until SHIP sees the step done. Returns the checklist as it now stands. A token that acts for no person is refused.

NameRequiredDescriptionDefault
markYesopen removes the caller's mark.
stepNoThe milestone id from get_setup_progress. Give step or group.
groupNoA group id: marks every open step of the group.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover the safety profile (idempotent, non-destructive, not read-only), and the description adds genuinely useful behavior: the return value (the checklist as it now stands), the refusal case for a token that acts for no person, and the semantic that 'reached' stays false until SHIP sees the step done.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but front-loaded: the core action and the step/group distinction come first, and side details (return value, refusal) come later. The semicolon-heavy run-on sentences are slightly harder to parse than ideal but every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by stating the return value, the refusal condition, and the relationship to setup progress. Nothing critical is missing for an agent to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and all three parameters are enum-documented, so the schema carries the load. The description clarifies the intent of step vs group (one milestone vs every open step of a group), which adds modest value beyond the schema but no new format or constraint detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (mark) and resource (a step of the setup checklist) with the exact state transitions available (completed/skipped/open). It is clearly distinguishable from the sibling get_setup_progress, which is pointed to as the source of step ids.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explains the two modes clearly: give 'step' for one milestone, or 'group' to mark every open step of a group (the console's Skip all). It also notes open removes the mark. It stops short of explicit when-not-to-use guidance, but the routing between step and group is well covered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

merge_missionMerge a mission's pull requestA
DestructiveIdempotent
Inspect

Merge the mission's open pull request on GitHub with the repository's own merge method; the mission then becomes done, and the merge cannot be undone from here. When the branch is out of date with its base, answers 202 with status 'updating_branch' and a message: SHIP updates the branch and merges once CI passes, with no further call. Answers 409 with the reason for any other blocker (missing reviews, failing checks, another branch protection). On a merge conflict SHIP starts the builder to fix it and merges once the mission is ready again, with no further call. Requires the missions:write scope.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (destructiveHint=true, idempotentHint=true, openWorldHint=true), the description adds the irreversible nature of the merge, the missions:write scope requirement, the auto-branch-update and auto-conflict-fix behavior, and the meaning of the 202/409 responses. That is rich behavioral context an agent cannot get from structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core action and its irreversibility lead, followed by outcome handling; every sentence carries operational information. It is dense but slightly long, and the response-handling clauses could be tightened without losing meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description covers the failure/async paths (202, 409, conflict auto-fix), permissions, and irreversibility, which is most of what an agent needs. It does not describe the shape of a successful merge response, which is a minor remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single required parameter with 100% schema coverage, so the schema already documents issueId fully (Linear key or GitHub Issues id). The description adds no parameter-level detail beyond the schema, which is the expected baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ('Merge the mission's open pull request on GitHub') and states the repository's own merge method is used, so the action is unmistakable. It does not differentiate itself from adjacent siblings such as set_mission_auto_merge or set_mission_disposition, which is the one gap preventing a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear conditional guidance on how to proceed in each outcome: 202 updating_branch means stop calling, 409 means a blocker exists and the reason is returned, and a merge conflict is handled automatically with no further call. What is missing is an explicit when-not-to-use or a named alternative to choose instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pause_missionPause a missionA
Idempotent
Inspect

Hold a mission before its next stage. The stage that runs now finishes; no agent is stopped. Use resume_mission to continue. Requires the missions:write scope and a role that can control missions.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With annotations already covering readOnlyHint=false, destructiveHint=false, and idempotentHint=true, the description adds valuable behavioral context: the current stage finishes, no agent is stopped, and the operation is reversible via resume_mission. It also discloses required permissions. It stops short of full transparency about edge cases like already-paused missions or effects on scheduled tasks.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is four short, front-loaded sentences with zero waste. The core behavior is stated first, followed by the key operational nuance and the alternative, then the prerequisite.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple pause tool with one fully described parameter, no output schema, and clear annotations, the description covers purpose, usage, behavioral nuance, and authorization requirements. No additional return-value or parameter detail is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the sole parameter issueId is fully documented in the schema, including format examples. The description adds no additional parameter semantics beyond what the schema provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (hold/pause), resource (mission), and scope (before its next stage). It distinguishes itself from the sibling resume_mission by naming it as the continuation alternative, and 'no agent is stopped' implicitly contrasts with stop_mission.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states when to use it ('before its next stage') and which alternative to use to resume ('Use resume_mission to continue'). It also gives the prerequisite scope and role, leaving no ambiguity about eligibility.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

refresh_mcp_server_toolsRefresh an MCP server's tool listA
DestructiveIdempotent
Inspect

Snapshot the bound server's current tools (with their readOnlyHint / destructiveHint annotations) for approval. Approvals of tools that no longer exist are dropped. The binding must be ready; a server inside the sandbox has no gateway tool list. Requires config:write.

NameRequiredDescriptionDefault
projectIdYesProject id, e.g. from list_projects
serverNameYesServer name as the repository's MCP file declares it (see list_mcp_servers)

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already flag destructiveHint=true and idempotentHint=true, and the description adds real context beyond them: approvals for tools that no longer exist are dropped (explains the destructive side effect), the binding must be ready (prerequisite), and 'Requires config:write' (auth scope). No contradiction with the mutation/destructive profile.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the action and its effect, followed by prerequisite and auth requirement. Dense but every clause carries distinct information with little waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, so return semantics are not required, and the description covers effect, prerequisite, exclusion, and auth scope. Missing only the approval workflow/return detail, which is minor for a well-annotated mutation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and both projectId and serverName are documented in the schema with examples and a pattern. The description adds no parameter-level syntax or format beyond the schema, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource: snapshotting the bound server's current tool list, including the readOnlyHint/destructiveHint annotations, for approval. It is clearly about tool-list refresh rather than generic config, though it doesn't name a sibling (e.g. list_mcp_servers or bind_mcp_server) to explicitly contrast with.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear context and an exclusion: the binding must be ready and 'a server inside the sandbox has no gateway tool list', which implies when this cannot be used. It stops short of naming an alternative tool for those cases, but the when/when-not conditions are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

report_setup_blockerReport why setup stopsAInspect

Record why the person this token acts for will not finish setting up SHIP, in their words: for example no repository yet, no wish to set up model keys or credits, or no time now. Call it only after the person said why. Ask them when get_setup_progress shows setup is not done and they say they will stop. Do not guess a reason. Each call is a new report. A token that acts for no person is refused.

NameRequiredDescriptionDefault
detailNoThe person's own words, short. Leave out names, keys and anything private.
reasonYesno-repository: no code or no GitHub repository to connect. want-templates: wants to start from a template. model-keys: does not want to set up model keys or credits with a model provider. github-access: does not want to give SHIP access to GitHub. too-complex: the setup is too long or unclear. price: the cost. not-now: only exploring, or will come back later. other: anything else, said in detail.

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds behavior beyond annotations: 'Each call is a new report' explains the non-idempotentHint=false semantics, and 'A token that acts for no person is refused' discloses an auth failure mode. It does not say what happens to reports afterwards or what the call returns, which with no output schema is a minor gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose, then preconditions, then the negative constraint and auth caveat. Every sentence carries information; only 'Each call is a new report' is close to restating the non-idempotent annotation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter write tool with no output schema and full annotation coverage, the description supplies the trigger, the sibling that gates it, the privacy constraint, the anti-guessing rule and the refusal condition. Nothing an agent needs in order to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so baseline is 3; the description adds value by mapping example situations ('no repository yet', 'no wish to set up model keys or credits', 'no time now') onto the reason enum values, and by framing the detail parameter as the person's own words. It does not add format guidance beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Record why...will not finish setting up') on a specific resource (setup blocker reason, in the person's own words), with concrete examples of what qualifies. It is clearly distinguishable from read-oriented siblings like get_setup_progress and mark_setup_step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit preconditions: 'Call it only after the person said why', 'Ask them when get_setup_progress shows setup is not done', plus a negative rule 'Do not guess a reason'. It names the sibling that triggers it and the condition that selects this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

request_more_creditsAsk for more creditsAInspect

Ask for more credits for the workspace, saying what they are for. Any member may ask. The answer says who has the request: SHIP for a sponsored or Enterprise workspace, its owners and admins otherwise. A message sent while a request is open joins it.

NameRequiredDescriptionDefault
messageYesWhat the credits are for.

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=false and idempotentHint=false but say nothing about routing or effect. The description adds real behavioral context beyond them: who may submit, that approval routes to SHIP for sponsored/Enterprise workspaces versus owners and admins otherwise, and that a new message joins an existing open request (explaining the non-idempotency). It stops short of describing the response payload or any limits on concurrent requests.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the action and scope, then eligibility, then routing and open-request behavior. Nothing is padding, though the routing sentence is telegraphic enough to require a second read.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter tool with no output schema, the description covers eligibility, routing of the answer, and deduplication behavior, which is most of what an agent needs. It does not explain what a successful call returns or what happens to a request that is declined, but the core is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single required parameter with 100% schema description coverage ('What the credits are for.'). The description restates the same intent ('saying what they are for') without adding format, length, or content expectations, so it neither helps nor hurts beyond the schema baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource tied to a named scope: 'Ask for more credits for the workspace.' An agent can tell this is a request/escalation action, not a purchase, but the description never names the closely related siblings (start_credit_checkout, list_credit_offers), so the differentiation must be inferred.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Any member may ask' establishes who is eligible, and 'A message sent while a request is open joins it' tells the agent not to open duplicates. However, there is no explicit when-to-use statement and no comparison to start_credit_checkout or list_credit_offers, so selection guidance is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

resume_missionResume a missionAInspect

Continue a paused mission. When a checkpoint holds it (get_mission shows pauseSource 'checkpoint:'), this approves that stage's result and starts the next stage. To ask for changes to the result instead, use send_mission_instruction. Requires the missions:write scope.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=false, and openWorldHint=true, so the safety profile is partly covered. The description adds genuine value beyond that: the checkpoint-approval side effect of advancing to the next stage, and the required missions:write scope. It doesn't spell out reversibility of the approval or error cases, but the added context is substantial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the core action, then the checkpoint condition, then the routing alternative, ending with the scope requirement. Every sentence earns its place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-param action tool with no output schema, the description supplies the operational context an agent needs: what it does, the exact state that triggers it, the sibling alternative, and the required auth scope. Combined with annotations, nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single issueId param is fully documented in the schema with examples. The description adds no syntax beyond that, so the baseline of 3 applies – the schema carries the parameter burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (continue/resume) and resource (mission), and adds a precise behavioral detail distinguishing it: at a checkpoint it approves the stage's result and advances. An agent can tell this apart from pause_mission, stop_mission, and send_mission_instruction without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit triggering condition – get_mission showing pauseSource 'checkpoint:<role>' – and names the alternative (send_mission_instruction) with the exact reason to pick it (wanting changes instead of approval). Nothing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_scheduled_taskRun a scheduled task nowAInspect

Fire a scheduled task immediately, bypassing both its schedule and the skip-if-previous-open guard — an explicit run always creates a new issue even while a prior fire's mission is still open. Requires the pipeline:control capability and, for a token caller, the missions:run scope — the same authorization starting a mission by hand requires, because it is the same act.

NameRequiredDescriptionDefault
scheduledTaskIdYesScheduled task id to fire

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond annotations by disclosing the skip-if-previous-open bypass, the guaranteed new-issue side effect even when a prior mission is open, and the exact authorization requirements (pipeline:control capability, missions:run scope for token callers). This is precisely the operationally important context annotations alone (readOnly/idempotent/destructive) cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action and its key override, and every clause carries meaning. The second sentence is dense with em-dash asides about authorization, slightly reducing readability, but nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation-triggering tool with no output schema, the description supplies the side-effect semantics and the full auth prerequisite set an agent needs before invoking. Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter, so the schema already documents scheduledTaskId. The description adds no format or sourcing detail for the id beyond what the schema states, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (fire/run) and resource (scheduled task) with the key modifier 'immediately', and the 'bypassing its schedule' clause cleanly separates it from the passive scheduled execution that create/update_scheduled_task imply. An agent can distinguish this trigger action from its CRUD siblings at a glance.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives strong context on when this behavior applies (manual override of schedule and the skip-if-previous-open guard), which implicitly frames the manual-trigger use case. It stops short of naming a sibling alternative or an explicit when-not-to-use condition, so it lands just below a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

save_sandbox_registryStore a private registry credentialA
DestructiveIdempotent
Inspect

Store (or replace) the credential for a registry host. The platform attaches it to sandbox requests for that host only; it never enters a sandbox and is never returned.

NameRequiredDescriptionDefault
hostYesRegistry host or URL, e.g. npm.pkg.github.com
schemeYesHow the registry takes the credential: 'bearer' sends secret as a token, 'basic' sends username and secret
secretYesToken or password
usernameNoRequired for basic

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, idempotentHint=true and readOnlyHint=false, so the safety profile is covered. The description adds genuinely useful non-annotation context: the credential is attached only to requests for that host, never enters a sandbox, and is never returned, which explains blast radius and why no read-back occurs. It stops short of stating permission requirements or explicitly tying 'replace' to the destructive hint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with zero filler, and the upsert behavior is front-loaded before the scoping facts. Every clause carries information the agent cannot get from the schema or annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a credential-store tool with a fully documented schema, no output schema and annotations covering mutation/idempotency, the description is nearly complete: it covers scope of attachment, secret non-exposure, and replace semantics. Missing only prerequisites (who may write credentials) and any note on rotation/verification behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema already documents host, scheme (including the bearer/basic enum semantics), secret and the conditional username. The description adds no parameter-level detail (e.g. that username is ignored under bearer), so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (store/replace) and resource (credential for a registry host), and the parenthetical clarifies upsert semantics so the agent knows this is not a pure create. It does not explicitly name the sibling it differs from (list_sandbox_registries, delete_sandbox_registry), leaving that differentiation to inference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by 'for a registry host' — the agent infers this is the way to supply credentials for host-scoped sandbox requests. However, there is no explicit when-to-use statement, no note that list_sandbox_registries should be consulted first, and no stated prerequisites for the host/scheme combination.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

save_sandbox_templateCreate or update a sandboxAInspect

Create a sandbox (omit templateId) or change one. A definition is Dockerfile steps on the platform base (RUN, ENV, ARG, WORKDIR, USER; no FROM). A changed definition or size becomes a new version and is built at once; the response carries that build. Refusals name the line or the size bound.

NameRequiredDescriptionDefault
nameNoRequired when creating
sizeNoA tier, or { vcpu, memoryMib, diskMb } within 1-4 vCPU, >=3072 MiB per vCPU, <=12288 MiB, 8000-20000 MB
starterIdNoCreate from a starter's definition and size
dockerfileNoThe Dockerfile that builds the sandbox image. Give this or starterId to create; it replaces the file on an update
templateIdNoSandbox id (sbt_...)

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations establish a non-read-only, non-idempotent, non-destructive write, and the description adds substantial context beyond them: that a changed definition or size becomes a new version built immediately, that the response carries that build, and that refusals identify the offending line or size bound. It does not disclose permission/auth requirements or what happens to prior versions on update.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four dense sentences, all front-loaded: the create/update distinction comes first, followed by definition constraints, versioning behavior, and error semantics. No filler and every sentence carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless-required, five-param mutation tool with no output schema, the description covers create vs update, definition constraints, version/build behavior, and error reporting, and even notes the response carries the build. It omits the shape of the returned build object and whether non-dockerfile fields (name, size) are partial updates, leaving minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all five parameters (name, size, starterId, dockerfile, templateId). The description reinforces the create-vs-update role of templateId and the Dockerfile rules, but adds little syntax or format detail the schema lacks; baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('Create a sandbox ... or change one') and immediately disambiguates the two modes via the templateId condition. It is clearly distinguishable from siblings like build_sandbox_template, delete_sandbox_template, and save_sandbox_registry without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly states the discriminator between create and update ('omit templateId'), which is real when-to-use guidance. It does not name alternative tools (e.g., build_sandbox_template or list_sandbox_starters) for related workflows, so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

send_mission_instructionSend a follow-up instruction to a missionAInspect

Tell a mission what to change. Use kind='follow_up' (the default) to ask for a change: the instruction reaches the agents AND dispatches the builder against the mission's open pull request, overriding cached review and QA verdicts. Use kind='note' to record context for whatever runs next without starting any work. A restart is a different thing: it re-evaluates an unchanged pull request against its cached verdicts, and so changes nothing. Returns a receipt saying what was recorded and what it will cause. This tool calls the SHIP API, documented at https://letsship.ai/docs/api. Requires the missions:write token scope.

NameRequiredDescriptionDefault
bodyYesWhat the agents should do. Travels verbatim into the prompt
kindNofollow_up (default) dispatches the builder; note records context and dispatches nothing
issueIdYesIssue id the mission is tracked under, e.g. SHIP-244 or GH-1280

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by disclosing that follow_up dispatches the builder against the mission's open pull request and overrides cached review and QA verdicts, that note dispatches nothing, that a receipt is returned, and that it requires the missions:write token scope against the external SHIP API. These are consequential side effects an agent cannot infer from readOnlyHint=false / idempotentHint=false alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action, then branches cleanly by kind, with the restart disambiguation placed after the primary modes. Slightly dense and the trailing doc-link and token-scope sentences could be woven more tightly, but no sentence is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-parameter mutation tool with no output schema, the description covers the missing pieces itself: what it returns ('a receipt saying what was recorded and what it will cause'), the auth requirement, and the external API reference. Nothing needed to call it correctly is absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning to kind by explaining the downstream consequences of each value (builder dispatch + verdict override vs. pure context recording) rather than restating the schema's enum text. body and issueId are already fully documented in the schema and get no extra treatment.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Tell a mission what to change') and immediately disambiguates the two operating modes (follow_up vs note) plus the confusingly similar 'restart' action. An agent can distinguish this from siblings like start_mission or pause_mission without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says when to use each mode: follow_up 'to ask for a change' (dispatches the builder), note 'to record context for whatever runs next without starting any work'. It also names a nearby alternative concept (a restart) and states why it is different ('re-evaluates an unchanged pull request ... changes nothing').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set_agents_overrideSet a per-issue agents-override for one roleA
DestructiveIdempotent
Inspect

Patch one role's harness/model/effort/prompt for a single mission, layered over ship.yml per-field (a role or field this call omits keeps resolving from ship.yml — SHIP-337). Use promptContent or promptPath(+promptRef) to run an ephemeral prompt experiment (spec D9) without committing to the target repo. Read-modify-write: this call is merged onto the previously stored override record (via get_agents_override), so a prior set for a DIFFERENT role — or a different field on this same role, e.g. an earlier harness pin — survives; the underlying PUT is a full-record replace and would otherwise silently drop it. This tool calls the SHIP API, documented at https://letsship.ai/docs/api.

NameRequiredDescriptionDefault
roleYesThe role this override entry patches
sizeNoContainer size for this role on this mission
modelNoThe model id this role uses for this mission. The harness must support it
effortNoReasoning effort for this role, mapped to the harness's own control at dispatch: low, medium, high or xhigh, and also max on claude-code. A value the harness does not take is refused at dispatch.
harnessNoThe code harness this role runs on for this mission
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-353 or WEB-42
sandboxNoSandbox name or id this role starts in for this mission (beats ship.yml and the project default)
rejectOnNoReview rejection threshold — only meaningful on the reviewer role
projectIdYesProject id the mission belongs to
promptRefNoGit ref for promptPath — only meaningful alongside promptPath.
baseBranchNoOverride the pipeline's fork/target branch for this mission
checkpointNoPause the pipeline after this role's stage completes
promptModeNoHow this prompt relates to the repo's own .ship/agents/<role>.md (or ship.yml-pointed) layer. replace (default) makes this the role's only role-prompt layer; extend resolves the repo's layer first and appends this content after it, as a separately attributed span. Only meaningful with promptContent or promptPath. NOTE: the merge is per FIELD, and prompt is one field — a later call that sets a new promptContent/promptPath without repeating promptMode replaces the whole stored prompt, reverting to replace.
promptPathNoRepo-relative path to an L2 prompt file. Without promptRef this is the SHIP-347 pointer form (resolved at the pinned base branch); with promptRef it's a path+ref experiment resolved at that ref.
promptContentNoInline prompt-experiment body, capped at 30,000 characters. Mutually exclusive with promptPath.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations (destructiveHint, idempotentHint, readOnlyHint) by disclosing the read-modify-write contract: the call is merged onto the previously stored record, a prior set for a different role or a different field survives, and the underlying PUT is a full-record replace that would otherwise silently drop it. It also notes a value the harness does not accept is refused at dispatch — exactly the behavioral risk an agent needs before calling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and the merge semantics before the API link, and every sentence carries real information. It is dense with parentheticals (SHIP-337, spec D9, 'e.g. an earlier harness pin') that add traceability but slightly slow parsing of an already long description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 15 parameters, no output schema, and a destructive+idempotent annotation profile, the description supplies the missing merge/replace semantics an agent must know to avoid silent data loss. The remaining gap is sibling routing: it names get_agents_override but omits clear_agents_override, the counterpart for removal.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, but the description adds field-level meaning beyond the schema: promptContent/promptPath are framed as an ephemeral experiment path and the merge is described as per-FIELD, which clarifies why an omitted parameter persists rather than reverting. The destructive/merge semantics of omitting a parameter are the key added value the schema's per-property text does not convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Patch one role's harness/model/effort/prompt for a single mission') and scopes it to a single role on a single mission. It also distinguishes itself from the read path by naming get_agents_override, so an agent can pick between read and write without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear conditions for the prompt-experiment path (promptContent or promptPath(+promptRef) to run an ephemeral experiment) and explains the layering-over-ship.yml model, plus points to get_agents_override for reads. It never mentions clear_agents_override, the sibling an agent would need to remove an override, so the when-not side is incomplete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set_mcp_config_pathSet a project's MCP file pathA
Idempotent
Inspect

Point the project at the repository file that declares its MCP servers, e.g. .cursor/mcp.json or .vscode/mcp.json. null restores the default (.mcp.json). The file is read, never edited. When the repository's ship.yml sets mcp.config, that wins and this call is refused (409); list_mcp_servers reports which one applies as configPathSource. Requires the config:write capability.

NameRequiredDescriptionDefault
projectIdYesProject id, e.g. from list_projects
configPathYesRepository-relative path, or null

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations mark it non-read-only but non-destructive and idempotent; the description adds rich context beyond that: the target file is read and never edited, null has a specific reset behavior, a 409 conflict path exists when ship.yml overrides, and it requires the config:write capability. This governs auth, failure, and side-effect expectations precisely.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four tight sentences, front-loaded with the primary action, followed by the reset behavior, the read-only-edit constraint, and the override/capability caveats. No filler; every clause carries operational weight.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter setter with no output schema and solid annotations, the description covers the action, the null/default semantics, the refusal path, and the required capability. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters are already documented; the description still adds meaning by naming example configPath values (.cursor/mcp.json, .vscode/mcp.json) and clarifying that null restores the .mcp.json default. That is real semantic enrichment above the baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Point the project at the repository file that declares its MCP servers') with concrete examples of the file. It clearly delimits this from siblings like bind_mcp_server (server binding) and list_mcp_servers (reporting), so an agent can distinguish it without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operating context: null restores the default, the 409 refusal when ship.yml declares mcp.config, and that list_mcp_servers reveals the effective source via configPathSource. However, it never states when to prefer this over alternatives (e.g., bind_mcp_server) or prerequisites for calling it, so explicit when-not guidance is thinner than a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set_mission_auto_mergeTurn auto-merge on or off for one missionA
DestructiveIdempotent
Inspect

Set whether SHIP merges this mission's pull request by itself once the checks, the review and the test pass. The setting is for this mission only and replaces the project's auto-merge for it; get_project shows the project's. While it is on, SHIP merges without asking again. Requires the missions:write scope.

NameRequiredDescriptionDefault
enabledYestrue merges the pull request once it is ready; false leaves the merge to a person
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations supply safety profile (destructiveHint=true, idempotentHint=true), and the description adds meaningful context beyond them: the override/replacement semantics of the setting, the consequential behavior ('SHIP merges without asking again') once enabled, and the required missions:write scope. This is real behavioral disclosure, though it doesn't discuss reversibility or rate/permission failure modes in depth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the core action, then scope/override semantics, then the consequence and auth requirement. Every sentence carries distinct information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter toggle with no output schema, the description covers what the setting does, its scope relative to projects, its behavioral consequence, and the required scope. Nothing needed to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters (enabled, issueId) are fully documented in the schema itself. The description adds no syntax or format detail beyond that, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (set) plus resource (this mission's auto-merge) and scope (this mission only). It explicitly distinguishes itself from the project-level setting ('replaces the project's auto-merge for it; get_project shows the project's') and from the sibling merge_mission by being a configuration toggle rather than an action.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly frames the context: per-mission override that supersedes the project default, and points to get_project to inspect the project setting. It doesn't explicitly say when to prefer this over merge_mission or when-not to enable it, so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set_mission_dispositionRecord why a mission endedA
DestructiveIdempotent
Inspect

Record a person's reason for how a mission ended, which the delivery statistics use: product (real work, judged on its merits), probe (a test on purpose), superseded (another mission replaced it: give supersededBy) or takeover (a person finished the work outside SHIP); null clears it. Set it only with the person's answer, never from your own guess. Requires the missions:write scope and the config:write capability.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412
dispositionYesproduct, probe, superseded or takeover; null clears a disposition set by mistake
supersededByNoThe issue id of the mission that replaced this one, for disposition superseded only

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the mutation profile (readOnlyHint=false, destructiveHint=true, idempotentHint=true), so the bar is lower. The description adds what the structured data cannot: the required missions:write scope and config:write capability, the fact that null clears an existing value, and the dependency of supersededBy on superseded. It does not describe concurrency or overwrite-warning behavior beyond the clear semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense but well-ordered passage: purpose first, then the value taxonomy, then the correction case, then the behavioral constraint and permissions. Every clause carries information, though the parenthetical explanations could be tightened slightly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-parameter mutation with no output schema, this covers purpose, allowed values, conditional parameters, clearing semantics, the human-answer constraint, and required scopes. Only the side effects on delivery statistics (mentioned as consuming the value but not how) remain unspecified, which is a minor omission.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description genuinely adds meaning beyond the schema: it explains the semantic intent of each enum value (real work judged on merits, a deliberate test, replaced by another mission, finished outside SHIP) and restates that supersededBy is conditional. The issueId format is left to the schema, which handles it well.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — recording a person's reason for why a mission ended — and then enumerates the exact vocabulary (product, probe, superseded, takeover, null) plus what each means. This is unambiguous and clearly separable from siblings like archive_mission, stop_mission or set_mission_auto_merge, which touch lifecycle state rather than delivery disposition.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when and when-not guidance: 'Set it only with the person's answer, never from your own guess' rules out speculative invocation, and it flags that supersededBy applies to superseded only and that null is for correcting a mistake. No sibling alternative is named for the case where the disposition is unknown, which is the only gap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

start_credit_checkoutStart buying credits or a monthly planA
Destructive
Inspect

Opens a payment page and returns its URL. Nothing is charged until the buyer pays on that page. It sells one credit pack, a monthly plan (offer team, with teamPlan starter or scale, a subscription that renews until cancelled), or extra credits for a workspace already on a plan (offer team_credits, for quantity credits at that plan's own price per credit, paid once, expiring with the plan month). Only a workspace on a plan can use team_credits. Only owners and admins can open a page. The server refuses the request while startNow is false, and the same idempotencyKey returns the same page.

NameRequiredDescriptionDefault
offerYesA pack, team or team_credits, from list_credit_offers.
quantityNoCredits for team_credits, within the range list_credit_offers gives; omit for a pack or a plan.
startNowYesRecords the buyer's own consent to start the credits at once, during the withdrawal period, against this text: "Start my credits now, so they can be used during the 14-day withdrawal period. If I withdraw, I pay for the credits I used; I lose the right to withdraw once all the credits I am buying are used." The server refuses the request while it is false.
teamPlanNoThe plan to start, for offer team only: starter or scale, from list_credit_offers.
idempotencyKeyNoReuse on retry to get the same page back.

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well past the annotations by disclosing the consent gate (server refuses while startNow is false), the no-charge-until-payment semantics, the renewal/expiry behavior of each offer, and the idempotencyKey retry contract. Note the annotations declare idempotentHint=false while the description offers key-based replay protection; these are compatible (the key makes retries safe, not the call itself), so no contradiction. No auth scope beyond owner/admin is described.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The primary effect (page URL, deferred charge) leads, followed by the offer breakdown and then the constraints. It is a single dense paragraph with heavy parenthetical nesting, which costs a little readability, but every clause carries distinct operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a side-effecting, open-world, permission-gated checkout tool with no output schema, the description covers who may call it, what each offer does, what is returned (a URL), the consent precondition, and retry behavior. It does not address interaction with an existing subscription or what happens to concurrent/duplicate checkouts beyond the idempotency key.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds interpretation the schema lacks: quantity applies only to team_credits and is priced at the plan's per-credit rate, paid once and expiring with the plan month, while teamPlan turns the purchase into a renewing subscription. It reinforces the startNow refusal and idempotencyKey reuse semantics in prose.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb and resource ("Opens a payment page and returns its URL") and immediately scopes out what it is not (nothing is charged until the buyer pays). It also enumerates the three distinct purchase shapes, so an agent can tell it apart from request_more_credits or update_team_subscription without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives real eligibility conditions: only a workspace on a plan may use team_credits, and only owners and admins can open a page. It also maps each offer to its use case (pack vs. subscription vs. top-up credits). It does not name sibling alternatives (request_more_credits, list_credit_offers, update_team_subscription) as explicit redirects, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

start_mcp_server_oauthStart OAuth for a remote MCP serverAInspect

Begin the OAuth consent for a server bound as remote. Returns an authorizeUrl that a HUMAN must open in a browser signed in to the SHIP console as the same user this token acts for; the provider returns to the console, which stores the credential. The link is single-use and expires (expiresAt). An agent cannot complete the consent itself. Requires config:write.

NameRequiredDescriptionDefault
scopeNoRequested scope; defaults to the server's own hint
returnToNoConsole path to land on after consent, e.g. /projects/<id>
projectIdYesProject id, e.g. from list_projects
serverNameYesServer name as the repository's MCP file declares it (see list_mcp_servers)

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only cover the safety profile (non-read-only, open-world, non-idempotent). The description adds substantial behavioral context beyond that: the return value (authorizeUrl), the human-in-the-loop requirement, single-use + expiring link (expiresAt), where the credential is stored, and the config:write requirement. This is exactly the extra disclosure the annotations can't carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action, then packed with the essential return-value and human-in-the-loop facts. Every clause earns its place, though the semicolon-separated density is slightly heavier than strictly necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by explaining the returned authorizeUrl and expiresAt, the auth requirement (config:write), and the critical constraint that an agent cannot finish consent. Nothing an agent needs to call and reason about this tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters including scope defaults and returnTo. The description adds no parameter-specific syntax or format detail, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Begin the OAuth consent for a server bound as remote') and immediately distinguishes its scope from sibling lifecycle tools like bind_mcp_server or refresh_mcp_server_tools. An agent knows exactly what action this performs without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides a clear precondition ('a server bound as remote') and an explicit constraint that 'an agent cannot complete the consent itself,' which tells the agent when this tool is and isn't the right call. It stops short of naming specific alternative tools, hence 4 rather than 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

start_missionStart a missionAInspect

Hand a mission off to the AI agent team. Omit startAt for the default full run (planner first); pass startAt='builder' with a plan to skip straight to coding; pass startAt='reviewer' or startAt='qa' with an existing prNumber to hand off an existing build. Requires the missions:run token scope.

NameRequiredDescriptionDefault
planNoHand-off plan markdown, for startAt='builder'
agentsNoPer-role harness/model override for THIS mission only; overrides ship.yml without changing it
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-244 or WEB-42
startAtNoPipeline stage to enter at; omit for the default full run starting at the planner
prNumberNoExisting open PR number, required for startAt='reviewer' or startAt='qa'
projectIdYesProject id the mission belongs to

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare openWorldHint=true, idempotentHint=false, destructiveHint=false. The description adds genuinely useful context beyond that: it states the required missions:run token scope (auth requirement) and explains the pipeline stage semantics of a hand-off. It doesn't describe reversibility or side effects of the started run, so not a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the core action, then the mode-selection rules, then the auth requirement. Every sentence earns its place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter tool with nested agents objects and no output schema, the description covers the trickiest parameter (startAt) plus the auth requirement. The agents override parameter is explained in the schema rather than the description, and no return behavior is given, but the schema compensates.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the per-parameter descriptions carry most meaning (baseline 3). The description goes further by explaining the conditional relationship between startAt and the plan/prNumber parameters, which the schema documents only in isolation, adding real semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action ("Hand a mission off to the AI agent team") that maps to the resource 'mission'. Clear verb+resource and it enumerates the pipeline stages the tool drives. It does not, however, distinguish itself from the sibling create_mission, which an agent might reasonably confuse with starting a run.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Strong conditional guidance: omit startAt for default planner-first run, use builder+plan to skip to coding, use reviewer/qa with prNumber for an existing build. This tells the agent which mode to select. Missing any exclusion relative to the create_mission alternative, so not a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

stop_agent_runStop an agent runA
DestructiveIdempotent
Inspect

Stop the agent in one run before it finishes. Its work in progress is lost, and the run's status becomes stopped a few seconds later. Needs an org admin and the sandboxes:stop scope. This stops one agent run, not the mission: to stop the whole mission, use POST /v1/missions/{issueId}/stop.

NameRequiredDescriptionDefault
sandboxIdYesAgent run id: the run's sandbox id, from list_agent_runs

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true, but the description adds context annotations cannot carry: in-progress work is lost and the run's status settles to 'stopped' a few seconds later (asynchronous completion). It also names the required role and scope, which is essential for a destructive mutation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the action and its consequence, then the requirements, then the disambiguation. Zero filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive one-parameter mutation with no output schema, the description covers the loss of work, the async status transition, the auth requirements, and the sibling disambiguation. Nothing needed to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single sandboxId parameter is already documented in the schema as 'the run's sandbox id, from list_agent_runs'. The description adds no syntax or format detail beyond that, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Stop the agent in one run before it finishes') and immediately scopes it against the sibling stop_mission. An agent can distinguish this from stop_mission and pause/resume without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says this stops one agent run and not the mission, and names the alternative route (POST /v1/missions/{issueId}/stop) for the mission-wide case. It also states the prerequisite role (org admin) and scope (sandboxes:stop).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

stop_missionStop a missionA
DestructiveIdempotent
Inspect

Hard-stop a mission: every agent run of it stops and its work in progress is lost. The pull request stays open, and start_mission starts the mission again. To stop one agent run only, use stop_agent_run. Requires the missions:write scope.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true, so the safety profile is covered. The description adds genuine context beyond that: work in progress is lost, the pull request stays open, and the mission can be restarted via start_mission, plus the required missions:write scope.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, no filler; the destructive consequence is front-loaded before the alternative route. Every clause carries distinct information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter mutation with no output schema, the description covers consequence, reversibility, scope requirement, and the sibling alternative. An agent has everything needed to call it correctly and safely.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single issueId parameter is fully documented with a format example (SHIP-412), so the schema carries the burden. The description adds nothing about the identifier beyond what the schema states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Hard-stop a mission") plus the concrete blast radius ("every agent run of it stops"). It immediately separates itself from the sibling stop_agent_run, which is a narrower operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes the agent: "To stop one agent run only, use stop_agent_run," and notes start_mission reverses the operation. It does not address pause_mission or archive_mission, the other nearby alternatives, so the routing is clear but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

unarchive_missionUnarchive a missionA
Idempotent
Inspect

Show an archived mission in list_missions and on the board again. Requires the missions:write scope and the config:write capability.

NameRequiredDescriptionDefault
issueIdYesThe mission's issue id: a Linear key or a GitHub Issues id, e.g. SHIP-412

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this is a non-read-only, idempotent, non-destructive, closed-world mutation. Beyond that, the description adds the authorization requirements (missions:write scope and config:write capability), which the agent cannot infer from annotations. It stops short of describing what happens if the mission is not currently archived or whether the call is reversible.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler. The behavioral effect is front-loaded, with the authorization requirement following as a secondary constraint. Every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter state-change tool with no output schema, the description covers the effect, the authorization path, and the visible outcome (list and board). It leaves unaddressed what happens on a non-archived mission or on permission failure, but annotations already carry the safety profile, so the gap is minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single parameter and schema description coverage is 100%, including the Linear key / GitHub id format example ('SHIP-412'). The description adds no additional meaning about issueId beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action (unarchive) on a specific resource (mission) and names the concrete observable effect: the mission reappears in list_missions and on the board. This implicitly and clearly distinguishes it from the sibling archive_mission, which does the opposite. An agent can pick between the two without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives the precondition context — it operates on an archived mission and requires the missions:write scope plus the config:write capability — which tells the agent when the call will succeed. It does not explicitly name archive_mission as the inverse alternative or state error behavior, so guidance is clear but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

unbind_mcp_serverRemove an MCP server bindingA
DestructiveIdempotent
Inspect

Delete a binding and its stored credential: no role receives the server from its next run. The repository's MCP file is not changed, so the server can be bound again. Requires config:write.

NameRequiredDescriptionDefault
projectIdYesProject id, e.g. from list_projects
serverNameYesServer name as the repository's MCP file declares it (see list_mcp_servers)

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With annotations already flagging destructive/idempotent/non-read-only, the description adds material extra context: the stored credential is deleted too, the repository's MCP file is NOT modified, and the operation is reversible via re-binding. It also discloses the config:write authorization requirement — exactly the kind of disclosure annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight clauses, front-loaded with the effect (delete binding and credential), followed by the non-effect (repo file untouched) and the prerequisite. No filler sentences.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity, two-parameter tool with no output schema, the description covers the action, the side effects, reversibility, and the required permission — everything an agent needs before invoking it. Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both projectId and serverName are already documented in the schema. The description adds no syntax, format, or constraint detail beyond implying that serverName identifies the binding to remove; the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (delete/unbind) and resource (an MCP server binding) and clearly differentiates itself from bind_mcp_server and list_mcp_bindings by describing the removal action and its consequence. An agent can pick it out of the sibling list without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear context: it removes the binding and credential so no role gets the server on the next run, and states the precondition 'Requires config:write.' It implies when to use it (to undo a binding) but never explicitly names bind_mcp_server as the counterpart operation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_projectChange a project's settingsA
DestructiveIdempotent
Inspect

Change the settings of one project; a field you leave out keeps its value. status paused or archived stops new missions in the project, and active starts them again. autoMerge sets whether SHIP merges a ready pull request by itself. defaultSandboxTemplateId sets the sandbox the agents start in (null for the platform base). repoFullName, linearTeamKey and jiraProjectId connect the project to another repository, Linear team or Jira project through the organization's existing connections. tracker moves the project to another tracker: linear with linearTeamKey, jira with jiraConnectorId and jiraProjectId, github-issues on its repository, or none; it is refused while a mission of the project runs. Change only what a person asked for. Answers the project as it now stands. Requires the projects:admin scope and the config:write capability.

NameRequiredDescriptionDefault
nameNoA new name
statusNoactive, paused or archived
trackerNoMove the project to linear, github-issues, jira or none; only when a person asked for the move
autoMergeNotrue merges a ready pull request without a person
issueCodeNoA GitHub Issues project's issue id prefix, 2-5 characters; only new ids use it
projectIdYesProject id, from list_projects
repoFullNameNoowner/repo of the repository to connect, through the organization's GitHub connection
jiraProjectIdNoA Jira project's numeric id on the project's Jira site, e.g. 10000; Jira-tracked projects only
linearTeamKeyNoThe key of the Linear team to connect, e.g. ENG
jiraConnectorIdNoThe Jira site's connection id, for tracker jira only
defaultSandboxTemplateIdNoA sandbox id (sbt_...) from list_sandbox_templates, or null for the platform base

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by explaining what status changes actually do (paused/archived stops new missions, active restarts them), what autoMerge enables, how tracker moves are gated on running missions, that connections go through the org's existing integrations, and the required projects:admin scope plus config:write capability. This is rich side-effect and precondition disclosure for a destructive, open-world mutation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core contract (settings change, omitted fields preserved) before field details, and nearly every sentence carries distinct information. It is a dense wall of semicolon-chained clauses, though, which costs some scannability versus a short structured list.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 11-parameter destructive mutation with no output schema, the description supplies the return shape ('Answers the project as it now stands'), partial-update semantics, per-field behavior, gating conditions, and auth requirements. An agent has everything needed to call this correctly and safely.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds genuine cross-parameter semantics the schema cannot express: which companion field each tracker value requires (linear with linearTeamKey, jira with jiraConnectorId and jiraProjectId, github-issues on its repository). It leaves a few parameters (issueCode, name) unaddressed in prose, relying on the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource with explicit scope: 'Change the settings of one project'. The field-by-field enumeration (status, autoMerge, tracker, connections) makes it unmistakable against siblings like update_scheduled_task or set_mission_auto_merge, which touch different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operating context: partial-update semantics ('a field you leave out keeps its value'), pairing rules for tracker values, and an explicit exclusion ('refused while a mission of the project runs'). It also states 'Change only what a person asked for'. It stops short of naming a sibling alternative for any of these fields, so not a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_scheduled_taskUpdate a scheduled taskA
DestructiveIdempotent
Inspect

Patch a scheduled task's name, prompt, schedule, skip-guard, or enabled state — only the fields passed are changed. To change the schedule, set interval/time/timezone together (plus weekday or dayOfMonth as the interval requires). This tool calls the SHIP API, documented at https://letsship.ai/docs/api. Requires the config:write capability.

NameRequiredDescriptionDefault
nameNoScheduled task name, shown in the console and used in the issue title
timeNo24-hour wall-clock time in `timezone`, e.g. '09:00'.
labelsNoTracker label names put on every issue this scheduled task files (replaces the list; [] clears it). The label must ALREADY exist in the project's tracker team — the platform attaches labels but cannot create them, so a name the workspace does not have is dropped.
promptNoReplaces the issue description verbatim
enabledNoSet false to pause the schedule without deleting the scheduled task
weekdayNoDay of week — required when interval is 'weekly'.
intervalNoFire cadence. 'weekly' requires weekday; 'monthly' requires dayOfMonth.
timezoneNoIANA timezone id, e.g. 'Europe/Amsterdam'.
dayOfMonthNoDay of month, 1-28 — required when interval is 'monthly' (capped at 28 so every month has the day).
scheduledTaskIdYesScheduled task id to update
skipIfPreviousOpenNoSkip a scheduled fire while the previous fire's mission is still open (default true — the guardrail is opt-out, not opt-in)

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (destructiveHint=true, idempotentHint=true), so the bar is lower; the description still adds the auth requirement (config:write), the partial-update semantics that explain the idempotent hint, and the external SHIP API dependency with a doc link. It does not explain what 'destructive' means here (e.g. that labels replaces the existing list), leaving one gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, no filler. The mutation scope and partial-update behavior are front-loaded, followed by the schedule coupling rule and then the operational prerequisites.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 11-parameter mutation tool with no output schema, the description covers update semantics, cross-field constraints, capability auth, and backend dependency. Remaining unknowns (failure behavior on an unknown id, replacement semantics for labels) are handled by the schema rather than the prose, so it is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage the baseline is 3, but the description adds genuine cross-field semantics the schema only states locally: the interval/time/timezone coupling and the weekday/dayOfMonth conditionality. That is meaning beyond the per-property descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Patch') plus resource ('a scheduled task') and enumerates the mutable surface (name, prompt, schedule, skip-guard, enabled state). The 'only the fields passed are changed' clause distinguishes it from create/delete/run siblings without needing their schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a real usage rule — schedule changes require interval/time/timezone set together, plus weekday or dayOfMonth as the interval dictates — and names the config:write capability prerequisite. It stops short of explicitly naming when to prefer a sibling (e.g. run_scheduled_task or delete_scheduled_task), so it lands at clear-context-without-exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_team_subscriptionChange the monthly plan subscriptionA
DestructiveIdempotent
Inspect

Change the workspace's monthly plan subscription: plan switches between starter and scale from the next renewal on (nothing is charged or refunded for the current month; passing the current plan drops a waiting switch), or cancelAtPeriodEnd true ends it when the month already paid for ends, false resumes it before then. Pass at least one. Owners and admins only. The answer is the subscription as it now stands, with any waiting switch under nextPlan.

NameRequiredDescriptionDefault
planNoThe plan from the next renewal on: starter or scale, from list_credit_offers.
cancelAtPeriodEndNotrue ends the subscription at the end of the paid month; false resumes it.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only give generic flags (destructive=true, idempotent=true, openWorld=true). The description adds substantive behavioral context the annotations cannot convey: no charge or refund for the current month, dropping a waiting switch by re-passing the current plan, the exact timing boundary for cancellation, the owners/admins-only authorization requirement, and the shape of the returned object including nextPlan.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

It is a single dense sentence with parenthetical asides, but every clause carries needed semantics (timing, billing, idempotence-by-re-pass, auth, return shape) and the mutation verb is front-loaded. Slightly heavy for one breath, but no wasted text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description explicitly covers what is returned and where the waiting switch appears (nextPlan), plus authorization and side-effect boundaries. For a two-parameter mutation tool this leaves nothing an agent needs to know before calling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so the baseline is 3, but the description goes further by explaining the effect of passing the current plan value (drops the waiting switch) and by compensating for the schema's zero required parameters with the 'Pass at least one' constraint. That is real meaning added over the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource -- changing the workspace's monthly plan subscription -- and immediately scopes it to plan switches and cancellation flips, so it is clearly distinguishable from billing siblings like get_billing, list_credit_offers, and start_credit_checkout. An agent can tell what state this tool mutates without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the prerequisite 'Pass at least one' and describes the conditions under which each mode applies (plan switch from next renewal vs cancelAtPeriodEnd ending/resuming). It also names list_credit_offers as the source of valid plan values. It does not explicitly name alternative tools for adjacent billing actions, so a 4 rather than 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

watch_agent_runWatch an agent runA
Read-onlyIdempotent
Inspect

Wait on one agent run and follow what its agent does, instead of polling get_agent_run. The call returns when the run ends, when its mission moves to another stage, or after maxSeconds. While it waits, each agent event (a thought, a tool call, a plan update, an error) is sent as a progress notification when the request has a progress token. The result has the status of the run, the stage of its mission, ended, the reason it returned, the last 20 events and a cursor. When the run is still running, call it again with since set to the cursor. A run that already ended returns at once. If your client ends a call after 60 seconds, set maxSeconds below that.

NameRequiredDescriptionDefault
sinceNoThe cursor from an earlier result. Only the events after it are sent
sandboxIdYesAgent run id: the run's sandbox id, from list_agent_runs or get_mission
maxSecondsNoThe longest time to wait, in seconds (default 300, at most 600)

Output Schema

NameRequiredDescription
hintNoWhat to do next, when the run is still running
endedYesTrue when the run is no longer running
stageYesThe stage of the run's mission when the call returned
cursorYesThe id of the last event. Give it as since to continue
eventsYesThe last 20 events at most, oldest first
reasonYesWhy the call returned: the run ended, its mission moved to another stage, or maxSeconds passed
statusYesThe status of the run when the call returned
sandboxIdYes

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnly/idempotent hints by disclosing the blocking contract (returns on run end, stage transition, or maxSeconds), the progress-notification streaming behavior gated on a progress token, the result payload shape, and the cursor/resume semantics. This is the kind of long-running-call behavior an agent cannot infer from annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core contract and the anti-polling rationale, and every sentence carries information. The result-field sentence ('the status of the run, the stage of its mission, ended, the reason it returned...') is slightly awkward as a run-on list, but there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a blocking, streaming, cursor-resumable tool, the description covers invocation conditions, resume mechanics, notification behavior, and timeout handling. It even describes return contents despite an output schema existing, leaving no gap an agent needs to close before calling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds real meaning: it ties 'since' to the cursor returned in a prior result and ties maxSeconds to the client's 60-second call cutoff, which is operational guidance the schema does not provide. It stops short of documenting 'sandboxId' beyond what the schema already says.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Wait on one agent run and follow what its agent does') and explicitly distinguishes itself from the sibling get_agent_run by naming the polling alternative it replaces. An agent can tell immediately that this is a blocking/following variant rather than a point-in-time read.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use guidance (instead of polling get_agent_run), the three conditions that end the wait, the resume pattern ('call it again with since set to the cursor'), the already-ended fast path, and a concrete client-timeout rule ('If your client ends a call after 60 seconds, set maxSeconds below that'). Nothing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

whoamiIdentify the calling principalA
Read-onlyIdempotent
Inspect

Returns the caller's identity, org, role, role-derived RBAC capabilities, and token scopes. Use this to check whether the current token may perform an action (e.g. config:write) before attempting it.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
roleYes
orgIdYes
scopesYes
userIdYes
projectIdYesThe project this credential is bound to, or null when it is org-wide. A bound token can read and act only on that project, and on nothing org-wide - so a caller can see the boundary it works within rather than discovering it through a 404.
capabilitiesYes
principalTypeYesagent: a self-registered agent nobody has claimed yet. It has no user, org, role or scopes, and reaches only /me and /scopes until a person claims it.

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds genuine context beyond that: the response includes RBAC capabilities and token scopes, and the tool is meant as a permission pre-check.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler. The return contents come first and the operational use case is front-loaded immediately after, which is exactly the right ordering for a 0-param identity tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described; annotations cover the safety profile; and the description states both purpose and usage. Nothing an agent needs to call this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline of 4 applies. Nothing in the schema needs compensating for, and the description doesn't introduce misleading param expectations.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (returns) plus the exact resource set: caller identity, org, role, role-derived RBAC capabilities, and token scopes. This is far more specific than its siblings like healthz or get_billing, so an agent can place it instantly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use guidance with a concrete motivating scenario: check whether the current token may perform an action (e.g. config:write) before attempting it. No sibling overlaps with this pre-flight permission-check role, so the absence of a named alternative is not a gap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool update
    • Addedanswer_qa_checkpoint
  2. 1 tool update
    • Addedreport_setup_blocker
  3. 1 tool update
    • Changedupdate_project2 fields changed
      • addedInput schema / properties / jiraConnectorId
        Added value: +{
        +  "description": "The Jira site's connection id, for tracker jira only",
        +  "minLength": 1,
        +  "type": "string"
        +}
      • addedInput schema / properties / tracker
        Added value: +{
        +  "description": "Move the project to linear, github-issues, jira or none; only when a person asked for the move",
        +  "enum": [
        +    "linear",
        +    "github-issues",
        +    "jira",
        +    "none"
        +  ],
        +  "type": "string"
        +}
  4. 3 tool updates
    • Changedget_project1 field changed
      • changedOutput schema / properties / tracker / enum
        Previous value: -[
        -  "linear",
        -  "github-issues",
        -  "none"
        -]New value: +[
        +  "linear",
        +  "github-issues",
        +  "jira",
        +  "none"
        +]
    • Changedlist_projects1 field changed
      • changedOutput schema / properties / data / items / properties / tracker / enum
        Previous value: -[
        -  "linear",
        -  "github-issues",
        -  "none"
        -]New value: +[
        +  "linear",
        +  "github-issues",
        +  "jira",
        +  "none"
        +]
    • Changedupdate_project1 field changed
      • addedInput schema / properties / jiraProjectId
        Added value: +{
        +  "description": "A Jira project's numeric id on the project's Jira site, e.g. 10000; Jira-tracked projects only",
        +  "pattern": "^\\d+$",
        +  "type": "string"
        +}
  5. 2 tool updates
    • Changedget_setup_progress2 fields changed
      • addedOutput schema / properties / groups / items / properties / milestones / items / properties / status
        Added value: +{
        +  "enum": [
        +    "open",
        +    "reached",
        +    "completed",
        +    "skipped"
        +  ],
        +  "type": "string"
        +}
      • changedOutput schema / properties / groups / items / properties / milestones / items / required
        Previous value: -[
        -  "id",
        -  "title",
        -  "description",
        -  "reached",
        -  "action",
        -  "docsUrl"
        -]New value: +[
        +  "id",
        +  "title",
        +  "description",
        +  "reached",
        +  "status",
        +  "action",
        +  "docsUrl"
        +]
    • Addedmark_setup_step
  6. 1 tool update
    • Changedlist_missions1 field changed
      • changedInput schema / properties / activity / enum
        Previous value: -[
        -  "queued",
        -  "planning",
        -  "building",
        -  "fixing_ci",
        -  "addressing_review",
        -  "addressing_qa",
        -  "ci_running",
        -  "ci_failed",
        -  "reviewing",
        -  "changes_requested",
        -  "deploying",
        -  "testing",
        -  "qa_failed",
        -  "ready_to_merge",
        -  "auto_merging",
        -  "needs_attention",
        -  "merged",
        -  "auto_merged",
        -  "declined",
        -  "paused",
        -  "stopped"
        -]New value: +[
        +  "queued",
        +  "planning",
        +  "building",
        +  "fixing_ci",
        +  "addressing_review",
        +  "addressing_qa",
        +  "ci_running",
        +  "ci_failed",
        +  "reviewing",
        +  "changes_requested",
        +  "deploying",
        +  "testing",
        +  "qa_failed",
        +  "ready_to_merge",
        +  "auto_merging",
        +  "needs_attention",
        +  "merged",
        +  "auto_merged",
        +  "declined",
        +  "pausing",
        +  "paused",
        +  "stopped"
        +]
  7. 5 tool updates
    • Changedbind_mcp_server1 field changed
      • addedInput schema / properties / transport / description
        Added value: +"How the gateway reaches the server: 'remote' proxies to remoteUrl, 'isolate' runs the server's npm package away from the sandbox so a secret stays out of it, and 'local' runs the command inside the sandbox with no credential. Omit it on an update to keep the server the binding is pinned to."
    • Changedsave_sandbox_registry1 field changed
      • addedInput schema / properties / scheme / description
        Added value: +"How the registry takes the credential: 'bearer' sends secret as a token, 'basic' sends username and secret"
    • Changedsave_sandbox_template1 field changed
      • addedInput schema / properties / dockerfile / description
        Added value: +"The Dockerfile that builds the sandbox image. Give this or starterId to create; it replaces the file on an update"
    • Changedset_agents_override3 fields changed
      • addedInput schema / properties / effort / description
        Added value: +"Reasoning effort for this role, mapped to the harness's own control at dispatch: low, medium, high or xhigh, and also max on claude-code. A value the harness does not take is refused at dispatch."
      • addedInput schema / properties / harness / description
        Added value: +"The code harness this role runs on for this mission"
      • addedInput schema / properties / model / description
        Added value: +"The model id this role uses for this mission. The harness must support it"
    • Changedupdate_scheduled_task2 fields changed
      • addedInput schema / properties / name / description
        Added value: +"Scheduled task name, shown in the console and used in the issue title"
      • addedInput schema / properties / skipIfPreviousOpen / description
        Added value: +"Skip a scheduled fire while the previous fire's mission is still open (default true — the guardrail is opt-out, not opt-in)"
  8. 71 tool updates
    • First observedactivate_sandbox_template_version
    • First observedapprove_mission_ceiling
    • First observedarchive_mission
    • First observedbind_mcp_server
    • First observedbuild_sandbox_template
    • First observedclear_agents_override
    • First observedcreate_mission
    • First observedcreate_scheduled_task
    • First observeddecline_mission
    • First observeddelete_sandbox_registry
    • First observeddelete_sandbox_template
    • First observeddelete_scheduled_task
    • First observeddestroy_environment
    • First observeddismiss_notice
    • First observedget_agent_run
    • First observedget_agent_run_logs
    • First observedget_agents_override
    • First observedget_artifact
    • First observedget_billing
    • First observedget_environment
    • First observedget_mission
    • First observedget_notices
    • First observedget_project
    • First observedget_sandbox_build_log
    • First observedget_sandbox_capacity
    • First observedget_scheduled_task
    • First observedget_setup_progress
    • First observedhealthz
    • First observedinspect_mission
    • First observedlist_agent_runs
    • First observedlist_artifacts
    • First observedlist_billing_ledger
    • First observedlist_credit_offers
    • First observedlist_environments
    • First observedlist_mcp_bindings
    • First observedlist_mcp_servers
    • First observedlist_missions
    • First observedlist_orgs
    • First observedlist_projects
    • First observedlist_sandbox_registries
    • First observedlist_sandbox_starters
    • First observedlist_sandbox_template_versions
    • First observedlist_sandbox_templates
    • First observedlist_scheduled_task_runs
    • First observedlist_scheduled_task_templates
    • First observedlist_scheduled_tasks
    • First observedmerge_mission
    • First observedpause_mission
    • First observedrefresh_mcp_server_tools
    • First observedrequest_more_credits
    • First observedresume_mission
    • First observedrun_scheduled_task
    • First observedsave_sandbox_registry
    • First observedsave_sandbox_template
    • First observedsend_mission_instruction
    • First observedset_agents_override
    • First observedset_mcp_config_path
    • First observedset_mission_auto_merge
    • First observedset_mission_disposition
    • First observedstart_credit_checkout
    • First observedstart_mcp_server_oauth
    • First observedstart_mission
    • First observedstop_agent_run
    • First observedstop_mission
    • First observedunarchive_mission
    • First observedunbind_mcp_server
    • First observedupdate_project
    • First observedupdate_scheduled_task
    • First observedupdate_team_subscription
    • First observedwatch_agent_run
    • First observedwhoami

Publisher details

Operator
SHIP
Operator website
https://letsship.ai
Vendor relationship
First-party
Restrictions
Unknown

Related MCP Connectors

Related MCP Servers

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.