Skip to main content
Glama

Stomio

Server Details

The beta program platform for hardware and IoT teams, who run structured betas with their own testers. Connect your AI assistant to build tasks and steps, screen applicants, triage tickets, read results and track tester progress. Actions that reach testers return a preview first and take effect only after confirmation. Sign in with OAuth, with read-only or read and write access. Stomio does not recruit testers.

Ownership verified
Status
Healthy
OAuth
Works in Glama
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
ChatGPT
–
Claude
–

TDQS

A3.8/5.0

Scored across 94 tools

Disambiguation4/5

Despite 94 tools, the surface is organized around distinct resources and actions: beta lifecycle, tasks/steps, testers/invitations, applicants/screening, tickets, results, labels/tags, and media. The descriptions actively disambiguate the riskiest overlaps (per-beta labels vs workspace tester tags vs ticket tags; task vs step media; summary vs detail results). A few read tools remain close cousins (e.g. getTaskResults vs getStepResults, getTicketDetails vs getTicketMessagesBatch), but their scopes are clear.

Naming Consistency4/5

The dominant pattern is consistent camelCase verb_noun (getBetaTasks, createTask, updateStep, deleteBetaLabel, inviteTesters). There are minor verb preferences (add vs create, set vs update, manage vs assign) and some domain prefixes vary between Beta and Product, but the style is predictable and readable throughout. No chaotic mixing of casing or naming schemes.

Tool Count1/5

94 tools is far beyond a manageable MCP surface and will impose substantial selection and context overhead even if each tool has a distinct role. The breadth reflects a full beta-testing platform, but the rubric's extreme-mismatch band fits a tool count this large. Consolidation by workflow or resource would greatly improve usability.

Completeness4/5

Coverage is unusually comprehensive: full lifecycle for betas, tasks/steps, testers, invitations, applicants, screening, tickets, announcements, agreements, labels/tags, media, and results. The main gaps are minor domain edges such as no permanent beta delete, no ticket delete/merge, and workspace tag/custom-field definitions remaining dashboard-only. Agents can work around these gaps with archive/lifecycle and existing read/write tools.

Available Tools

94 tools
addBetaManagerA
Destructive
Inspect

Assigns a workspace member as a manager of this beta project. They must already be a member of the workspace (see listOrganizationUsers; inviting new members to the workspace is not possible over MCP). They are notified by email and in-app. Their workspace role (owner/manager/viewer) is unchanged: viewers stay read-only. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
userIdYesThe user id of a workspace member, from listOrganizationUsers or listBetaManagers. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and destructiveHint=true; the description adds substantial context beyond that: the target user is notified by email and in-app, their workspace role is unchanged and viewers stay read-only, and the first call is a no-op preview. It also discloses token behavior (single-use, short-lived), which an agent cannot infer from the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the action and prerequisite, then the critical two-step protocol, with no filler sentences. It is dense and slightly repetitive with the schema's confirmationToken description, which is why it stops short of a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers prerequisites, side effects (notifications), safety caveats, and the confirmation flow, which is strong for a 3-parameter mutation. However, with no output schema, it never says what the preview or success response contains (e.g., what fields the agent should surface to the user), leaving a gap for a two-step approval workflow.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning the schema cannot convey: confirmationToken must be paired with the SAME arguments, is single-use, and expires after a few minutes. It also cross-references where valid userId values come from (listOrganizationUsers / listBetaManagers).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Assigns a workspace member as a manager of this beta project.' That is clearly distinguishable from siblings like removeBetaManager, listBetaManagers, and inviteTesters, so an agent can select it without opening other schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives both prerequisites ('must already be a member of the workspace (see listOrganizationUsers)') and an explicit exclusion ('inviting new members to the workspace is not possible over MCP'), pointing to the right alternative. It also spells out the two-step when-to-call sequence: first without a token to get a preview, then again only after explicit user approval.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

addScreeningQuestionAInspect

Adds a question to this beta project's screening form (the public application form). Only usable BEFORE the form is live: publish it with publishScreeningForm once the questions are ready. Allowed types: MultipleChoices, SingleChoice, FreeText, YesNo, Rate, Instructions (file and ranking are not supported on screening forms). Auto-review rules can then reference the question by its questionId (getScreeningSetup). Requires the screening setup to exist (configureScreening). Each question carries ready: false means it fails authoring validation (a choice question with no options, say) and blocks publishScreeningForm until fixed. ready is input for your reasoning, not for users: when everything passes, do not mention it; when something fails, say what is missing in plain words instead of quoting the flag.

NameRequiredDescriptionDefault
textYesThe question shown to applicants.
typeYesThe answer type.
contentNoThe step configuration, shaped by type: for MultipleChoices/SingleChoice an array of at least 2 { choice } options with non-empty texts, optionally plus ONE final { choice: "Other", other: true } option that lets the tester type a free answer (its label is fixed: choice must be the literal text "Other", any other wording is rejected); for Rate an object { scale (number, 3 or more), symbol ("Thumb", "Star" or "Square"), lowerText?, higherText? (labels for the scale ends, up to 20 characters, default empty) }; for Ranking an array of at least 2 items { id (a v4 UUID you generate), text } with unique ids and unique texts; for TriggeredEmail an object { to, subject, message, cc? } where to is { allManagers?: boolean, allTesters?: boolean, tagsIds?: string[], personFieldIds?: string[] } and names at least one audience, subject is plain text of at most 128 characters and takes no variables (nothing replaces them there, so the recipient would read the braces), message is the email body as basic HTML reading at most 4096 characters of text, and cc is a list of Person custom field ids. Any other key of the email object or of to is rejected rather than dropped, so bcc and to.emails are not accepted: nothing would send to them. tagsIds are tester tag ids from listTesterTags, and ONLY the entries it marks assignable: a computed smart tag is rejected because a triggered email cannot be sent to one. personFieldIds and cc are ids of Person-type fields from listCustomFields; a personFieldIds field routes the email to the manager-persons that tester recorded in it, cc puts them in copy. The email goes out the first time a tester answers or skips the step before it (skipping a whole optional task sends nothing), one email per tester, and the audiences reach: allTesters ONLY that tester, never every tester in the beta project; tagsIds that same tester, only if they carry one of the tags (tester tags never reach managers, and beta labels are not tags); allManagers every manager on the beta project. To notify a manager when a tester finishes a task, make this the task's last step and use allManagers or personFieldIds. message may use these tester variables: {{USERNAME}}, {{FULL_NAME}}, {{ADDRESS}}, {{EMAIL}}, {{PHONE}}, {{PROJECT_NAME}}, {{WORKSPACE}}, {{REMAINING_PROJECT_DAYS}}, {{CURRENT_WEEK_DAY}}. The {{MANAGER_*}} ones belong to invitations and are rejected here. On an SDK beta project the audience must be managers only (allManagers), because its testers are not workspace testers and every other audience would reach nobody. REQUIRED for MultipleChoices, SingleChoice, Rate, Ranking and TriggeredEmail; omitting it is rejected. Must be OMITTED for FreeText, Instructions and File; passing one is rejected. Omit it for YesNo too (its answers are fixed to Yes / No, set automatically); passing that exact pair back is accepted, anything else is rejected. Pass the array or object value itself, not a JSON-encoded string. Call getBetaTasks to see the current shape of existing steps.
requiredNoWhether the applicant must answer. Defaults to true.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
descriptionNoSupporting detail shown under the question (basic HTML allowed).

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, it discloses important behavior: allowed and unsupported question types, the pre-live constraint, the readiness validation that blocks publishScreeningForm, and the auto-review linkage by questionId. The annotations only provide basic mutation hints, so this added operational context is substantial.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded and efficient for a tool with complex authoring rules. It contains useful constraints and only a small amount of meta-guidance about interpreting the ready flag, which is justified by the behavior it describes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex authoring tool, it covers the key prerequisites, timing window, allowed types, readiness validation, and follow-up actions. It only lightly implies the return value via questionId and does not fully explain the response shape, but the core invocation context is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the input schema's content property already documents type-dependent content rules extensively. The description mainly reiterates the allowed types and readiness behavior rather than adding new parameter-level semantics beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the exact verb and resource: 'Adds a question to this beta project's screening form (the public application form).' It also distinguishes the tool from publishScreeningForm and getScreeningSetup by explaining the authoring phase and prerequisites.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It clearly states when the tool is usable: only BEFORE the form is live, after the screening setup exists. It also points to publishScreeningForm as the next step, but it does not explicitly route edits to updateScreeningQuestion or deletes to deleteScreeningQuestion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

archiveBetaA
Destructive
Inspect

Archive this beta project: its testers' in-app notifications for it are removed, its managers are emailed, and its applicant screening form (if any) is closed. Archiving freezes the beta's content (tasks and steps), its schedule and details, its tester list, manager removal, its NDA document, its opt-in link, and its screening setup. It does NOT freeze everything: cancelling pending invitations, applicant review and rules, labels and tags, announcements, the per-tester NDA bypass, and the ticket conversation all still work, and a manager can still be added. A finished beta cannot be archived (it is already frozen). Reversible with restoreBeta. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare destructiveHint=true, and the description adds substantial context beyond that: exactly which resources are frozen, which remain mutable (invitations, applicant review, labels, announcements, NDA bypass, ticket conversation), and that manager addition still works. It also discloses side effects like tester notifications being removed and managers being emailed, plus token single-use and expiry behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core action and effects before the freeze list and two-step instructions. The freeze/no-freeze enumeration is long but each item carries real behavioral information; only minor trimming would improve it.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive, two-step mutation with no output schema, the description covers consequences, invariants (cannot archive a finished beta), reversibility, the full confirmation workflow, and token lifetime. An agent has everything needed to call it safely and correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both productId and confirmationToken are already documented in the schema. The description reinforces the two-step token workflow (omit first, resend with same arguments), but adds no syntax or format detail beyond what the schema provides, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Archive this beta project') and immediately enumerates the concrete effects (notifications removed, managers emailed, screening form closed). It is clearly distinguished from siblings like pauseBeta and restoreBeta through the freeze/no-freeze breakdown.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states a hard precondition ('A finished beta cannot be archived'), names the reversal tool ('Reversible with restoreBeta'), and gives an explicit two-step invocation protocol with when to call with and without the token. Nothing about when to use this tool is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cancelInvitationsA
Destructive
Inspect

Withdraws THIS beta project's pending tester invitations, so those people can no longer join it. Get the targets from listPendingInvitations: name them by invitationIds, which works for every pending invitee, or by emails, which only works for the invitees whose address listPendingInvitations reports (an invitee who already holds a workspace tester account is identified by id and name instead). At least one of the two is required and they combine freely; naming the same person both ways is one target, not two, and one call withdraws at most 100 pending invitations. An email or id with no pending invitation on this beta project comes back as cancelled:false and nothing happens to it: people who already accepted and became testers are unaffected (use removeTester for those), and invitations to OTHER beta projects are left untouched. Nobody is emailed about the cancellation.

NameRequiredDescriptionDefault
emailsNoThe email addresses whose pending invitations to withdraw, from listPendingInvitations.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
invitationIdsNoThe ids of the pending invitations to withdraw, from listPendingInvitations. Use these for invitees whose address is not reported.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare destructiveHint=true and readOnlyHint=false; the description adds concrete consequences the agent cannot infer: a cap of 100 withdrawals per call, no email notification to anyone, cancelled:false plus no-op for unmatched targets, and scope limited to one beta project. These are exactly the operational details an agent needs before performing a destructive action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The key constraint (what is withdrawn and where targets come from) is front-loaded, and every sentence carries information. It is a single dense paragraph with stacked em-dash clauses, which is slightly harder to scan than a short bulleted structure would be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, yet the description still explains the meaningful return signal (cancelled:false for non-matching targets). Combined with the source of targets, the 100-item cap, and the interaction with removeTester, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and each parameter is documented, but the description adds cross-parameter meaning absent from the schema: emails only work for addresses listPendingInvitations reports while invitationIds work for every invitee, the two inputs combine freely, at least one is required, and naming the same person both ways counts as one target. That is genuine added value, though per-field detail largely duplicates the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource scoped precisely: "Withdraws THIS beta project's pending tester invitations." The emphasis on THIS and pending separates it from removeTester and from actions on other beta projects without requiring the agent to open a sibling schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes the agent to listPendingInvitations as the source of targets, and names the alternative for a related case ("people who already accepted and became testers are unaffected (use removeTester for those)"). It also states the exclusion that invitations to OTHER beta projects are untouched, so when-to-use and when-not-to-use are both covered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

commitTaskChangesA
Destructive
Inspect

Publishes ALL staged task and step changes of the beta project to its testers, exactly as the Publish button in Stomio does. A staged task that is not ready (no steps yet) blocks the publish: finish it or delete it from the staging area first. The confirm publishes exactly the staged set the preview was issued for: if anything is staged, un-staged, or committed by anyone between the preview and the confirm, the confirm fails and you must preview again. Testers are not notified, the content simply updates. If the staged changes include deleted tasks or steps, committing PERMANENTLY deletes all tester answers, attachments, and completion records collected for them. The response's totalTasks is the beta's total task count after the commit, not a count of changes; diff getBetaTasks live vs unpublished BEFORE committing to know what will change. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description discloses important behavior: testers are not notified, deleted tasks or steps permanently delete tester answers, attachments, and completion records, and totalTasks is the post-commit beta task count rather than a change count. It also explains single-use, expiring confirmation tokens and concurrent-change failure behavior, which materially improves safe invocation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long but front-loads the core action and uses clear sentences for blocking conditions, destructive effects, totalTasks semantics, and the two-step flow. Some wording duplicates the confirmationToken schema description, keeping it just short of maximally concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, but the description explains the important response semantics via totalTasks and the preview/token flow. For a destructive two-step mutation with only two parameters, it gives enough context about permissions-like constraints, side effects, and concurrency to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters are already documented in the schema. The description mostly repeats the two-step confirmationToken usage already present in the schema and adds little new parameter-level meaning beyond noting token expiry in minutes rather than merely 'short-lived'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it publishes ALL staged task and step changes of the beta project to its testers. It also distinguishes the operation from a live beta publish by describing it as the equivalent of Stomio's Publish button and by emphasizing staged versus live changes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is explicit: call without confirmationToken first to preview, show the preview to the user, then call again with the same arguments plus the token only after explicit approval. It also gives when-not conditions, such as blocked publish when a staged task has no steps, and the need to preview again if the staged set changed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

configureOptInA
Destructive
Inspect

Configure this beta project's public opt-in join link: anyone with the link can sign up as a tester (subject to the domain restriction). Provide at least one of enabled and allowedDomains. enabled: true creates the link if none exists (an existing link is kept unchanged); enabled: false DELETES the link, and anyone holding it can no longer join — re-enabling later mints a DIFFERENT link, the old one stays dead. allowedDomains restricts signups to email addresses on those domains (like acme.com, at most 50); an empty list removes the restriction. The link itself (optInLink, also on getProductOverview) only becomes visible and joinable once the beta is published, running, or paused; configuring it on a draft is fine and it goes live at publish.

NameRequiredDescriptionDefault
enabledNoWhether the public opt-in link exists. false deletes it permanently; re-enabling creates a NEW link.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
allowedDomainsNoEmail domains allowed to sign up (like acme.com, at most 50). An empty list removes the restriction.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond destructiveHint=true by spelling out exactly what is destroyed: enabled:false permanently deletes the link, holders can no longer join, and re-enabling mints a different link while the old one stays dead. It also discloses the publish-state gating of link visibility, which annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and the two irreversible branches are explained crisply, but the dense run-on sentence packs several independent facts together and repeats schema-level details (50-domain cap, empty list behavior) that an agent already gets from the parameters.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-parameter mutation with no output schema, the description covers everything an agent needs to act safely: required parameter combination, permanence of deletion, link regeneration, domain semantics, and the publish-state dependency of the resulting link.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, and the description largely restates the schema's enabled and allowedDomains meanings (including the 50-domain cap and empty-list behavior). It does add one real cross-parameter rule not expressible in the schema — at least one of enabled/allowedDomains must be provided — but most of the per-parameter text duplicates structured fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a precise verb+resource: 'Configure this beta project's public opt-in join link.' It is clearly distinct from nearby siblings like configureScreening or setInvitationEmail, and even names the related read surface (getProductOverview exposes optInLink).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

States the precondition that at least one of enabled/allowedDomains must be supplied, explains the true vs false branches, and clarifies that configuring on a draft is valid and the link only goes live at publish/running/paused. It does not explicitly redirect to a sibling for viewing the link, but the operational context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

configureScreeningAInspect

Create or update this beta project's applicant screening setup. On first call it creates the screening source, applicant pool, and an EMPTY screening form; author its questions with addScreeningQuestion, then make it live with publishScreeningForm (the public link only accepts applications once the form is live). active gates the public application link (false pauses new applications without deleting anything). customThankYouTitle (up to 200 characters) and customThankYouSubtitle (up to 1000 characters) override the copy applicants see after submitting; pass an empty string to restore the default copy. Configuring sends nothing and notifies nobody.

NameRequiredDescriptionDefault
activeNoWhether the public application link accepts new applications (given the form is also published).
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
customThankYouTitleNoTitle applicants see after submitting. Empty string restores the default.
customThankYouSubtitleNoSubtitle applicants see after submitting. Empty string restores the default.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well past the annotations (readOnlyHint=false, destructiveHint=false) by disclosing that the first call creates three entities plus an empty form, that active=false pauses applications without deleting anything, and that configuring 'sends nothing and notifies nobody.' The empty-string restore behavior for the thank-you copy is also spelled out.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action, then the create/author/publish workflow, then parameter caveats. Dense but every clause carries operational information; nothing is repeated from the schema verbatim.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a four-parameter mutation tool with no output schema, the description covers creation side effects, ordering constraints, public-link gating, and non-notification guarantees. An agent has everything needed to call it correctly and in the right order.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description earns an extra point by attaching consequences to parameters — active gates the link without deletion, and empty strings restore default copy. It stops short of adding anything for productId beyond what the schema already says.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Create or update this beta project's applicant screening setup.' It also names the sibling tools it hands off to (addScreeningQuestion, publishScreeningForm), so an agent can separate it from the question-level and publish-level tools immediately.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly gives the sequence: first call creates source/pool/empty form, then author questions with addScreeningQuestion, then publish via publishScreeningForm, and notes the public link only accepts applications once live. That is a when-to-use and what-happens-next chain with no inference required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

createApplicantRuleAInspect

Create an auto-review rule for this beta project's applicant screening. When a new application matches ALL of the rule's conditions, the applicant's SUGGESTED status becomes actionSuggestedStatus and the suggested labels are attached; suggestions never invite or reject anyone by themselves, a manager still reviews (reviewApplicants). Each condition is { type: "question" | "emailDomain", operator: "equals" | "contains" | "exists" | "gt" | "lt", questionId?, value? }. type "question" compares an applicant's answer to a screening form question (questionId required, from getScreeningSetup); type "emailDomain" compares the applicant's email domain (contains not supported). "exists" takes no value; every other operator requires one. gt/lt are numeric and question-only (like a Rate answer). A rule matches when ALL its conditions match. Existing applicants are re-evaluated immediately (refreshedCount says how many suggestions changed). Requires the screening setup to exist (getScreeningSetup / configureScreening).

NameRequiredDescriptionDefault
nameYesA short manager-facing rule name.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
conditionsYesThe conditions that must ALL match.
actionSuggestedStatusYesThe status to suggest when the rule matches.
actionSuggestedLabelIdsNoPer-beta label ids (listBetaLabels) to suggest when the rule matches.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnlyHint=false, destructiveHint=false), it discloses that existing applicants are re-evaluated immediately on creation and that refreshedCount reports how many suggestions changed — non-obvious side effects an agent must anticipate. It also makes the non-destructive scope explicit: suggestions never invite or reject anyone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with purpose and match semantics before drilling into condition grammar, which is good ordering. It is dense and slightly repetitive — the ALL-conditions match rule is stated twice — and embeds a schema-like structure in prose rather than relying on the input schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a write tool with no output schema, it covers prerequisites, match semantics, per-field condition grammar, downstream effects, and even names the return field refreshedCount. An agent has everything needed to construct a valid call and anticipate its effects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description goes well past it by fully specifying the condition object grammar (type, operator, questionId, value) and constraints the schema only hints at: questionId required for type "question", "contains" unsupported for emailDomain, "exists" takes no value, and gt/lt are numeric and question-only. This is meaning the schema alone does not supply.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names a specific verb and resource ("Create an auto-review rule") and scopes it to this project's applicant screening, which cleanly separates it from updateApplicantRule and deleteApplicantRule in the sibling list. It immediately explains what the rule does downstream (suggested status + labels on match).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the prerequisite (screening setup must exist, pointing to getScreeningSetup / configureScreening) and clarifies the rule only produces suggestions that a manager still reviews via reviewApplicants. It does not explicitly say when to use create versus updateApplicantRule, so a small routing gap remains.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

createBetaAInspect

Create a new beta project in this Stomio workspace as a draft. You become its manager and AI features are enabled on it, so it is immediately usable with the other tools. The draft is invisible to testers until it is published. Typical flow: createBeta, then write its tasks and steps yourself with createTask and createStep, then inviteTesters, then publishBeta (schedule) or launchBeta (start now). Only name is needed to create the project; everything else can be added afterwards.

NameRequiredDescriptionDefault
codeNoTicket reference prefix (2 to 4 characters, unique within the workspace). Tickets in the beta are numbered from it, like APP-1, APP-2. If omitted, a prefix is generated from the beta name when the beta is published.
nameYesBeta project name, unique within the workspace (up to 64 characters).
objectivesNoWhat the manager wants to learn or validate (hypotheses, decisions, focus areas). Manager-facing context stored on the project, never sent to testers. Optional, and editable later with updateBetaDetails.
descriptionNoWhat this beta project is about: the product or feature under test, who the testers are, what they will be doing. Manager-facing context stored on the project, never displayed to testers. Optional, and editable later with updateBetaDetails.
customFieldsRequiredNoWhether testers must fill the workspace's custom profile fields during onboarding. Defaults to true, so they are required unless you pass false. The field definitions themselves are workspace-level configuration, not editable over MCP.
shippingInfoRequiredNoWhether testers must provide a shipping address during onboarding (for betas that ship physical hardware). Defaults to false.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only say non-readonly, non-destructive; the description adds real behavioral context: the caller becomes manager, AI features are auto-enabled, and the project stays invisible to testers until published. It does not state what the call returns or what happens on a duplicate name (the schema hints name is unique), which keeps it from a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with what the tool creates and the resulting state, then the workflow. Most sentences carry information, though a phrase like 'so it is immediately usable with the other tools' is mildly redundant with the AI-features sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter creation tool with no output schema, the description covers lifecycle position, ownership, visibility, and defaults well. The one real gap is the return value (e.g. the created beta's identifier), which an agent would need to chain into createTask or publishBeta.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so the baseline is 3, but the description adds cross-parameter semantics the schema cannot: only name is required at creation and every other field can be supplied later via updateBetaDetails. It does not elaborate on the boolean defaults, which the schema already covers.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Create a new beta project') plus the resulting state ('as a draft') and the effect on the caller ('You become its manager'). This distinguishes it clearly from sibling mutations like duplicateBeta, archiveBeta, or restoreBeta.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit ordered workflow naming the actual sibling tools (createBeta -> createTask -> createStep -> inviteTesters -> publishBeta/launchBeta), and even branches on schedule-vs-start-now. It also states the minimum viable call ('Only name is needed').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

createBetaLabelsAInspect

Creates labels in this beta project, inside one label category (existing or new). Labels are what manageTaskLabels and manageTesterLabels assign, and they gate task visibility: a task labeled X is only visible to testers labeled X, so to segment testers, create the labels here first, then label the testers and the tasks. Pass categoryId for an existing category (from listBetaLabels) OR categoryName to reuse a category by name or create it. Label names are unique across the whole beta project, not just the category. Colors are hex like '#0F77F0'; omit color to get one assigned from the app's palette. Categories, labels, and their assignments are manager-internal and never notify anyone.

NameRequiredDescriptionDefault
labelsYesThe labels to create (1 to 20 per call).
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
categoryIdNoID of an existing label category, from the categoryId field of listBetaLabels. Provide exactly one of categoryId or categoryName.
categoryNameNoCategory name. Reuses the existing category with this name (case-insensitive) or creates a new one. Provide exactly one of categoryId or categoryName.
groupResultsNoOnly when creating a NEW category. When true, testers sharing a label of this category are aggregated into a single group in the results summary (completion shows 0% or 100%, tester counts capped at 1). Defaults to false.
shareTicketsNoOnly when creating a NEW category. When true, testers sharing a label of this category can see and interact with each other's tickets (except chats). Defaults to false.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare non-read-only, non-destructive. The description adds real behavioral context beyond that: passing categoryName can create a new category as a side effect, uniqueness is enforced project-wide (implying duplicate-name failures), color is auto-assigned from a palette when omitted, and the whole operation is manager-internal and sends no notifications. It stops short of saying what happens on a partial failure or duplicate name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then relationships, then parameter rules, then side-effect caveats — a logical progression. It is somewhat long and repeats the color/hex detail already in the schema, but no sentence is pure filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter nested-array creation tool with no output schema, the description covers workflow, uniqueness, category creation side effects, and notification behavior well. The notable gap is the return value — with no output schema, an agent does not learn whether created label IDs or categories are returned.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description still adds meaning: it clarifies that name uniqueness is 'across the whole beta project, not just the category' (sharpening the schema's looser wording) and frames categoryName as a reuse-or-create switch rather than a plain string.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource+scope: 'Creates labels in this beta project, inside one label category (existing or new)'. It immediately distinguishes itself from updateBetaLabel/deleteBetaLabel and from manageTaskLabels/manageTesterLabels by explaining that this tool creates the label entities those tools merely assign.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit sequencing guidance ('create the labels here first, then label the testers and the tasks') and routes the agent to siblings: listBetaLabels for existing category IDs, manageTaskLabels/manageTesterLabels for assignment. It also states the either/or rule for categoryId vs categoryName, so an agent knows how to choose between the two paths.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

createStepAInspect

Adds a step to a task and appends it to the end of that task's step list. A step is one thing a tester reads, answers, or uploads, except TriggeredEmail: that one is invisible to testers and sends an email the moment a tester answers the step immediately before it, so it takes no text and is rejected as the first step of a task. Creating always appends, so create steps in the order testers should work through them, or reorder them afterwards with updateStepsOrder. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
textNoThe step's question or title, as the tester sees it (up to 128 characters). Omit it for a TriggeredEmail step, which testers never see; passing one is rejected.
typeYesThe kind of step. MultipleChoices, SingleChoice, Rate, Ranking and TriggeredEmail REQUIRE a content shaped for the type (a choices array, a choices array, a scale object, an items array, an email object). FreeText, Instructions and File take no content and reject one. YesNo takes no authored content either: omit it and the fixed Yes / No pair is set automatically, though echoing that exact pair back is accepted so a step read from getBetaTasks round-trips. TriggeredEmail is not a step testers see or answer: it sends an email the moment a tester answers the step immediately before it, so it takes no "text" and can never be the first step of a task.
taskIdYesID of the task, from getBetaTasks. Never invent ids.
contentNoThe step configuration, shaped by type: for MultipleChoices/SingleChoice an array of at least 2 { choice } options with non-empty texts, optionally plus ONE final { choice: "Other", other: true } option that lets the tester type a free answer (its label is fixed: choice must be the literal text "Other", any other wording is rejected); for Rate an object { scale (number, 3 or more), symbol ("Thumb", "Star" or "Square"), lowerText?, higherText? (labels for the scale ends, up to 20 characters, default empty) }; for Ranking an array of at least 2 items { id (a v4 UUID you generate), text } with unique ids and unique texts; for TriggeredEmail an object { to, subject, message, cc? } where to is { allManagers?: boolean, allTesters?: boolean, tagsIds?: string[], personFieldIds?: string[] } and names at least one audience, subject is plain text of at most 128 characters and takes no variables (nothing replaces them there, so the recipient would read the braces), message is the email body as basic HTML reading at most 4096 characters of text, and cc is a list of Person custom field ids. Any other key of the email object or of to is rejected rather than dropped, so bcc and to.emails are not accepted: nothing would send to them. tagsIds are tester tag ids from listTesterTags, and ONLY the entries it marks assignable: a computed smart tag is rejected because a triggered email cannot be sent to one. personFieldIds and cc are ids of Person-type fields from listCustomFields; a personFieldIds field routes the email to the manager-persons that tester recorded in it, cc puts them in copy. The email goes out the first time a tester answers or skips the step before it (skipping a whole optional task sends nothing), one email per tester, and the audiences reach: allTesters ONLY that tester, never every tester in the beta project; tagsIds that same tester, only if they carry one of the tags (tester tags never reach managers, and beta labels are not tags); allManagers every manager on the beta project. To notify a manager when a tester finishes a task, make this the task's last step and use allManagers or personFieldIds. message may use these tester variables: {{USERNAME}}, {{FULL_NAME}}, {{ADDRESS}}, {{EMAIL}}, {{PHONE}}, {{PROJECT_NAME}}, {{WORKSPACE}}, {{REMAINING_PROJECT_DAYS}}, {{CURRENT_WEEK_DAY}}. The {{MANAGER_*}} ones belong to invitations and are rejected here. On an SDK beta project the audience must be managers only (allManagers), because its testers are not workspace testers and every other audience would reach nobody. REQUIRED for MultipleChoices, SingleChoice, Rate, Ranking and TriggeredEmail; omitting it is rejected. Must be OMITTED for FreeText, Instructions and File; passing one is rejected. Omit it for YesNo too (its answers are fixed to Yes / No, set automatically); passing that exact pair back is accepted, anything else is rejected. Pass the array or object value itself, not a JSON-encoded string. Call getBetaTasks to see the current shape of existing steps.
requiredNoWhether the tester must answer this step. Defaults to true.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
descriptionNoSupporting detail shown under the step (up to 1024 characters of text; basic HTML is allowed).

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, which only declare this as a non-read-only, non-destructive mutation, the description discloses substantial behavior: staged versus live changes, staged-id responses with liveVersionId links, TriggeredEmail sending semantics and timing, audience routing constraints, and edit restrictions on finished or archived projects. This is exactly the kind of operational context annotations do not provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded and organized, with the core append behavior and staging model stated early. However, it is very long and repeats TriggeredEmail and content-shape details that are also present in the schema, reducing conciseness for an already complex definition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity, including staging semantics, TriggeredEmail rules, audience restrictions, and extensive content validation, the description is complete enough for an agent to call it correctly. The absence of an output schema is mitigated by the description's note that responses return staged rows and unexpected ids.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the description largely restates the detailed schema guidance for content, type, text, and audience fields rather than adding substantial new parameter meaning. It does add useful cross-references and behavioral context, but the parameter-level semantics are already carried by the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it adds a step to a task and appends it to the end of that task's step list. It distinguishes the special TriggeredEmail step type and points to sibling tools such as updateStepsOrder and commitTaskChanges, so an agent can identify the operation without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly tells the agent to create steps in tester order because creation always appends, and to use updateStepsOrder to reorder afterwards. It also explains immediate application on draft/scheduled projects versus staged changes on running/paused projects, names commitTaskChanges and revertTaskChanges as the publishing/discarding alternatives, and states that finished or archived projects cannot be edited.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

createTaskAInspect

Creates a new task in this beta project and appends it to the end of the task list. A task is a container for the steps the testers work through; add those with createStep. The response's ready is false until the task has at least one step: that is expected right after creation, not an error, and nothing to tell the user about; just continue with createStep. Creating always appends, so create tasks in the order testers should meet them, or place them afterwards with updateTasksOrder. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
nameYesThe task name (1 to 128 characters).
optionalNoWhether testers may skip this task: they get a Skip control and a skipped task counts as done. Defaults to false. Only tasks of Stomio-placement beta projects can be optional, and a task cannot be both optional and scheduled with availableDate: the two are mutually exclusive.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
descriptionNoWhat the testers should do in this task (up to 512 characters of text; basic HTML is allowed).
availableDateNoISO 8601 date-time before which the task shows as locked and testers cannot start it. Omit for available immediately; null (updateTask) clears an existing date. A date past the beta end date is accepted here but blocks publishBeta and commitTaskChanges until fixed. Mutually exclusive with optional: a task can be optional or scheduled, not both.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With annotations only saying this is a non-read-only, non-destructive mutation, the description adds substantial behavior beyond them: staging on running or paused projects, the need for commitTaskChanges to publish, revertTaskChanges to discard, and getBetaTasks with version 'unpublished' to inspect staged state. It also explains that responses return staged ids with liveVersionId links, and that ready=false after creation is expected rather than an error.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the creation action and appending behavior, then moves into step guidance and staging semantics. It is longer than typical but dense with operational detail; most sentences carry necessary context, though the staging paragraph is quite dense.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no output schema, the description covers the important response behavior (ready=false, staged ids, liveVersionId), lifecycle edge cases (draft versus running projects, archived projects), and next-step guidance. It gives an agent everything needed to invoke the tool correctly in context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, and the description adds useful cross-parameter meaning: optional and availableDate are mutually exclusive, and optional only works for Stomio-placement beta projects. It also clarifies ordering behavior around availableDate, though most individual field semantics remain in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: creates a new task in this beta project and appends it to the end of the task list. It also distinguishes the tool from siblings by directing step creation to createStep and reordering to updateTasksOrder, so an agent can identify its unique role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit when-to-use context: create tasks in tester order because creation always appends, use updateTasksOrder for later placement, and use createStep to add steps. It also explains lifecycle constraints: edits are immediate on draft or scheduled projects, staged on running or paused projects, and blocked for finished or archived projects.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

createTicketA
Destructive
Inspect

Creates a ticket in this beta project. A "chat" ticket opens a direct conversation with ONE tester, who is notified by email and in-app — it requires that tester's user id in userId (from getBetaTesters). Every other type creates an internal ticket visible to the beta project's managers until someone makes it public in Stomio, and it notifies them. Creating a ticket also fires any connected Slack, Webex, Zapier and Jira integrations (only a Jira failure is reported back, as jiraSyncFailed; the others are fire-and-forget). The beta project must be running — paused is not enough. In the response, code is the ticket's human-visible reference (like PROD-123); it is absent on chat tickets, which are never numbered. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
typeYesThe ticket type. Use "chat" only to open a direct conversation with a tester, and pass that tester in userId.
titleYesThe ticket title (1 to 128 characters).
userIdNoREQUIRED for type "chat" and forbidden otherwise: the user id of the tester to open the conversation with, from getBetaTesters.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
descriptionNoThe ticket body (up to 1024 characters of text; basic HTML is allowed).
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare it is a non-read-only, destructive write. The description goes far beyond: it discloses notification side effects (email + in-app to the tester, or to managers), that Slack/Webex/Zapier/Jira integrations fire with only jiraSyncFailed reported back, the active-project precondition, and the two-step preview/confirm semantics with single-use expiring tokens.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but efficient: behavior type split, side effects, precondition, return-value caveat (code absent on chat tickets), and the two-step protocol are each covered in one or two sentences with no filler. Minor redundancy with the schema on the token being 'single-use and short-lived,' but nothing that wastes meaningful space.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter mutation with no output schema and heavy side effects, the description covers everything an agent needs: preconditions, the write's blast radius, which failures surface (jiraSyncFailed), and what 'code' means in the response. The two-step confirmation protocol, the most likely source of misuse, is spelled out explicitly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter including enum values and conditional userId requirements. The description reinforces that userId is the single tester's id from getBetaTesters and that it is required only for chat, but adds little beyond what the schema already states, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a specific verb and resource ('Creates a ticket in this beta project') and immediately splits the resource into two distinct behaviors (chat vs. internal ticket), which is exactly what an agent needs to distinguish it from siblings like updateTicket, replyToTicket, or makeTicketPublic.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states explicit when-to-use conditions ('chat' only to open a direct conversation with one tester; every other type for internal tickets), hard preconditions ('the beta project must be running — paused is not enough'), and a mandatory call sequence. Alternatives are not named by tool name, but no ambiguity remains about which path to choose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteApplicantRuleA
Destructive
Inspect

Delete an auto-review rule of this beta project's applicant screening. Existing applicants are re-evaluated against the remaining rules immediately (refreshedCount says how many suggestions changed); already APPROVED reviews are never touched. This cannot be undone, but a rule can be recreated with createApplicantRule.

NameRequiredDescriptionDefault
ruleIdYesThe ruleId, from getScreeningSetup. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the destructiveHint annotation: it discloses that existing applicants are immediately re-evaluated, that APPROVED reviews are never affected, and that the operation is irreversible. This is the side-effect information an agent needs before invoking a destructive tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the action and scoping, followed by consequences and recoverability. No filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description helpfully explains the refreshedCount return value; combined with the mutation semantics and recovery path, nothing material is missing for a two-parameter destructive tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both ruleId and productId documented including provenance (getScreeningSetup, listProducts). The description adds no parameter meaning beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Delete an auto-review rule') and narrows scope to 'this beta project's applicant screening', which cleanly separates it from deleteScreeningQuestion, deleteStep, and updateApplicantRule.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies the use case (removing an unwanted screening rule) and points to createApplicantRule as the recovery path, but never states the alternative for modifying rather than removing a rule (updateApplicantRule), leaving that routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteBetaLabelA
Destructive
Inspect

PERMANENTLY deletes one label of this beta project and every assignment it holds: it is taken off the testers and the tasks carrying it, off any pending invitation waiting to apply it, and out of the suggested labels of the applicant screening rules. This cannot be undone, and the label's category survives it. Task visibility is label-gated, so a task that had only this label becomes visible to EVERY tester of the beta project. The change applies immediately to live and staged tasks alike, and nobody is notified. The confirm deletes exactly the assignments the preview counted: if the testers, tasks, or pending invitations carrying this label change in between, the confirm fails and you must preview again. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
labelIdYesID of the label to delete, from listBetaLabels.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes far beyond the destructiveHint annotation: it enumerates every assignment that is removed (testers, tasks, pending invitations, screening-rule suggested labels), discloses irreversibility, the surviving category, no notifications, immediate effect on live and staged tasks, and the optimistic-concurrency failure mode where the confirm fails if assignments changed since the preview.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but front-loaded: the destructive scope comes first, the consequence for task visibility second, and the two-step protocol last. The capitalised 'TWO-STEP TOOL' block is heavy, but nearly every clause carries operational information rather than filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description compensates by explaining what the first call returns (a preview plus a confirmationToken) and what happens on the second. The main uncovered detail is the preview's payload shape, which an agent would otherwise only discover at runtime.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, including the confirmationToken field, so the baseline is 3. The description reinforces the token lifecycle (single-use, expires after a few minutes, same arguments required) but adds little meaning about labelId or productId beyond what the schema already says.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise verb and resource ('PERMANENTLY deletes one label of this beta project') and immediately bounds scope by noting 'the label's category survives it', which separates it from deleteBetaLabelCategory and updateBetaLabel. An agent knows exactly which entity is destroyed without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit invocation conditions for the two-step flow (call without confirmationToken, show preview, get user approval, re-call with the same arguments) and a re-preview condition when underlying assignments change. It does not explicitly route to sibling alternatives such as updateBetaLabel for non-destructive edits or manageTaskLabels for per-task removal, so the when-not guidance is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteBetaLabelCategoryA
Destructive
Inspect

PERMANENTLY deletes one label category of this beta project along with the labels inside it. This cannot be undone. It works only while none of those labels is assigned to a tester, a task, or a pending invitation; otherwise it fails and names the labels still in use, which you clear with manageTesterLabels and manageTaskLabels or delete with deleteBetaLabel. The confirm deletes exactly the labels the preview counted: if a label is added to the category in between, the confirm fails and you must preview again. Nobody is notified. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
categoryIdYesID of the category to delete, from the categoryId field of listBetaLabels.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and readOnlyHint=false, and the description adds substantial context beyond them: irreversibility, the failure message naming in-use labels, that nobody is notified, conditional-confirm failure on changed category contents, and token single-use/expiry. The only minor gap is no detail on the preview payload's shape, though that is largely inferable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the destructive consequence, then failure modes, then the two-step protocol, with zero filler sentences. Slight redundancy exists because the two-step confirmation flow is restated in the confirmationToken schema description, making the long final paragraph partly duplicative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive, irreversible mutation with no output schema, the description covers everything an agent needs: preconditions, alternatives, failure behavior, notification semantics, and the full preview/confirm lifecycle including race conditions and token expiry.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all three parameters; the description still adds meaning by requiring the 'SAME arguments' on the confirm call and by tying token validity to the previewed label set. That goes beyond the schema's generic 'single-use and short-lived' note.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('PERMANENTLY deletes one label category ... along with the labels inside it'), which distinguishes it from the sibling deleteBetaLabel (single label) and updateBetaLabelCategory. Scope and side effect (labels inside are removed) are explicit in the first sentence.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Names the preconditions for success (no label assigned to a tester, task, or pending invitation) and the exact alternatives for clearing blockers: manageTesterLabels, manageTaskLabels, or deleteBetaLabel. It also spells out the required two-call sequence and the re-preview requirement if a label is added mid-flow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteScreeningQuestionA
Destructive
Inspect

Deletes a screening form question. Only usable before the form is live. Auto-review rules referencing the question have that condition dropped, exactly like the app: a rule left with no conditions is deleted (rulesTouched reports how many rules were affected).

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
questionIdYesThe questionId of a screening form question, from getScreeningSetup. Never invent ids.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, and the description goes beyond them by disclosing the cascade: referencing auto-review rules lose that condition and empty rules are deleted, mirroring app behavior. The mention of rulesTouched also hints at side-effect reporting. It does not state error behavior when the form is live (rejection vs. no-op), leaving one meaningful gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the core action and constraint before the cascade detail. 'Exactly like the app' is slightly informal but conveys that the behavior matches the UI, which is useful. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and only two required params, the description covers what an agent actually needs: the precondition, the destructive cascade, and the meaning of the rulesTouched field. Nothing material is missing for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and both parameters carry their own descriptions including the 'Never invent ids' warning, so the description adds nothing about parameter meaning. Baseline 3 applies since the schema does the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Deletes a screening form question'), which cleanly separates it from sibling deletions like deleteStep, deleteTask, and deleteApplicantRule. The added precondition sharpens rather than muddies the purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a hard usage boundary: only usable before the form is live, which is the decisive gating condition an agent needs. It does not explicitly point to updateScreeningQuestion or configureScreening as the alternative when the form is already live, so it stops just short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteStepA
Destructive
Inspect

Deletes a single step from a task. Deleting is permanent: it also removes ANY answers and uploads the testers already submitted for that step. A deletion that would leave a triggered-email step first is refused, because such a step is sent when a tester answers the step right before it. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited. On a running or paused beta project the deletion is only STAGED, and the answers are destroyed when commitTaskChanges publishes it.

NameRequiredDescriptionDefault
stepIdYesID of the step to delete, from getBetaTasks. Never invent ids.
taskIdYesID of the task, from getBetaTasks. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the destructiveHint=true annotation by disclosing cascade destruction of answers and uploads, the triggered-email refusal rule, and the staging model on running/paused projects. It also warns that responses return staged rows with ids the caller did not pass in. It loses a point only for not clarifying how long staging persists or whether the refusal is checked at delete time or commit time.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the destructive consequence and well organized into constraint, workflow, and id behavior. However, the final sentence restates the staging point already made earlier ('On a running or paused beta project the deletion is only STAGED'), which is redundant and could be merged.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-parameter mutation tool with no output schema, the description covers destruction scope, preconditions, project-state branching, the staging lifecycle, and id echo behavior. An agent has everything needed to call it correctly or to know it should defer to commitTaskChanges.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline would be 3, but the description adds real meaning not in the schema: during staging both the live id and the staged id are accepted, and the response echoes the staged id with liveVersionId linking back. That is actionable id-semantics an agent would otherwise get wrong.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Deletes a single step from a task') and immediately scopes it apart from siblings like deleteTask, updateStep, or removeStepMedia by making clear it removes the step and all its submitted answers and uploads.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-not conditions (refused if it would leave a triggered-email step first; blocked entirely on finished or archived projects) and names the exact alternatives for the staged workflow: commitTaskChanges to publish, revertTaskChanges to discard, getBetaTasks with version 'unpublished' to inspect.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteTaskA
Destructive
Inspect

Deletes a task from this beta project. Deleting is permanent: it also removes every step inside the task and ANY answers, uploads and completion records the testers already submitted for them. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited. On a running or paused beta project the deletion is only STAGED, and the answers are destroyed when commitTaskChanges publishes it — the point of no return is the commit, which is why commitTaskChanges is the confirmation-gated call.

NameRequiredDescriptionDefault
taskIdYesID of the task, from getBetaTasks. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare destructiveHint=true; the description goes far beyond by disclosing the exact blast radius (all steps plus tester answers, uploads and completion records are destroyed), the staging vs immediate application rules, and that the commit — not this call — is the point of no return. It also warns that responses return STAGED rows with unexpected ids and liveVersionId, which is non-obvious behavior an agent would otherwise misread.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The critical facts are front-loaded (permanent deletion, cascade, cannot-edit archived), but the staging rule is stated twice ('the change is STAGED' near the top and again in the final sentence about the answers being destroyed at commit), and the id-handling paragraph is somewhat tangential to the delete action itself. It is dense and mostly earns its space, but the repetition and length work against scannability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive two-parameter mutation with no output schema, the description covers everything that matters: consequences, reversibility window, environment-dependent behavior, id semantics, and the blocked-state exclusion. Nothing an agent needs to call this safely is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so the baseline is 3. The description adds real meaning beyond the schema by explaining that while staging, both live and staged ids of a task are accepted but the response always carries the staged row — context the parameter docs do not provide.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Deletes a task from this beta project') and immediately scopes it against siblings by explaining the cascade (steps, answers, uploads, completion records), which distinguishes it from deleteStep and archiveBeta without requiring schema inspection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly names the conditions that change behavior (draft/scheduled = immediate; running/paused = staged), names the alternative tools that govern the staged outcome (commitTaskChanges, revertTaskChanges, getBetaTasks with version 'unpublished'), and states a hard exclusion: content of finished or archived beta projects cannot be edited.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

duplicateBetaAInspect

Create a new draft beta project as a copy of this one. Copied: all tasks and steps (with their media), the description, and the tester onboarding flags (override them with the optional arguments). NOT copied: testers, invitations, tickets, results, labels, managers other than you, the ticket code prefix, objectives, the NDA/agreement, the opt-in link, and the custom invitation email. The copy is a draft with AI features enabled, owned by you, immediately usable with the other tools.

NameRequiredDescriptionDefault
nameYesName for the copy, unique within the workspace (up to 64 characters).
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
customFieldsRequiredNoOverride the copied custom-fields requirement.
shippingInfoRequiredNoOverride the copied shipping-info requirement.

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only tell the agent this is a non-read, non-destructive write. The description goes far beyond that with an explicit copied/not-copied inventory (testers, invitations, tickets, results, labels, managers, ticket code prefix, objectives, NDA, opt-in link, custom invitation email) and states the resulting state: a draft, AI features enabled, owned by you. That is exactly the kind of side-effect disclosure a mutation tool needs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action, then the copied/not-copied inventories and the end state. The enumerated exclusion list is long but each item prevents a real surprise about missing data, so it earns its space; the closing sentence about ownership and usability is slightly redundant but still informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 4-parameter mutation with no output schema, the description covers side effects, resulting state, and ownership thoroughly. The main gap is the return value: the agent is not told that a new productId comes back, which matters because every sibling tool needs an ID and there is no output schema to fall back on.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all four parameters are already documented, including the 'override the copied X requirement' semantics. The description only alludes to them generically ('override them with the optional arguments') and labels them 'tester onboarding flags', which is a slightly different framing from the schema's custom-fields/shipping-info wording, so it adds little beyond the structured data.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Create a new draft beta project as a copy of this one') and immediately establishes the scope as a duplication rather than a from-scratch creation, which cleanly separates it from createBeta in the sibling list. An agent can tell what it produces (a draft copy) without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clear context for when to use it (copying an existing project) and a useful post-condition ('immediately usable with the other tools'). It does not explicitly name createBeta as the from-scratch alternative or state exclusions such as 'do not use for cross-workspace copies', so it stops short of full routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

flagAnswerAInspect

Flag or unflag one tester answer in this beta project. A flag is a manager-only bookmark for an answer worth coming back to (unclear, off-topic, or suspicious): the tester is not notified, the answer keeps counting in every result, and nothing about it changes for anyone else. listFlaggedAnswers reads the flagged set back, and every answer entry in the results reads already carries its own flagged state. answerId values come from the reads that list individual answers: free-text, file, instructions and ranking entries carry one, while choice and rate steps report per-option counts and the testers behind them, not answer ids.

NameRequiredDescriptionDefault
flaggedYestrue to flag the answer, false to remove an existing flag.
answerIdYesThe answer to flag or unflag. answerId values come from the reads that list individual answers: free-text, file, instructions and ranking entries carry one, while choice and rate steps report per-option counts and the testers behind them, not answer ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only say readOnlyHint=false and destructiveHint=false; the description goes well past that by declaring the flag is a manager-only bookmark, the tester is not notified, the answer keeps counting in every result, and nothing changes for anyone else. Those are exactly the non-obvious consequences an agent needs before mutating state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The behavioral constraint is front-loaded in the first sentence, followed by effect and then id provenance. Slightly long, and the answerId sentence duplicates the schema description of the same field, but no sentence is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-parameter mutation with no output schema, the description covers effect, side-effect visibility, notification behavior, and input provenance. It stops short of stating idempotency (flagging an already-flagged answer) or error behavior for an invalid or non-flaggable answerId, which would fully close the loop.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so productId, answerId and flagged are already documented, and the description's answerId provenance sentence is repeated almost verbatim from the schema. It adds no new syntax, format, or edge-case meaning beyond what structured fields provide, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Flag or unflag one tester answer in this beta project" gives a specific verb pair, a single-resource scope, and an owning entity (beta project). It is immediately distinguishable from listFlaggedAnswers, which reads the flagged set rather than mutating it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description states the precondition that governs invocation: answerId values only exist for free-text, file, instructions and ranking entries, while choice and rate steps expose per-option counts and no answer ids. It also points to listFlaggedAnswers and notes that every answer entry in results already carries its flagged state, telling the agent when it does not need this tool to inspect flags.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getAnswerAttachmentA
Read-only
Inspect

Fetch one file uploaded by a tester as a step answer, using a resourceId taken from the answer resources returned by getTaskResults or getStepResults. Images are returned as viewable image content so you can inspect a screenshot directly; videos, files and oversized or unsupported images come back as metadata plus a time-limited download URL: share it with the user, or fetch it yourself when the user asks you to read or analyze the file. A metadata-plus-URL result is the intended answer for those cases, not an error — do not retry it. Download URLs differ on every call; the resourceId is the stable identity.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
resourceIdYesThe UUID of the answer attachment (the id of an answer resource from getTaskResults or getStepResults).

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint, so the description carries the rest — and it does: images return as viewable content, other types return metadata plus a time-limited download URL, and URLs change on every call while resourceId is the stable identity. The anti-retry instruction is exactly the kind of behavioral detail annotations cannot express and prevents a common agent failure loop.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then return-shape behavior, then the retry caveat — a sensible escalation. It is denser and longer than a two-sentence ideal, but no sentence is redundant or repeats structured fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read tool with no output schema, the description covers what comes back in each case, how to pass the URL along, and the retry pitfall. Nothing an agent needs in order to call it correctly and interpret the result is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description adds genuine meaning by framing resourceId as a stable identity sourced from answer resources. productId is left entirely to the schema, which is acceptable given its own description points at listProducts.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Fetch one file uploaded by a tester as a step answer") and pins provenance by saying the resourceId comes from getTaskResults or getStepResults. It does not explicitly differentiate itself from attachment/media siblings such as getTicketAttachment, getStepMedia or getTaskMedia, so the distinction has to be inferred from the 'step answer' wording.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use guidance (share the URL with the user, or fetch it yourself when the user asks you to read or analyze the file) and an explicit when-not case: a metadata-plus-URL result is the intended answer, not an error, and must not be retried. It also points to the upstream tools that produce a valid resourceId, leaving no routing ambiguity for the common case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getApplicantsOverviewA
Read-only
Inspect

Get a summary of this beta's applicant screening pipeline. Applicants are people who filled out the public screening form (no sign-in required) and can be reviewed, then invited to become testers. Returns whether a screening form is set up, applicant counts by stage, and how many auto-suggestion rules are configured. active means the form accepts new submissions; published means the form page is live — the two are independent flags. applicantCounts.pending bundles what listApplicants reports separately as the new and needsReview statuses (pending is not itself a valid listApplicants status filter). Returns screeningSetUp=false for betas with no screening form.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=true; the description goes well beyond that by explaining the semantics of the two independent flags (active vs published), how pending aggregates new+needsReview, and that screeningSetUp=false is returned for betas without a form. With no output schema present, this return-shape disclosure carries real behavioral weight.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The first sentence front-loads the purpose and the detail sentences each convey distinct facts (flag independence, pending bundling, screeningSetUp=false). It is slightly dense but no sentence is pure filler; the final sentence is the closest to redundant with the earlier 'returns whether a screening form is set up'.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read tool with no output schema, the definition supplies the return contents, the meaning of each flag, the caveat about pending, and the empty-state behavior. There is nothing material an agent would need before invoking it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is only one parameter and schema coverage is 100%; the schema already tells the agent to get valid IDs from listProducts. The description adds no additional meaning about productId, so the baseline 3 for a fully documented schema is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: a summary of a beta's applicant screening pipeline. It then enumerates exactly what the summary contains (form-setup flag, counts by stage, auto-suggestion rule count), which lets an agent distinguish it from listApplicants (individual records) and getScreeningSetup (form configuration) without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Defines what an applicant is (filled out public screening form, no sign-in, reviewed then invited) and clarifies the relationship to listApplicants, explicitly noting that pending is not a valid listApplicants status filter. It never states a clean when-to-use-this-vs-sibling rule (e.g. 'use this for aggregate counts, use listApplicants for per-applicant data'), but the surrounding context makes the selection clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getBetaTasksA
Read-only
Inspect

Get the task list for the current beta, including steps, labels, and sync metadata. ready=false on a task or step means it fails authoring validation (a task needs a name and at least one complete step) and blocks publishing/launching until fixed. ready is input for your reasoning, not for users: when everything passes, do not mention it at all; when something fails, say what is missing in plain words instead of quoting the flag. optional=true means testers may skip the task (a skipped task counts as done); availableDate locks the task from testers until that date; a task never has both. A task or step may carry an instruction media "resource" whose URLs are time-limited download links meant for the human user; to look at it yourself, pass its id to getTaskMedia or getStepMedia. Description fields may contain HTML: the Stomio editor stores rich text and re-saves plain text wrapped in tags, so an HTML-wrapped version of text you wrote is the same content, not corruption. Staged tasks and steps always carry their own media resource ids (creating the draft duplicates media), so a differing resource id between live and staged is NOT a media edit; compare media by presence and type only. The unpublished list carries a synced flag: true means the staging area matches the live content and committing publishes nothing. The commitTaskChanges preview is the authoritative statement of what a commit changes.

NameRequiredDescriptionDefault
versionNoOmit for published tasks (what testers see, has results). Pass "unpublished" for the manager's staged draft. The unpublished list is the COMPLETE next version of the beta: committing replaces the live set with it, so a live task with no staged counterpart (no unpublished task whose liveVersionId equals the live task's id) is deleted on commit. Its CONTENTS say nothing about whether anything is pending, though, so never infer that from them: read the response's synced flag instead. synced: true means the staged version matches live and committing publishes nothing, and the list is then either a full copy of the live set or empty (no staging area has been created yet); neither shape means a pending deletion. The commitTaskChanges preview is the authoritative statement of what a commit would change. Staged entries carry liveVersionId pointing at the live entry they replace; a staged entry without liveVersionId is newly added. Only a running or paused beta project has a staging area at all: on any other state (draft, scheduled, finished, archived) this returns the live list under tasks, with a note, and no unpublishedTasks or synced key.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Consistent with readOnlyHint=true and adds substantial context beyond it: what ready=false blocks, that optional=true means skippable, that availableDate withholds a task, that URLs in media resources are time-limited links for the human user, that HTML wrapping is normal, that staged media ids legitimately differ from live ones, and that non-running/paused states return the live list with no unpublishedTasks/synced key. These are exactly the non-obvious traits an agent would otherwise misread.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The ready/optional/availableDate guidance and the media/HTML traps are each worth stating, but they are packed into one dense paragraph with no structure, and the staging-area/synced explanation is repeated almost verbatim from the schema. Front-loading is fine, but the length and repetition cost readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description does much of the work: it names the tasks/unpublishedTasks/synced keys, explains liveVersionId on staged entries, and flags the media resource shape. It does not enumerate step or label fields despite promising them in the first sentence, so an agent still learns some of the response shape by inspection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters are already documented in detail, and much of the description's text about the unpublished list (liveVersionId, synced, deletion semantics) largely restates the schema description rather than adding new meaning. The behavior of the version flag under different beta states is the main added nuance, so this sits at the baseline rather than above it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence gives a specific verb and resource: 'Get the task list for the current beta, including steps, labels, and sync metadata.' That clearly separates it from mutation siblings like commitTaskChanges and from media siblings it explicitly points to. It never contrasts itself with results/performance siblings (getTaskResults, getStepPerformanceSummary), so the differentiation is good but not complete.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives real routing guidance: pass a media resource id to getTaskMedia or getStepMedia rather than fetching the time-limited URL yourself, and treat commitTaskChanges' preview as the authoritative statement of pending changes. The published-vs-staged selection criteria live in the schema rather than the description, and it does not say when to call this versus updateTask/createStep.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getBetaTestersA
Read-only
Inspect

Get the list of testers ALREADY ATTACHED to the current beta, including invitation status, completion stats, and segmentation labels. Invitees without an account are NOT listed (use listPendingInvitations for them), and invitationStatus reads "accepted" for testers who joined without an invitation (opt-in link, import). completion is each tester's percentage of the tasks VISIBLE TO THEM — label-gated visibility means two testers' percentages can rest on different task sets, so do not compare them as if on one denominator. In tags, testers' tags are grouped under opaque tag-category ids; labels carry categoryName. customFields holds the workspace profile answers this tester gave, each entry naming its field (fieldId, name, type) alongside the value; a field the tester left unanswered is absent, and listCustomFields has the full definitions including the available choices. location is the country only. lastAnswerAt is when the tester last answered, changed, or skipped a step in this beta project, the same activity the drop-off counts use; it is absent for a tester who has not answered anything yet.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only supply readOnlyHint, so the description carries the burden and does so thoroughly: it explains that invitationStatus reads 'accepted' for opt-in/imported testers, that completion percentages are computed over label-gated visible-task sets and are not cross-comparable, that tag categories are opaque ids resolved via labels.categoryName, that unanswered custom fields are simply absent, and that location is country-only.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and the sibling exclusion before field-level caveats, and every sentence carries non-obvious semantics. The single dense paragraph and the long compound final sentence about lastAnswerAt make it harder to scan than it needs to be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must document return shape, and it does — covering invitationStatus, completion denominators, tag/category structure, customFields entries and their absence semantics, location granularity, and lastAnswerAt. Nothing an agent needs to interpret the response is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single productId parameter is fully described in the schema (100% coverage, including where to obtain valid IDs from listProducts), and the description adds nothing about it. With the schema doing all parameter work, the baseline of 3 applies rather than a penalty.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb ('Get the list') + resource ('testers') + scope ('ALREADY ATTACHED to the current beta'), plus an enumeration of the payload (invitation status, completion stats, segmentation labels). It explicitly draws the boundary against listPendingInvitations, so an agent can separate this from the nearest sibling without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit exclusion rule — invitees without an account are not returned, use listPendingInvitations instead — and points to listCustomFields when full field definitions are needed. It does not address other plausible alternatives in the sibling set (e.g. getTesterFunnelDetail, listApplicants), so routing guidance is clear but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getProductOverviewA
Read-only
Inspect

Get the setup and configuration of this beta project: its name, description, objectives, lifecycle state, schedule, and the requirements testers must meet (agreement, shipping info, custom fields). Use getBetaTasks for what the testers are asked to do. description and objectives are manager-facing setup context that steers Stomio's AI task generation; testers do not see them. state is one of: 'draft' (being set up, invisible to testers), 'published' (scheduled, waiting for its start date — testers cannot participate yet), 'running' (live), 'paused' (manager-paused, testers blocked until resumed), 'finished' (end date passed), 'archived'. paused stays true even while the state still reads 'published'. startAt/endAt are absent while the beta is an unscheduled draft. optInLink (the public join link) is present only while the beta is published, running, or paused AND opt-in is set up; its absence does not mean opt-in is disabled. code is the ticket-code prefix (the PROD in PROD-123). customInvitationEmail (absent when never configured) is the custom tester invitation email template (setInvitationEmail); when its activated is true, every invitation email this beta sends uses it. agreementUrl (present only when hasAgreement) is a time-limited download link to the NDA/agreement document for the human user; it differs on every call, hasAgreement is the stable fact. url, testersUrl, announcementsUrl, applicantsUrl, and resultsUrl are permanent links to this beta project's dashboard pages (sign-in gated, usable by any manager of this workspace); unlike the signed links above they never expire.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With readOnlyHint already declaring the safety profile, the description goes well past the annotations: it explains absence semantics (startAt/endAt missing while unscheduled, optInLink present only in published/running/paused and not implying opt-in is disabled, customInvitationEmail absent when never configured), link lifetime behavior (agreementUrl is time-limited and differs on every call while hasAgreement is the stable fact; url/testersUrl/etc. never expire and are sign-in gated), and visibility rules (description/objectives are manager-facing and hidden from testers). These are exactly the non-obvious traits an agent cannot infer from structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and siblings are front-loaded in the first two sentences, and nearly every subsequent clause disambiguates a specific returned field, so the length is largely earned. The downside is that it is one undifferentiated paragraph of dense clauses with no grouping or line breaks, which makes the field-semantics section harder to scan than it needs to be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the full burden of explaining return values, and it does so comprehensively: the six-value state enum with meaning per value, the paused/state interaction, the opt-in link conditions, ticket-code prefix, invitation email template, and signed vs. permanent URLs. Nothing an agent needs in order to interpret the response correctly appears to be missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single parameter productId is documented there, including the pointer to listProducts. The description body never mentions productId or adds any format or constraint detail beyond the schema, so this rests at the baseline for a fully covered schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource ('Get the setup and configuration of this beta project') and then enumerates exactly which fields come back (name, description, objectives, lifecycle state, schedule, requirements). It explicitly carves out the adjacent territory by naming getBetaTasks for task content, so an agent can distinguish it from the most easily confused sibling without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit routing rule ('Use getBetaTasks for what the testers are asked to do') and the input schema directs the agent to listProducts for valid IDs. It does not address the other plausible neighbors (getBetaTesters, getApplicantsOverview, getScreeningSetup) or state any precondition for calling this tool, so the guidance is clear but not exhaustive across a very large sibling set.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getScreeningSetupA
Read-only
Inspect

Get this beta project's applicant screening configuration: the source (active flag, published flag, the public application link, custom thank-you copy), the screening form's questions (their questionIds are what rule conditions reference), and the auto-review rules. published means the screening form is live and accepting applications; the form only accepts applications when BOTH active and published are true. Author the form with addScreeningQuestion (and edit/delete/reorder siblings) while it is not yet live, then publishScreeningForm; a LIVE form is edited from the Stomio dashboard only. Returns configured: false when screening was never set up (use configureScreening to create it). Each question carries ready: false means it fails authoring validation (a choice question with no options, say) and blocks publishScreeningForm until fixed. ready is input for your reasoning, not for users: when everything passes, do not mention it; when something fails, say what is missing in plain words instead of quoting the flag.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Consistent with readOnlyHint=true. The description adds substantial context the annotation cannot carry: the semantics of active/published (both must be true to accept applications), that questionIds are what rule conditions reference, the configured: false return case, and that ready: false blocks publishing and indicates a validation failure. It stops short of describing pagination or the exact shape of the returned rules, but the behavioral picture is well covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and return contents are front-loaded before the workflow guidance. Dense but largely purposeful; the closing instruction about how to phrase the ready flag for users is agent-behavior guidance that adds length without adding tool semantics.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, yet the description defines the return shape in words (configured flag, source fields, questions with ready state, rules) and explains the validation gating. For a single-parameter read tool this leaves nothing an agent needs missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter (productId) at 100% schema coverage, and the schema already points to listProducts for valid IDs. The description adds no additional meaning about productId, so the baseline 3 applies even though the single-parameter surface makes the gap minor.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get this beta project's applicant screening configuration') and then enumerates exactly what is returned: source flags and public link, thank-you copy, form questions with questionIds, and auto-review rules. That level of detail separates it cleanly from siblings like configureScreening, addScreeningQuestion, and publishScreeningForm.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit routing: use configureScreening when screening was never set up (signalled by configured: false), author questions with addScreeningQuestion while the form is not live, then publishScreeningForm, and note that a LIVE form can only be edited from the Stomio dashboard. It names alternatives and the exact conditions that select them.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getStepMediaA
Read-only
Inspect

Fetch the media a manager attached to a step for testers to follow, using the id of the step's resource as getBetaTasks returns it (published or unpublished). Images are returned as viewable image content so you can inspect a screenshot directly; videos, files and oversized or unsupported images come back as metadata plus a time-limited download URL: share it with the user, or fetch it yourself when the user asks you to read or analyze the file. A metadata-plus-URL result is the intended answer for those cases, not an error — do not retry it. Download URLs differ on every call; the resourceId is the stable identity.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
resourceIdYesThe UUID of the step media (resource.id on a step from getBetaTasks).

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint, so the description carries the real burden and does: it distinguishes inline image content from metadata-plus-URL fallbacks, notes URL time-limiting and per-call variation, and preempts the main failure mode by declaring that a metadata-plus-URL result is the intended answer, not an error.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the action and the id source, then the two result shapes, then the anti-retry warning. Dense but nearly every clause earns its place; the final sentence about stable identity could have been folded into the preceding clause.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description fully compensates by describing both possible return shapes and how to use them. For a two-parameter read tool, an agent has everything needed to call it and interpret the response.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters are already documented. The description still adds meaning beyond the schema: that resourceId is the step's resource.id from getBetaTasks and is the stable identity while download URLs are not, which disambiguates which value to persist.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Fetch the media a manager attached to a step') and pins the scope with 'using the id of the step's resource as getBetaTasks returns it (published or unpublished)'. An agent can tell this apart from the task-media siblings without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete when-to guidance on the returned URL ('share it with the user, or fetch it yourself when the user asks you to read or analyze the file') and an explicit prohibition ('do not retry it'), plus where the required id comes from (getBetaTasks). Little is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getStepPerformanceSummaryA
Read-only
Inspect

Summarize response counts and percentages for each step, highlighting bug-heavy prompts. answersPercentage is computed against the testers who could see the step, a denominator that can differ per step. bugTickets counts the OPEN bug tickets reported on the step (beta projects only; absent otherwise). taskId groups the steps by their task.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, so safety is covered; the description goes further by explaining that answersPercentage uses a per-step denominator of testers who could see the step, which can vary between steps. It also discloses that bugTickets counts only OPEN tickets and is beta-projects-only and otherwise absent, which is exactly the kind of conditional behavior an agent cannot derive from annotations. It stops short of describing ordering, pagination, or response shape.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four tight sentences with the core purpose front-loaded, followed by the metric caveats. Dense but every sentence carries information an agent needs; minor cost is that the output-field caveats precede any statement of when to call the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly takes on the burden of explaining key return fields — answersPercentage denominator, bugTickets scope and absence, and taskId grouping. For a single-parameter read-only summary tool this is close to complete, though the remaining response fields (raw response counts per answer option) are only gestured at.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single productId parameter is fully documented in the schema, including where to get valid IDs, so the baseline is 3. The description adds no further semantics for productId; its mention of taskId refers to a returned grouping field rather than an input, which is useful output context but not parameter guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Summarize response counts and percentages for each step,' and adds the distinguishing focus 'highlighting bug-heavy prompts.' It does not explicitly contrast itself with nearby siblings such as getStepResults or getTaskPerformanceSummary, which report on different granularity, so an agent must infer the boundary from the name and text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use or when-not-to-use guidance and no mention of alternatives like getStepResults or getTaskPerformanceSummary. The only implicit cue is that the output is a per-step summary, leaving the agent to guess when this is preferable to a detailed results call.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getStepResultsA
Read-only
Inspect

Inspect an individual step to review grouped answers and other per-step analytics. answers is ABSENT only for triggered-email steps, which collect no answers but do return triggeredEmails, holding the recipients the email was sent to and the ones who opened it; absence means the type has no answers, not that nobody answered. For instructions steps, answers lists the testers who completed the step: acknowledgments, not collected input (this drill-down is the only place they are listed; getTaskResults reports only their count). Choice and rate steps return per-option counts seeded with every offered option (0 = offered, unchosen; a key marked other:true is a free-typed answer). Skipped testers are not listed here; use getStepPerformanceSummary for skip counts. An invalid tagId/testerUserId filter is rejected with an error, never silently ignored.

NameRequiredDescriptionDefault
tagIdNoOptional tag filter matching the UI filters.
stepIdYesUUID of the step to inspect.
taskIdYesUUID of the task that owns the step.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
testerUserIdNoOptional tester user ID filter.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnlyHint annotation, the description explains missing answers for triggered-email steps, the special meaning of answers for instructions steps, seeded choice/rate counts with zero values, the other:true free-typed marker, and validation behavior for invalid filters. This gives an agent substantial behavioral context about what to expect and what errors mean.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded in the first sentence, and the remaining sentences each cover a distinct return-shape or filtering behavior. It is dense and could use sentence breaks, but no sentence is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of explaining return values, and it does so for answers, triggeredEmails, choice/rate option counts, and instructions acknowledgments. It does not fully document every possible per-step analytics field, but it is complete enough for an agent to call the tool and interpret the main response shapes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all five parameters. The description adds filter behavior that the schema does not: an invalid tagId or testerUserId filter is rejected with an error rather than silently ignored, which changes how an agent should treat bad filter values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Inspect an individual step to review grouped answers and other per-step analytics.' It clearly distinguishes this drill-down from getTaskResults and getStepPerformanceSummary by naming both siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly routes skip-count questions to getStepPerformanceSummary and explains that getTaskResults reports only counts of instructions acknowledgments. The usage context is clear, though it does not enumerate every situation where a caller should choose this tool over getTaskResults.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getTaskMediaA
Read-only
Inspect

Fetch the media a manager attached to a task for testers to follow, using the id of the task's resource as getBetaTasks returns it (published or unpublished). Images are returned as viewable image content so you can inspect a screenshot directly; videos, files and oversized or unsupported images come back as metadata plus a time-limited download URL: share it with the user, or fetch it yourself when the user asks you to read or analyze the file. A metadata-plus-URL result is the intended answer for those cases, not an error — do not retry it. Download URLs differ on every call; the resourceId is the stable identity.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
resourceIdYesThe UUID of the task media (resource.id on a task from getBetaTasks).

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnlyHint annotation: it explains what is returned inline (images as viewable content), what degrades to metadata plus a time-limited URL, that such a result is success rather than an error, and that URLs are unstable across calls while resourceId is the stable identity. This is exactly the behavioral context an agent needs to avoid misinterpreting results.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the purpose and then layers the return-behavior rules; every sentence carries information, including the retry warning that prevents a common failure loop. It is dense and somewhat long, but there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and a read-only tool, the description fully covers what comes back in each case and how to interpret it, plus the stability caveat on the URL. Nothing an agent needs in order to call it and handle the result is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is already high, and the description adds provenance meaning for resourceId ('the id of the task's resource as getBetaTasks returns it') that helps the agent map it to the correct field. It does not add format or constraint detail beyond that, so it sits just above baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Fetch the media a manager attached to a task') and scopes it to testers following instructions. It names the source tool for the identifier (getBetaTasks) and is clearly distinguishable from siblings like getStepMedia, setTaskMedia, and removeTaskMedia.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly directs the agent to get the resource id from getBetaTasks (published or unpublished) and describes the two usage branches: share the download URL with the user, or fetch it yourself when asked to read/analyze. It also gives a negative instruction — do not retry when a metadata-plus-URL result comes back.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getTaskPerformanceSummaryA
Read-only
Inspect

Get completion, drop-off, and ticket counts for every task in the beta. testersCompleted is the number of testers who completed the task; testersDropped is the number who answered inside the task, then submitted no answer anywhere in the beta for 14 days, AND still have unanswered steps left in the beta, so someone who stopped more recently than that is not counted yet and someone who answered everything before going quiet is never counted. completion percentages are computed per task against the testers who can SEE that task (label-gated visibility), so tasks are not all on the same denominator. stepsCount counts the answerable steps of the task only: a triggered-email step sends an email instead of collecting an answer, so it is left out, and this is the same number the results view of the dashboard shows as steps. Tasks with no steps, or whose only steps are triggered-email steps, are omitted, so this list can be shorter than getBetaTasks. That difference is not missing data. An instructions-only task is listed, since its steps collect completion acknowledgments.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
onlyCompletionNoSet to true to return only id, name, and completion for each task — every other field is omitted (faster on large projects).

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations supply only readOnlyHint, and the description does the heavy lifting: it defines the 14-day drop-off window precisely, discloses that denominators vary per task due to label-gated visibility, and explains that stepsCount excludes triggered-email steps. It also explains omission behavior (no-step and email-only tasks excluded) and that instructions-only tasks are still listed, which is exactly the kind of subtle behavior an agent could not infer.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but almost every clause carries real semantic weight; the purpose is front-loaded. It is a single long paragraph where the denominator, drop-off, stepsCount, and omission rules would be easier to scan as separate sentences or a short list.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the return-value burden, and it defines testersCompleted, testersDropped, stepsCount, and completion semantics thoroughly. For a read-only summary tool with two simple inputs, nothing essential an agent needs to interpret or call it is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both productId and onlyCompletion are already documented in the schema with their effects. The description adds no additional parameter-level meaning — its detail is about returned fields, not inputs. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get completion, drop-off, and ticket counts for every task in the beta') and implicitly scopes it against getBetaTasks and getStepPerformanceSummary by noting the list can be shorter and that counts are per task, not per step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear context: use it for a per-task view of completion/drop-off/tickets, and explains why this list can be shorter than getBetaTasks ('That difference is not missing data'), which heads off a common misuse. It also points at the faster path via onlyCompletion. It does not explicitly state exclusions or name every alternative (e.g., getTaskResults), so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getTaskResultsA
Read-only
Inspect

Get detailed results for a specific task, including step-level answers and ticket counts. Reading the results: answers only contains real answers — skipped testers are counted in skippedCount but never listed; choice and rate steps return per-option counts seeded with EVERY offered option (a 0 means offered but unchosen; an extra key marked other:true is a free-typed answer); free-text and file steps list per-tester entries. completion: 0 also appears when no tester could see the task. File answer URLs are time-limited and differ on every call — the stable identity is the answer resource id (usable with getAnswerAttachment over MCP). testersCompleted and testersDropped (at task level and on each step) mean the same as in getTaskPerformanceSummary: dropped counts testers who answered here, then submitted no answer anywhere in the beta for 14 days, and still have unanswered steps left in the beta, so recent drop-offs and testers who answered everything are not counted. Instructions steps never carry answers entries in THIS tool, but their answersCount still counts completion acknowledgments; call getStepResults on the step to see who acknowledged and when. An invalid tagId/testerUserId filter is rejected with an error, never silently ignored.

NameRequiredDescriptionDefault
tagIdNoOptional: Filter to testers assigned to a given tag.
taskIdYesThe UUID of the task to inspect.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
testerUserIdNoOptional: Filter results down to a single tester user id.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds substantial behavior beyond the readOnlyHint annotation: skipped testers are counted but not listed, choice/rate options are seeded with every offered option, other:true marks free-typed answers, completion:0 can mean no tester could see the task, file URLs are time-limited and unstable, and invalid tagId/testerUserId filters are rejected rather than ignored.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but front-loads the purpose and then explains result-reading semantics in a single long paragraph. Most clauses earn their place because there is no output schema, though the semicolon-heavy structure is less scannable than it could be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of an output schema and the complexity of task result semantics, the description thoroughly covers skipped counts, per-step answer shapes, completion edge cases, time-limited file URLs, tester drop definitions, instruction-step exceptions, and filter validation. It is complete enough for an agent to call and interpret the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all four parameters, including tagId and testerUserId as optional filters. The description adds only a narrow validation note that invalid tagId/testerUserId filters are rejected with an error; it does not otherwise deepen parameter meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Get detailed results for a specific task, including step-level answers and ticket counts.' It also references sibling tools such as getTaskPerformanceSummary, getStepResults, and getAnswerAttachment, helping an agent distinguish this task-level results tool from related reporting tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear contextual routing: use getStepResults on an instruction step to see acknowledgments, and use getAnswerAttachment with a stable answer resource id for file attachments. It does not explicitly state when to choose this over getTaskPerformanceSummary versus when to use both, but the use context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getTesterFunnelDashboardA
Read-only
Inspect

Get tester funnel counts, engagement rate, and top tester leaderboards in a single response. Reading the summary: keys are OMITTED when not applicable — a beta with no NDA reports accepted and no ndaAccepted at all; a beta with an NDA reports ndaAccepted, and accepted disappears only once ndaAccepted rises above 0 (the NDA milestone supersedes it), so ndaAccepted: 0 appears alongside accepted. engagementRate is null when the beta is not active, has no testers, or has no visible tasks. topPerformers is the (at most) 5 testers who completed EVERY task visible to them, fastest first; topTicketCreators is the top 5 by tickets opened; both keys are omitted when nobody qualifies.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnlyHint annotation by explaining which keys are omitted under which conditions, when engagementRate is null, and how the top-5 leaderboards are ranked and truncated. It stops short of defining what engagementRate actually measures or what the count fields are.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose sentence is front-loaded and the return-value notes follow in a logical order (summary keys, then engagementRate, then leaderboards). The middle sentence about the NDA milestone is dense and takes effort to parse, but every clause conveys real conditional behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description properly carries the return-shape burden and documents key omission and null rules that would otherwise be surprising. It is incomplete on engagementRate's definition and on what funnel count fields exist, leaving the agent to discover the full key set.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

A single productId parameter with 100% schema coverage that already points to listProducts for valid IDs. The description adds no further meaning about the parameter, so the baseline applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource plus the exact contents returned: funnel counts, engagement rate, and top-tester leaderboards in one response. It is distinguishable from getTesterFunnelDetail only by implication ('in a single response', leaderboards), and never names the sibling.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Use is only implied — an aggregated summary for a beta project — with no explicit when-to-use or when-not-to-use guidance. The sibling getTesterFunnelDetail is the obvious alternative for per-tester breakdowns, but the description never routes the agent to it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getTesterFunnelDetailB
Read-only
Inspect

List every tester milestone (invites, NDA, completions) with timestamps.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds that timestamps accompany each milestone, which is real behavioral value about the payload shape, but it says nothing about volume, pagination, or whether results are per-tester rows or aggregates for the whole product.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with the verb first and the milestone examples plus timestamp note packed in. No filler or redundant restatement of the name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read-only list tool with full schema coverage and no output schema, the description is minimally adequate. It leaves unresolved the key ambiguity of scope (all testers in the product vs. a single tester) and its relationship to the funnel dashboard sibling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the single parameter, and the schema itself tells the agent to source productId from listProducts. The description adds no scoping detail beyond what the schema provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('List') and resource ('tester milestone') and enumerates the milestone types (invites, NDA, completions) plus the fact that timestamps are included. It does not distinguish itself from the very close sibling getTesterFunnelDashboard, so an agent cannot tell the two apart from the text alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no statement of when this is preferable to getTesterFunnelDashboard or getApplicantsOverview, and no prerequisites beyond the schema's note about productId. The only implicit guidance is 'list milestones', which the name already conveys.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getTicketAttachmentA
Read-only
Inspect

Fetch one attachment from a ticket conversation, using a resourceId taken from the attachment metadata on a message returned by getTicketDetails or getTicketMessagesBatch. Images are returned as viewable image content so you can inspect a screenshot directly; videos, files and oversized or unsupported images come back as metadata plus a time-limited download URL: share it with the user, or fetch it yourself when the user asks you to read or analyze the file. A metadata-plus-URL result is the intended answer for those cases, not an error — do not retry it. Download URLs differ on every call; the resourceId is the stable identity.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
resourceIdYesThe UUID of the ticket attachment (attachment.resourceId from a ticket message).

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond readOnlyHint=true: it discloses that images come back as viewable image content, that videos/files/oversized/unsupported images return metadata plus a time-limited download URL, and that this is the intended answer rather than an error. The explicit 'do not retry it' and the note that URLs change every call while resourceId is stable are exactly the behavioral facts an agent needs and cannot get from annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads what the tool does, then the input provenance, then the return-shape contract. Every sentence carries non-redundant operational information (result shape, non-error status, retry prohibition, URL volatility); nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the full burden of explaining return values and does so by enumerating the two result shapes and clarifying which is not an error. For a two-parameter read tool, nothing an agent needs in order to call it or interpret its result is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real semantics the schema lacks: the provenance of resourceId (attachment.resourceId on a message) and the fact that it is the stable identity while download URLs rotate. productId is left to the schema, which already points at listProducts.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Fetch one attachment from a ticket conversation') and scopes it to ticket attachments as opposed to answer attachments, so it is distinguishable from siblings like getAnswerAttachment. It also tells the agent exactly where the resourceId comes from, so the tool's role in the workflow is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit preconditions ('using a resourceId taken from ... getTicketDetails or getTicketMessagesBatch') and clear post-fetch guidance (share the URL with the user, or fetch it yourself if asked to analyze the file). It does not name a sibling alternative for non-ticket attachments, so it stops short of full when/when-not routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getTicketDetailsA
Read-only
Inspect

Get ticket metadata plus the newest conversation messages in a single call: messages is the most recent window (compare messagesReturned with totalMessages to see truncation; getTicketMessagesBatch returns full histories). System entries (state, assignee, title changes) are never included and are not counted in totalMessages. ticket state is open or closed. In ticketDetails.tags, group is an internal tag-category id with no name available on this surface. Each message carries senderRole — the role its author was acting in when posting: a workspace-team role ("owner", "manager", "viewer"), a participant role ("tester", "sdkTester"), or "guest" (external, invited by email). The same person can hold a team role and be a tester in the same beta, so attribute statements by senderRole, not by who the person is. Messages include image, video and file entries, each with an attachment object ({ resourceId, type, url }); url is a time-limited download link for the human user and differs on every call — resourceId is the stable identity. Over MCP, an image attachment can also be viewed by calling getTicketAttachment with its resourceId. ticketDetails.url is the ticket page in the Stomio dashboard: permanent, and the right link to hand the user.

NameRequiredDescriptionDefault
ticketIdYesTicket UUID to inspect (use id from listTickets).
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint, so the description carries real behavioral weight and does so: truncation is detectable via messagesReturned vs totalMessages, system entries are excluded and not counted in totalMessages, and tag groups are unnamed internal ids. The attachment semantics are unusually strong — url is a time-limited per-call link while resourceId is the stable identity, and ticketDetails.url is the permanent human-facing link.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The truncation guidance is correctly front-loaded and nearly every clause carries substantive information, but the whole definition is a single unbroken block that runs well past what a two-parameter tool needs, and the senderRole explanation is a lengthy detour buried mid-paragraph.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must explain the return shape, and it does so thoroughly: message envelope, senderRole taxonomy, attachment object shape, and the two distinct URL types. Given the tool's complexity as a combined metadata-plus-messages read, nothing an agent needs to interpret the response is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for both required parameters, so ticketId and productId are already fully documented in the schema, including where to source the ids. The description adds no syntax, format, or edge-case guidance for either parameter, which is the expected baseline 3 when the schema does the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb and resource ('Get ticket metadata plus the newest conversation messages') and immediately scopes it as a 'most recent window' rather than full history. The sibling it could be confused with, getTicketMessagesBatch, is named and distinguished in the same sentence, so an agent can route without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It names two alternatives with their selecting conditions: getTicketMessagesBatch when full histories are needed, and getTicketAttachment when an image attachment must be viewed over MCP. It stops short of a full when-not-to-use statement (e.g., no guidance on when listTickets is the better starting point), which keeps it just below the top score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getTicketMessagesBatchA
Read-only
Inspect

Fetch conversations for multiple tickets (up to 25 at a time) to reduce the number of tool calls needed. Returns each ticket's FULL conversational history (unlike getTicketDetails, which windows to the newest messages); system entries such as state and assignee changes are never included, so an empty messages array means no conversation, not an empty ticket history. Ids beyond the first 25 are reported in ignoredTicketIds, unprocessed. Each message carries senderRole — the role its author was acting in when posting: a workspace-team role ("owner", "manager", "viewer"), a participant role ("tester", "sdkTester"), or "guest" (external, invited by email). Messages include image, video and file entries, each with an attachment object ({ resourceId, type, url }); the url is a time-limited download link for the human user and differs on every call — resourceId is the stable identity. Over MCP, an image attachment can also be viewed by calling getTicketAttachment with its resourceId.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
ticketIdsYesArray of ticket UUIDs. Only the first 25 unique ids are processed.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond readOnlyHint=true, it discloses the 25-ID processing limit, ignoredTicketIds behavior, exclusion of system entries, and the correct interpretation of an empty messages array. It also documents senderRole values, attachment structure, time-limited and changing URLs, and resourceId as the stable attachment identity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose, batch limit, and key sibling distinction are front-loaded, and each sentence supplies distinct operational or return-value information. It is nevertheless a long, dense single paragraph with several semicolon-heavy sentences, so bullets or shorter sentences would improve scanability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates unusually well by documenting messages, ignoredTicketIds, senderRole, attachment fields, URL volatility, and empty-result semantics. Together with the complete input schema and read-only annotation, it provides the context needed to select, invoke, and interpret the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, including the productId source and the ticketIds UUID and uniqueness constraints. The description adds useful operational context about batching and explains that excess IDs appear unprocessed in ignoredTicketIds, though it adds little further information about productId.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource—fetching conversations for multiple tickets—and clarifies that it returns full conversational history. It explicitly distinguishes the tool from getTicketDetails, which only returns a window of the newest messages.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explains when to prefer the batch tool: retrieving conversations for multiple tickets, up to 25, to reduce tool calls. It also guides selection between getTicketDetails for windowed history and getTicketAttachment for viewing an image attachment by resourceId.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

inviteApplicantsA
Destructive
Inspect

Invites shortlisted applicants to join the beta project as testers, sending each of them the beta invitation email. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing is sent and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes. Only applicants whose status is "shortlisted" can be invited — review them with reviewApplicants first. The beta project must be published, running, or paused; draft, finished, and archived beta projects refuse invitations. Each id is reported individually, so a partial failure still invites the rest. An applicant whose address cannot receive mail is still invited but carries emailDelivery "undeliverable" and is counted in undeliverable: the invitation exists, no email went out, and pinging will not help, so cancel it and invite the corrected address.

NameRequiredDescriptionDefault
matchIdsYesApplicant match ids from listApplicants (1 to 100). Only shortlisted applicants can be invited.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond readOnlyHint=false and destructiveHint=true, it discloses that the first call sends nothing, the executing call sends email, tokens are single-use and short-lived, and failures can be partial per applicant. The undeliverable case is especially transparent: the invitation persists without email, pinging will not help, and cancellation followed by reinvitation is needed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose and confirmation workflow are front-loaded, followed logically by prerequisites, failure behavior, and edge-case handling. The single dense paragraph and lengthy final sentence reduce scanability, and a few details duplicate the schema, but nearly all of the content is operationally useful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a consequential mutation with no output schema, it supplies the dry-run/execute protocol, approval requirement, eligibility and project-state preconditions, token lifecycle, partial-failure semantics, and response-field interpretation for undeliverable mail. Explicitly naming cancelInvitations for the final remediation would be a small improvement, but the essential invocation context is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema coverage is 100%, the description adds operational constraints not captured there: productId must identify a published, running, or paused project, the second call must reuse the same arguments, and match ids can succeed or fail independently. Some shortlist and token details merely repeat the schema, so the added parameter value is not quite maximal.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence specifies the action, population, outcome, and side effect: shortlisted applicants are invited to become beta testers and receive invitation emails. This applicant-specific scope, reinforced by the reviewApplicants prerequisite, distinguishes it from inviteTesters and the list/review siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly requires a preview call, user review and approval, and a second call with identical arguments plus the token. It also states when invitations are refused based on applicant and project status, directs the agent to reviewApplicants first, and explains how to handle undeliverable addresses.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

inviteTestersA
Destructive
Inspect

Invites people to join this beta project as testers. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing is sent and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use, expire after a few minutes, and are bound to the people resolved and the beta project's lifecycle state at preview time: if either changes before the confirm, it fails asking for a fresh preview. Name people either by email (invitees, addresses supplied by the user) or, for people who are already testers somewhere in this workspace, by their userId (userIds, from listOrganizationTesters — since tester emails are never exposed here, this is how you re-invite testers from another beta project; its productIds field shows which beta projects each tester already belongs to). The two combine freely; at least one is required. Duplicates are resolved server-side: someone who already belongs to this beta comes back as "alreadyTester" and an email that already holds a pending invitation comes back as "invited" WITHOUT being emailed again (use pingTesters to re-send those). An invitee whose address cannot receive mail is still "invited" but carries emailDelivery "undeliverable" and is counted in undeliverable: the invitation exists, no email went out, and pinging will not help, so cancel it and invite the corrected address. While the beta project is still a DRAFT the invitations are created but no emails go out; they are only sent once the beta is published or running. Finished and archived beta projects refuse invitations. When the beta has an activated custom invitation email template (getProductOverview customInvitationEmail, configured with setInvitationEmail), the invitation emails use it. Results for userId-sourced people are reported by name and userId, never by email; the pending invitation they leave behind is likewise reported by id and name in listPendingInvitations, and pingTesters and cancelInvitations take that id.

NameRequiredDescriptionDefault
userIdsNouserId values of existing workspace testers to invite, as returned by listOrganizationTesters. Use this for people whose email you do not know (tester emails are not exposed over this connection).
inviteesNoThe people to invite by email address. Use real email addresses supplied by the user; never fabricate them.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and destructiveHint=true; the description adds the full behavioral picture: two-step preview/confirm gating, single-use short-lived tokens bound to resolved people and beta lifecycle state, no emails in DRAFT, refusal on finished/archived betas, server-side dedupe returning 'alreadyTester'/'invited' without re-emailing, undeliverable handling, and custom template usage. This is far more than the annotations convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The two-step procedure is front-loaded and nearly every clause carries load-bearing detail (token binding, dedupe outcomes, DRAFT suppression). It is dense and long, with several long run-on sentences and parentheticals that could be split for readability, but little is genuinely redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 4-parameter mutation with no output schema, the description covers preconditions, the confirmation handshake, per-person result states (alreadyTester, invited, emailDelivery 'undeliverable'), and downstream follow-ups (pingTesters, cancelInvitations, listPendingInvitations). Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, but the description adds meaning the schema does not: userIds and invitees 'combine freely' with at least one required (a union constraint not expressible in the schema), why userIds exist (tester emails are never exposed), and how confirmationToken interacts with identical arguments on the second call.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Invites people to join this beta project as testers') and distinguishes itself from nearby siblings by naming inviteApplicants-adjacent tools (pingTesters, cancelInvitations, listOrganizationTesters). An agent can tell exactly which operation this performs without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly prescribes the call sequence (call without confirmationToken first, show preview, get user approval, then re-call with the token) and routes agents to alternatives by condition: use userIds for existing workspace testers from listOrganizationTesters, pingTesters to re-send pending invitations, cancelInvitations for undeliverable ones. It also states when the tool refuses (finished/archived) and when emails are suppressed (DRAFT state).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

launchBetaA
Destructive
Inspect

Start this PUBLISHED (scheduled) beta project immediately instead of waiting for its start date. Testers with project-status emails enabled are notified that the beta launched. If the beta is already running this is a no-op. A draft cannot be launched directly: publishBeta it first. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare destructive=true and readOnly=false, and the description goes well beyond them: it discloses that testers with project-status emails enabled get notified, that the first call is a pure preview with no state change, and that tokens are single-use and expire in minutes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action and its preconditions, then the two-step flow. Every sentence carries information, though the token mechanics are restated in both the description and the confirmationToken schema entry, adding mild redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive mutation with no output schema, the description covers the state precondition, side effects (tester notifications), no-op behavior, and the full confirmation workflow — everything an agent needs to call it safely.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description adds real meaning by explaining that confirmationToken is obtained from the first call and must be sent with the SAME arguments, which the schema alone only partially conveys.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (start a PUBLISHED/scheduled beta project immediately) with scope clarified as overriding the start date, which cleanly separates it from siblings like publishBeta, pauseBeta, and resumeBeta.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit preconditions and alternatives: only published projects can be launched, a draft requires publishBeta first, and calling it on an already-running beta is a no-op. The two-step confirmation protocol and the instruction to only re-call after explicit user approval are spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listAnnouncementsA
Read-only
Inspect

List the announcements already sent in this beta project, with their text, sender, audience tags, and reach. recipientsCount includes managers who received a courtesy copy; view tracking exists only for testers, so compute open rates as viewedCount out of testerRecipientsCount, never out of recipientsCount. Every announcement is always listed with these counts. Recipient lists are opt-in because they grow with testers times announcements: pass recipientsSince (for example 7 days ago) and each announcement sent at or after it also carries testerRecipients, the testers it reached who are still in this beta project (userId, name, username, and viewed: whether they opened it). Managers who received a courtesy copy are not in it, and a tester removed from the beta project drops out of it and of testerRecipientsCount and viewedCount. audienceTags lists the tag names the announcement targeted; an EMPTY list means no tag targeting, which covers both a send to all testers and a send to a hand-picked list of testers. Which of the two it was is not stored anywhere and cannot be worked out afterwards: the beta's membership moves and a hand-picked list can contain every tester, so do not present a guess about how recipients were chosen as fact. date is when the announcement was created/sent. attachmentUrl is a time-limited download link for the human user and differs on every call; hasAttachment is the stable fact.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
recipientsSinceNoOptional: ISO 8601 date-time. Announcements sent at or after it also carry testerRecipients; omit it for counts only.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well past readOnlyHint=true by disclosing non-obvious data quirks: recipientsCount includes courtesy-copy managers, view tracking exists only for testers so open rates must use testerRecipientsCount, recipients drop out when removed from the beta, and attachmentUrl changes every call while hasAttachment is stable. It also flags that the hand-picked-vs-all-tester distinction is unrecoverable and must not be guessed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose and generally dense with earning content, but it runs long and repeats the courtesy-copy exclusion twice ('recipientsCount includes managers...' and 'Managers who received a courtesy copy are not in it'). A tighter ordering around recipient-list caveats would read better.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the full burden of return-value explanation and does so thoroughly: counts, testerRecipients shape, audienceTags semantics, date, and attachmentUrl versus hasAttachment. Given only two simple parameters and a read-only annotation, nothing an agent needs to call or interpret this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description still adds real meaning: recipientsSince not only enables testerRecipients but qualifies which announcements carry the list ('sent at or after it'), what fields appear in each entry (userId, name, username, viewed), and who is excluded from it. That is substantive semantics beyond the schema's one-line note.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List the announcements already sent in this beta project') plus the fields returned (text, sender, audience tags, reach). The 'already sent' framing cleanly separates a read of past announcements from the sibling sendAnnouncement write tool, so an agent can pick between them without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit conditional guidance for the optional parameter ('pass recipientsSince ... omit it for counts only') and warns against presenting an inferred sending method as fact. It never names an alternative sibling tool or states a when-not-to-call condition for the tool as a whole, which keeps it below a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listApplicantsA
Read-only
Inspect

List the people who applied to this beta project through its screening form, with the answers they gave. Each applicant moves through a pipeline: "new" and "needsReview" are undecided, "shortlisted" and "rejected" are manager decisions that stay changeable, and "invited" then "joined" are final. The matchId of each entry is the id that reviewApplicants and inviteApplicants take. Use getApplicantsOverview when only the per-status counts are needed. suggestedStatus is what the workspace's auto-suggestion rules propose (advisory only, nothing is applied until a manager reviews); approvedStatus records the reviewed decision. screeningAnswers is empty when no application record exists for the match, not when someone answered nothing. The result is paginated: when total is larger than the number of applicants returned, call again with a higher offset. counts is the UNFILTERED per-status census of the whole pipeline, while total counts only what the current status filter matches — they intentionally disagree when a filter is set.

NameRequiredDescriptionDefault
limitNoHow many applicants to return (default 50, maximum 200).
offsetNoHow many applicants to skip, for paging through the list (default 0).
statusNoOnly return applicants currently in this pipeline status. Omit to return all of them.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnlyHint annotation, the description explains pipeline state semantics, the advisory nature of suggestedStatus versus the reviewed approvedStatus, when screeningAnswers is empty, and how pagination and counts/total can disagree. These are meaningful behavioral traits that an agent would not get from the annotations or schema alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose and every sentence contributes useful semantics about statuses, output fields, pagination, or filtering. It is dense and slightly long for a list tool, but the length is largely justified by the absence of an output schema and the complexity of the applicant pipeline.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a paginated list tool with no output schema, the description covers the key return semantics: matchId, suggestedStatus, approvedStatus, screeningAnswers emptiness, pagination behavior, and the intentional difference between counts and total. It also names the relevant overview alternative, leaving little an agent would need to infer before calling it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is already 100%, so the baseline is 3, but the description adds real meaning about the status enum by explaining which states are undecided, changeable manager decisions, or final. It also clarifies how offset is used when total exceeds the returned applicant count, though it does not add much beyond the schema for productId and limit.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb and resource: list people who applied to this beta project through its screening form, including their answers. It explicitly distinguishes this tool from getApplicantsOverview and ties the returned matchId to reviewApplicants and inviteApplicants, so an agent can identify its role among siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly says to use getApplicantsOverview when only per-status counts are needed, which gives a clear alternative and implies this tool is for per-applicant detail. It also explains the pagination follow-up behavior and how the status filter interacts with total versus counts, giving practical invocation guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listBetaLabelsA
Read-only
Inspect

List the labels of this beta project with their categoryId. Labels belong to a single beta project and appear on its testers (getBetaTesters) and tasks (getBetaTasks) — tickets use the workspace tags from listTicketTags instead. Labels also gate task visibility: a task labeled X is only visible to testers labeled X, which is why completion percentages are per-tester. Create new labels with createBetaLabels.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already covers the safety profile, yet the description adds real domain behavior: labels are scoped to one beta project, they appear on testers and tasks, and they gate task visibility so completion percentages are per-tester. That visibility-gating rule is context an agent cannot get from annotations or schema, though return shape details (beyond categoryId) are absent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core action is front-loaded in the first clause and each following sentence carries routing or domain information. The label-gating explanation is slightly tangential for a list call but earns its place by explaining why per-tester completion exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read tool with no output schema, the description covers scope, the notable returned field (categoryId), the relationship to testers/tasks, and the correct sibling for creation. Only the full return structure and whether categories are always populated are left implicit.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter already documents itself (productId, sourced from listProducts). The phrase "of this beta project" merely restates the schema's meaning and adds no format, constraint, or edge-case guidance beyond it, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

"List the labels of this beta project with their categoryId" gives a specific verb, resource, scope, and even the key returned field. It explicitly separates itself from ticket tags (listTicketTags) and names the creation sibling (createBetaLabels), so an agent can distinguish it from the surrounding list* tools without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description tells the agent when to use a different tool: tickets use workspace tags via listTicketTags instead, and new labels are made with createBetaLabels. It does not mention updateBetaLabel/deleteBetaLabel as follow-ups for mutation, but the primary confusion risk (label vs. tag) is resolved.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listBetaManagersA
Read-only
Inspect

List the managers assigned to this beta project, with their workspace role (managerRole: owner, manager, or viewer — a workspace-level role, not per-beta). name can be null; fall back to username when addressing someone. Use listOrganizationUsers for the whole workspace.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond readOnlyHint=true, it discloses useful result behavior: managerRole has three named values, the role is workspace-level, and name may be null with username as the addressing fallback. It does not describe pagination, ordering, or the overall return shape, but the disclosed quirks are substantive for this simple read operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the operation and scope, followed by two important result caveats and the scoped alternative. Every clause contributes actionable selection, interpretation, or invocation information without repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read-only list tool, the combined definition covers project scope, how to obtain the required ID, read-only safety, key result fields and their interpretation, nullable-name handling, and the workspace-wide alternative. Although there is no output schema, the description supplies the most important result semantics needed to select and use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and productId is already documented as the beta project ID with valid IDs available from listProducts. The description's reference to "this beta project" adds no meaningful parameter detail beyond that schema, so the high-coverage baseline applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource—listing managers assigned to a beta project—and clarifies that the returned role is workspace-level rather than per-beta. It also distinguishes this tool from the sibling listOrganizationUsers by workspace-wide scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly routes agents to listOrganizationUsers when they need the whole workspace rather than one beta's managers. The instruction to fall back to username when name is null also gives concrete guidance for using the results.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listCustomFieldsA
Read-only
Inspect

List the workspace's custom profile fields: the questions testers answer about themselves during onboarding when a beta project has customFieldsRequired. Each field's id is the fieldId on the customFields entries of getBetaTesters rows, so use this to read what an answer means. type is one of MultipleChoices, SingleChoice, Alphanumeric, Link, Date, Numeric, Person; choice fields list their choices, and Person fields list the workspace members offered as answers (options that are a plain email address rather than a member are not listed here). isForTesters false means the field is manager-maintained and testers never see it, in which case its question and description are absent. The definitions are workspace-level configuration managed in the Stomio dashboard, so an empty list means no custom fields exist yet.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only supply readOnlyHint and a title, so the description carries the real burden and does: workspace-level configuration scope, that the list can legitimately be empty, that Person fields omit plain-email options, and that isForTesters=false fields have absent question/description. Those are non-obvious behavioral traits an agent could not infer from the schema or annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause and each subsequent sentence carries distinct information (type enumeration, Person-option caveat, isForTesters semantics, empty-list meaning). The sentences are long and clause-dense, which slightly hurts scanability, but there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must explain the return shape, and it does thoroughly: id, type with its full enum, choices for choice fields, Person member options with an exclusion, the isForTesters flag and its consequence, and the empty-list case. Nothing an agent needs to interpret the response is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is no parameter meaning for the description to add; this hits the baseline for a parameterless tool. The descriptive detail it does provide is about return values rather than inputs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opening clause gives a specific verb and resource ('List the workspace's custom profile fields') and immediately grounds it in the domain ('the questions testers answer about themselves during onboarding when a beta project has customFieldsRequired'). It also distinguishes itself from the related sibling by explaining that field ids match fieldId on getBetaTesters rows, so an agent can tell which tool answers which question.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the trigger for calling it ('use this to read what an answer means' after seeing customFields entries on getBetaTesters rows) and what an empty result implies. It lacks an explicit when-not or a named alternative to prefer in edge cases, which keeps it just short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listFlaggedAnswersA
Read-only
Inspect

List the answers managers flagged in this beta project, newest first, each with the answer text, who gave it, and the task and step it belongs to. A flag is a manager-only bookmark and carries no reason or author, so it says an answer was singled out, not why. Set and clear flags with flagAnswer.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, so safety is covered; the description goes further by defining what a flag actually signifies (a manager-only bookmark with no reason or author, indicating selection but not rationale) and by specifying newest-first ordering and the returned fields. It does not mention pagination, result caps, or behavior when no answers are flagged, which are the remaining gaps for a list tool with no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and scope, then the flag semantics and sibling pointer. No filler and every clause carries information an agent would otherwise have to guess.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by enumerating the returned fields and their ordering, and it explains the domain meaning of a flag. Only pagination/volume behavior is unaddressed, which is a minor omission for so simple a one-parameter read.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter is documented in the schema itself, including where to obtain valid IDs (listProducts). The description only restates that the ID refers to a beta project, adding no syntax or filtering detail beyond the schema, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (list the answers managers flagged), scopes it to a beta project, and adds ordering (newest first) plus returned fields (answer text, author, task, step). This clearly distinguishes it from flagAnswer, the mutation sibling, and from generic result tools like getStepResults.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes the mutating counterpart to flagAnswer ('Set and clear flags with flagAnswer'), so an agent knows the division of labor between reading and writing flags. It does not compare against other list/result tools (getTaskResults, getStepResults, listApplicants), which leaves some inference for the retrieval case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listOrganizationTestersA
Read-only
Inspect

List the tester directory of this Stomio workspace: every tester who belongs to at least one AI-enabled beta project, with those beta projects. Testers whose beta projects all have AI features disabled are not listed, and each row shows only its AI-enabled beta memberships. productIds spans the whole workspace, so it can include projects you are not assigned to yourself: those ids read as not found in the project-scoped tools, and only ids also returned by listProducts are accessible to you. The result is paginated: when total is larger than the number of testers returned, call again with a higher offset to get the next page. Out-of-range limit/offset values are clamped to the advertised bounds, not rejected — read the echoed limit/offset back from the response. name can be null; fall back to username.

NameRequiredDescriptionDefault
limitNoHow many testers to return (default 50, maximum 200).
offsetNoHow many testers to skip, for paging through the directory (default 0).

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnlyHint annotation: it discloses the AI-enabled filtering rule, that rows show only AI-enabled memberships, and that out-of-range limit/offset are clamped rather than rejected, with the echoed values as the source of truth. It also flags that name may be null and username is the fallback, which is exactly the kind of post-call behavior an agent would otherwise get wrong.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and scope are front-loaded, and every sentence carries substantive information about filtering, pagination, or result interpretation. It runs as one dense block with a long middle sentence about productIds that takes a second read, which keeps it just under top marks.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read tool with no output schema, the description supplies everything needed to call it correctly and interpret the response: pagination mechanics, clamping, the meaning of total, the null-name fallback, and the workspace-versus-project id caveat. Nothing material is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters are documented there, so the baseline is 3. The description adds real meaning beyond it: the clamping behavior for limit/offset, the need to read back echoed values, and the paging loop condition based on total versus returned count.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List the tester directory of this Stomio workspace') and immediately narrows scope: only testers belonging to at least one AI-enabled beta project, with a stated exclusion rule. An agent can distinguish this workspace-wide directory listing from the project-scoped tester tools without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear conditions for re-calling with a higher offset, and explains how returned productIds relate to project-scoped tools ('those ids read as not found... only ids also returned by listProducts are accessible'), which steers the agent away from misusing sibling tools on this output. It stops short of naming a specific alternative such as getBetaTesters vs listOrganizationUsers, leaving that comparison to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listOrganizationUsersA
Read-only
Inspect

List the members (managers) of this Stomio workspace with their role (managerRole: owner, manager, or viewer) and the AI-enabled beta projects they are assigned to, plus how many manager invitations are still pending. name can be null; fall back to username. betaProjectIds covers ALL members, so it can include projects you are not assigned to yourself: those ids read as not found in the project-scoped tools, and only ids also returned by listProducts are accessible to you. Invited manager email addresses are not exposed here; pendingInvitations entries carry the invite date only.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the readOnlyHint annotation, the description discloses the returned member roles, project assignment semantics, the name-null/username fallback, pending invitation behavior, and privacy limitations around invited emails. It also explains an important access caveat: betaProjectIds can include projects the caller cannot access, which will read as not found in project-scoped tools.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but front-loaded, starting with the core listing behavior and then covering fallback, access, and privacy caveats. Every sentence adds useful information, though the colon-heavy third sentence could be easier to scan if split into shorter statements.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no parameters, the description carries the full burden of explaining what is returned. It does so thoroughly: member roles, beta project ids and their access implications, pending invitation count/date behavior, and the missing invited-email detail are all covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

This tool has zero parameters, so there is no parameter semantics to clarify. The baseline for a parameterless tool is 4, and the description appropriately spends its space on return-value meaning instead.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: list members/managers of this Stomio workspace, including role, assigned AI-enabled beta projects, and pending invitation count. It is clear enough to distinguish this organization-wide roster from beta-scoped or tester-focused siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the workspace-level scope, but the description does not explicitly say when to use this instead of siblings like listBetaManagers or listPendingInvitations. It does provide useful context about cross-referencing listProducts for accessible project ids, but that is result-interpretation guidance rather than direct tool-selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listPendingInvitationsA
Read-only
Inspect

List all pending (not yet accepted) tester invitations for the beta project. Returns id, name, invitedAt, lastPingAt and hasAccount for each; the id matches the invitationId that the funnel tools report for a pending invitee. email is present only when the address does not belong to a tester of this workspace (hasAccount: false); an invitee who already holds a tester account here is identified by id and name instead, because tester email addresses are not exposed over this connection. Pass the ids to pingTesters and cancelInvitations to act on any row. Use this to identify testers who haven't joined yet and may need a reminder — these people don't appear in getBetaTesters.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

readOnlyHint already covers safety, and the description still adds substantial behavioral context: the exact returned fields, the conditional presence of email keyed to hasAccount, and the reason (tester email addresses are not exposed over this connection). The id-to-invitationId cross-reference with the funnel tools is non-obvious operational knowledge found nowhere in the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads what the tool lists, then the return shape, then the email caveat, then the routing guidance. The email/hasAccount sentence is long and slightly dense, but every clause carries real information for handling rows, so nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the return-shape burden and does so precisely (id, name, invitedAt, lastPingAt, hasAccount, conditional email). For a single-parameter read tool, an agent needs nothing further to call and interpret it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter and schema coverage is 100%, so the schema already fully documents productId, including how to obtain valid IDs. The description reinforces that the query is scoped to a beta project but adds no format or constraint detail beyond the schema, making 3 the correct baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (list pending, not-yet-accepted tester invitations) scoped to the beta project, and explicitly separates itself from getBetaTesters. An agent can distinguish it from listOrganizationTesters or inviteTesters without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit use case (identify testers who haven't joined and may need a reminder), names the follow-on tools for acting on results (pingTesters, cancelInvitations), and states the exclusion that these invitees do not appear in getBetaTesters. When-to-use and when-not-to-use are both covered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listProductsA
Read-only
Inspect

List the beta projects you manage in this Stomio workspace. Returns each project id, name, lifecycle state, and url (its page in the Stomio dashboard, a permanent link any manager of this workspace can open). Call this first to obtain the productId required by the other tools. Only beta projects with AI features enabled are listed, so the manager may have further beta projects that are not accessible here. state is one of: 'draft' (being set up, invisible to testers), 'published' (scheduled, waiting for its start date — testers cannot participate yet), 'running' (live), 'paused' (manager-paused, testers blocked until resumed), 'finished' (end date passed), 'archived'.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnlyHint annotation: it warns that the manager may have other beta projects not visible here, and defines each lifecycle state's tester-visible consequence ('draft' invisible to testers, 'published' testers cannot participate yet, 'paused' testers blocked). That is exactly the operational context annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and the 'call this first' routing cue before the detail. The state enumeration is long but each entry carries distinct meaning since there is no output schema; still, the list is the largest block and could be slightly tightened.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by naming the returned fields and fully decoding the state enum. Combined with the visibility caveat and the downstream productId dependency, an agent has everything needed to call and interpret it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline is 4; there is nothing extra the description could add here. It correctly does not waste space restating an empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List the beta projects you manage in this Stomio workspace') plus the exact fields returned (id, name, lifecycle state, url). This clearly separates it from sibling list tools such as listBetaManagers and listOrganizationUsers.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly instructs 'Call this first to obtain the productId required by the other tools', giving both ordering and rationale. It also discloses the exclusion rule (only beta projects with AI features enabled), so the agent knows when results will be incomplete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listTesterTagsA
Read-only
Inspect

List the workspace-level tester tags, which are shared across all beta projects, shown with how many of this beta project's testers carry each one. Their ids are the valid tagId filters for getTaskResults and getStepResults. Each entry has an "assignable" flag: assignable tags are manager-managed tags that manageTesterTags can add or remove, and their testersCount is a stored assignment count; non-assignable entries are computed smart tags (segments recomputed from live behavior — their counts change as testers work) that manageTesterTags will reject, and they additionally carry a slug. A tag whose category is configured for both surfaces also appears in listTicketTags under the same id: one tag, two surfaces, not a collision.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint, but the description adds substantial behavior: assignable tags are manager-managed with stored counts, non-assignable entries are live-computed segments whose counts drift as testers work, they carry an extra slug field, and manageTesterTags will reject them. It also explains that overlapping ids in listTicketTags are one tag across two surfaces rather than a collision, which prevents a real misdiagnosis.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and each clause carries non-obvious information (filter validity, assignable semantics, slug, cross-surface id overlap). The text is dense but justified; the final sentence about listTicketTags is valuable though slightly tangential to listing behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, and the description compensates by describing entry contents (assignable flag, testersCount semantics, slug on smart tags) so the agent knows what comes back. For a single-parameter read tool with only readOnlyHint annotations, nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With a single required parameter at 100% schema coverage, the schema already documents productId fully, including where to obtain valid IDs. The description adds only indirect context by tying counts to 'this beta project's testers', so a baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening clause gives a specific verb and resource ('List the workspace-level tester tags') and immediately scopes it by project ('how many of this beta project's testers carry each one'). It also distinguishes the two entry kinds (assignable vs computed smart tags) and names the related siblings getTaskResults, getStepResults, manageTesterTags, and listTicketTags, so an agent can place the tool precisely.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the practical downstream use ('their ids are the valid tagId filters for getTaskResults and getStepResults') and clarifies that manageTesterTags can only mutate assignable tags while rejecting non-assignable ones, which is strong when-to-use guidance. It also pre-empts confusion with listTicketTags. It stops short of an explicit 'call this before filtering' imperative, so a 4 rather than 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listTicketsA
Read-only
Inspect

Get list of tickets (feedback items) for the beta testing program. Tickets can be bugs, ideas (feature requests), questions, or chat messages. The returns could include a "step" field which indicates the specific step where the feedback was reported. Each ticket carries url: its page in the Stomio dashboard, a permanent link any manager of this workspace can open. Every ticket of the beta project is returned, with no cap and no paging; the optional filters narrow the list server-side and combine (a ticket must match all of them). Conversation state comes back on every ticket: updatedAt is when it last changed, messagesCount counts the conversation entries testers and managers wrote, attachments included and automatic events such as state or assignee changes excluded, and equals the totalMessages that getTicketDetails reports; lastMessageAt / lastMessageFrom / lastMessageInternal describe the newest of those entries, so automatic events such as a state or assignee change never move them, and they are absent on a ticket nobody has written in yet. lastMessageFrom is "tester", "manager", or "guest"; lastMessageInternal true means the newest entry is a manager-only note testers cannot see. awaitingManagerReply is on every ticket and is true when the newest entry the tester can actually see came from a tester or a guest: it ignores manager-only notes, so writing a private note on a ticket does not make it look answered, and it is false on a ticket nobody has written in. Use it, not lastMessageFrom, to find the tickets still waiting on a manager. lastPublicMessageAt is when that newest tester-visible entry was written, absent when there is none: on a ticket awaiting a reply it is the moment the wait began, so measure waiting time from it, never from lastMessageAt, which a later manager-only note moves. Read the thread itself with getTicketDetails or getTicketMessagesBatch.

NameRequiredDescriptionDefault
typeNoOptional: Filter by ticket type. Omit to get all tickets.
stateNoOptional: Filter by ticket state. Omit to get tickets in both states.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
updatedSinceNoOptional: ISO 8601 date-time; keeps only tickets changed at or after it. A ticket changes when someone writes in it or edits it.
assigneeUserIdNoOptional: Filter to the tickets assigned to one manager, by their userId from listBetaManagers, or to the unassigned ones with the literal value "unassigned". Any other value is rejected.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=true; the description adds substantial behavioral context beyond that: no cap and no paging, filters applied server-side and combined conjunctively (a ticket must match all), and precise semantics for derived fields such as messagesCount excluding automatic events and awaitingManagerReply ignoring manager-only notes. This is exactly the disclosure an agent needs for a non-paginated bulk read.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and the dense field semantics are logically ordered (identity/link, enumeration scope, conversation state, then reply-waiting). It is long and repeats itself in places, explaining 'automatic events such as state or assignee change' twice and restating lastMessageFrom's value set, which slightly weakens the every-sentence-earns-its-place test.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of explaining the returned shape and does so thoroughly: url, updatedAt, messagesCount, lastMessageAt/From/Internal, awaitingManagerReply, lastPublicMessageAt. An agent can interpret and act on the results without further documentation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description adds meaning beyond it by stating that optional filters narrow results server-side and combine with AND semantics, which the schema does not say. It does not add per-parameter detail on updatedSince or assigneeUserId beyond the schema, so it stops short of a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource ('Get list of tickets (feedback items) for the beta testing program') and clarifies the ticket domain (bugs, ideas, questions, chat). It also distinguishes itself from the thread-reading siblings getTicketDetails and getTicketMessagesBatch by naming them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit routing: use this to list/filter tickets, and 'Read the thread itself with getTicketDetails or getTicketMessagesBatch' for message content. It also warns to use awaitingManagerReply rather than lastMessageFrom when finding tickets awaiting a manager. It lacks guidance on when not to use it (e.g., single known ticket vs. getTicketDetails), but the alternative routing is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listTicketTagsA
Read-only
Inspect

List the workspace-level tags that managers put on tickets — these are the tags shown in the "tags" field of the ticket reads, and their ids are what manageTicketTags accepts. They are shared across all beta projects and are never visible to testers. A tag category can be configured for tickets, for testers, or for both, so a tag id that also appears in listTesterTags is the SAME tag serving both surfaces, not a different tag that happens to share an id. The per-beta labels from listBetaLabels are a separate concept entirely. The tag definitions are workspace-level configuration managed in the Stomio dashboard; MCP cannot create or rename them, so an empty list means no ticket tags exist yet.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Read-only annotations are already provided, yet the description still adds substantial context beyond them: the tags are shared across all beta projects, never visible to testers, and are dashboard-managed configuration the MCP layer cannot mutate. It also explains the empty-result semantics, which is exactly the kind of behavior an agent cannot infer from a readOnlyHint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core statement and the manageTicketTags id linkage are front-loaded, and the sibling distinctions follow logically. The sentence about tag categories being configurable for tickets, testers, or both is dense but earns its place as the explanation for id overlap; the overall passage is longer than typical but has no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the return-value burden and does so: it identifies the tags as the ones visible in the ticket reads' "tags" field, emphasizes ids as the actionable output, and defines the empty-list case. For a one-parameter read tool this leaves nothing an agent needs unresolved.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With a single parameter and 100% schema description coverage, the schema already fully documents productId and how to obtain valid IDs. The description adds only an indirect clarification that the tag listing itself is workspace-wide despite the per-project productId, which is useful but marginal. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ("List the workspace-level tags that managers put on tickets") and immediately scopes it as workspace-level rather than per-beta. It explicitly disambiguates from two siblings, listTesterTags and listBetaLabels, so an agent can choose correctly without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives the practical trigger for calling the tool by explaining that the returned ids are what manageTicketTags accepts, and it rules out confusion with listTesterTags (same tag, both surfaces) and listBetaLabels (a separate concept). It also warns that MCP cannot create/rename tags, so an empty list means "none exist yet" rather than an error. What's missing is an explicit statement of when not to use it or whether it should be called before every manageTicketTags invocation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

loadRequiredConventionsA
Read-only
Inspect

REQUIRED FIRST STEP: call this once per conversation, before any other tool. Returns the conventions this server requires (naming, vocabulary, concurrency handling) and the playbooks for common workflows. The rules it returns are binding for every other tool call in the conversation.

NameRequiredDescriptionDefault

No parameters

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds the non-obvious behavioral facts: idempotent-per-conversation usage and that the returned rules are binding on all subsequent calls. It does not state what happens if called twice or whether a second call returns the same payload, which keeps it just short of 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, with the critical imperative front-loaded in caps so an agent scanning tool names cannot miss the ordering requirement. Zero redundant wording.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of explaining the return value, and it does so adequately by naming the categories of content returned. It stops short of describing the shape or size of the payload, which an agent might want before deciding how to consume it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to clarify and the baseline is 4. No parameter detail is needed or given.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action (returns server conventions and workflow playbooks) with a concrete payload list (naming, vocabulary, concurrency handling). Because it is a bootstrap tool, the description immediately distinguishes it from every CRUD sibling in the list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit instruction: 'call this once per conversation, before any other tool,' plus the consequence that the returned rules are binding for every other call. There is no ambiguity about when to invoke it or how often.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

makeTicketPublicA
Destructive
Inspect

Makes an internal ticket public: EVERY tester of this beta project can then see it, respond on it, and upvote it, and they are notified it was posted. This is ONE-WAY — a public ticket can never be made private again (the app enforces the same rule). Chat tickets cannot be made public (they are private conversations), and tickets in App (SDK) projects cannot be public at all. The change is announced as a message in the ticket thread. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
ticketIdYesID of the ticket, from listTickets or getTicketDetails. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond readOnlyHint=false and destructiveHint=true, it discloses visibility to every tester, notifications, the thread announcement, permanent loss of privacy, eligibility restrictions, and token expiry. It also explains the non-obvious dry-run behavior: the first call changes nothing and returns a preview plus a confirmation token.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Although relatively long, every sentence communicates a distinct consequence, restriction, or invocation requirement, and the core action is front-loaded. Emphasis on "ONE-WAY" and "TWO-STEP TOOL" appropriately highlights safety-critical information without wasting prose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an irreversible mutation without an output schema, the description covers the effects, exclusions, permanence, consent requirement, preview behavior, and token lifecycle. Together with the fully documented input schema and consistent annotations, it provides sufficient context for safe selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents all three parameters, including identifier sources and the basic confirmation-token workflow. The description adds operational meaning by connecting the token to a user-visible preview, explicit approval, and expiration "after a few minutes," though it adds little new information about productId or ticketId.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening clause identifies the exact resource and transition: "Makes an internal ticket public." It then clarifies the resulting visibility and interactions, distinguishing this operation from generic siblings such as updateTicket, updateTicketState, and replyToTicket.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly excludes chat tickets and tickets in App (SDK) projects, while the irreversibility warning helps the agent assess whether publication is appropriate. The preview-and-explicit-approval instructions precisely describe when and how to proceed with the executing call.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

manageTaskLabelsAInspect

Adds and/or removes per-beta labels on a single task, and returns the task's resulting labels. Get assignable ids from listBetaLabels (create new labels with createBetaLabels) — these are NOT the workspace ticket tags from listTicketTags or the tester tags from listTesterTags. Labels are manager-internal and assigning one sends no notification, but on a running or paused beta project a label change does mark the beta as having unpublished changes (commitTaskChanges will report something to publish). Adding a label the task already has, or removing one it does not have, is a no-op. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
taskIdYesID of the task, from getBetaTasks. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
addLabelIdsNoLabel ids to add, from listBetaLabels.
removeLabelIdsNoLabel ids to remove, from listBetaLabels.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and destructiveHint=false; the description carries everything else of value: no notification is sent, a label change marks the beta as having unpublished changes, duplicate adds/missing removals are no-ops, and edits are staged until commitTaskChanges (discardable via revertTaskChanges). It even flags a surprising response behavior: staged rows are returned even when live ids are passed, with liveVersionId linking back.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause, and every sentence carries operational information rather than filler. It is dense and long, with the staging/id-echo material packed into the final sentences, so an agent must read to the end, but nothing is truly redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a non-destructive mutation tool with no output schema, the definition covers input sourcing, side effects, no-op semantics, staging lifecycle, response shape, and the terminal archived/finished restriction. Nothing an agent needs to invoke it safely is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds the decisive semantic that the label id parameters use manager-internal beta label ids rather than workspace ticket tags or tester tags, which the schema does not say. It also clarifies the no-op conditions for addLabelIds/removeLabelIds. It does not repeat the 50-item cap, which is already in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb pair and resource ('adds and/or removes per-beta labels on a single task') plus the return value, and explicitly separates itself from the similarly named label tools (listBetaLabels, createBetaLabels, listTicketTags, listTesterTags). An agent can route correctly without opening any sibling schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Names the prerequisite tools for obtaining ids ('Get assignable ids from listBetaLabels (create new labels with createBetaLabels)') and explicitly excludes the wrong namespaces (ticket tags, tester tags). It also states the conditions that change behavior: draft/scheduled projects apply immediately, running/paused projects stage the change, and finished/archived projects are read-only.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

manageTesterLabelsAInspect

Adds and/or removes per-beta labels on a single tester of this beta project, and returns their resulting labels. Get assignable ids from listBetaLabels (create new labels with createBetaLabels) — these are NOT the workspace tester tags from listTesterTags or the ticket tags from listTicketTags. Labels are manager-internal: the tester sees nothing and no notification is sent. Adding a label the tester already has, or removing one they do not have, is a no-op.

NameRequiredDescriptionDefault
userIdYesThe tester's user id, from getBetaTesters. This is the user id, not the tester id, and never an invented one.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
addLabelIdsNoLabel ids to add, from listBetaLabels.
removeLabelIdsNoLabel ids to remove, from listBetaLabels.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnlyHint=false, destructiveHint=false), the description discloses the visibility model ('manager-internal: the tester sees nothing and no notification is sent') and idempotency semantics ('adding a label the tester already has, or removing one they do not have, is a no-op'). It also states the return: the tester's resulting labels. That is exactly the behavioral context annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, each carrying a distinct payload: what it does, where ids come from plus the disambiguation, and the behavior/visibility notes. The verb+resource lead is front-loaded and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no output schema, the description supplies the mutation target, id provenance, side-effect profile, idempotency, and the shape of the response. Nothing an agent needs to invoke it correctly is absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline would be 3; the description earns more by pinning down the correct source of label ids and by warning that these are not tester tags or ticket tags — a distinction the schema's generic 'from listBetaLabels' note only partly makes. It adds no detail on the 50-item cap or ordering, but the schema covers those.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Adds and/or removes per-beta labels on a single tester of this beta project.' The scope qualifier ('per-beta labels', 'single tester') and the explicit exclusion of workspace tester tags and ticket tags separate it cleanly from the many sibling label/tag tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives concrete prerequisites — assignable ids come from listBetaLabels, new labels come from createBetaLabels — and explicitly rules out the wrong id sources (listTesterTags, listTicketTags). What is missing is an explicit steer against the near-sibling manageTesterTags, which an agent could plausibly confuse with this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

manageTesterTagsAInspect

Adds and/or removes workspace tester tags on a single tester of this beta project, and returns their resulting tags. Get ids from listTesterTags and use only entries whose "assignable" flag is true — non-assignable entries are computed smart tags and will be rejected. These are NOT the per-beta labels from listBetaLabels or the ticket tags from listTicketTags. Tester tags are workspace-level, so the change follows the tester across every beta project. Be aware of one automatic side effect: if the tag's category also applies to tickets, Stomio adds or removes that tag on ALL of the tester's private tickets in this workspace and notifies each of those tickets' audiences. Adding a tag the tester already has, or removing one they do not have, is a no-op. Tags are applied one at a time and are not rolled back as a group, so if the call fails partway the tags processed before the failure stay applied; re-running with the same ids is safe.

NameRequiredDescriptionDefault
userIdYesThe tester's user id, from getBetaTesters. This is the user id, not the tester id, and never an invented one.
addTagIdsNoTester tag ids to add, from listTesterTags.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
removeTagIdsNoTester tag ids to remove, from listTesterTags.

TDQS

A4.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Strong disclosure beyond annotations: cross-project workspace scope, an automatic side effect that mutates all private tickets and notifies their audiences, per-tag application without group rollback, and idempotent no-ops. However, it does not state permission requirements or which roles can call it, so it is not fully complete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action, then constraints and side effects in decreasing priority. Dense but nearly every clause carries operational value; the sibling-disambiguation sentence is long, though still informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, but the description states it returns the tester's resulting tags. Scope, idempotency, partial-failure semantics, and side effects are all covered for a 4-parameter mutating tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning: tag ids must come from listTesterTags and be assignable, and the userId must be the user id rather than a tester id. It does not restate the 50-item caps or explain interaction between addTagIds and removeTagIds.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Adds and/or removes workspace tester tags on a single tester') and names the exact siblings it is not: listBetaLabels and listTicketTags. An agent can distinguish it from manageTesterLabels/manageTicketTags without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives the source for ids (listTesterTags), the filter to apply (only entries with assignable=true, otherwise rejected), and no-op conditions for already-present/absent tags. It also states that re-running with the same ids is safe after a partial failure, which is exactly the recovery guidance an agent needs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

manageTicketTagsAInspect

Adds and/or removes workspace ticket tags on a single ticket, and returns the ticket's resulting tags. Ticket tags are manager-internal: they are never shown to testers and assigning one sends no notification. Get assignable ids from listTicketTags — these are NOT the same as the per-beta labels from listBetaLabels or the tester tags from listTesterTags. The tag definitions themselves are workspace-level configuration managed in the Stomio dashboard: MCP cannot create, rename, or delete them, so when listTicketTags is empty there is nothing to assign until a manager creates tags in the dashboard. Adding a tag the ticket already has, or removing one it does not have, is a no-op.

NameRequiredDescriptionDefault
ticketIdYesID of the ticket, from listTickets or getTicketDetails. Never invent ids.
addTagIdsNoTicket tag ids to add, from listTicketTags.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
removeTagIdsNoTicket tag ids to remove, from listTicketTags.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, the description discloses that ticket tags are manager-internal, never shown to testers, and send no notification. It also states that the tool returns the ticket's resulting tags and that adding an existing tag or removing a missing tag is a no-op.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core action and return value, then adds only relevant context about visibility, id sources, dashboard limitations, and idempotency. Every sentence contributes distinct operational guidance without repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no output schema, the description is complete enough: it explains what changes, what is returned, who can see the result, where ids come from, what cannot be done through MCP, and how redundant operations behave. No critical agent decision is left unaddressed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all four parameters. The description still adds meaningful semantic guidance by distinguishing listTicketTags ids from per-beta labels and tester tags, and by clarifying no-op behavior for addTagIds and removeTagIds.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: adding and/or removing workspace ticket tags on a single ticket. It also clearly distinguishes this tool from listTicketTags, listBetaLabels, and listTesterTags, so an agent can identify its purpose without opening sibling schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It explicitly tells the agent where to get assignable ids (listTicketTags) and names the alternatives that are not appropriate (listBetaLabels, listTesterTags). It also covers the edge case where listTicketTags returns empty and no assignment is possible until a manager creates tags in the dashboard.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pauseBetaAInspect

Pause this RUNNING beta project. While paused, testers cannot submit answers or open tickets, and the beta shows as paused; managers can keep editing content in sketch mode. No emails are sent. Reversible with resumeBeta.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations (readOnlyHint=false, destructiveHint=false) by disclosing concrete side effects: testers cannot submit answers or open tickets, the beta displays as paused, managers retain sketch-mode editing, and no emails are sent. It also flags reversibility, which is exactly the context an agent needs before a state mutation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the action and its precondition, followed by effects and reversibility. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter state-mutation tool with no output schema, the description covers precondition, side effects, notification behavior, and reversibility. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the single productId parameter, so the schema already carries the semantics and the description adds nothing about the parameter. Baseline 3 applies since the description does not supplement the parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Pause') and resource ('this RUNNING beta project') with an explicit precondition on state. An agent can distinguish it from siblings like resumeBeta, archiveBeta, and launchBeta without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear precondition (project must be RUNNING) and names the counterpart tool 'resumeBeta' as the way to reverse it, which routes the agent well. It does not explicitly exclude archiveBeta or restoreBeta, so the alternative-space is not fully enumerated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

pingTestersA
Destructive
Inspect

Resends the beta invitation email to one or more testers who still have a PENDING invitation (they were invited but have not joined yet). TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing is sent and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes. Only testers with a pending invitation can be pinged; accepted or rejected testers cannot. Get the targets from listPendingInvitations (pending invitees do not appear in getBetaTesters): name them by invitationIds, which works for every pending invitee, or by emails, which only works for the invitees whose address listPendingInvitations reports (an invitee who already holds a workspace tester account is identified by id and name instead). At least one of the two is required and they combine freely; naming the same person both ways is one target, not two. One call pings at most 100 pending invitations. Never fabricate emails or ids. The batch is all-or-nothing: if ANY provided email or id lacks a pending invitation on this beta project, the whole call is rejected and nothing is sent.

NameRequiredDescriptionDefault
emailsNoEmail addresses to ping. Use real addresses from listPendingInvitations. Only pending invitations can be pinged.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
invitationIdsNoInvitation ids to ping, from listPendingInvitations. Use these for invitees whose address is not reported.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the non-read-only and destructive annotations, it discloses that the first call sends nothing, the second sends emails, tokens are single-use and short-lived, and batches are limited to 100 invitations. The all-or-nothing rejection behavior and guarantee that nothing is sent on validation failure are especially valuable operational details.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose and confirmation workflow are front-loaded, and essentially every sentence supplies actionable information without filler. However, the dense single paragraph and especially long target-discovery sentence reduce scanability; separating workflow, targeting, and validation into short paragraphs or bullets would improve structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a consequential four-parameter email-sending tool with no output schema, the description covers the preview result, execution workflow, target sources, eligibility, parameter interaction, limits, token lifecycle, and failure behavior. Together with the fully documented schema, it provides everything needed to select and invoke the tool safely.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema coverage is 100%, the description adds substantial cross-parameter semantics: at least one of emails or invitationIds is required, they can be combined, duplicate identification counts as one target, and confirmationToken must accompany the same arguments. It also explains the differing coverage of the two targeting methods and the atomic validation across every supplied email and id.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence precisely states the action and scope: resend beta invitation emails to testers whose invitations are still PENDING. This clearly distinguishes the tool from initial invitation workflows and from getBetaTesters, which the description explicitly says does not contain pending invitees.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit eligibility rules, excluding accepted or rejected testers, and prescribes a two-step workflow requiring a preview and explicit user approval before sending. It also routes target discovery to listPendingInvitations and explains when invitationIds must be used instead of emails.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

publishBetaA
Destructive
Inspect

Schedule this draft beta project to run between startAt and endAt. The beta must be ready: at least one task, every task with at least one step, and at least one tester added or invited — otherwise this fails with the missing piece and nothing changes. On confirm, every pending tester invitation is emailed and the opt-in link (if any) goes live. If the beta has no ticket code prefix yet, one is generated from its name so tickets get references like APP-1 (set it yourself beforehand with updateBetaDetails). Testers can start only once startAt is reached; use launchBeta to start a published beta immediately. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
endAtYesISO 8601 date-time when the beta ends. Must be after startAt.
startAtYesISO 8601 date-time when the beta starts. Must be in the future.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare destructiveHint=true, but the description adds far more: the exact readiness preconditions (one task, each task with a step, at least one tester) and the failure behavior ("fails with the missing piece and nothing changes"). It also discloses side effects on confirm — pending invitations emailed, opt-in link goes live, ticket prefix auto-generated — and that tokens are single-use and expire in minutes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and preconditions before side effects and the confirmation protocol, and every sentence carries operational information. The two-step token explanation slightly duplicates the confirmationToken schema description, but the repetition is defensible for a destructive tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive, no-output-schema mutation tool, the description covers preconditions, failure modes, side effects, alternatives, and the confirmation handshake. An agent has everything needed to call it safely without inferring anything from annotations alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, and the description adds behavioral meaning beyond the schema: startAt governs when testers can begin, and endAt bounds the run window. It does not, however, clarify the productId/beta-project naming mismatch present in the schema description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource: "Schedule this draft beta project to run between startAt and endAt." It immediately distinguishes itself from siblings by naming launchBeta for immediate starts and updateBetaDetails for the ticket prefix, so an agent can route correctly without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use and when-not-to-use guidance: use launchBeta to start a published beta immediately, use updateBetaDetails beforehand to set the ticket code prefix, and testers only start once startAt is reached. It also spells out the required two-call confirmation protocol step by step.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

publishScreeningFormA
Destructive
Inspect

Makes this beta project's screening form live: the public application link (getScreeningSetup publicLink) starts accepting applications, provided the source is also active. Requires at least one question on the form. A no-op (changed: false) when the form is already live. Publishing cannot be undone from MCP; pause applications with configureScreening active: false instead. Once live, the form's questions are edited from the Stomio dashboard only.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare destructiveHint=true and readOnlyHint=false; the description earns credit by explaining exactly what that means here — publishing cannot be undone from MCP — and by naming the reversible path (configureScreening active: false). It also discloses the idempotent return signal (changed: false) and a post-state workflow consequence: questions become dashboard-only once live.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core action and its effect lead, followed by preconditions, idempotency, irreversibility, and the post-publish constraint. Every clause carries distinct decision-relevant information; there is no restating of the name or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter, irreversible mutation with no output schema, the description covers preconditions, result signaling (changed: false), rollback limitations, and the follow-up editing path. Nothing an agent needs in order to call it safely or interpret the result is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single productId parameter, and the schema already tells the agent to source IDs from listProducts. The description only reinforces that the target is a 'beta project', adding no new syntax or format meaning beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Makes this beta project's screening form live') and immediately identifies the concrete effect: the public application link from getScreeningSetup's publicLink begins accepting applications. It is clearly distinguishable from siblings like configureScreening, which it explicitly cites.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives preconditions (source must be active, at least one question required), the no-op case (already live returns changed: false), and the explicit alternative for the opposite intent ('pause applications with configureScreening active: false instead'). An agent can decide whether to call this tool without opening another schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

removeBetaAgreementA
Destructive
Inspect

Remove this beta project's NDA/agreement document: testers no longer have to accept an agreement during onboarding, and the stored document file is deleted. Only allowed while the beta is a draft or scheduled. A no-op (changed: false) when no agreement is set.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond readOnlyHint=false and destructiveHint=true, the description discloses two concrete consequences: testers no longer accept an agreement during onboarding and the stored document file is deleted. It also documents the lifecycle restriction and empty-state behavior, including the changed:false result, none of which is available from the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description front-loads the action and its consequences, then uses two short sentences for the precondition and empty-state result. Every clause supplies operational information, with no unnecessary restatement or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter destructive operation, the definition covers the target, tester-visible effect, file deletion, allowed beta states, and behavior when no agreement is configured. Although there is no output schema, the changed:false detail supplies the most important response behavior, and nothing essential to selecting or invoking the tool is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter, productId, is already fully described in the schema, including how to obtain valid IDs from listProducts, so the high-coverage baseline applies. The description clarifies what happens to the identified beta project but adds no further identifier syntax, format, or selection semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening clause uses a specific verb and resource—"Remove this beta project's NDA/agreement document"—and immediately defines the resulting onboarding behavior. Removing the agreement for the entire beta and deleting its document also distinguish the purpose from setting an agreement or bypassing agreement acceptance for an individual tester.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The draft-or-scheduled restriction gives an explicit lifecycle condition for when the operation is available, while the no-agreement case tells the agent what to expect instead of implying an error. It does not explicitly route agents to setBetaAgreement when they want to add or replace a document, so the guidance is strong but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

removeBetaManagerA
Destructive
Inspect

Removes a manager from this beta project. They are removed as a participant from the beta's tickets and their notifications for it are deleted, but they stay a member of the workspace and of any other beta project, and nothing they created is deleted. You cannot remove yourself, and a beta project must keep at least one manager. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
userIdYesThe user id of a workspace member, from listOrganizationUsers or listBetaManagers. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well past the destructiveHint=true annotation by enumerating exactly what is destroyed (ticket participation, notifications) and what is explicitly preserved (workspace membership, other betas, authored content). It also discloses the two-step token behavior including single-use and expiry, and the invariant constraints that can cause failure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the action, then side effects, then invariants, then the protocol in a clearly flagged "TWO-STEP TOOL" segment. Every sentence carries distinct operational information with no restatement of the tool name or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

A destructive mutation with no output schema needs side effects, preconditions, and the safety protocol fully spelled out, and this description provides all three, including what a successful first call returns (a preview plus token). Nothing an agent needs to call it safely is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so productId and userId provenance plus the confirmationToken semantics are already documented in the schema. The description reinforces the token workflow (same arguments, explicit approval) but adds no new meaning about individual parameters, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Removes a manager from this beta project") and immediately scopes it against neighbors: the person stays a workspace member and a member of other beta projects, and is only stripped of ticket participation. That distinguishes it cleanly from addBetaManager and from workspace-level removals in the sibling list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear preconditions for use: the target must be a manager, the caller cannot remove themselves, and the beta must retain at least one manager. It also prescribes the exact invocation protocol (preview call, user approval, then execute call). It does not name an alternative tool for cases where the user should instead be removed from the workspace entirely, which keeps it from a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

removeStepMediaA
Destructive
Inspect

Removes the media (image, video, or file) shown on a step and deletes its stored file, like setStepMedia with attachmentId=null. A no-op (cleared: false) when the step has no media. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
stepIdYesID of the step within the task, from getBetaTasks. Never invent ids.
taskIdYesID of the task, from getBetaTasks. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the destructiveHint annotation: it states the stored file is deleted, that an empty step returns cleared: false, that changes are staged on live betas until commitTaskChanges, and that finished/archived projects cannot be edited. That is exactly the mutation risk profile an agent needs before calling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but front-loaded: the destructive action comes first, then edge cases, then the staging contract, then the hard limit. Every sentence carries operational information and none restates the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description still tells the agent what a response looks like (cleared: false, staged rows with liveVersionId) and when the call is refused. There is no meaningful gap for a destructive three-parameter mutation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so all three ids are documented, but the description adds real meaning: staging accepts both live and staged ids yet always returns the staged row with a liveVersionId back-link, so the agent should expect unfamiliar ids in the response.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Starts with a specific verb and resource ('Removes the media... shown on a step and deletes its stored file') and names the sibling it mirrors (setStepMedia with attachmentId=null), which distinguishes it from removeTaskMedia and getStepMedia.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explains the no-op case, the draft/scheduled immediate path, the running/paused staging path, and which sibling tools publish or discard staged changes (commitTaskChanges, revertTaskChanges, getBetaTasks). It stops short of explicitly contrasting step-level vs task-level removal with removeTaskMedia, so a 4 rather than 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

removeTaskMediaA
Destructive
Inspect

Removes the media (image, video, or file) shown on a task and deletes its stored file, like setTaskMedia with attachmentId=null. A no-op (cleared: false) when the task has no media. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
taskIdYesID of the task, from getBetaTasks. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark this as destructive and non-read-only, and the description adds substantial context beyond them: it deletes the stored file, stages changes on running/paused projects, returns cleared:false as a no-op, and reports staged IDs rather than necessarily the IDs passed in. It also discloses that finished or archived beta project content cannot be edited.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core action is front-loaded in the first sentence, followed by behavior that matters for safe invocation: no-op handling, immediate versus staged execution, and archived-project restrictions. The density is justified by the staging complexity, and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive mutation with no output schema, the description covers the action, side effect on stored files, no-op result, staging lifecycle, related tools, ID behavior, and editability restrictions. An agent has enough information to invoke it correctly and interpret the staged response.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so taskId and productId are already documented in the input schema. The description still adds non-obvious meaning by noting that taskId may be either the live or staged ID and that the response returns the staged row with liveVersionId linking back.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it removes media from a task and deletes the stored file. It also distinguishes the operation from setTaskMedia by describing it as equivalent to setTaskMedia with attachmentId=null, so an agent can identify the exact action without opening another schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains the no-op case when the task has no media, the immediate-apply case for draft/scheduled projects, and the staged case for running/paused projects. It names the related commitTaskChanges, revertTaskChanges, and getBetaTasks behaviors that determine how staged removals are published or discarded.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

removeTesterA
Destructive
Inspect

PERMANENTLY removes a tester from this beta project and deletes everything they contributed to it: their answers, their uploaded files, their task progress, and their notifications for it. This cannot be undone. Their tickets are kept for the managers, but the tester is removed as a participant on them. The tester is NOT notified, and they stay a member of the workspace and of any other beta project. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
userIdYesThe tester's user id, from getBetaTesters. This is the user id, not the tester id, and never an invented one.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already flag destructiveHint=true, yet the description goes far beyond them, spelling out exactly what is destroyed (answers, uploaded files, task progress, notifications) and that the action is irreversible. It also discloses non-obvious side effects: tickets are retained for managers, the tester is not notified, and workspace membership is untouched. Token single-use and expiry behavior is an additional useful disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the irreversible consequence, then scope exclusions, then the two-step procedure. Sentences are dense but every one carries distinct information; nothing is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive mutation with no output schema, the description covers what the first call returns (preview plus token), what the second call does, and all side effects an agent must communicate to the user. Nothing needed to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds workflow-level meaning to confirmationToken that a bare schema cannot convey: the preview-then-execute cycle, reusing the SAME arguments, and single-use short-lived tokens. The userId and productId meanings are largely restated from the schema, limiting it to a 4 rather than 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource, 'PERMANENTLY removes a tester from this beta project', and immediately enumerates the scope of the deletion. It distinguishes itself from siblings such as removeBetaManager by stating the tester remains a workspace member and stays in other beta projects.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit operational guidance: call without confirmationToken first, show the preview to the user, and only re-call with the same arguments after explicit approval. The only gap is that it does not name a sibling alternative for reversible or lesser removals, but the described prerequisites (get user id from getBetaTesters, project id from listProducts) are clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reorderScreeningQuestionsAInspect

Reorders the screening form questions. Pass EVERY questionId of the form (from getScreeningSetup) in the desired order. Only usable before the form is live, and only when the form keeps its questions in a single group (forms restructured into several groups in the dashboard must be reordered there).

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
questionIdsYesThe complete list of the form's questionIds in the desired order.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and destructiveHint=false; the description adds the real behavioral content: this must be a complete-list reorder, and it is time-gated (pre-launch) and structure-gated (single group). It stops short of saying what happens if an ID is omitted or unknown, or whether the call is atomic, but it covers the constraints that most affect whether the call will succeed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the action and the completeness requirement, then the two preconditions. No filler and no repetition of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 2-parameter mutation with no output schema, the description covers the preconditions, the full-replacement requirement, and the fallback path. The only real gap is return behavior (what the form looks like after reordering, or how failures are reported), which an agent would otherwise have to discover.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the schema already says questionIds is 'the complete list of the form's questionIds in the desired order', so the description's 'Pass EVERY questionId' largely restates it. The added value is pointing at getScreeningSetup as the source, though the schema points at listProducts for productId, creating mild split guidance between the two sources.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Reorders the screening form questions'), which distinguishes it from updateScreeningQuestion, addScreeningQuestion, deleteScreeningQuestion and updateStepsOrder (steps, not questions). It does not explicitly name those siblings, so the differentiation is inferred rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use and when-not-to-use conditions: only before the form is live, and only for single-group forms. It also names getScreeningSetup as the source of questionIds and routes multi-group forms to the dashboard, which is a concrete alternative path.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

replyToTicketA
Destructive
Inspect

Posts a reply on a ticket. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing is sent and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes. A normal reply notifies the ticket's audience by email and in-app. Setting internal=true posts a note visible only to managers in Stomio, but it is still synced to connected integrations (Jira, Webex), so it leaves Stomio too. Only open tickets accept new messages; close or reopen a ticket with updateTicketState. To attach a file, upload it first with uploadMedia (target 'ticket') and pass the attachmentId; an attachment is always posted as its own follow-up message (with text and attachment given, TWO messages are posted), and message may be omitted when attachmentId is provided.

NameRequiredDescriptionDefault
messageNoThe reply body (1 to 4096 characters).
internalNoWhen true, posts a manager-only internal note instead of a reply the testers see. Defaults to false.
ticketIdYesID of the ticket, from listTickets or getTicketDetails. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
attachmentIdNoAttachment handle from uploadMedia (target 'ticket'). Posted as its own image/video/file message after the text (if any).
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and destructiveHint=true; the description goes far beyond by disclosing that a reply triggers email and in-app notification, that internal notes are still synced to Jira/Webex and thus leave Stomio, that tokens are single-use and short-lived, and that a text+attachment call produces two messages.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and the two-step protocol, then layered constraints. It is dense and somewhat long, but nearly every clause carries operative information an agent needs; only the 'TWO-STEP TOOL' label is mildly redundant with the sentences that follow.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex, irreversible, externally-visible mutation with six parameters and no output schema, the description covers safety (preview/confirm), side effects (email, Jira/Webex sync), state preconditions, and attachment mechanics. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so a 3 is the floor, but the description adds real meaning: the consequences of internal=true (visible to managers yet still synced externally), the attachmentId follow-up-message behavior, and the fact that message may be omitted when an attachment is supplied. It does not add syntax details for ticketId/productId, but those are already well documented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Posts a reply on a ticket') and immediately draws the internal-note vs. public-reply distinction. It is clearly separable from siblings like updateTicket, updateTicketState, and uploadMedia, each of which is named for its own role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit routing: use updateTicketState to close/reopen, uploadMedia first to attach a file, and set internal=true for manager-only notes. It also states the precondition that only open tickets accept messages and spells out the exact when-to-call sequence for the two-step confirmation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

restoreBetaAInspect

Restore this ARCHIVED beta project with a new end date (defaults to 30 days from now). It returns to its pre-archive scheduling: scheduled if its start date is still in the future, otherwise running (or paused if it was paused when archived). Its applicant screening form becomes reachable again but applications stay closed until reopened from the dashboard. No emails are sent.

NameRequiredDescriptionDefault
endAtNoISO 8601 date-time for the new end date. Must be in the future. Defaults to 30 days from now.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by describing the resulting state (scheduled vs running vs paused preserved), the default end date, that the screening form becomes reachable while applications remain closed, and that no emails are sent. These are exactly the side effects an agent needs before committing a mutation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, each carrying distinct information, with the target state and default front-loaded before the secondary form/email details. No filler or restatement of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a non-destructive mutation with full schema coverage and no output schema, the description covers preconditions, defaults, and downstream effects thoroughly. It omits any indication of what the call returns or how errors (e.g. non-archived project) are surfaced, which would complete the picture.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the schema already documents both parameters including the ISO 8601 format and the 30-day default; the description only repeats the default for endAt. It adds no guidance on productId sourcing or date edge cases, so it stays near baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Restore this ARCHIVED beta project') plus the state transition it performs, which cleanly separates it from archiveBeta, resumeBeta, and pauseBeta in the sibling list. An agent can identify the operation without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The word 'ARCHIVED' establishes the precondition for use, and the return-to-scheduling sentence clarifies the outcome versus a plain resume. It does not explicitly name resumeBeta or say what to do if the project is not archived, so routing is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

resumeBetaAInspect

Resume this PAUSED beta project so testers can participate again. No emails are sent.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already establish that this is not read-only and not destructive. The description adds a useful side-effect disclosure, 'No emails are sent', which is meaningful behavioral context beyond the annotations, though it does not cover permissions or reversibility.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with the core action and state condition front-loaded. Every phrase adds useful information, and there is no redundant wording.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter mutation tool with no output schema, the description provides the essential state requirement, expected effect, and email side-effect behavior. The remaining gap is that it does not clarify how this differs from restoreBeta or launchBeta.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the single productId parameter is already fully documented in the schema, including how to obtain valid IDs from listProducts. The description adds no additional parameter meaning beyond that.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb, 'Resume', and resource, 'PAUSED beta project', and states the outcome that testers can participate again. It clearly contrasts with pauseBeta via the PAUSED condition, but it does not explicitly distinguish itself from nearby siblings such as restoreBeta or launchBeta.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The condition for use is clear: the beta must be PAUSED. However, there is no explicit when-not guidance or named alternative such as restoreBeta, so the agent must infer edge cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

revertTaskChangesA
Destructive
Inspect

PERMANENTLY discards every staged task and step change of the beta project, including changes staged by other managers in the Stomio app, and restores the staging area to match the live content testers currently see. The confirm discards exactly the staged set the preview was issued for: if the staged set changes between preview and confirm, the confirm fails and you must preview again. Nothing that is already live changes, and the discarded edits cannot be recovered. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the destructiveHint annotation, it discloses that the discard is irreversible, that it affects other managers' staged edits, that live content is untouched, that the operation is token-bound to the previewed set, and that tokens are single-use and short-lived. This is unusually rich behavioral context for a destructive mutation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The critical facts (permanent, irreversible, two-step) are front-loaded in caps, and the paragraph is dense with signal. It is a bit long and partly restates the schema's confirmationToken description, costing a point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description still explains what the first call returns (a preview and a token) and what the confirm call does, plus the failure mode when the staged set changes. Nothing an agent needs to invoke this correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description adds genuine meaning by explaining the confirmationToken's two-step, single-use, expiring nature and that the second call must use the SAME arguments, plus that productId targets the beta project's staging area.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: it permanently discards every staged task and step change and restores the staging area to live content. It also scopes the blast radius (affects changes staged by other managers) and contrasts with the live state, which makes it easy to tell apart from commitTaskChanges.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear, explicit protocol: call without confirmationToken first, show the preview to the user, then call again with the same arguments plus the token after approval. It also states a failure condition (staged set changed → re-preview). It never names the commitTaskChanges alternative explicitly, which keeps it from a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reviewApplicantsAInspect

Sets the review status of one or more applicants to "shortlisted", "rejected", or "needsReview". This is an internal pipeline change only: nothing is sent to the applicant, and rejecting is reversible — a rejected applicant can be shortlisted later. Applicants who have already been invited or joined cannot be re-reviewed. Shortlisting does NOT invite anyone; inviteApplicants is a separate step. Get match ids from listApplicants. Each id is reported individually, so a partial failure still applies the rest.

NameRequiredDescriptionDefault
statusYesThe review status to apply to every listed applicant.
matchIdsYesApplicant match ids from listApplicants (1 to 50).
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and destructiveHint=false; the description goes well beyond that by disclosing that nothing is sent to the applicant, that rejection is reversible, that non-reviewable applicants exist, and that results are reported per id so a partial failure still applies the rest. These are exactly the mutation-side effects an agent needs; only the response shape is left implicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Semantically dense but front-loaded: the core action comes first, then constraints, then the sibling pointer, then the failure semantics. No filler sentences and no repetition of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-parameter mutation with no output schema, it covers side effects, reversibility, eligibility, and partial-failure behavior. The only shortfall is that the return payload (per-id results) is described only qualitatively rather than indicating what a success or failure entry looks like.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so status enum, 1-50 match id bounds, and the productId provenance are already documented in the schema. The description adds the useful framing that status applies to 'every listed applicant', but otherwise restates rather than extends parameter meaning — the correct baseline of 3 when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (sets) and resource (review status of applicants) plus the exact enum values it accepts, so the action is unambiguous. It also explicitly distinguishes itself from the nearest sibling by stating that shortlisting does not invite and that inviteApplicants is a separate step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit prerequisites (get match ids from listApplicants) and explicit exclusions (applicants already invited or joined cannot be re-reviewed). It also names the alternative tool, inviteApplicants, for the adjacent task, leaving little to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

sendAnnouncementA
Destructive
Inspect

Creates an announcement and dispatches it (email + in-app) to testers of the beta project. The announcement is recorded and its delivery is queued; individual email or in-app delivery failures are logged and do not surface here. The response carries the announcementId that listAnnouncements reports (with recipient and view counts). TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing is sent and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes. Audience targeting (pick exactly one): allTesters=true reaches every tester; usersIds targets specific testers by their user ids from getBetaTesters; tagIds targets everyone carrying any of the given tester tags from listTesterTags (smart tags included), narrowed to this beta project's testers.

NameRequiredDescriptionDefault
textYesThe announcement body (1 to 600 characters). Only bold, italic, underline, line-break and span tags survive; every other tag, links included, is removed. A URL written as plain text stays in the text.
audienceYesWho receives the announcement. Set exactly one of: allTesters=true, usersIds, or tagIds.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
attachmentIdNoOptional attachment handle from uploadMedia (target 'announcement').
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only flag this as a non-read-only, destructive operation. The description goes much further: delivery is queued, individual email or in-app failures are logged and do not surface here, tokens are single-use and expire after minutes, and a tag set matching no tester is refused rather than silently sent to nobody. That is exactly the irreversibility and failure-mode context an agent needs before a write action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The two-step protocol and the response/ID lineage are front-loaded, and the closing audience paragraph covers remaining targeting modes. It is dense but readable; a little of the audience paragraph duplicates the schema's own guidance, which is the only slack.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter tool with a nested audience object and no output schema, the description supplies both the return value (announcementId, verifiable via listAnnouncements with recipient and view counts) and the multi-call confirmation lifecycle. Nothing an agent needs to invoke it safely is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description still adds value by tying usersIds to getBetaTesters, tagIds to listTesterTags (smart tags included), and by noting tags are narrowed to this beta project's testers. The audience paragraph largely restates the schema's own exactly-one-of rule, which keeps it from a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ('Creates an announcement and dispatches it (email + in-app) to testers of the beta project'), including the delivery channels and audience. That is enough to separate it from listAnnouncements and pingTesters without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It lays out the exact two-step protocol: first call without confirmationToken yields a preview, the user must explicitly approve, then the same arguments plus the token. Audience selection rules (exactly one of allTesters, usersIds, tagIds) and the source tools for each ID type are spelled out, with a clear condition for picking each.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

setBetaAgreementAInspect

Set or replace this beta project's NDA/agreement document. Testers must accept it during onboarding before they can participate. Upload the PDF first with uploadMedia (target 'agreement', application/pdf only) and pass the attachmentId here; a previously set agreement document is replaced and its file deleted. Only allowed while the beta is a draft or scheduled: the agreement is frozen once the beta starts. The returned agreementUrl is a time-limited download link for the human user and differs on every call.

NameRequiredDescriptionDefault
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
attachmentIdYesAttachment handle from uploadMedia with target 'agreement'.

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by disclosing that a previously set agreement is replaced and its file deleted, that the agreement freezes after launch, and that agreementUrl is a time-limited link that differs each call. Minor tension: the file-deletion side effect sits awkwardly against destructiveHint=false, though it is cleanup of a superseded file rather than destruction of the beta data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four tight sentences, front-loaded with the action, each carrying unique information: prerequisites, replacement semantics, state restriction, and return-value caveat. No filler or restatement of the name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, yet the description explains the returned agreementUrl and its time-limited nature. Combined with the documented prerequisites and lifecycle restriction, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description nonetheless adds real meaning—the attachmentId must originate from uploadMedia with target 'agreement' and be application/pdf, and the ID identifies the beta whose agreement is being frozen or replaced.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource (set/replace the beta project's NDA/agreement document) with clear scope. It is trivially distinguishable from siblings like removeBetaAgreement, setTesterAgreementBypass, and uploadMedia, which are referenced by role.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

States the prerequisite workflow explicitly (upload the PDF first with uploadMedia, target 'agreement', application/pdf only, then pass the attachmentId) and the state precondition (only while the beta is draft or scheduled; frozen once the beta starts). The agent knows both when the call will succeed and what to do beforehand.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

setInvitationEmailAInspect

Configure this beta project's custom tester invitation email. When activated, EVERY invitation email this beta sends (inviteTesters, inviteApplicants, the publish-time fan-out, and invitations sent from the Stomio dashboard) uses this subject and message instead of the stock Stomio email; when not activated the stock email is used and the stored draft is kept. Provide only the fields to change. The message (basic HTML, up to 1024 characters of rendered text) may use ONLY these template variables, written in double curly braces: {{PROJECT_NAME}}, {{WORKSPACE}}, {{MANAGER_USERNAME}}, {{MANAGER_FULL_NAME}} — any other {{...}} placeholder is rejected. The subject (plain text, up to 128 characters) supports NO template variables at all: variables are only substituted in the message, so a {{...}} placeholder in the subject is rejected here (it would reach testers unreplaced). Activating requires both a subject and a message (already-stored values count). Invitation emails are sent from a no-reply address, so a message that invites testers to "just reply to this email" reaches nobody: point them at the Stomio dashboard, or at an address written in the message itself. Configuring the template sends nothing by itself. Read the current template via getProductOverview (customInvitationEmail).

NameRequiredDescriptionDefault
messageNoThe email body (basic HTML; template variables allowed).
subjectNoThe email subject (plain text, up to 128 characters; {{...}} template variables are NOT supported in subjects and are rejected).
activatedYesWhether invitation emails use the custom template. Required.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations (which only say it is not read-only and not destructive) by disclosing blast radius: every invitation path — inviteTesters, inviteApplicants, publish fan-out, and dashboard sends — switches to this template when active. It also warns that mail is sent from a no-reply address so reply-based CTAs fail, and notes the draft is preserved on deactivation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first sentence and the remaining sentences are ordered from effect to validation to caveats. It is dense and em-dash-heavy, and a few clauses (e.g. the explanation of why subject placeholders are rejected) could be trimmed, but almost every sentence carries operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, yet the description covers side effects, validation constraints, mode behavior, and where to read the current state. For a mutation tool with a 4-parameter schema, nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning: the exact whitelist of allowed template variables, the rendered-text character cap for the message, the rule that subjects support no variables at all, and the cross-field activation requirement. Partial-update semantics ("provide only the fields to change") are also stated beyond the schema's field-level notes.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource: "Configure this beta project's custom tester invitation email." The scope (per-beta-project template for invitation emails) is unambiguous and clearly distinct from the read-side sibling getProductOverview, which the description names for reading the current value.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the two modes (activated vs not activated and what happens to the stored draft in each), that only fields to change need to be provided, that activation requires both subject and message, and that configuring sends nothing by itself. It also names getProductOverview as the way to inspect the current template. When-to-use and how-to-use are both covered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

setStepMediaAInspect

Sets or clears the media (image, video, or file) shown on a step. Pass an attachmentId from uploadMedia (target 'step') to set it — any existing media on the step is replaced — or attachmentId=null to remove the current media. A TriggeredEmail step shows no media and is rejected here. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
stepIdYesID of the step within the task, from getBetaTasks. Never invent ids.
taskIdYesID of the task, from getBetaTasks. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
attachmentIdYesAttachment handle from uploadMedia (target 'step'), or null to remove the step's current media.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond readOnlyHint=false, it discloses replacement of existing media, immediate versus staged writes by project state, delayed tester visibility, commit and revert behavior, and the editing ban for terminal project states. The explanation of staged response rows, unexpected IDs, and liveVersionId is especially valuable behavioral context not supplied by the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The set-and-clear behavior is front-loaded, followed logically by prerequisites, exclusions, lifecycle effects, and response-ID behavior. Although relatively long, each sentence supplies an operational rule needed for this stateful mutation rather than filler or marketing language.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a four-parameter mutation with no output schema, the definition supplies the required workflow, invalid states, replacement behavior, publication lifecycle, and important staged-response semantics. Together with the fully described input schema, it gives an agent enough context to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and much of the attachmentId guidance repeats that schema, so the baseline is 3. The description earns additional credit by explaining that task and step IDs may be either live or staged during staging, although it adds little specific semantics for productId.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence precisely identifies the operation and resource: setting or clearing an image, video, or file on a step. Its step scope distinguishes it from task-media tools, while the replacement and null-removal behavior makes the mutation unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit set and clear workflows, including the prerequisite uploadMedia call with target 'step'. It also explains when not to use it, for TriggeredEmail steps and finished or archived projects, and names commitTaskChanges, revertTaskChanges, and getBetaTasks for the staged workflow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

setTaskMediaAInspect

Sets or clears the media (image, video, or file) shown on a task. Pass an attachmentId from uploadMedia (target 'task') to set it — any existing media on the task is replaced — or attachmentId=null to remove the current media. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
taskIdYesID of the task, from getBetaTasks. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
attachmentIdYesAttachment handle from uploadMedia (target 'task'), or null to remove the task's current media.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes far beyond the annotations by disclosing that existing media is replaced, that running/paused projects stage the change until commitTaskChanges publishes it, that revertTaskChanges discards it, that both live and staged ids are accepted while responses return the staged row with liveVersionId, and that finished/archived projects cannot be edited. Note a mild tension worth flagging to callers: the annotations set destructiveHint=false, yet the call overwrites or removes existing media irreversibly enough that re-upload is required — the description, not the annotation, carries that risk information.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and the two usage modes, then the staging consequences; every sentence carries information. It is somewhat dense and the live-vs-staged id paragraph is shared boilerplate across this tool family, which keeps it just short of ideal brevity for a single three-parameter call.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no output schema and only three fully documented parameters, the description covers the mutation semantics, the staging lifecycle, the id round-trip behavior, and the hard restriction on archived projects. Nothing an agent needs in order to call it safely and interpret the response is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, yet the description still adds real meaning: the provenance of attachmentId (uploadMedia with target 'task'), the null-as-removal semantic, and the non-obvious id contract where live or staged task ids are accepted but the response returns staged ids with a liveVersionId back-reference.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource plus the media types covered: "Sets or clears the media (image, video, or file) shown on a task." This clearly separates it from getTaskMedia (read), removeTaskMedia (dedicated removal), setStepMedia (step scope) and uploadMedia (creates the attachment handle) without needing the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete when-to-use guidance: obtain an attachmentId from uploadMedia with target 'task' to set media, or pass null to clear it, plus the staging context on running/paused betas and the commitTaskChanges / revertTaskChanges / getBetaTasks alternatives. The one gap is that the sibling removeTaskMedia appears to overlap with attachmentId=null, and the description never says which to prefer or whether they differ.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

setTesterAgreementBypassAInspect

Exempts one tester of this beta project from accepting the NDA/agreement during onboarding (bypass: true), or re-requires it (bypass: false). Only meaningful when the beta project has an agreement set (getProductOverview hasAgreement). The tester is not notified. Setting the value it already has is a no-op (changed: false).

NameRequiredDescriptionDefault
bypassYestrue exempts the tester from the agreement; false re-requires it.
userIdYesThe tester's user id, from getBetaTesters. This is the user id, not the tester id, and never an invented one.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, it discloses that the tester is not notified and that setting the existing value is a no-op with changed:false. It does not cover authorization requirements or other edge cases, but adds meaningful behavioral context for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences, front-loaded with the action and its two modes, followed by the precondition and behavioral notes. No unnecessary wording.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple three-parameter mutation with no output schema, the description covers the main behavioral traits: agreement precondition, notification behavior, and no-op result. It could say more about error behavior when no agreement is set, but is otherwise complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the schema already documents all three parameters, including bypass semantics and userId source. The description reinforces the bypass meaning and the agreement precondition, but adds little beyond the structured fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: exempting or re-requiring one tester's NDA/agreement during onboarding, with bypass:true/false semantics. This clearly distinguishes it from siblings like setBetaAgreement and removeBetaAgreement.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear precondition: only meaningful when the beta project has an agreement set (getProductOverview hasAgreement). It does not explicitly name setBetaAgreement as the alternative for creating/removing the project-level agreement, so guidance is strong but not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

setTicketAssigneesA
Destructive
Inspect

Sets who is assigned to a ticket. The list REPLACES the current assignees: pass every user id that should end up assigned, or an empty list to unassign everyone. Assignees must be managers of this beta project — get their user ids from listBetaManagers. Each addition and removal is recorded as a message in the ticket thread, so it is visible to everyone on the ticket and notifies them; on a public ticket that audience is every tester of the beta project. Chat tickets cannot be assigned. If the ticket's assignees change between the preview and the confirm (for example from the Stomio dashboard), the confirm fails and you must preview again. In the response, name is the person's display handle (it can be their username when no name is set). TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
ticketIdYesID of the ticket, from listTickets or getTicketDetails. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
assigneeUserIdsYesThe complete list of manager user ids to assign, from listBetaManagers. An empty list unassigns everyone.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only say non-read-only and destructive; the description goes far beyond by disclosing that the list fully replaces existing assignees, that each add/remove posts a visible notification message to the ticket thread, that on a public ticket the audience is every beta tester, that tokens are single-use and short-lived, and that stale previews fail. This is exactly the kind of side-effect and auth-context detail an agent needs before a destructive mutation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the mutation semantics, then constraints, then the two-step protocol — a sensible ordering for a destructive tool. It is dense and long, but nearly every sentence carries an operational constraint; the closing note about the 'name' field is the one sentence that could be trimmed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive, token-gated mutation with no output schema, the description covers replacement semantics, id sourcing, eligibility constraints, notification side effects, failure/re-preview behavior, and token lifetime. Nothing an agent needs to call it safely and correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema already documents replacement semantics, the manager-only source, and the confirmationToken flow, so the description mostly restates rather than extends it. The only marginal addition is the note about the response 'name' field being a display handle, which is not a parameter at all. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Sets who is assigned to a ticket') and immediately clarifies the replace-not-append semantics, which distinguishes it from the addBetaManager/removeBetaManager siblings. An agent knows exactly what state change this produces without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the prerequisite source for ids (listBetaManagers), states the manager-only constraint, states the chat-ticket exclusion, and lays out the full two-step call protocol with the exact condition for the second call (user explicitly approves, same arguments plus token). It even covers the concurrency case where the confirm fails and the agent must re-preview.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateApplicantRuleAInspect

Update an auto-review rule of this beta project's applicant screening; provide only the fields to change (conditions and actionSuggestedLabelIds REPLACE the current lists). Each condition is { type: "question" | "emailDomain", operator: "equals" | "contains" | "exists" | "gt" | "lt", questionId?, value? }. type "question" compares an applicant's answer to a screening form question (questionId required, from getScreeningSetup); type "emailDomain" compares the applicant's email domain (contains not supported). "exists" takes no value; every other operator requires one. gt/lt are numeric and question-only (like a Rate answer). A rule matches when ALL its conditions match. Existing applicants are re-evaluated immediately (refreshedCount says how many suggestions changed); already APPROVED reviews are never touched.

NameRequiredDescriptionDefault
nameNoA short manager-facing rule name.
ruleIdYesThe ruleId, from getScreeningSetup. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
conditionsNoThe full replacement list of conditions that must ALL match.
actionSuggestedStatusNoThe status to suggest when the rule matches.
actionSuggestedLabelIdsNoThe full replacement list of per-beta label ids (listBetaLabels) to suggest.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations (which only say readOnlyHint=false, destructiveHint=false): it discloses immediate re-evaluation of existing applicants, that refreshedCount reports how many suggestions changed, and that already-APPROVED reviews are never touched — a key safety boundary for an agent. It also flags the unusual replace-vs-patch semantics for conditions and actionSuggestedLabelIds.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but zero-waste: purpose and the partial-update/replace distinction are front-loaded, then the condition grammar, then the side-effect on existing applicants. Every clause carries information the agent would otherwise have to guess.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter mutation tool with no output schema and a nested conditions structure that the schema leaves undefined, the description covers purpose, payload shape, replacement semantics, return signal (refreshedCount), and the APPROVED-review exclusion. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline would be 3, but the description adds substantial meaning the schema lacks: the conditions array has no item schema at all, and the description supplies its full grammar (type, operator, questionId, value), per-type constraints (questionId required for question, contains unsupported for emailDomain), operator-specific value rules (exists takes none; gt/lt are numeric and question-only), and the AND semantics across conditions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ("Update an auto-review rule") scoped to "this beta project's applicant screening", which cleanly separates it from createApplicantRule and deleteApplicantRule in the sibling list without needing the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operative guidance: supply only the fields to change, and names getScreeningSetup as the source of questionId. It does not explicitly route the agent to createApplicantRule for new rules or deleteApplicantRule for removal, so the alternative-selection guidance is contextual rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateBetaDetailsAInspect

Update this beta project's name, description, objectives, ticket code prefix, schedule (startAt, endAt), or tester onboarding requirements (shipping info, custom fields); provide only the fields to change. description and objectives are manager-facing context, never shown to testers: the manager sees them in the dashboard, and they carry what this project is about across sessions and surfaces. Read the current values first (getProductOverview) and revise them rather than replace them — the manager may have edited them in the app. Keep them concise, current, human-readable statements of what the project is and what the manager wants to learn; do not store conversation notes or internal bookkeeping in them. Which dates are editable depends on the state: a draft takes neither (publishBeta schedules it), a scheduled beta takes both, and a running or paused beta takes endAt only. state is one of: 'draft' (being set up, invisible to testers), 'published' (scheduled, waiting for its start date — testers cannot participate yet), 'running' (live), 'paused' (manager-paused, testers blocked until resumed), 'finished' (end date passed), 'archived'.

NameRequiredDescriptionDefault
codeNoTicket reference prefix (2 to 4 characters, unique within the workspace). Tickets are numbered from it, like APP-1. CAUTION: changing it renames the references of every existing ticket in this beta project.
nameNoNew beta project name, unique within the workspace (up to 64 characters).
endAtNoISO 8601 date-time when the beta ends. Editable while the beta is scheduled, running, or paused. Must be in the future and after the start date; no task may have an available date past it.
startAtNoISO 8601 date-time when the beta starts. Editable only while the beta is scheduled (state published), and only to a future date: launchBeta is what starts a scheduled beta now. A draft is scheduled with publishBeta, and a running or paused beta keeps the start date it began on.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
objectivesNoWhat the manager wants to learn or validate (hypotheses, decisions, focus areas).
descriptionNoWhat this beta project is about.
customFieldsRequiredNoWhether testers must fill the workspace's custom profile fields during onboarding. Defaults to true at creation, so they are required unless this is set to false. The field definitions themselves are listed by listCustomFields and are managed in the Stomio dashboard.
shippingInfoRequiredNoWhether testers must provide a shipping address during onboarding (for betas that ship physical hardware). Defaults to false at creation.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and destructiveHint=false; the description goes well beyond them by disclosing the state machine that gates edits (draft/scheduled/running/paused/finished/archived), the blast radius of changing the ticket code prefix, and that description/objectives are manager-only and never tester-visible. Minor gap: it doesn't say what happens when a field is sent in a state that disallows it (rejected vs silently ignored).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and partial-update rule are front-loaded, but the closing paragraph is bloated: the six-state enumeration restates rules already given for startAt/endAt in both the description and the schema, and the read-modify guidance is repeated twice. Roughly a third of the text is redundant with structured fields.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter mutation tool with full schema coverage and no output schema, this is largely complete: prerequisite read step, partial-update contract, state gating, and side effects are all covered. What remains unstated is the editability of name/code by state and the failure behavior on an invalid update.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds semantics the schema does not carry: description/objectives are manager-facing context persisted across sessions and must not hold conversation notes, and date editability varies by state. The customFieldsRequired/shippingInfoRequired defaults are left entirely to the schema, keeping this just under a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Update this beta project's') and enumerates the exact editable surface: name, description, objectives, ticket code prefix, schedule, onboarding requirements. That field list distinguishes it from siblings like updateTicket, updateStep, and updateTask, which share the 'update' verb but touch different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states partial-update semantics ('provide only the fields to change'), prescribes the read-first alternative ('Read the current values first (getProductOverview)'), and routes scheduling actions to publishBeta and launchBeta instead of this tool. It also gives when-not conditions via the per-state date editability rules.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateBetaLabelAInspect

Rename or recolor one label of this beta project; provide only the fields to change. Renaming keeps every assignment it already has, on testers, tasks and pending invitations alike, and notifies nobody. Label names are unique across the whole beta project, not just within their category. A label cannot be moved to another category; delete it and create it again in the category you want. To change what a category itself does, use updateBetaLabelCategory; to create labels, use createBetaLabels.

NameRequiredDescriptionDefault
nameNoNew label name, unique within the beta project (case-insensitive).
colorNoNew hex color like '#0F77F0'.
labelIdYesID of the label to change, from listBetaLabels.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations only declare non-read-only and non-destructive; the description goes well beyond that by disclosing side effects an agent could not otherwise know: assignments on testers, tasks and pending invitations survive a rename, nobody is notified, and names are unique project-wide. It stops short of stating validation behavior (e.g. what happens on a duplicate name or invalid hex) or the response payload, which keeps it off a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, all load-bearing, with the core action front-loaded before the behavioral notes and alternatives. Minor redundancy with the schema on case-insensitive name uniqueness is the only slack.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a non-destructive mutation with no output schema, the definition covers side effects, uniqueness constraints, immutability of category, and sibling routing, which is enough to call it safely. The remaining gap is failure behavior on duplicate names or malformed colors, which an agent would have to discover at runtime.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but "provide only the fields to change" supplies partial-update semantics that the schema does not make explicit, and the project-wide uniqueness note reinforces the scope of the name constraint. The color and labelId semantics are left entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Rename or recolor one label of this beta project" gives a specific verb pair and a precisely scoped resource, and the closing sentence explicitly separates it from updateBetaLabelCategory (category-level changes) and createBetaLabels (creation). An agent can distinguish it from every nearby sibling without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states when to use this tool (rename/recolor an existing label, passing only changed fields), names the two alternatives with their conditions, and gives an explicit when-not plus the workaround: a label cannot be moved between categories, so delete and recreate. Nothing about routing is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateBetaLabelCategoryAInspect

Rename a label category of this beta project or change what it does; provide only the fields to change. The labels inside it keep their names, colors and assignments. groupResults and shareTickets take effect immediately for everyone: turning shareTickets on lets testers sharing a label of this category read each other's tickets (chats excluded), and turning it off takes that access away again.

NameRequiredDescriptionDefault
nameNoNew category name.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
categoryIdYesID of the category to change, from the categoryId field of listBetaLabels.
groupResultsNoWhen true, testers sharing a label of this category are aggregated into a single group in the results summary (completion shows 0% or 100%, tester counts capped at 1).
shareTicketsNoWhen true, testers sharing a label of this category can see and interact with each other's tickets (except chats).

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only mark this as non-read-only and non-destructive; the description adds real value beyond that by stating the update is partial, that contained labels are unaffected, that changes propagate immediately to all testers, and that shareTickets grants and can later revoke cross-tester ticket visibility (chats excluded). It stops short of stating who may perform the change or what permissions are required for a setting with access-control consequences.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the action and scope, then the invariants, then the behavioral detail on the two boolean flags. Some content duplicates the schema wording for shareTickets and groupResults, but nothing is filler and the ordering supports quick scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter mutation tool with no output schema, the description covers partial-update semantics, invariants, blast radius, and immediate-effect behavior. What is missing is failure behavior (e.g., invalid categoryId) and whether a role or permission is required, which an agent would need for a permission-altering call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so all five parameters are already documented, including the aggregation behavior of groupResults and the ticket-sharing behavior of shareTickets. The description mainly reinforces timing ('take effect immediately for everyone') and reversibility for shareTickets, adding modest meaning beyond the schema rather than new semantics for name, productId, or categoryId.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource ('Rename a label category ... or change what it does') and clarifies the scope by noting the labels inside keep their names, colors and assignments, which distinguishes a category update from a label update. It never names the closely related siblings updateBetaLabel or deleteBetaLabelCategory, so an agent must infer the boundary rather than being routed to it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear operational context: 'provide only the fields to change' establishes partial-update semantics, and it explains that groupResults and shareTickets take effect immediately for everyone, which signals this is a live-effect configuration call rather than a staged edit. It offers no explicit when-not guidance or pointer to alternative tools such as updateBetaLabel or deleteBetaLabelCategory.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateScreeningQuestionAInspect

Edits a screening form question: its text, description, answer type, content, or whether it is required. Pass only the fields to change, except that changing type also requires the matching content for the new type (unless the new type takes none). Only usable before the form is live; live forms are edited from the Stomio dashboard. Each question carries ready: false means it fails authoring validation (a choice question with no options, say) and blocks publishScreeningForm until fixed. ready is input for your reasoning, not for users: when everything passes, do not mention it; when something fails, say what is missing in plain words instead of quoting the flag.

NameRequiredDescriptionDefault
textNoThe question shown to applicants.
typeNoThe new answer type.
contentNoThe step configuration, shaped by type: for MultipleChoices/SingleChoice an array of at least 2 { choice } options with non-empty texts, optionally plus ONE final { choice: "Other", other: true } option that lets the tester type a free answer (its label is fixed: choice must be the literal text "Other", any other wording is rejected); for Rate an object { scale (number, 3 or more), symbol ("Thumb", "Star" or "Square"), lowerText?, higherText? (labels for the scale ends, up to 20 characters, default empty) }; for Ranking an array of at least 2 items { id (a v4 UUID you generate), text } with unique ids and unique texts; for TriggeredEmail an object { to, subject, message, cc? } where to is { allManagers?: boolean, allTesters?: boolean, tagsIds?: string[], personFieldIds?: string[] } and names at least one audience, subject is plain text of at most 128 characters and takes no variables (nothing replaces them there, so the recipient would read the braces), message is the email body as basic HTML reading at most 4096 characters of text, and cc is a list of Person custom field ids. Any other key of the email object or of to is rejected rather than dropped, so bcc and to.emails are not accepted: nothing would send to them. tagsIds are tester tag ids from listTesterTags, and ONLY the entries it marks assignable: a computed smart tag is rejected because a triggered email cannot be sent to one. personFieldIds and cc are ids of Person-type fields from listCustomFields; a personFieldIds field routes the email to the manager-persons that tester recorded in it, cc puts them in copy. The email goes out the first time a tester answers or skips the step before it (skipping a whole optional task sends nothing), one email per tester, and the audiences reach: allTesters ONLY that tester, never every tester in the beta project; tagsIds that same tester, only if they carry one of the tags (tester tags never reach managers, and beta labels are not tags); allManagers every manager on the beta project. To notify a manager when a tester finishes a task, make this the task's last step and use allManagers or personFieldIds. message may use these tester variables: {{USERNAME}}, {{FULL_NAME}}, {{ADDRESS}}, {{EMAIL}}, {{PHONE}}, {{PROJECT_NAME}}, {{WORKSPACE}}, {{REMAINING_PROJECT_DAYS}}, {{CURRENT_WEEK_DAY}}. The {{MANAGER_*}} ones belong to invitations and are rejected here. On an SDK beta project the audience must be managers only (allManagers), because its testers are not workspace testers and every other audience would reach nobody. REQUIRED for MultipleChoices, SingleChoice, Rate, Ranking and TriggeredEmail; omitting it is rejected. Must be OMITTED for FreeText, Instructions and File; passing one is rejected. Omit it for YesNo too (its answers are fixed to Yes / No, set automatically); passing that exact pair back is accepted, anything else is rejected. Pass the array or object value itself, not a JSON-encoded string. Call getBetaTasks to see the current shape of existing steps.
requiredNoWhether the applicant must answer.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
questionIdYesThe questionId of a screening form question, from getScreeningSetup. Never invent ids.
descriptionNoSupporting detail shown under the question (basic HTML allowed).

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare the write, non-destructive profile; the description adds real behavioral context beyond that: the pre-live-only window, the type/content coupling, partial-update semantics, and the existence of a `ready` validation flag that blocks publishScreeningForm. It stops short of saying what the mutation returns or what error surfaces on a rejected content payload, so a 5 is not warranted.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the verb, resource, and editable fields; the pre-live restriction follows before the validation discussion. The final paragraph about how to phrase the `ready` flag for users is materially longer than the rest and is presentation guidance rather than tool behavior, which slightly dilutes the structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter mutation tool with full schema coverage and no output schema, the description covers the gaps that matter: when the operation is legal, how partial updates behave, the type/content dependency, and the validation state an agent must reason about after the call. An agent can call this correctly and interpret the result without further context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents each field and its constraints, setting a baseline of 3. The description still adds cross-parameter meaning the schema does not: that omitted fields are left untouched and that `type` and `content` must be supplied together for content-bearing types, which is decision-relevant semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Edits a screening form question') and enumerates exactly which attributes are editable (text, description, answer type, content, required). The phrase 'screening form question' plus the reference to publishScreeningForm / live forms distinguishes it from the task-step editing siblings like updateStep.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit when-not constraint ('Only usable before the form is live; live forms are edited from the Stomio dashboard') and a partial-update rule ('Pass only the fields to change'), plus the conditional requirement that a type change needs matching content. It does not name sibling tools such as addScreeningQuestion, deleteScreeningQuestion, or configureScreening as the alternatives, so routing is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateStepAInspect

Edits an existing step: its text, description, answer type, content, or whether it is required. Pass only the fields you want to change, except that changing type also requires the matching content for the new type (unless the new type takes none — YesNo gets its fixed Yes/No pair automatically). Changing a step to TriggeredEmail drops its text along with its old content, and is rejected when that step is the task's first one; changing a TriggeredEmail step to another type discards its email configuration. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
textNoThe step's question or title (up to 128 characters). A TriggeredEmail step has none; passing one for it is rejected.
typeNoThe kind of step. MultipleChoices, SingleChoice, Rate, Ranking and TriggeredEmail REQUIRE a content shaped for the type (a choices array, a choices array, a scale object, an items array, an email object). FreeText, Instructions and File take no content and reject one. YesNo takes no authored content either: omit it and the fixed Yes / No pair is set automatically, though echoing that exact pair back is accepted so a step read from getBetaTasks round-trips. TriggeredEmail is not a step testers see or answer: it sends an email the moment a tester answers the step immediately before it, so it takes no "text" and can never be the first step of a task.
stepIdYesID of the step to edit, from getBetaTasks. Never invent ids.
taskIdYesID of the task, from getBetaTasks. Never invent ids.
contentNoThe step configuration, shaped by type: for MultipleChoices/SingleChoice an array of at least 2 { choice } options with non-empty texts, optionally plus ONE final { choice: "Other", other: true } option that lets the tester type a free answer (its label is fixed: choice must be the literal text "Other", any other wording is rejected); for Rate an object { scale (number, 3 or more), symbol ("Thumb", "Star" or "Square"), lowerText?, higherText? (labels for the scale ends, up to 20 characters, default empty) }; for Ranking an array of at least 2 items { id (a v4 UUID you generate), text } with unique ids and unique texts; for TriggeredEmail an object { to, subject, message, cc? } where to is { allManagers?: boolean, allTesters?: boolean, tagsIds?: string[], personFieldIds?: string[] } and names at least one audience, subject is plain text of at most 128 characters and takes no variables (nothing replaces them there, so the recipient would read the braces), message is the email body as basic HTML reading at most 4096 characters of text, and cc is a list of Person custom field ids. Any other key of the email object or of to is rejected rather than dropped, so bcc and to.emails are not accepted: nothing would send to them. tagsIds are tester tag ids from listTesterTags, and ONLY the entries it marks assignable: a computed smart tag is rejected because a triggered email cannot be sent to one. personFieldIds and cc are ids of Person-type fields from listCustomFields; a personFieldIds field routes the email to the manager-persons that tester recorded in it, cc puts them in copy. The email goes out the first time a tester answers or skips the step before it (skipping a whole optional task sends nothing), one email per tester, and the audiences reach: allTesters ONLY that tester, never every tester in the beta project; tagsIds that same tester, only if they carry one of the tags (tester tags never reach managers, and beta labels are not tags); allManagers every manager on the beta project. To notify a manager when a tester finishes a task, make this the task's last step and use allManagers or personFieldIds. message may use these tester variables: {{USERNAME}}, {{FULL_NAME}}, {{ADDRESS}}, {{EMAIL}}, {{PHONE}}, {{PROJECT_NAME}}, {{WORKSPACE}}, {{REMAINING_PROJECT_DAYS}}, {{CURRENT_WEEK_DAY}}. The {{MANAGER_*}} ones belong to invitations and are rejected here. On an SDK beta project the audience must be managers only (allManagers), because its testers are not workspace testers and every other audience would reach nobody. REQUIRED for MultipleChoices, SingleChoice, Rate, Ranking and TriggeredEmail; omitting it is rejected. Must be OMITTED for FreeText, Instructions and File; passing one is rejected. Omit it for YesNo too (its answers are fixed to Yes / No, set automatically); passing that exact pair back is accepted, anything else is rejected. Pass the array or object value itself, not a JSON-encoded string. Call getBetaTasks to see the current shape of existing steps.
requiredNoWhether the tester must answer this step.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
descriptionNoSupporting detail shown under the step (up to 1024 characters of text; basic HTML is allowed).

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Far beyond the annotations, it discloses that writes are immediate on draft/scheduled projects but STAGED on running/paused ones, that responses return the staged row with ids the caller never passed, and that liveVersionId links back. It also flags data loss explicitly: converting to TriggeredEmail drops text and discards email configuration on the reverse conversion, and it explains rejection conditions such as a TriggeredEmail step being the task's first. The destructiveHint=false annotation is about resource deletion, not field overwrites, so this is not a contradiction, but the description correctly carries the risk disclosure the annotation does not.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action and the partial-update rule before the conditional and staging details. The prose is dense and runs as a single long block, and some staging/tool-cross-reference material could arguably live in a shared conventions doc, but nearly every clause carries operational consequence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter mutation tool with no output schema, the description covers change semantics, staging lifecycle, id-return behavior, permission/state restrictions, and the sibling tools needed to supply valid ids and complete the workflow. Little an agent needs in order to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, but the description adds meaning the per-parameter schema cannot: the conditional dependency between 'type' and 'content', and the rule that fields you omit are left untouched. It also routes the agent to getBetaTasks to inspect the current shape rather than guessing.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence names a specific verb (Edits) and resource (an existing step) and enumerates the mutable facets: text, description, answer type, content, required. An agent can immediately distinguish this from createStep, deleteStep, and updateStepsOrder without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states the partial-update contract ('Pass only the fields you want to change'), the cross-field dependency (changing type requires matching content), the terminal case that cannot be edited (finished or archived beta projects), and it names the correct companions for each follow-up need: commitTaskChanges, revertTaskChanges, getBetaTasks, listTesterTags, listCustomFields.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateStepsOrderAInspect

Reorders the steps inside one task. Pass the step ids in the order the testers should work through them. A subset is allowed: the steps you list are placed first, in the order given, and every step you leave out lands after the listed ones (their relative order among themselves is not guaranteed). A triggered-email step cannot be first: it is sent when a tester answers the step right before it. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
taskIdYesID of the task, from getBetaTasks. Never invent ids.
stepIdsYesStep ids of this task, from getBetaTasks, in the desired order.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only say readOnlyHint=false and destructiveHint=false; the description goes far beyond that by disclosing staging semantics, the commit/revert workflow, the triggered-email-first constraint, and that finished or archived beta projects cannot be edited. It also warns that staged responses return STAGED rows with liveVersionId links, so callers expect ids they did not pass in.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first sentence, then ordering rules, then the constraint, then staging behavior. Dense and mostly waste-free, though the staging/id-echo sentence is long and packs several distinct ideas that could be split for faster scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a non-destructive reorder mutation with three fully documented params and no output schema, nothing material is missing: ordering semantics, subset behavior, invalid-position rule, staging lifecycle, and edit restrictions on finished/archived projects are all covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds real meaning: stepIds is explicitly a partial subset, listed steps go first, omitted steps land after in unspecified relative order. It also clarifies why returned ids may differ from the ones supplied during staging.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Starts with a specific verb and resource plus scope ('Reorders the steps inside one task'), which immediately separates it from updateTasksOrder (reorders tasks) and updateStep (edits a single step). The operand is named explicitly: 'Pass the step ids in the order the testers should work through them.'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear situational context: the change applies immediately on draft/scheduled betas but is STAGED on running/paused betas, and it names commitTaskChanges, revertTaskChanges and getBetaTasks as the follow-up tools. It does not, however, explicitly route to a sibling for the task-level analogue or state when-not to use it beyond the archived/finished exclusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateTaskAInspect

Edits a task: its name, description, optional flag, or availableDate schedule. Pass only the fields you want to change. On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
nameNoThe new task name (1 to 128 characters).
taskIdYesID of the task, from getBetaTasks. Never invent ids.
optionalNoWhether testers may skip this task: they get a Skip control and a skipped task counts as done. Defaults to false. Only tasks of Stomio-placement beta projects can be optional, and a task cannot be both optional and scheduled with availableDate: the two are mutually exclusive.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
descriptionNoThe new description (up to 512 characters of text; basic HTML is allowed).
availableDateNoISO 8601 date-time before which the task shows as locked and testers cannot start it. Omit for available immediately; null (updateTask) clears an existing date. A date past the beta end date is accepted here but blocks publishBeta and commitTaskChanges until fixed. Mutually exclusive with optional: a task can be optional or scheduled, not both.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations (readOnlyHint=false, destructiveHint=false) by disclosing the staging lifecycle, that staged edits are invisible to testers until commit, and that responses return the STAGED row with ids you did not pass plus a liveVersionId linking to the live row — a non-obvious and genuinely surprising behavior. The one gap is that it never says what happens to staged edits if the session ends or whether staging is per-user or shared across managers.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and the partial-update rule, then the conditional behavior, then the constraint — a sensible information hierarchy with no filler sentences. It is fairly dense and the staging paragraph is long, but each clause carries operational meaning an agent needs before calling.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no output schema, the description covers the full decision surface: what can be edited, what cannot, when changes take effect, which companion tools publish or revert them, and why returned ids may differ from those passed. Little an agent needs in order to call this correctly is left unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so name, taskId, optional, productId, description, and availableDate are all documented at the field level; the description mainly restates the field list. It does add one useful semantic not in the schema — the dual-id acceptance and staged-row echo behavior — but that is response semantics rather than parameter meaning. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource plus the exact editable fields (name, description, optional flag, availableDate schedule), so an agent can distinguish it from createTask, updateStep, and updateTasksOrder without reading any schema. The 'pass only the fields you want to change' framing also pins down that this is a partial update, not a replace.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when the edit applies immediately (draft/scheduled beta) versus when it is staged (running/paused beta) and names the follow-up tools: commitTaskChanges to publish, revertTaskChanges to discard, getBetaTasks with version 'unpublished' to inspect. It also gives a clear exclusion: content of finished or archived beta projects cannot be edited.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateTasksOrderAInspect

Reorders the tasks of this beta project. Pass the task ids in the order the testers should see them. A subset is allowed: the tasks you list are placed first, in the order given, and every task you leave out lands after the listed ones (their relative order among themselves is not guaranteed). On a draft or scheduled beta project the change is applied immediately. On a running or paused beta project the change is STAGED: testers do not see it until commitTaskChanges publishes all staged changes (revertTaskChanges discards them; getBetaTasks with version 'unpublished' shows the staged state). While staging, these tools accept BOTH the live id and the staged id of a task or step, but always respond with the STAGED row — expect ids you did not pass in, with liveVersionId linking back to the live row. Content of finished or archived beta projects cannot be edited.

NameRequiredDescriptionDefault
taskIdsYesTask ids from getBetaTasks, in the desired order.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations only say the call is not read-only and not destructive; the description supplies the real behavioral contract — immediate application on draft/scheduled projects versus staging on running/paused ones, and the fact that responses carry staged rows with liveVersionId. That is substantial context beyond the annotations, though it does not address permissions or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action, then ordering rules, then staging behavior, then the id caveat — a logical progression with no filler sentences. The staging/id paragraph is dense and could be broken up, but no sentence is dispensable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter mutation with no output schema, the description covers ordering semantics, staging lifecycle, how to publish or revert, how to inspect staged state, id round-tripping, and the archived-project restriction. Nothing an agent needs in order to call it safely is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds genuine meaning the schema lacks: taskIds defines an order and may be a subset, with omitted tasks appended in unspecified relative order. That subset/append semantics cannot be inferred from 'in the desired order' alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Reorders the tasks of this beta project') and immediately clarifies that it operates on task ordering rather than task content, which separates it from updateTask and updateStepsOrder among the siblings. An agent can pick this tool without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly allows a subset and defines the resulting order, names commitTaskChanges as the publish path and revertTaskChanges as the discard path, and points at getBetaTasks with version 'unpublished' to inspect staged state. It also states a hard exclusion: content of finished or archived beta projects cannot be edited.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateTicketA
Destructive
Inspect

Updates a ticket's title, type, external link, or (public tickets only) whether testers may post responses. Pass only the fields you want to change. A changed title or type is announced as its own message in the ticket thread (changing both posts two), visible to everyone on the ticket and notifying them in-app; on a public ticket that audience is every tester of the beta project. A changed link or allowResponses posts no thread message but notifies the ticket's audience in-app. If any of these fields changes between the preview and the confirm (for example from the Stomio dashboard), the confirm fails and you must preview again. A ticket cannot be converted into a chat. Use updateTicketState to open or close a ticket, and makeTicketPublic to make it public. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes.

NameRequiredDescriptionDefault
linkNoAn external URL shown on the ticket (like an issue or document link, up to 256 characters). Stored lowercased. Pass an empty string to remove it.
typeNoThe new ticket type.
titleNoThe new ticket title (1 to 128 characters).
ticketIdYesID of the ticket, from listTickets or getTicketDetails. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
allowResponsesNoWhether testers may post responses on this ticket. Only applies to PUBLIC tickets.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only say readOnlyHint=false and destructiveHint=true; the description adds substantial non-obvious behavior: which field changes post thread messages, how many messages, who is notified (whole beta project audience on public tickets), that link/allowResponses notify without posting, and that concurrent edits invalidate the confirm step. It also discloses token single-use and expiry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the field list, then side effects, then exclusions, then the two-step protocol — a sensible order. It is dense and long, but nearly every sentence carries a distinct behavior; only the wording of the notification rules could be tightened slightly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive mutation with no output schema, the description covers everything an agent needs: what changes, what side effects occur, what fails, how confirmations work, and when a different tool is required.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already defines every parameter including the confirmationToken mechanics. The description still adds meaning the schema lacks: allowResponses only applies to public tickets, and the announcement side effects differ per field, which affects how an agent should batch changes.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb plus the exact set of updatable fields ('title, type, external link, or ... whether testers may post responses') and scopes allowResponses to public tickets. It explicitly separates itself from updateTicketState and makeTicketPublic, so an agent can pick the right tool without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use alternatives ('Use updateTicketState to open or close', 'makeTicketPublic to make it public'), a hard exclusion ('A ticket cannot be converted into a chat'), and a full two-step protocol with when to preview versus when to execute.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateTicketStateA
Destructive
Inspect

Opens or closes a ticket. TWO-STEP TOOL: call it WITHOUT confirmationToken first — nothing changes and you receive a preview plus a confirmationToken. Show the preview to the user; only after the user explicitly approves, call again with the SAME arguments plus the confirmationToken. Tokens are single-use and expire after a few minutes. The testers on the ticket are notified of the change in-app, and the status is synced to a connected Jira issue. A closed ticket accepts no new replies. In the response, jiraSyncFailed appears only when a linked Jira issue could not be updated; its absence means synced or no Jira link.

NameRequiredDescriptionDefault
stateYesThe state to set: "open" or "closed".
ticketIdYesID of the ticket, from listTickets or getTicketDetails. Never invent ids.
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.
confirmationTokenNoTwo-step confirmation token. Omit on the first call to receive a preview plus a token; then call again with the SAME arguments plus this token to execute. Single-use and short-lived.

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by disclosing side effects (testers are notified in-app, status syncs to a linked Jira issue), the irreversibility consequence of closing (no new replies), and token lifetime/single-use semantics. The destructiveHint=true annotation is consistent with the described state-changing behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The two-step procedure is front-loaded and stated imperatively, followed by consequences and response semantics in a tight sequence. Almost every clause carries operational value; nothing is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by explaining that jiraSyncFailed appears only on a failed Jira sync and that absence means synced or unlinked. Combined with the confirmation flow and side-effect disclosure, an agent has everything needed to invoke it safely.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so each parameter including confirmationToken is already documented in the schema. The description reinforces the token's single-use and expiry behavior and the 'SAME arguments' requirement, but adds little per-parameter meaning beyond the schema baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Opens or closes a ticket" gives a specific verb pair and resource, and the state enum confirms the scope. It does not explicitly distinguish itself from siblings like updateTicket or replyToTicket, so an agent must infer that this tool is only for state transitions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives unusually precise invocation guidance: first call without confirmationToken to get a preview, show it to the user, then re-call with identical arguments plus the token after explicit approval. It does not state when to choose this tool over near-neighbors such as updateTicket, which keeps it from a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

uploadMediaAInspect

Upload a media file for this beta project and get an attachmentId to pass to replyToTicket, sendAnnouncement, setTaskMedia, setStepMedia, or setBetaAgreement. The upload target must match the tool you will attach with. Two modes with different size limits: pass data (base64) to upload directly (small files, up to 3145728 bytes), or omit data to receive an uploadUrl (large files such as videos, up to 524288000 bytes) — then HTTP PUT the raw bytes to it sending EXACTLY the headers the response lists (Content-Type and x-goog-content-length-range; missing or different headers make the PUT fail), and use the attachmentId afterwards. Uploading alone changes nothing visible; the file only appears once attached. attachmentIds are single-use, bound to this beta project, and expire after about an hour. Every upload is virus-scanned before it can be attached (inline uploads are scanned immediately; URL-mode uploads are scanned when attached). The response fields differ by mode: inline uploads report sizeBytes and scanStatus, URL-mode uploads report uploadUrl, requiredHeaders, and maxBytes.

NameRequiredDescriptionDefault
dataNoThe file content as standard base64 (no data: prefix). Omit to receive a signed uploadUrl instead.
targetYesThe surface this file will be attached to. Must match the attach tool used next. Target 'agreement' (the beta NDA document for setBetaAgreement) accepts application/pdf only.
mimeTypeYesThe exact MIME type of the file (e.g. image/png, video/mp4, application/pdf). Accepted families: images (jpeg, png, gif, webp, heic), video (mp4, quicktime), and files (pdf, plain text, csv, html, pcap, the zip, gzip, tar, 7z and rar archives, and Microsoft Office documents).
productIdYesID of the beta project to query. Get valid IDs from the listProducts tool.

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and destructiveHint=false; the description goes well beyond that with single-use attachmentIds, ~1 hour expiry, beta-project binding, virus scanning (immediate for inline, deferred to attach time for URL mode), and the fact that an upload alone changes nothing visible. It also warns that the PUT must send exactly the listed headers or fail, which is the kind of operational detail annotations cannot carry. It stops short of describing rate limits or failure/retry semantics, so it is not a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The text is front-loaded with purpose and downstream tools, then modes, then caveats, and ends with per-mode response fields — a logical order with no filler sentences. It is dense and long, but for a two-mode upload protocol nearly every clause carries actionable information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description pre-empts the question of what comes back by listing sizeBytes and scanStatus for inline mode and uploadUrl, requiredHeaders, and maxBytes for URL mode. Combined with expiry, scanning, and single-use rules, an agent has everything needed to call this correctly across both paths.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline would be 3, but the description adds data the schema omits: the concrete size ceilings for each mode, the mode switch driven by presence or absence of `data`, and the consequence that target must line up with the next attach call. Those are real semantic additions rather than restatements of the field descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb and resource (upload a media file) and its concrete outcome (an attachmentId), then names the five sibling tools that consume that ID. An agent can distinguish this from getTicketAttachment, removeStepMedia, or any other media sibling without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit mode-selection criteria — pass base64 for small files up to 3145728 bytes, omit data for an uploadUrl up to 524288000 bytes such as videos — plus the constraint that the target must match the attach tool used next. When-to-use and when-not-to-use (inline vs URL mode) are both spelled out rather than implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 94 tool updates
    • First observedaddBetaManager
    • First observedaddScreeningQuestion
    • First observedarchiveBeta
    • First observedcancelInvitations
    • First observedcommitTaskChanges
    • First observedconfigureOptIn
    • First observedconfigureScreening
    • First observedcreateApplicantRule
    • First observedcreateBeta
    • First observedcreateBetaLabels
    • First observedcreateStep
    • First observedcreateTask
    • First observedcreateTicket
    • First observeddeleteApplicantRule
    • First observeddeleteBetaLabel
    • First observeddeleteBetaLabelCategory
    • First observeddeleteScreeningQuestion
    • First observeddeleteStep
    • First observeddeleteTask
    • First observedduplicateBeta
    • First observedflagAnswer
    • First observedgetAnswerAttachment
    • First observedgetApplicantsOverview
    • First observedgetBetaTasks
    • First observedgetBetaTesters
    • First observedgetProductOverview
    • First observedgetScreeningSetup
    • First observedgetStepMedia
    • First observedgetStepPerformanceSummary
    • First observedgetStepResults
    • First observedgetTaskMedia
    • First observedgetTaskPerformanceSummary
    • First observedgetTaskResults
    • First observedgetTesterFunnelDashboard
    • First observedgetTesterFunnelDetail
    • First observedgetTicketAttachment
    • First observedgetTicketDetails
    • First observedgetTicketMessagesBatch
    • First observedinviteApplicants
    • First observedinviteTesters
    • First observedlaunchBeta
    • First observedlistAnnouncements
    • First observedlistApplicants
    • First observedlistBetaLabels
    • First observedlistBetaManagers
    • First observedlistCustomFields
    • First observedlistFlaggedAnswers
    • First observedlistOrganizationTesters
    • First observedlistOrganizationUsers
    • First observedlistPendingInvitations
    • First observedlistProducts
    • First observedlistTesterTags
    • First observedlistTickets
    • First observedlistTicketTags
    • First observedloadRequiredConventions
    • First observedmakeTicketPublic
    • First observedmanageTaskLabels
    • First observedmanageTesterLabels
    • First observedmanageTesterTags
    • First observedmanageTicketTags
    • First observedpauseBeta
    • First observedpingTesters
    • First observedpublishBeta
    • First observedpublishScreeningForm
    • First observedremoveBetaAgreement
    • First observedremoveBetaManager
    • First observedremoveStepMedia
    • First observedremoveTaskMedia
    • First observedremoveTester
    • First observedreorderScreeningQuestions
    • First observedreplyToTicket
    • First observedrestoreBeta
    • First observedresumeBeta
    • First observedrevertTaskChanges
    • First observedreviewApplicants
    • First observedsendAnnouncement
    • First observedsetBetaAgreement
    • First observedsetInvitationEmail
    • First observedsetStepMedia
    • First observedsetTaskMedia
    • First observedsetTesterAgreementBypass
    • First observedsetTicketAssignees
    • First observedupdateApplicantRule
    • First observedupdateBetaDetails
    • First observedupdateBetaLabel
    • First observedupdateBetaLabelCategory
    • First observedupdateScreeningQuestion
    • First observedupdateStep
    • First observedupdateStepsOrder
    • First observedupdateTask
    • First observedupdateTasksOrder
    • First observedupdateTicket
    • First observedupdateTicketState
    • First observeduploadMedia

Publisher details

Operator
Stomio. inc · Publisher source
Vendor relationship
First-party
Restrictions
paid plan

Related MCP Servers

  • A
    license
    Not graded
    quality
    C
    maintenance
    Remote MCP server that lets AI assistants read tracked competitive intelligence — competitor news, reviews, website changes, social posts and ads — and act on it by managing projects, competitors, battle cards, reports and canvases. Every answer carries a source and a date, and write actions only publish when explicitly shared.
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables brand visibility monitoring across major AI platforms like ChatGPT, Claude, Gemini, and Perplexity. It allows users to track visibility scores, analyze competitor data, and receive actionable insights to improve AI-generated brand recommendations.
    16
    23 npm
    1
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Browse IndustryLens's published competitive-intelligence reports and head-to-head competitor comparisons from any AI agent — real, source-backed data.
    MIT
Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources