Server Quality Checklist

Tool Scores

• spec_statusA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that verification is done entirely offline, which is useful contextual behavior. However, it does not explicitly state whether the operation is read-only or if it has side effects, though 'status' implies a safe query. More explicit disclosure of non-mutating behavior and authorization needs would improve transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently packs the purpose and the returned fields. It is front-loaded with the main query, but the list of fields makes it slightly dense. Still, every word contributes to the meaning, and there is no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has zero parameters and an output schema exists, the description provides enough context about the core purpose and included attributes. It does not mention any prerequisites or failure scenarios, but for a simple status query, the description is sufficiently complete within the existing structured context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so there are no parameters to document. According to the rubric, zero parameters earns a baseline score of 4, and the description adds no irrelevant parameter information. No compensation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly indicates this tool checks for and reports on an approved spec bundle, listing specific fields (version, source phase, approver, story count, approval signature state). It distinguishes itself from sibling tools by focusing on bundle approval status rather than individual stories. However, the question format ('Is there...?') is slightly less direct than a verb like 'Get' or 'List'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for verifying the state of an approved spec bundle, but it does not explicitly state when to use this tool versus siblings like check_story or list_stories. There are no exclusions or alternative tool references. The 'verified entirely offline' hint suggests a context but not comparative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_storyA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It mentions the engine equivalence and return values (pass|fail|needs-signoff), but does not explicitly state whether it modifies anything or any prerequisites, leaving some behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and engine, and contains no filler words. Every phrase adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a single parameter, no annotations, and an output schema present, the description adequately covers purpose, return format, and usage timing. It lacks minor details like prerequisites or edge cases, but remains complete for a simple check tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage for the only parameter story_id is 0%, and the description only refers to 'this story' without explaining the ID format or how to obtain it. Since the schema lacks descriptions, the description should compensate but doesn't.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a story's acceptance criteria using the same engine as the merge gate, specifying the verb 'Run' and the resource 'acceptance criteria'. It distinguishes itself from sibling tools like get_story or spec_status by focusing on the local check before pushing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use it before pushing to avoid discovering failures at the merge, providing clear temporal context. It doesn't name alternatives but the instruction is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_storyA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the returned fields and the key behavioral fact that the criteria are the approved definition of done. However, it does not explicitly state read-only behavior, potential errors, or authorization requirements, though 'Read this' implies a non-mutating operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with front-loaded output summary and a clear usage directive. Every sentence adds value, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the key output fields and usage timing, and an output schema exists for return details. It stops short of explaining error/not-found behavior or explicitly comparing to sibling tools, but for a simple single-id retrieval tool it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one required story_id with 0% schema description coverage. The description does not elaborate on the parameter, leaving the agent to infer that story_id identifies the story. This adds no meaning beyond the parameter name, so it does not compensate for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns 'One approved story in full' with title, epic, and every acceptance criterion. This specifies a verb (retrieve/get), resource (story), and the detailed scope, distinguishing it from sibling tools like list_stories which would provide a listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to read this tool 'BEFORE implementing the story,' establishing the intended usage context. It does not name alternatives or exclusions, but the emphasis on 'approved definition of done' implies this is the authoritative source for implementation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• next_storyA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It mentions the dependency on 'local story state' and the selection logic, but it does not explicitly state whether the operation is read-only or has side effects (e.g., marking the story as in progress). This ambiguity is a notable gap for an agent deciding whether to call this tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no redundancy. The first sentence defines the output and selection criteria; the second provides usage guidance. Every word earns its place, and the structure is front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (no parameters) and presence of an output schema, the description covers the essential aspects: what is returned (next approved, not done), the dependency (local state), and the use case (picking up work). It lacks explicit edge-case behavior (e.g., behavior when no approved story exists) but this is a minor omission for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is empty. The description correctly focuses on behavior rather than parameters. According to the calibration, a zero-parameter tool gets a baseline of 4, and the description does not need to add parameter information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's function: returning 'the next approved story that is not yet done, per local story state.' This specific selection criteria (approved and not done) distinguishes it from siblings like list_stories (all stories) and get_story (a specific story). The verb 'pick up work' implies retrieval of a single next item.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Use this to pick up work without guessing at priorities.' This provides a clear use case and implies this tool is preferable over manual priority guessing from list_stories. However, it does not explicitly name alternative tools or exclude when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_story_statusA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It transparently states that this tool writes only the local tracker-sync artifact and never external trackers or criteria. This reveals side-effect scope clearly, though it does not detail permissions, reversibility, or return behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, and every word adds value. It avoids redundancy and is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple setter with an output schema, the description adequately covers scope, status values, and side-effect limitations. It does not mention prerequisites (e.g., story must exist) or error cases, but these are not critical for basic usage given the output schema exists and sibling tools are available.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds crucial semantics by restricting status to 'doing | review | done', which is not in the schema as an enum. However, it does not explain the optional comment parameter or clarify the story_id format beyond its obvious meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Record story progress', which is a specific verb+resource pairing. It also enumerates the valid status values (doing | review | done) and explicitly scopes the tool to local story state, distinguishing it from sibling tools like check_story or spec_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states when this tool is appropriate (recording local story progress) and provides explicit exclusions: 'never a configured external tracker, and never the approved criteria.' However, it does not name sibling alternatives explicitly, so it stops short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_storiesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses output fields, includes the local status enum, and reveals the key behavioral trait that this list is the sole approved scope. It does not explicitly state read-only behavior or ordering, but the scope is well communicated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, with the most important information front-loaded ('All approved stories'), and no filler. Every word contributes to understanding the tool's output and scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with an existing output schema, the description fully covers purpose, return fields, status semantics, and the concept of approved scope, making it complete without needing to explain return values or parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and schema coverage is 100%, so no parameter clarification is needed. The baseline for no-parameter tools is 4 and the description adds no unnecessary parameter noise.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('list') and resource ('approved stories in this repository'), enumerates the exact fields and status values returned, and clarifies that it represents the approved scope, clearly distinguishing it from sibling tools like get_story or spec_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'The list IS the approved scope' provides clear context for when this tool is appropriate, but it does not explicitly name alternatives or state when not to use it, falling just short of full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.