Server Quality Checklist

Tool Scores

• recall_projectB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure, but it only says 'recall' without detailing return format, sorting, filtering, authentication needs, or what kind of summaries are included. This is a minimal disclosure that leaves agents uncertain about operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no fluff. It efficiently communicates the core purpose, using only seven words while still distinguishing the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists, the description does not explain how the kind or limit parameters affect results, what types of 'prior research' or 'reviews' are included, or how to filter by project. For a retrieval tool with three parameters, this is too incomplete for reliable selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention any of the three parameters (project_id, kind, limit). The agent receives no guidance on how to set or interpret these parameters beyond the schema's basic types and constraints, so the description fails to compensate for the coverage gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('recall') and names the resource type ('prior research, delivery summaries, and reviews for a project'), clearly distinguishing it from sibling tools like research_niche (which creates research) and remember_delivery (which stores delivery info). This gives the agent a precise idea of what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving historical project information but does not explicitly state when to use this tool versus alternatives like inspect_reference or review_implementation. It lacks explicit exclusions or alternative tool references, so guidance is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remember_deliveryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It indicates a write operation ('persist') but does not mention whether it overwrites existing records, whether it is idempotent, if authorization is required, or any side effects. It adds only minimal context beyond the tool's name and purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the key verb ('Persist') and contains no filler or redundant information. Every word contributes to explaining the core functionality, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters and no schema-level descriptions, the description is too sparse to provide complete context. Even though an output schema exists, the description does not explain how the tool fits into the workflow, what constitutes 'delivery', or what data structures are involved. The minimal text leaves important gaps for an agent deciding when and how to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% for 4 parameters, and the description does not compensate. It makes no mention of project_id, summary, files, or research_id, nor explains their roles or relationships. This leaves the input schema as the only source of parameter information, which is insufficient for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('persist') and identifies the resource ('what the agent implemented'). It also states the intended outcome ('so later reviews can compare intent and delivery'), which clearly distinguishes it from sibling tools like review_implementation (which likely performs the comparison) and recall_project (which likely retrieves prior records).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: after implementing something, persist it for later reviews. However, it does not explicitly state when not to use the tool or mention alternatives. It lacks direct guidance on choosing between remember_delivery and its siblings, though the purpose is clear enough to infer the intended timing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• review_implementationB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It introduces 'deterministic' and 'remember' but fails to explain what 'remember' means (e.g., side effects, storage, idempotency) or what the review output looks like. This is significant for a tool that implies state changes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. Every phrase ('deterministic', 'quality review', 'remember actionable feedback') contributes to conveying the purpose. Structurally, it is concise and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives a two-part operation but is vague on the 'remember' side-effect and the scope of the review. Since an output schema exists, return-value details are not needed, but the lack of annotation and incomplete side-effect disclosure leave gaps in understanding the tool's complete behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does not mention project_id or html at all, leaving their semantics to be inferred from the tool name and the word 'HTML'. This is minimal compensation and falls short.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action (run a quality review) on a specific resource (HTML) and adds the distinguishing trait 'deterministic' plus the outcome of remembering feedback. It stands apart from siblings like inspect_reference or research_niche, though it could be more explicit about what 'quality review' entails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: the tool is used when you need to run an HTML quality review and persist feedback. However, it does not explicitly state when not to use it or mention any alternatives among the sibling tools, so it only reaches the 'implied usage' level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• research_nicheB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavior. It states the tool 'returns' specific outputs but does not disclose side effects, required permissions, persistence, or failure behavior, leaving significant behavioral ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundancy or irrelevant detail, effectively front-loaded with the action and deliverables.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters and no annotation context. The description states the high-level purpose but omits how project_id integration works, what 'original components' means, and whether research is persisted. However, the output schema likely covers return structure, so overall completeness is borderline adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%; the description only clarifies the 'niche' parameter by naming it, while leaving 'language', 'project_id', and 'max_references' unexplained beyond their names and schema constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Research' with the resource 'niche' and enumerates output types (references, design patterns, guidance, original components), which clearly distinguishes it from siblings like inspect_reference and recall_project.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing to research a niche, but provides no explicit guidance on when to use this tool over siblings or any exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_referenceA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It implies a read-only inspection of a public page but does not clarify what 'aggregate visual/UI patterns' means, whether any network/fetch limits apply, or how errors are handled. The presence of an output schema mitigates but does not fully compensate for the lack of behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states the tool's function without redundancy or filler. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and an output schema, the description covers the essential purpose and scope. The vague phrase 'aggregate visual/UI patterns' and the lack of usage alternatives prevent a perfect score, but overall it is adequately complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage for the url parameter is 0%, so the description must compensate. It does add the semantic constraint that the page must be 'public,' which goes beyond the parameter name. However, it does not specify URL format or validation rules, leaving some gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'inspect' with a specific resource ('one public page') and the intended outputs ('metadata and aggregate visual/UI patterns'). This distinguishes it from sibling tools like research_niche and review_implementation, which suggest different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for inspecting a single public page but provides no explicit when-to-use versus alternatives, nor any exclusions. The scoping constraint 'one public page' gives some context, but no comparative guidance is offered.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.