Server Quality Checklist

Tool Scores

• list_scenariosA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds that it lists 'every published situation' (a scope detail) and the returned fields. However, it does not discuss pagination, ordering, or potential edge cases (e.g., empty list). The behavioral disclosure is adequate but minimal beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 12 words, front-loaded with the verb 'List'. It conveys essential information (what is listed, what fields) without any extraneous content. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, an output schema exists), the description is nearly complete. It states the scope ('every published situation') and the fields included. Minor missing context includes ordering, default listing behavior, and what happens if no situations exist, but the presence of an output schema likely covers the return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters, so there is nothing to document. The description does not need to add parameter semantics. The baseline for 0 parameters is 4, and the description meets this without requiring further elaboration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and the resource ('every published situation'), and specifies the output fields (title, category, slug, canonical URLs). This distinguishes it from siblings route_emergency (routing) and get_scenario (single scenario retrieval).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like get_scenario. It does not mention exclusions, prerequisites, or typical use cases. With sibling tools present, such context would be valuable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• route_emergencyA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint=true, idempotentHint=true, destructiveHint=false, so no contradiction. The description adds valuable behavioral context beyond annotations: it explains the return structure (who to call, order, first steps, costs, dispatch phone) and the conditional availability of 24/7 dispatch numbers. This is informative but does not cover edge cases like unrecognized problems or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, each earning its place. The first sentence front-loads the core functionality and output, while the second adds the optional parameter nuance. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (2 params, annotations present, output schema exists), the description covers the essential aspects: input format, output contents, and optional state usage. It does not explain output schema shape (but that is handled by output schema), nor does it address potential failures or multi-match cases. Almost complete for the intended use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with basic descriptions for both parameters. The description adds meaningful context for the 'state' parameter, noting it enables region-aware dispatch numbers, which the schema description does not convey. For 'problem', the description restates the schema's intent without adding detail, so the net improvement is moderate but solid.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: it routes a home emergency by returning matching situations with who to call, order, first steps, costs, and dispatch phone. The verb 'route' is specific and the scope is well-defined. It implicitly distinguishes from siblings (get_scenario, list_scenarios) by focusing on actionable routing rather than simply fetching or listing scenarios.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a user describes a home emergency in plain words, and optionally when a US state is known. However, there is no explicit guidance on when not to use this tool versus alternatives (e.g., get_scenario might be preferred for retrieving a single scenario without dispatch info). The usage context is clear but lacks exclusions or differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_scenarioA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, idempotentHint=true, destructiveHint=false. Description adds useful behavioral context by specifying the return format (structured answer + markdown guide). However, it doesn't mention performance characteristics or any dependencies on other tool calls if inputs are not from siblings.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise (two sentences) and front-loaded with the primary purpose. Every word adds value - the first sentence explains what is returned, the second tells where inputs come from. No redundant language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (2 required string params, no enums, no nested objects), the presence of both input and output schemas, and comprehensive annotations, the description is complete. It clearly explains what the tool returns and how to properly call it, leaving no ambiguity for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already covers both parameters with 100% coverage (category and slug). Description adds value by stating they are 'category slug' and 'scenario slug' with examples like 'water' and 'ceiling-leaking', which clarifies the role of each field beyond their type definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it fetches one scenario with full details including structured answer, costs, FAQ, sources, and step-by-step guide as markdown. It distinguishes itself from siblings by specifying it retrieves a single scenario in full versus list_scenarios that lists multiple scenarios, and route_emergency which handles routing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use category and slug values from route_emergency or list_scenarios, providing clear context on when to invoke this tool and where to get the required input parameters. This avoids confusion with its sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.