Server Quality Checklist

Tool Scores

• compass_decide_fitA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint, idempotentHint, and openWorldHint, which the description does not contradict. The description adds valuable behavioral context: it is 'conservative' and returns 'unknown' rather than overclaiming, which goes beyond the annotations by clarifying the decision philosophy.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the key action and outputs, and contains no wasted words. Every sentence contributes essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters with a nested object, annotations, output schema), the description adequately covers the core behavior. It explains the conservative nature and the three possible outcomes. Missing details like what 'reason codes' and 'evidence' entail are likely covered by the output schema, so the description is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema description coverage is 100%, so the schema already documents all parameters. The description adds high-level behavioral context (conservatism, 'unknown' return) but no additional parameter-specific meaning beyond what the schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'decide' and resource 'restaurant fit', and clearly states it returns 'fit', 'not_fit', or 'unknown' with additional artifacts. It distinguishes itself from the sibling tools (compass_search finds restaurants, compass_enrich_restaurant enriches data) by focusing on the fit decision.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a conservative decision approach and warns against overclaiming, but does not explicitly state when to use this tool versus its siblings or any prerequisites. It lacks explicit context on when not to use it, leaving the agent to infer from the purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compass_searchA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond readOnlyHint and idempotentHint from annotations, the description adds key behavioral details: conservative handling by returning 'unknown' when evidence is insufficient and not returning certification/free-from facts.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with action and result, no wasted words. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a fully described schema and output schema (assumed), the description covers key behavior and results. Minor gap: how to use results with siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description only adds context to the query parameter via 'natural-language dietary query'; no additional meaning for other parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches restaurants by natural-language dietary query, returning ranked results with VeganScore, evidence, and confidence. It distinguishes from siblings (decide_fit, enrich) through its search focus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus siblings. It implies use for dietary queries but lacks when-not or alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compass_enrich_restaurantA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly, idempotent, and open world. The description adds value by disclosing that the tool does not return certification/free-from facts, and explains the matching confidence threshold behavior. This provides useful behavioral context beyond what annotations offer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each adding essential information: purpose and returned data, exclusions, and low-confidence behavior. It is front-loaded and concise without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description covers all necessary context: purpose, input options, constraints, and edge-case behavior. It is complete for a read-only enrichment tool with no need for additional state or authentication details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are well-documented. The description adds semantic value by explaining that google_place_id provides the highest match confidence, and that compass_id is for direct lookup. This enhances the agent's understanding of parameter use.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specific verb 'Match' and resource 'restaurant enrichment data' clearly defines the tool's function. It lists returned data types (VeganScore, vegan dietary profile, evidence) and explicitly states what is not returned (certification/free-from facts), providing a precise purpose that distinguishes it from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the input parameters (name and address as primary matching; compass_id or google_place_id as alternatives) and describes the behavior when confidence is low (returns 'matched: false' with candidates). While it doesn't explicitly state when not to use this tool, the context for enrichment vs. search/decide is implied by the sibling tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.