Server Quality Checklist

Tool Scores

• clear_cacheA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond annotations: it explains that clearing is 'granular' for both individual items and list caches, which helps the agent understand the tool's capabilities. While annotations cover idempotency and non-destructive nature, the description provides practical usage context that complements them without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with three sentences that each add distinct value: stating the core purpose, providing usage context, and explaining granular capabilities. No wasted words, and information is front-loaded with the primary function stated first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with comprehensive annotations and full schema coverage, the description provides adequate context about purpose, usage scenarios, and behavioral characteristics. The main gap is the lack of output schema information, but given the tool's relative simplicity and good annotation coverage, the description is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema already fully documents the single parameter's purpose, enum values, and default. The description mentions 'granular cache clearing for both individual items and list caches' which aligns with but doesn't significantly expand upon the schema's detailed enum descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Clear') and target resource ('internal cache for stored data'), distinguishing it from sibling tools like list, retrieve, or search. It provides additional context about what the cache contains (stored data) and the granularity of clearing operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use the tool ('Useful for debugging or when you know the data has changed'), giving clear context for its application. However, it doesn't specify when NOT to use it or mention alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• listA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already cover read-only, non-destructive, idempotent, and open-world properties. The description adds valuable behavioral context beyond annotations: 'intelligent caching for performance', 'pagination for large datasets exceeding token limits', and 'returns only essential identifier fields for efficient browsing'. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with core purpose in the first sentence, followed by supporting details. Every sentence adds value: scope of models, return format, performance features, and pagination support. No wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (list operation with pagination), rich annotations, and 100% schema coverage, the description is largely complete. It explains the tool's optimization approach and caching behavior. The main gap is lack of output schema, but the description partially compensates by stating 'returns only essential identifier fields'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing full parameter documentation. The description adds minimal extra semantics, mentioning 'highly optimized responses' and 'intelligent caching' which relate to overall tool behavior rather than parameter specifics. Baseline 3 is appropriate as the schema carries the burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('fetch all records') and resource ('single Metabase resource type'), specifying the exact models supported (cards, dashboards, tables, databases, collections). It distinguishes from siblings like 'search' by emphasizing 'complete lists' for 'overview purposes' rather than filtered results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('for overview purposes', 'efficient browsing', 'highly optimized responses'), but does not explicitly state when not to use it or name alternatives like 'search' or 'retrieve' from the sibling list. It implies usage for bulk retrieval vs. specific lookups.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• retrieveA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond annotations: 'intelligent concurrent processing', 'optimized caching', and 'table pagination for large databases exceeding token limits'. While annotations cover safety (readOnlyHint, destructiveHint) and idempotency, the description provides practical implementation details that help the agent understand performance characteristics and limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in three sentences: purpose statement, key capabilities/limitations, and special handling for edge cases. Every sentence adds value without redundancy, and the most important information (what it does) comes first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the rich annotations (readOnlyHint, openWorldHint, idempotentHint) and comprehensive schema coverage, the description provides good contextual information about capabilities and limitations. The main gap is the absence of an output schema, but the description compensates somewhat by indicating what kind of details will be fetched. For a read-only retrieval tool, this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the schema already documents all parameters thoroughly. The description mentions 'multiple IDs (max 50 per request)' and 'table pagination for large databases' which aligns with but doesn't significantly expand upon the schema's parameter descriptions. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Fetch') and resource ('additional details for supported models') with specific model types listed. It distinguishes from siblings like 'list', 'search', and 'export' by focusing on retrieving details for specific IDs rather than listing, searching, or exporting data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (fetching details for specific models with IDs) and mentions limitations (max 50 IDs, pagination for large databases). However, it doesn't explicitly state when NOT to use it or name specific alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds valuable behavioral context beyond annotations: it specifies what types of content are searched (cards, dashboards, tables, etc.), mentions the return format includes 'search metrics, recommendations, and clean results organized by model type,' and indicates this is a comprehensive search tool. Annotations already cover safety (readOnly, non-destructive, idempotent) and openness, so the description appropriately focuses on operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in three sentences: first states the core functionality and scope, second provides usage guidance, third describes the return format. Every sentence adds value with zero waste, making it front-loaded and appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the comprehensive annotations (readOnly, openWorld, idempotent, non-destructive) and 100% schema coverage, the description provides good contextual completeness. It explains the tool's purpose, when to use it, and what it returns. The main gap is the lack of output schema, but the description partially compensates by mentioning return content organization.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema already thoroughly documents all 9 parameters including their purposes, defaults, and restrictions. The description doesn't add any parameter-specific information beyond what's in the schema, so it meets the baseline of 3 where the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches across all Metabase items using the native search API, listing specific resource types (cards, dashboards, tables, collections, databases, and more). It distinguishes from siblings by specifying this is the primary search tool ('Use this first for finding any Metabase content'), unlike list or retrieve tools that might fetch specific items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides usage guidance: 'Use this first for finding any Metabase content.' This tells the agent when to prefer this tool over alternatives like list or retrieve. It establishes this as the primary search mechanism for broad content discovery.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• executeA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructiveHint=true and readOnlyHint=false. The description adds explicit security warnings about destructive SQL operations, mentions read-only mode rejection, and clarifies behavior for both modes. This goes beyond annotations to provide essential safety context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise (~150 words), front-loaded with primary purpose, then logically organizes modes, limitations, warnings, and exceptions. Every sentence serves a purpose—no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool complexity (dual modes, security concerns, no output schema), description covers all critical aspects: modes, parameters, row limits (and export alternative), security warning, and read-only mode handling. Leaves no major gaps for safe usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema documents all parameters. Description adds context by explaining the card_parameters format in detail and clarifying mode-specific parameters (e.g., database_id for SQL only). Slightly redundant on row_limit since schema already has bounds, but overall adds value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states it is a 'Unified command to execute SQL queries or run saved cards', specifying verb (execute) and resources (SQL queries, saved cards). Distinguishes two clear modes and sets expectations for row limits, making it highly specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides guidance on when to use Card vs SQL mode based on filter needs, and mentions export tool for larger datasets. However, it does not compare to sibling tools (clear_cache, search, retrieve, list) or state when not to use this tool, leaving room for ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.