Server Quality Checklist

Tool Scores

• describe_tableA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It adds an important guarantee: 'Returns actual SQLite metadata only — never fabricates fields such as country', which communicates trustworthiness and avoids hallucinated results. It also describes the return shape. It does not explicitly state that the operation is read-only, but 'returns metadata' strongly conveys that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: purpose first, then when to use it, then parameter provenance, then output shape. Every sentence earns its place, with no filler or repeated structural information beyond what is useful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one string parameter, an output schema exists, no annotations), and the description gives enough to invoke it correctly: what it returns, where the parameter value comes from, and why to use it before joins. It could add explicit note about error behavior for invalid table names, but that is not essential for a straightforward describe/metadata tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides only a type string and a title 'table', so the description adds crucial meaning: table is 'the name from list_tables'. This tells the agent where valid values come from, effectively linking describe_table to list_tables. That is exactly the kind of semantic enrichment the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Describe') and precise resources (columns, types, primary keys, foreign keys) for 'one table', clearly distinguishing it from list_tables (listing) and execute_readonly_sql (querying). The phrase 'one table' prevents confusion with table-listing siblings, and the mention of metadata-only output further clarifies its exact scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use before writing joins', which gives a clear context and timing for using this tool. It also states that the table name should come from list_tables, effectively providing a prerequisite. However, it does not explicitly name alternatives or say when not to use this tool (e.g., when listing tables or running arbitrary SQL).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the behavioral burden well: it mentions internal SQLite tables are omitted and that columns are not invented. This guards against false expectations, though it does not address side effects or access requirements, which are minor for a read-only listing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Each sentence earns its place: purpose, usage timing, exclusions, anti-invention behavior, and return shape. The description is front-loaded and compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool with an output schema, this description is complete. It tells the agent when to use it, what it will not include, and what the response looks like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so detailed parameter descriptions are unnecessary. The description usefully documents the output contract instead: a table list of names and purposes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb and resource: 'List user tables in shop.db', and clarifies the output is a one-line purpose per table. It also differentiates itself by explicitly targeting user tables rather than internal ones, which separates it from schema/query tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Use this first to discover the real schema' gives a clear and explicit usage signal. It does not, however, explicitly state when it should not be used or name alternatives like describe_table or execute_readonly_sql.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shop_analyticsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full behavioral burden. It reveals important behavior beyond a naive reading: spend, units, and revenue are computed as quantity * unit_price, cancelled orders are excluded, and year is taken from order_date. It also implies a read-only aggregation nature, though it doesn't explicitly state side-effect safety, cancellation behavior, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, structured, and front-loaded with the main purpose. It lists operations in a scannable format, then adds only the necessary computation caveats. Every sentence carries substantive information with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an output schema exists, the description doesn't need to formally define return fields, and it does cover operation options, defaults, a special requirement, and calculation semantics. Remaining minor gaps are ambiguous behavior such as how customer_most_orders handles the limit parameter and what happens when an invalid combination of operation and year is given, but these do not block correct use in common cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 0% description coverage, so the description must explain the parameters, and it does: operation values map to concrete aggregations, limit has per-operation defaults, year is required for revenue_by_year, and all calculation semantics are explained. This goes well beyond the bare enum/schema and provides enough detail to invoke each operation correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that this tool provides shop aggregations without hand-written SQL, and it enumerates exactly five named operations with their output fields. This distinguishes it from the SQL-oriented sibling tools (execute_readonly_sql) while making the tool's exact scope immediately evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without hand-written SQL' effectively tells the agent to use this tool for shop aggregation reporting instead of writing SQL, and it names concrete operations such as top_customers_by_spend and revenue_by_year. It provides explicit per-operation constraints like the year requirement and default limits, though it does not explicitly state when to use a sibling tool like describe_table or execute_readonly_sql.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_readonly_sqlA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does so thoroughly: it discloses that the database is opened read-only, that refused mutations leave the file unchanged, states that stacked statements are rejected, and explains pagination defaults and the response format. This goes well beyond what the schema or annotations alone would reveal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but structured: it front-loads the primary action, then lists allowed/rejected statements, pagination behavior, and return contract. Every sentence contributes a distinct constraint with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter tool with an output schema and no annotations, the description covers the SQL grammar constraints, safety guarantees, pagination limits, error behaviors, and guidance for choosing a sibling tool. Nothing essential to correctly invoking it is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by defining the 'sql' parameter in terms of allowed statement types and by providing concrete limits for 'limit' (default 100, max 1000) and 'offset' (default 0). All three parameters gain usable meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with an exact verb and resource: 'Run a single read-only SQL statement against shop.db.' It further specifies the allowed statement forms (SELECT, WITH/CTE, EXPLAIN) and explicitly distinguishes itself from the shop_analytics sibling for rankings/revenue, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to 'Prefer shop_analytics for rankings and yearly revenue,' naming an alternative and the scenario where it should be used. It also sets clear boundaries for this tool by enumerating what is allowed (read-only SELECT, WITH, EXPLAIN) and rejecting writes/DDL and stacked statements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.