Server Quality Checklist

Tool Scores

• find_customerA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It implies a read operation ('search') but does not explicitly state safety, rate limits, or side effects. It adds some behavioral context (returns oid, next step) but not full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: first states purpose and inputs, second states output and next step. No fluff, front-loaded, and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter search tool with no output schema, the description covers the main inputs, output (oid), and next step. However, it omits mention of the limit parameter and details like pagination, error handling, or return structure. Adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%: only the 'name' parameter has a description with examples. The tool's description repeats that 'name' can be partial or ID but adds no new meaning. The 'limit' parameter lacks description in both schema and tool description, so the description fails to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for customers by partial name or ID, and distinguishes from siblings by specifying the return of an internal oid for use with customer_insights.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit context: use partial name/id to search, then pass oid or name to customer_insights. It does not explicitly state when not to use, but the guidance is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• customer_insightsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It enumerates the returned data but omits details like read-only nature, permissions, or performance implications. It adequately describes output components but lacks explicit behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loading the core purpose and listing returned data. The example at the end adds clarity without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two parameters and no output schema, the description sufficiently explains what the tool returns. It covers the main components but could be more precise about edge cases or default behavior for the notes_limit parameter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema describes both parameters fully, achieving 100% coverage. The description adds no additional meaning beyond the schema, so it meets the baseline without extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: returning a consolidated insight profile for a customer, listing specific components (identity, contracts, etc.) and providing an example query. It distinguishes from siblings like find_customer, which likely returns basic info.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a concrete example ('get the insights of the customer Max'), giving clear context for when to use this tool. However, it does not explicitly mention when not to use it or suggest alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility. It declares read-only behavior and automatic LIMIT, but does not detail error handling, performance, or return format. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences front-load the core action, table context, and parameter usage. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and simple tool purpose, description adequately covers read-only constraint, table schema hint, parameter binding, and automatic limit. Minor gap on error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all 3 parameters (100% coverage). Description enhances meaning by explaining ? placeholders with params array and default max_rows. Adds value over schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it executes read-only SQL queries against DuckDB, distinguishing it from sibling tools that provide predefined insights or table metadata. The verb 'Execute' and resource 'read-only SQL query' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for custom SQL queries but does not explicitly state when to use this tool over siblings like customer_insights or describe_table. Absence of when-not or alternative guidance limits clarity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_tableA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description fully discloses the tool's behavior: it returns column names and data types, and it works on tables or views. No destructive side effects are implied. However, it omits details like permission requirements or scope limits, which is acceptable for a simple read-only schema tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long with no extraneous words. Every sentence adds value: first states functionality, second gives usage guidance. It is perfectly concise for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter, full schema coverage, no output schema, and clear sibling context, the description covers all necessary aspects: purpose, when to use, and what to expect. No further details are needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the parameter 'table' is already described in the schema. The description adds no extra parameter-level meaning beyond what the schema provides (e.g., the example 'dms_customer' is already in schema). Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Return the column names and data types for a given table or view', which identifies the specific verb (return) and resource (schema of a table/view). It uses 'describe_table' and provides a use case distinct from sibling tools like list_tables (which lists table names) and query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises 'Use this to learn the schema before writing a query', giving a clear context for when to invoke the tool. It does not list when not to use or mention alternatives, but the purpose is well-directed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool lists all tables and views (a read-only operation) and includes row counts. It does not mention potential limitations like large result sets or ordering, but for a simple listing tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with action, no fluff. Every sentence adds value: the first states what it does, the second advises when to use it.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and no output schema, the description is mostly complete. It tells the agent to use it first for discovery. It could specify the return format (e.g., list of names with counts) but is sufficient for an effective choice.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so per guidelines baseline is 4. The description does not need to add parameter meaning as the schema is empty and coverage is 100%.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all tables and views in the warehouse with row counts, using a specific verb 'List' and resource 'tables and views'. It distinguishes from sibling tools like describe_table (describes a specific table) and query (queries data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this first to discover what data is available', giving a clear when-to-use instruction. However, it does not explicitly mention when not to use or provide alternatives, though context from siblings implies describe_table for specific tables.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.