Server Quality Checklist

Tool Scores

• describe_tableC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. 'Get table structure' implies a read-only operation, but it doesn't specify whether this requires specific permissions, what the output format is (e.g., JSON, table), or if there are any limitations (e.g., rate limits, error handling). The description is too brief to adequately inform the agent about behavioral traits beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just three words, making it front-loaded and efficient. However, it borders on under-specification given the tool's purpose and lack of annotations, which slightly reduces its effectiveness. Every word earns its place, but more detail could improve clarity without sacrificing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a read operation with 2 parameters), no annotations, and no output schema, the description is incomplete. It fails to explain what 'table structure' includes (e.g., columns, data types, indexes), the return format, or any behavioral aspects like error cases. This leaves significant gaps for the agent to understand how to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters ('schema' and 'table') with their types and requirements. The description adds no additional semantic context beyond implying that 'table' is the primary identifier. Since the schema does the heavy lifting, a baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get table structure' clearly states the action (get) and target (table structure), which is better than a tautology. However, it's somewhat vague about what 'structure' entails (e.g., columns, types, constraints) and doesn't distinguish it from sibling tools like 'list_tables' or 'query', which might also provide structural information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a database connection via 'connect_db'), differentiate it from 'list_tables' (which lists table names) or 'query' (which might retrieve data), or specify use cases like schema exploration. This leaves the agent with minimal context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool lists tables, implying a read-only operation, but doesn't specify whether it returns all tables, includes system tables, requires specific permissions, or handles errors. This leaves significant gaps in understanding the tool's behavior beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—a single, clear sentence that directly states the tool's purpose without any unnecessary words. It is front-loaded and efficiently communicates the core functionality, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that interacts with a database. It doesn't address key contextual aspects like what the output looks like (e.g., a list of table names, metadata), error conditions, or dependencies on other tools (e.g., 'connect_db'). This leaves the agent with insufficient information to use the tool effectively in complex scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'schema' documented as 'Schema name (default: public)'. The description adds no additional meaning about parameters beyond what the schema provides, such as explaining the significance of the schema parameter or how it affects the listing. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('tables in the database'), making the tool's purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'list_schemas' or 'describe_table', which would require specifying what makes this listing operation unique.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a database connection via 'connect_db'), nor does it explain how it differs from 'list_schemas' (which might list schemas instead of tables) or 'describe_table' (which might provide detailed metadata).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Execute a SELECT query') but doesn't describe what happens: whether it returns results, error handling, performance implications, or security constraints (e.g., read-only access). For a query execution tool, this leaves critical behavioral traits unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it immediately understandable without unnecessary elaboration. Every word earns its place by directly conveying the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a database query tool with no annotations and no output schema, the description is incomplete. It doesn't explain return values (e.g., result sets, error formats), usage context (e.g., requires prior connection), or limitations (e.g., query timeout, row limits). For a tool that executes arbitrary SQL, this leaves significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('sql' and 'params') with clear descriptions. The description adds no additional meaning beyond what the schema provides, such as examples of valid SQL syntax or parameter binding details. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Execute') and resource ('SELECT query'), making the purpose unambiguous. It distinguishes from siblings like 'connect_db' or 'list_tables' by focusing on query execution rather than connection or metadata listing. However, it doesn't explicitly differentiate from 'execute' which might handle non-SELECT queries, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'execute' (which might handle INSERT/UPDATE) or 'describe_table' (for schema inspection). It lacks context about prerequisites (e.g., requires an established database connection) or exclusions (e.g., only for SELECT queries, not data modification).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_schemasB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states it's a list operation but doesn't mention whether it requires authentication, returns paginated results, includes metadata, or has any side effects. For a tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core functionality without any wasted words. It's appropriately sized for a simple list operation and front-loads the essential information immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is insufficiently complete. It doesn't address what the return format looks like (e.g., list of schema names, full schema objects, pagination), authentication requirements, or error conditions. For a database tool with no structured metadata, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the schema already fully documents the input structure. The description appropriately doesn't add parameter information beyond what's in the schema, maintaining focus on the tool's purpose rather than redundant details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('all schemas in the database'), making the purpose immediately understandable. It doesn't explicitly distinguish from sibling tools like 'list_tables' or 'describe_table', but the resource specificity (schemas vs tables) provides implicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'list_tables' or 'describe_table'. The description only states what it does without context about appropriate scenarios or prerequisites, leaving the agent to infer usage from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• executeB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool executes queries but doesn't disclose critical behavioral traits: whether it requires authentication, what permissions are needed, if changes are reversible, transaction behavior, error handling, or what happens on success/failure. For a mutation tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (one sentence) with zero waste. It's front-loaded with the core purpose and includes essential parameter syntax guidance. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what happens after execution (e.g., returns row count, success/failure, error messages), security implications, or transactional behavior. For a tool that modifies data, this leaves critical gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (sql and params) with their types and purposes. The description adds minimal value beyond what the schema provides, mentioning the same query types and parameter placeholders ($1, $2). Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool executes SQL queries (INSERT, UPDATE, DELETE), providing specific verb+resource. It distinguishes from the 'query' sibling tool by specifying the types of queries it handles (data manipulation vs. likely SELECT queries). However, it doesn't explicitly mention database operations or contrast with all siblings like 'connect_db'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for data modification queries (INSERT/UPDATE/DELETE) versus the 'query' tool likely for SELECT queries, but doesn't explicitly state when to use this tool versus alternatives. No guidance on prerequisites (like needing an established connection) or exclusions is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• connect_dbA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the existence of a default connection and failure conditions, which adds useful behavioral context. However, it doesn't disclose other important traits like authentication requirements (implied by password parameter), potential side effects of establishing connections, or error handling behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with zero waste. The first states the purpose, the second provides crucial usage guidance. Every word earns its place, and the structure is front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a connection tool with 5 parameters and no output schema, the description provides good contextual completeness. It covers purpose and usage guidelines well. The main gap is lack of output information (what happens after connection), but given the tool's nature and absence of output schema, the description does reasonably well.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. This meets the baseline of 3 when schema does the heavy lifting, but no extra value is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Connect to PostgreSQL database') and resource ('PostgreSQL database'), making the purpose immediately understandable. It doesn't explicitly distinguish from sibling tools like 'execute' or 'query', but the connection focus is clear enough to avoid confusion with execution or querying operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Default connection exists - only use when requested or if other commands fail.' This clearly indicates when to use (when requested or after failure) and when not to use (when default connection works), helping the agent avoid unnecessary invocations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.