Server Quality Checklist

Tool Scores

• executeB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only mentions that no results are returned, but crucially omits that executing DDL/DML can be destructive, requires certain permissions, may affect database state, and does not indicate error handling or side effects. This lack of transparency is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently conveys the core purpose and a key constraint (no results). It is front-loaded and without wasted words. However, it could be slightly restructured or bulleted for clarity, but overall it is concise and effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of output schema and annotations, the description should provide more contextual completeness. It fails to mention return value on success (e.g., affected rows), error behavior, prerequisites (e.g., user attachment), or whether the operation is idempotent. For a potentially destructive SQL execution tool, this is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters adequately. The tool description adds minimal value beyond what's in the schema—it reiterates that the SQL is DDL/DML and that userId is per user. This meets the baseline for high schema coverage but does not exceed it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Execute', the resource 'SQL statement (DDL/DML)', the scope 'for a specific user', and the outcome 'without returning results'. This effectively distinguishes it from sibling tools like 'query' which likely return results, and from administrative tools like 'detachUser' or 'listUsers'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings, especially 'query'. It does not mention that 'execute' is for DDL/DML operations without result sets, while 'query' is for SELECT statements that return results. There are no when-not-to-use instructions or alternatives listed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• queryB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry full behavioral burden. It only states it executes and returns results, but does not disclose whether queries are read-only, destructive, or require specific permissions, which is critical for a SQL tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the verb. However, it could be expanded with more context without harming conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of SQL execution and lack of output schema or annotations, the description is inadequate. It omits details on error handling, permission requirements, and result format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the schema already describes both parameters adequately. The description adds little beyond 'for a specific user', which is already hinted in the userId description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Execute' and the resource 'SQL query', and specifies it is for a specific user, which distinguishes it from a general SQL execution tool like 'execute'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this versus sibling tools like 'execute' or 'getUserStats'. The description implies user-specific queries but does not provide explicit usage context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• isAttachedB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must bear the full burden. It indicates a read-only check (no side effects), which is appropriate. However, it does not disclose error conditions, authentication requirements, or the meaning of 'cached'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently communicates the tool's purpose. It is appropriately sized for a simple check operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks information about the return value (expected to be boolean) and error handling (e.g., if user not found or default user used). Given the simplicity, it is moderately complete but could be enhanced.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with a description for 'userId'. The tool description adds no extra context beyond what the schema already provides, so a baseline score of 3 is warranted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('check') and the resource ('user's database cached in memory'). It effectively distinguishes from sibling tools like 'detachUser' and 'execute' by implying a read-only query, though it could be more explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., 'detachUser' or 'listUsers'). The agent is left to infer usage context from the name and siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getUserStatsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavioral traits. It only states that the tool gets statistics, but does not indicate whether it is read-only, requires authentication, has rate limits, or is destructive. The mention of 'etc.' is vague.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that concisely conveys the tool's purpose. There is no redundant or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and only one optional parameter, the description is minimally adequate. It tells the agent what the tool does, but lacks details on return format, error handling, or usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100% (the userId parameter has a description). The tool description does not add significant meaning beyond the schema, but it aligns with the parameter by mentioning 'a user's database'. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get statistics about a user's database (memory usage, query count, etc.)'. It uses a specific verb (Get) and resource (statistics about a user's database), and distinguishes it from sibling tools like detachUser, execute, and listUsers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, context, or scenarios where this tool is appropriate, nor does it exclude cases where other tools should be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• detachUserA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry full behavioral disclosure. It mentions detaching and freeing resources, but does not detail side effects, error conditions, or required state (e.g., user must be attached).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no extraneous information, efficiently conveying the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema or annotations, the description covers the core functionality. However, it lacks details on prerequisites and error states, which slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single optional parameter 'userId'. The tool description does not add additional meaning beyond the schema, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'detach', the resource 'user\'s database', and the outcome 'free resources'. It distinguishes from sibling tools like 'isAttached' and 'query'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for freeing resources by detaching, but does not explicitly state when to use or avoid, nor does it mention alternatives or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• listUsersA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It states 'list all currently cached users and cache statistics', which suggests a read-only operation, but it does not disclose whether there are side effects, authentication needs, rate limits, or exactly what 'cache statistics' entails. The description is adequate but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It is front-loaded with the core action and result. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description does not elaborate on the format or content of the returned data, especially 'cache statistics'. For a simple listing tool the description is minimally adequate, but it could be more complete by specifying what statistics are included.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so schema coverage is trivially 100%. The description adds no parameter-level detail because there are none. According to guidelines, 0 parameters warrants a baseline of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' and resource 'all currently cached users and cache statistics', clearly distinguishing from sibling tools like detachUser, execute, getUserStats, isAttached, and query. It precisely communicates the tool's scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need a list of cached users and overall cache stats, but it does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives among siblings. Only implied context is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.