Server Quality Checklist

Tool Scores

• execute_sqlC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavioral traits. However, it omits critical details such as whether the query is read-only or can mutate data, permission requirements, or error behavior, leaving the agent unaware of potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (8 words) but at the cost of crucial information. It lacks any structure to convey important context, making it under-specified rather than efficiently concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, SQL execution), the absence of annotations, and the fact that an output schema exists but is not referenced, the description fails to provide adequate context about return format, pagination, or behavior, leaving significant gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond the schema's parameter descriptions, but does not detract either.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Execute' and the resource 'SQL query against a Datasette instance', accurately capturing the tool's function. It effectively distinguishes from siblings like describe_database and search_table which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like search_table. There is no mention of safety considerations (e.g., whether SQL can modify data) or prerequisites, leaving the AI agent without decision-making context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_tableC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It only states 'full-text search' without clarifying whether the operation is read-only, if it requires authentication, or any side effects like logging. The mention of 'Datasette's search functionality' is vague and assumes prior knowledge.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that immediately states the core action ('search') and the target resource ('table'). It is front-loaded and contains no extraneous words, achieving maximum conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 11 parameters, including pagination (next_token) and optional column filters, the description is too brief. It does not explain how the tool handles pagination, the output shape, or the behavior of parameters like 'raw_mode'. An output schema exists but is not referenced, leaving the agent underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All parameters have descriptions in the schema (100% coverage), so the description does not need to add much. It adds no parameter-specific meaning beyond what the schema already provides, earning the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'full-text search within a table' using Datasette's search functionality. However, it does not differentiate from sibling tools like 'execute_sql' which could also perform searches, leaving ambiguity about when to use this specific tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. For instance, it does not explain when to prefer search_table over execute_sql for searching, or mention any prerequisites or constraints.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_databasesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It only states the action without confirming it's read-only, safe, or idempotent. It does not mention whether authentication is needed, rate limits, or any side effects. This leaves an agent uncertain about safety and constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, complete sentence that efficiently conveys the tool's purpose. It is front-loaded and contains no unnecessary words, earning its place as a concise and well-structured definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a simple function (list databases) and an output schema (existence noted in context), so the description minimally covers functionality. However, it lacks behavioral context such as read-only guarantee, pagination, or error conditions. With no annotations, the description should provide more completeness to fully inform the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (the single parameter 'instance' has a clear schema description: 'Name of the Datasette instance (from config)'). The tool description adds no extra meaning beyond the schema, merely restating the instance context. With full coverage, the baseline is 3, and no additional value is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List all databases in a Datasette instance' clearly states the action (list), the resource (databases), and the scope (a specific instance). It effectively distinguishes from sibling tools like describe_database (specific db), execute_sql (queries), list_instances (instances), and search_table (searches table) by focusing on listing all databases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings. It does not mention prerequisites, limitations, or context such as 'use this before describe_database to discover available databases'. Without such context, an AI agent may not understand the best scenario for invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_databaseA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only states a read operation but omits permissions, rate limits, or performance implications (e.g., potentially heavy schema retrieval).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that front-loads the purpose and scope. No filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity, output schema present, and full parameter coverage, the description is largely sufficient. It would benefit from mentioning that it returns all schemas for the database, but it's already adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (instance and database have descriptions). The description does not add new parameter meaning beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves complete database metadata including table schemas and column information. It distinguishes from sibling tools like execute_sql (executes queries) and list_databases (lists databases).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when/when-not guidance is provided. The context of sibling tools implies usage but the description does not help choose between describe_database and search_table for column-level details.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_instancesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description accurately indicates a read-only listing operation. It states it returns configuration details, which implies no side effects. However, it does not explicitly declare safety (e.g., read-only), but the behavioral intent is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences, front-loading the core action. Every word is essential, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool with an output schema present, the description is complete. It covers purpose and return value sufficiently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters and schema coverage is 100%. The description adds no parameter details, but none are needed. Baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all configured Datasette instances. The verb 'list' and resource 'Datasette instances' are specific, and the tool differentiates well from siblings that deal with databases or SQL.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like describe_database or list_databases. It simply states what it does, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.