Server Quality Checklist

Tool Scores

• explain_queryA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosing behavior. It does not state whether the query is actually executed (e.g., EXPLAIN vs. EXPLAIN ANALYZE), whether it is read-only, or what permissions are required. This lack of disclosure is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences that front-load the core purpose and then add a use case. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The presence of an output schema covers return values, but the description misses important context such as whether the plan is estimated or actually executed, any prerequisites, or limits. While simple, the tool's behavior is under-specified for a no-annotation case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only 'sql' with no description, and the description adds that it must be a SELECT statement, which is a key constraint. However, it does not explain syntax, options, or edge cases, so it provides only partial compensation for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Return the Postgres query plan for a SELECT statement.' This is specific and distinguishes it from siblings like run_query (which executes queries) and describe_table (which describes table structures).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Useful for debugging slow queries,' providing clear context for when to use the tool. However, it does not explicitly name alternatives or state when not to use it, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_tableA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the output scope (columns, types, nullability) but does not mention whether it is a read-only operation, any potential errors, or performance considerations. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long and front-loaded with the core action. Every word earns its place, with no unnecessary padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, and the output schema presumably defines the return format, so the description does not need to explain return values. It covers the purpose, usage, and basic behavior. It is complete enough for a straightforward describe operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, and the description does not explain the 'table' or 'schema' parameters. While 'table' is implied, 'schema' (with a default of 'public') is not mentioned at all, so the description fails to compensate for the lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to describe a table's columns, types, and nullability. This is a specific action on a specific resource, and it distinguishes itself from sibling tools like 'list_tables' (which lists tables) and 'sample_rows' (which returns data).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this before writing a query against an unfamiliar table,' which provides clear context for when to use the tool. However, it does not mention when not to use it or explicitly compare to alternatives, but this is sufficient for a simple introspection tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sample_rowsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the row limit (max 50) but does not explicitly state that the operation is read-only, safe, or how sampling is performed (e.g., random vs. arbitrary). It implies non-destructive use but does not explicitly confirm the behavioral profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, and every sentence adds value. No waste or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and the description covers its main purpose and constraint. However, it omits the `schema` parameter and does not explicitly confirm read-only behavior. Given an output schema exists, return-value documentation is not needed, so the description is mostly complete but has a couple of minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the `limit` and `table` parameters, but the `schema` parameter is not mentioned at all. This partial compensation places it at a mid-level score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Return up to `limit` sample rows from a table', using a specific verb and resource. It also differentiates from sibling tools like run_query (full query execution) and describe_table (schema inspection) by emphasizing the 'sample' aspect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on when to use the tool: 'before running real analytical queries'. It implies the alternative is to use a real query tool, but does not explicitly name alternatives or exclusion cases, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• user_activityA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility. It discloses the data source (materialized view), date semantics (inclusive/exclusive), return shape (one row per host+date), and filter behavior. It does not mention limit behavior or potential data staleness explicitly, but the materialized view hint and detailed row semantics cover most important behavioral aspects for a query tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a compact single paragraph that front-loads the core purpose, uses clear punctuation (em dashes) to separate concerns, and includes illustrative examples without becoming verbose. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the data source, filtering options, date boundaries, and output granularity, which is largely sufficient for an agent to select and invoke it. The availability of an output schema likely covers column details, but the unexplained limit parameter leaves a small completeness gap for correct invocation in scenarios needing large result sets.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully explain parameters. It explains host_email (single user vs team), and since/until with ISO-8601 and inclusive/exclusive formatting. However, it does not describe the limit parameter at all, which is a gap given the absence of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as providing per-user daily call activity from the mv_user_daily materialized view, with specific examples of questions it answers. It distinguishes itself from generic query and search sibling tools by focusing on user-centric call metrics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool (e.g., 'how many calls did X have this week'), and explains the host_email filter for single-user vs team view. However, it does not name alternative sibling tools or explicitly state when not to use this tool, so it lacks exclusions but has strong usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_schemasA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It reveals that the tool filters out system schemas, which is a meaningful behavioral trait beyond simply listing schemas. It does not state return format, but the output schema exists to cover that, and 'list' implies read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the action and resource. Every word earns its place ('non-system', database name, and the usage hint), with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, output schema present), the description is complete: it states what the tool does, its scope, and when to call it. No further behavioral or result details are necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the baseline is 4. The description correctly avoids any parameter details since there are none to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and clearly specifies the resource ('non-system schemas in the gong-nl-db database'). It distinguishes from sibling tools like list_tables by targeting schemas rather than tables, and the scope is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The explicit instruction 'Call this first when exploring' provides clear context for when to use the tool. It does not mention alternatives or exclusions, but for a simple discovery tool, this guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the transparency burden. It discloses that the tool defaults to 'public' and enumerates both tables and views, adding behavioral context beyond a generic listing tool. However, it does not mention permissions or whether system schemas are included, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, succinct and front-loaded: first states the purpose, second provides usage guidance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with an output schema, the description adequately covers what it does, the default behavior, and prerequisite tool. It doesn't mention potential filtering or permissions, but these are not critical given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates by explaining the schema parameter's default ('public') and directing users to list_schemas for valid values. This adds meaning beyond the schema's default value, though not extensive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists tables and views in a schema, using the specific verb 'List' and resource 'tables and views'. It distinguishes from siblings like list_schemas and describe_table by focusing on schema-level object enumeration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs to use list_schemas first to find valid schema names, providing an alternative tool for schema discovery. It also notes the default schema, giving guidance on when no parameter is needed. This is clear usage context without needing exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_queryA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full disclosure responsibility. It explicitly states the tool is read-only, describes rejection behavior for non-query statements (INSERT/UPDATE/DELETE/DDL), and discloses result capping at limit (max 1000). This fully covers safety and side-effect profile.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and every clause adds essential information about allowed queries, rejection, and result limits. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a SQL runner with 2 parameters and an existing output schema, the description covers all critical behaviors: allowed query types, rejection, and row cap. It does not need to explain return values (output schema exists) or enumerate every possible query. The description is complete for safe and correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must add meaning. It explains the limit parameter's role in capping results and specifies a maximum (1000), which the schema does not. The sql parameter is self-evident from the description, though not elaborated further. This meaningfully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a read-only SQL query against a specific database (gong-nl-db), with a specific verb and resource. It distinguishes from siblings by framing the tool as the direct SQL execution option, while siblings are schema exploration or semantic tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description defines clear usage boundaries: only SELECT, WITH (terminating in SELECT), and set-operation queries are allowed, with a 1000-row cap. It implies this is for read-only analysis, but does not explicitly name alternative tools for other operations (e.g., semantic search). This is clear context without explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_searchA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Even with no annotations, the description discloses meaningful behavior: uses Vertex AI text-embedding-005 and cosine similarity, finds conceptually related content without exact matches, and specifies optional filters. It doesn't mention error cases or performance limitations, but the key behavioral characteristics are transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but well-structured, with every sentence providing essential information: mechanism, use cases, differentiation from sibling, and parameter constraints. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema exists, the description needn't detail return values. It covers the tool's purpose, technology, use cases, distinctions from alternatives, and all parameter ranges/behaviors, making it complete for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description carries the full burden. It explains all parameter semantics: since/until are ISO-8601 dates (optional), host_email filters to one rep's calls, limit caps results 1–20 (default 10). This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly identifies a specific verb+resource: semantic search across call transcript chunks. It explicitly contrasts with search_transcripts (FTS keyword matching), distinguishing its purpose from its sibling tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use: 'Use this when the user asks to find calls about a topic or concept' with concrete examples. It also names the alternative tool and explains the key difference, making the choice between semantic_search and search_transcripts unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_transcriptsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the behavioral disclosure burden. It explains the internal mechanism (GIN FTS index), return content (matching segments joined to call metadata), query syntax (websearch), and filtering options (since/until/host_email). This goes well beyond a simple 'search' statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact yet information-dense: purpose in one clause, usage guidance in one sentence, and parameter syntax in one final sentence. There is no filler, and critical details are front-loaded. Every sentence serves a distinct purpose without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 5 parameters and an output schema, the description covers all essential aspects: what it searches, when to use it, how it behaves, and what each parameter means. The existence of an output schema relieves the need to describe return fields, and the description successfully equips an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite the schema having no property descriptions, the description adds rich meaning to 4 of 5 parameters: query (websearch syntax with examples), since/until (ISO-8601 format), and host_email (filters by owner). Only limit is left to schema, but its default is visible and self-explanatory, so the description compensates admirably.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Full-text search across transcript_segments', which clearly states the tool's function and target resource. It further distinguishes itself from alternatives by highlighting the GIN FTS index and speed advantage, making its purpose unambiguous in the context of sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance: 'Prefer this over ILIKE when searching for phrases in calls' and explains why (GIN FTS index, ~100x faster). This gives clear when-to-use direction and explicitly names an alternative (ILIKE), satisfying the criterion for explicit alternatives and usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.