Server Quality Checklist

Tool Scores

• list_gaps_by_priorityB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the description adds value by describing the ranking logic. However, it does not disclose other behavioral traits like pagination, error handling, or input validation. Given the read-only nature, the description is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single 12-word sentence that efficiently conveys the core functionality and ranking formula. Every word earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema, the description lacks parameter explanations and usage context. The tool's simplicity is undermined by the missing semantics for 'vault_path', making it incomplete for confident agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 2 parameters with 0% description coverage, and the description does not mention either parameter. 'vault_path' and 'limit' are left completely unexplained, forcing the agent to infer their meaning from names alone. The description fails to add any semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns gaps ranked by a specific priority formula (frequency × diversity × novelty), identifying a unique resource and action. This distinguishes it from sibling tools like 'find_gaps' which likely just list gaps without ranking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool over siblings like 'find_gaps' or 'export_review_queue', nor does it specify prerequisites or exclusions. The agent must infer usage from context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• export_review_queueB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false, consistent with the description stating it writes to output_csv. The description adds that it returns the row count, but does not disclose potential side effects beyond writing the CSV file.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the key purpose and output. No unnecessary words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 3 parameters and no schema descriptions, the description is incomplete. While output schema exists (mitigating need to describe return format), the omission of parameter details leaves the agent underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any of the three parameters (vault_path, output_csv, limit). The agent cannot infer their meaning or constraints from the description alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool exports a CSV of top-priority gap concepts, with specific verb 'export' and resource 'CSV of gap concepts'. It differentiates from sibling tools like list_gaps_by_priority by focusing on export, not listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions the tool is for 'Anki or other spaced-repetition tools' implying a use case, but does not contrast with siblings like find_gaps or list_gaps_by_priority. No guidance on when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_gapsA
  3.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, which is consistent. The description adds value by noting 'Applies fuzzy canonicalization and noise filtering', revealing processing behavior beyond a bare scan. No side effects or limitations are mentioned, but the read-only intent is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first states purpose and scope, second adds algorithmic details. No redundancy, front-loaded with key action. Every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the core task and processing but omits parameter details. An output schema exists (not shown), so return value clarity is less critical. For a tool with 3 parameters and a discovery function, the description is adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only implicitly references 'vault_path' via 'Scan a markdown vault' but does not explain 'min_references' or 'limit'. The meaning of these parameters is left to the agent to infer, which is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it scans a markdown vault to find concepts mentioned in multiple notes without dedicated notes. This distinguishes it from siblings like 'list_gaps_by_priority' (which lists existing gaps) and 'export_review_queue' (exporting). The verb 'scan' and 'return concepts' clarifies its discovery role, though explicit differentiation is missing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for identifying undocumented concepts but provides no when-to-use or when-not-to-use guidance relative to sibling tools like 'list_gaps_by_priority' or 'generate_research_questions'. No explicit context or exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_research_questionsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=true, so description's 'generate' is consistent as a read operation. Adds detail about output (priority score, factor breakdown), but could mention it doesn't modify state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words, front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With output schema present, description adequately covers inputs and output characteristics. Could mention prerequisites (e.g., gaps exist) but not critical for read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage. Description explains 'n' as top N gaps, but 'vault_path' is only inferred from its title. Partial compensation for low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'generate' and resource 'prioritized research questions for the top N gaps', distinguishing it from sibling tools like find_gaps or list_gaps_by_priority.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage when gaps are available (top N), but does not explicitly specify when not to use or compare to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• surprise_research_topicA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=true, which is consistent with the description's 'pick' action being non-destructive. The description adds behavioral context by explaining the sortition sampling method and the purpose of countering bias, going beyond the annotation alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences with no wasted words. The [Pro] prefix efficiently denotes premium tier, and the content is front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema, the description covers the essential aspects: what it does, why to use it, and the key parameter (n). It lacks only a brief note on vault_path, but overall is sufficient for agent understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters. It mentions 'n random gaps' clarifying the 'n' parameter's role as the count, but does not explain 'vault_path' or its format. Partial compensation for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool picks random gaps from the low-priority long tail, using the specific verb 'pick' and resource 'gaps'. It distinguishes itself from siblings like 'list_gaps_by_priority' and 'generate_research_questions' by emphasizing random sampling of low-priority items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context for use by noting it breaks confirmation bias and surfaces topics one wouldn't pick themselves. This implies a scenario where the agent wants to explore unexpected areas. However, it does not explicitly state when not to use it or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.