Server Quality Checklist

Tool Scores

• search_docsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It only mentions searching by title or summary, lacking details on pagination, access restrictions, or result format. This is minimal for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first sentence states the action and resource, second adds usage context with an example. No redundant words, efficiently front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with one parameter, the description covers basic purpose and usage context. However, lack of behavioral details limits completeness for autonomous agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must clarify the parameter. It implies the 'query' parameter searches title and summary fields, but does not specify format, length, or allowed values. Insufficient compensation for schema gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for documentation sections by title or summary, with an example ('AES encryption'). However, it does not explicitly differentiate from sibling tools like get_doc_structure or list_docs, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a use case: 'Use this when you aren't sure where a specific feature is documented.' This gives clear context for when to use, but does not explicitly state when not to use or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_doc_structureB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full responsibility. It implies a read operation but does not explicitly state it is read-only, non-destructive, or discuss error conditions, permissions, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loaded with the action and purpose. Every sentence earns its place with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, the description need not detail return values. However, it lacks parameter guidance and behavioral details, making it minimally adequate for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, and the description does not explain the 'doc_id' parameter (e.g., format, source, or how to obtain it). It adds no meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get the structure/nodes') and the specific resource ('a specific document'). It distinguishes from siblings by mentioning 'before resolving', which differentiates from 'resolve_section' and others.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: 'Use this to see what sections are available before resolving.' It implies temporal usage but does not explicitly exclude other contexts or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_docsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only says 'list all available...'. It does not disclose side effects, authentication needs, rate limits, or behavior like whether listing respects workspace boundaries. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is direct and front-loaded with the core action. No extraneous words or structure issues.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no parameters and an existing output schema, the description is adequate but not thorough. It does not mention authentication, output format preview, or any usage context like whether it requires a connection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so schema coverage is 100%. The description could still mention that no arguments are needed, but the absence is not a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list' and the resource 'all available SquadCo API documentation files', distinguishing it from sibling tools like get_doc_structure or search_docs, which imply specific retrieval or search operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool over alternatives or any exclusions. It simply states what it does, leaving the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_sectionA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only states 'Fetch the full content', indicating a read operation, but lacks details on authentication, rate limits, or error handling. Since an output schema exists, return values may be defined there, but additional behavioral context is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no extraneous information. It is front-loaded with the purpose and immediately provides parameter guidance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with an output schema, the description is minimally adequate. It explains what the tool does and how to use the parameters, but it could be more complete by addressing potential issues like section not found or required permissions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It provides concrete examples for doc_id ('payments') and section_id ('standard-test-cards'), adding meaning beyond names and types. However, it does not explain format constraints or possible values beyond examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch the full content of a specific documentation section', using a specific verb and resource. Sibling tools (get_doc_structure, list_docs, search_docs) have distinct purposes, making this tool's role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the user to provide doc_id and section_id with examples, implying when to use this tool. However, it does not explicitly state when not to use it or mention alternatives like search_docs for finding sections.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.