Server Quality Checklist

Tool Scores

• helpB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only vaguely mentions explaining server purpose and usage. It doesn't disclose what format the explanation comes in (e.g., text, structured data), whether it's interactive, or any behavioral traits like response time or dependencies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It's front-loaded with the core function and efficiently states what the tool does without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate but lacks depth. It doesn't explain what 'help' output looks like or how it integrates with sibling tools, leaving gaps for an AI agent to fully understand its use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate given the absence of parameters, warranting a baseline score above 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool 'explains the purpose of this server and how to use it', which provides a general purpose but lacks specificity about what 'help' actually returns or does. It doesn't clearly distinguish this from sibling tools like 'version' which might also provide server information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. The description implies it's for understanding the server, but it doesn't specify if this should be used first, after errors, or instead of exploring sibling tools like 'version'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search-docsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: it returns 'matching documentation routes with titles, content snippets, and relevance scores', which helps the agent understand the output format. However, it lacks details on permissions, rate limits, error handling, or pagination (though 'maxResults' in schema hints at result limiting), leaving some gaps for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, the second describes the return format, and the third provides usage context. Each sentence adds distinct value without redundancy, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search with 2 parameters), no annotations, and no output schema, the description is fairly complete: it covers purpose, return format, and usage context. However, it lacks details on behavioral aspects like error cases or performance, and the output schema absence means return values are only partially described, leaving minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'keyword' which relates to the 'query' parameter, adding semantic context. However, it does not explain the 'maxResults' parameter or provide details on query syntax, format, or relevance scoring. The description adds some value but does not fully compensate for the low schema coverage, meeting the baseline for partial compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search'), resource ('PDFDancer SDK documentation'), and scope ('by keyword'), distinguishing it from sibling tools like 'get-docs' (likely for retrieving specific docs), 'help' (general assistance), and 'version' (system info). It explicitly mentions what it searches for ('features, APIs, and usage examples'), making the purpose unambiguous and well-differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('to find information about PDFDancer features, APIs, and usage examples'), which implicitly suggests it's for exploratory searches rather than direct retrieval. However, it does not explicitly state when not to use it or name alternatives (e.g., 'get-docs' for known documentation routes), missing full explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• versionA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool returns the server version, which implies a read-only, non-destructive operation, but does not disclose behavioral traits like rate limits, authentication needs, or response format. The description adds basic context but lacks detailed behavioral information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that front-loads the purpose with zero waste. It is appropriately sized for a simple tool with no parameters, earning its place by succinctly conveying the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is complete enough for basic use. It explains what the tool does, but could be enhanced with details on return format or error handling to achieve a perfect score, though it's adequate for this low-complexity context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, with 100% schema description coverage (empty schema). The description does not need to add parameter semantics, so a baseline score of 4 is appropriate as it efficiently handles the lack of parameters without unnecessary detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Returns') and resource ('current version of the pdfdancer-mcp server'), distinguishing it from sibling tools like get-docs, help, and search-docs which handle documentation rather than server metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (checking server version), but does not explicitly state when to use this tool versus alternatives or any exclusions. It's clear from the purpose that this is for version queries, but no direct guidance on alternatives or prerequisites is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-docsA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool retrieves 'complete markdown content including code examples, detailed explanations, and API references,' which gives some context about return format. However, it doesn't disclose potential limitations like authentication requirements, rate limits, error conditions, or whether this is a read-only operation (though implied by 'retrieve').

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each serve distinct purposes: the first states the core functionality, the second provides usage guidance. There's no wasted language, and information is front-loaded with the primary purpose stated immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read operation with no annotations or output schema, the description provides good context about what the tool does and when to use it. It explains the relationship with sibling tools and describes the return content format. However, it doesn't address potential error cases or provide examples of the route parameter format, leaving some gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 1 parameter with 0% description coverage, so the description must compensate. It explains that the 'route' parameter corresponds to 'a specific route' for documentation retrieval, adding meaningful context beyond the schema's pattern constraint. However, it doesn't provide examples of valid route formats or explain the pattern requirement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Retrieve the full documentation content') and resource ('for a specific route'), distinguishing it from sibling tools like 'search-docs' which only finds relevant documentation rather than retrieving complete content. It explicitly mentions what the tool does beyond just the name/title.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool vs. alternatives: 'After finding relevant documentation with search-docs, use this tool to get the complete markdown content...' This clearly establishes the workflow relationship between sibling tools and specifies the appropriate context for invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.