Server Quality Checklist

Tool Scores

• fetch_openapi_specC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool fetches content but omits critical details: whether it handles errors (e.g., invalid URLs), supports authentication, has rate limits, returns raw text or parsed data, or includes metadata like HTTP status. For a fetch operation with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('fetches') and resource ('OpenAPI spec URLs'), with zero wasted words. It's appropriately sized for the tool's complexity, making it easy for an agent to parse quickly without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations and no output schema, the description is incomplete. It doesn't explain what the return values look like (e.g., structured data, raw text), error handling, or behavioral constraints like network timeouts. For a fetch operation that could involve multiple URLs and potential failures, more context is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the schema detailing that 'input' can be a URL string, URL object, or array of URLs/objects. The description adds minimal value beyond this, only noting it fetches from 'one or more OpenAPI spec URLs', which aligns with the schema's array support. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, as the description doesn't provide additional syntax or format details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('fetches') and resource ('content of one or more OpenAPI spec URLs'), making the purpose immediately understandable. It distinguishes from sibling tools like 'list_openapi_spec_sources' by focusing on fetching content rather than listing sources. However, it doesn't specify the format of the fetched content (e.g., JSON/YAML), which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'fetch_llms_txt' or 'list_openapi_spec_sources'. It doesn't mention prerequisites, such as whether URLs need to be accessible or authenticated, or clarify use cases like bulk fetching versus single spec retrieval. This leaves the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fetch_llms_txtB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions that llms.txt files can compile lists of URLs and suggests following references, which adds some behavioral context. However, it lacks details on error handling, rate limits, authentication needs, or what happens if URLs are invalid, leaving significant gaps for a tool that fetches external content.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that convey key information: the core function and a usage tip about following links. It's front-loaded with the main purpose, though the second sentence could be more tightly integrated to avoid slight redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that fetches external URLs. It lacks details on return values (e.g., content format, error responses), authentication, rate limits, or how to handle multiple URLs, making it inadequate for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the parameter 'input' well-documented as accepting URL strings, objects, or arrays. The description doesn't add any parameter-specific semantics beyond what the schema provides, such as format examples or constraints, so it meets the baseline for high schema coverage without extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('fetches the content') and resource ('one or more llms.txt URLs'), making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'list_llms_txt_sources' which might list URLs rather than fetch content, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by explaining that llms.txt files may contain links to other files, suggesting this tool is for following such references. However, it doesn't explicitly state when to use this tool versus alternatives like 'fetch_openapi_spec' or 'list_llms_txt_sources', providing only contextual hints rather than clear guidelines.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_openapi_spec_sourcesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool lists URLs but does not describe the return format (e.g., list structure, pagination), potential errors, or any operational constraints like rate limits or authentication needs. This leaves significant gaps in understanding how the tool behaves beyond its basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that directly states the tool's function without unnecessary words. It is front-loaded and efficiently communicates the core purpose, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate but lacks depth. It covers the basic purpose but does not address behavioral aspects like return format or usage context, which could be helpful for an AI agent. The absence of an output schema means the description should ideally hint at what is returned, but it does not, leaving some contextual gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100%, so no parameter documentation is needed. The description does not add parameter details, which is appropriate here, but it could slightly enhance clarity by noting the absence of inputs. A baseline of 4 is given as it adequately handles the zero-parameter case without redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'lists all available source urls where an OpenAPI spec can be fetched,' which is a specific verb ('lists') and resource ('source urls'). However, it does not explicitly differentiate from its sibling 'list_llms_txt_sources,' which has a similar structure but for a different resource type, leaving room for slight ambiguity in sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention its sibling tools (e.g., 'fetch_openapi_spec' for fetching specs or 'list_llms_txt_sources' for listing different sources), nor does it specify any prerequisites or contexts for usage, resulting in minimal guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_llms_txt_sourcesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the tool's behavior as listing sources and implies it's a read-only operation without side effects, but doesn't explicitly state safety aspects like whether it requires authentication, has rate limits, or what the output format looks like. It adds some context about workflow but lacks detailed behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose but includes verbose workflow instructions that extend beyond the tool's scope (e.g., 'ask the user if they want to use other tools'). While informative, some sentences could be trimmed for conciseness, as they describe post-tool actions rather than the tool itself.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is fairly complete. It explains what the tool does, how to use it in context, and next steps. However, it lacks details on output format or error handling, which could be useful for an agent. The absence of an output schema means the description should ideally cover return values, but it doesn't.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so no parameter documentation is needed. The description doesn't mention parameters, which is appropriate. Baseline is 4 for 0 parameters, as it doesn't need to compensate for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'lists all available source urls where an llms.txt can be fetched.' It specifies the verb ('lists') and resource ('source urls'), but doesn't explicitly distinguish it from sibling tools like 'list_openapi_spec_sources' beyond the resource type difference. The purpose is specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool and alternatives: it instructs to use 'fetch_llms_txt' after reading sources, mentions preferring 'llms.txt' but checking 'llms-full.txt' or 'llms-mini.txt' if inadequate, and suggests asking the user about other tools for missing sources. This covers when to use, next steps, and fallback options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.