Server Quality Checklist

Tool Scores

• get_ietf_docC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'pagination support' and describes the return format ('A dictionary containing the document content and metadata'), which adds some context. However, it lacks details on error handling (e.g., what happens if the RFC number doesn't exist), rate limits, authentication needs, or whether this is a read-only operation. For a tool with 3 parameters and no annotations, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized, with a clear purpose statement followed by 'Args:' and 'Returns:' sections. Each sentence adds value, such as explaining pagination and parameter defaults. There's no redundant or wasted text, making it efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, no annotations, no output schema), the description is moderately complete. It covers the basic purpose, parameters, and return format, but lacks details on behavioral aspects like error handling, rate limits, and differentiation from siblings. Without annotations or an output schema, more context on usage and behavior would improve completeness for reliable agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly lists all 3 parameters with brief explanations (e.g., 'The RFC number str', 'The line number to start from'), adding meaning beyond the input schema, which has 0% description coverage. However, it doesn't fully compensate for the schema's lack of descriptions—for example, it doesn't clarify the format of 'number' (though it gives an example '1234'), validate ranges for 'start_line' or 'max_lines', or explain edge cases. With 0% schema coverage, the description provides basic but incomplete parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get an RFC document by its number in RFC editor Index with pagination support'. This specifies the verb ('Get'), resource ('RFC document'), and key functionality ('pagination support'). However, it doesn't explicitly differentiate from sibling tools like 'list_ietf_docs_number' or 'search_ietf_rfc_by_keyword', which likely serve different purposes (listing vs. searching).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions pagination support but doesn't explain when to use this tool over 'list_ietf_docs_number' (which might list documents) or 'search_ietf_rfc_by_keyword' (which might search by content). There's no mention of prerequisites, exclusions, or specific contexts for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ietf_docs_numberB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but lacks details on traits such as rate limits, authentication needs, data freshness, or error handling. For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the key action and resource. It avoids unnecessary words and directly communicates the tool's function, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is minimally adequate. It explains the purpose but lacks behavioral context and output details (e.g., what format the count is returned in). For a tool with no structured data to rely on, it should provide more completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the input schema has 100% description coverage (though empty). The description doesn't need to explain parameters, so it appropriately focuses on the tool's purpose. A baseline of 4 is justified as there are no parameters to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('total number of IETF RFC documents'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_ietf_doc' (which likely retrieves specific documents) or 'search_ietf_rfc_by_keyword' (which searches content), though the focus on 'total number' implies a distinct counting function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description implies it returns a count, but it doesn't specify scenarios (e.g., for inventory checks or before bulk operations) or contrast with siblings like 'search_ietf_rfc_by_keyword' for filtered results. This leaves the agent without explicit usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_ietf_rfc_by_keywordB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the search scope (keyword in titles) and return format (list of matching RFCs with numbers and titles), but lacks details on critical behaviors like pagination, rate limits, error handling, or whether the search is case-sensitive. For a search tool with zero annotation coverage, this leaves significant gaps in understanding its operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and concise, with zero wasted words. It front-loads the core purpose in the first sentence, followed by clear sections for 'Args' and 'Returns'. Each sentence earns its place by providing essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (a search function with one parameter) and lack of annotations or output schema, the description is minimally adequate. It covers the basic purpose, parameter use, and return format, but doesn't address behavioral aspects like search limitations or result ordering. Without an output schema, the description should ideally detail the structure of returned RFC objects, which it only partially does by mentioning 'numbers and titles'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context beyond the input schema. The schema only defines 'keyword' as a string with 0% description coverage. The description clarifies that this keyword is used to search 'in their titles' and specifies the source ('RFC Editor Index'), which helps the agent understand the parameter's role and constraints. With only one parameter, this compensation is effective, though it could benefit from examples or format hints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for IETF RFC documents from RFC Editor Index by keyword in their titles.' It specifies the verb (search), resource (IETF RFC documents), and scope (RFC Editor Index, keyword in titles). However, it doesn't explicitly distinguish this from sibling tools like 'get_ietf_doc' or 'list_ietf_docs_number', which likely retrieve documents by specific identifiers rather than keyword searches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools or contexts where this search method is preferred over others, such as when users have only partial title information. Without such guidance, the agent must infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.