Server Quality Checklist

Tool Scores

• search_by_tagsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose behavioral traits like authentication needs, rate limits, or error handling. It only describes parameters without behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and structured as a docstring with Args. It is front-loaded with the main purpose and parameter details. However, the Pythonic format may be less familiar to some agents.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains all parameters but lacks information about return format, pagination behavior beyond page parameter, and error handling. Given the number of parameters and no output schema, it is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the schema: it defines tag format with examples, constrains limit (1-50) and page (minimum 1), enumerates sort values, and gives site examples. Schema coverage is 0%, so the description fully compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches StackOverflow questions by programming tags, with a specific verb and resource. It distinguishes from siblings like search_questions by focusing on tag-based filtering.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description does not mention when to prefer this over search_questions or other siblings, nor does it provide exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_questionA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only lists parameters. It does not disclose behavioral traits such as safety (read-only), error handling (missing ID), rate limits, or return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the main purpose. No redundant sentences, but it could be more structured (e.g., bullet points) for faster scanning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters and no output schema, the description covers parameters but lacks information about the return structure (e.g., what fields are returned) and error handling. This is a moderate gap for a retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds meaningful explanations for all four parameters (e.g., 'Maximum number of answers to include (1-20)'). This compensates well, though defaults and interactions are not explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves detailed information for a specific question, using a specific verb-resource pair. It distinguishes from siblings like get_answers (answers only) and get_related (related questions).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like search_questions or get_answers. Context is implied (when you have a question ID) but not stated, and no exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_questionsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It mentions combined keyword+tag search behavior but omits details on rate limits, authentication, or result handling. The parameter descriptions are thorough but not behavioral.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a clear one-liner purpose, a secondary usage note, and a bulleted parameter list. It is concise and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description does not explain the return value format or structure, which is critical since no output schema exists. It also lacks details on pagination behavior or how to handle results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description fully carries the burden. It provides a complete parameter list with meaningful explanations for all 10 parameters, including usage nuances like 'empty string for tag-only search' for query and 'list of tag names' for tags.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches StackOverflow questions by keywords, and explicitly mentions combined keyword+tag search. It differentiates from siblings like search_by_tags by explaining how to perform tag-only searches with an empty query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a usage hint for tag-only searches (empty query) but does not explicitly compare with the sibling tool search_by_tags or state when not to use this tool. It lacks clear guidance on alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_answersA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses that only answers are fetched (not question body), which is a key behavioral trait. However, it does not mention potential rate limits, authentication requirements, or pagination behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise with clear sections: a title line, a purpose sentence, usage guidance, and a structured Args list. No redundant information. Could be slightly more compact but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description provides adequate information for invocation: purpose, when to use, and parameter details. However, it lacks description of return format or example usage, which would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage, so description must compensate. It adds meaning for each parameter: specifies question_id type, max_answers range (1-50), sort order options (votes, creation, activity), and convert_to_markdown purpose. This significantly enhances schema understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get answers for a specific StackOverflow question' and distinguishes from sibling tools like 'get_question' by noting it fetches only answers without re-fetching the question body.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this when you already have the question context and just need the responses', providing clear when-to-use guidance. Also specifies constraints like max_answers range (1-50) and sort order options.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_relatedA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It states that results are lightweight summaries (no HTML bodies) and that StackExchange does not support pagination on these endpoints. This is important behavioral context beyond basic functionality. It implicitly indicates read-only behavior (fetching related content). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively concise, with three short paragraphs covering purpose, return content summary, and pagination constraint, followed by parameter descriptions. Each sentence adds value; there is no redundancy. The parameter descriptions could be integrated inline, but the structure is clear and not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema or annotations, the description provides adequate context: what it returns (titles, scores, tags), the pagination limitation, and parameter meanings. However, it could be improved by explicitly stating that the output is a list of items or by noting any authentication requirements. Still, it sufficiently enables an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, meaning the input schema only provides types and titles but no descriptions. The description's Args section provides meaningful natural language explanations for both parameters: question_id is 'StackOverflow question ID' and include_linked clarifies 'Whether to also include explicitly linked questions'. This adds significant value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool finds related and linked StackOverflow questions for discovery. It specifies the verb 'find' and the resource 'related and linked StackOverflow questions'. The name and description distinguish it from sibling tools like get_questions (single question) or search_questions (query-based search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates the tool is for discovery of related content, but does not explicitly state when to use it over alternatives like search_questions or search_by_tags. It provides no guidance on when not to use it or under what conditions another tool would be preferable. The context is clear but not prescriptive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.