Server Quality Checklist

Tool Scores

• search_by_queryB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description only documents parameters and return format. It does not disclose behavioral traits such as rate limits, authentication needs, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args and Returns sections and is front-loaded with the main purpose. However, it is somewhat lengthy due to detailed parameter documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (12 parameters) and lack of output schema, the description adequately explains all inputs and return format. It covers the essential context for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing clear, concise explanations for each of the 12 parameters, including types, defaults, and purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Stack Overflow for questions matching a query. It specifies verb+resource, but does not differentiate from sibling tools like advanced_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lacks any guidance on when to use this tool versus alternatives. No when/when-not information is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_stack_traceA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It only states the high-level purpose and parameter explanations, omitting details such as side effects, permissions, rate limits, or how the stack trace is processed. The absence of such information limits transparency for a tool that likely queries an external API.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and well-structured, with a short introductory sentence followed by a clean bullet-point list of parameters. It front-loads the purpose and uses minimal prose, earning every sentence's place. No extraneous information is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters) and lack of annotations and output schema, the description adequately explains parameter semantics but fails to describe the return format in sufficient detail ('Formatted search results' is vague). It also does not explain the overall workflow or how results are structured, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description includes an Args section with a one-line explanation for each of the 9 parameters, adding meaning beyond the input schema (which lacks descriptions). All parameters are covered, with clear explanations of their roles (e.g., 'Programming language of the stack trace' for 'language'). However, some explanations are minimal and could benefit from examples or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('analyze') and resource ('stack trace'), and explicitly identifies the target platform (Stack Overflow). It distinguishes itself from sibling tools like 'search_by_error' and 'search_by_query' by focusing on stack trace analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus its siblings (e.g., 'advanced_search', 'search_by_error'). It lacks explicit 'when-to-use' or 'when-not-to-use' instructions, leaving the AI agent without context for choosing the appropriate tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• advanced_searchA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description covers parameter details and return format but omits broader behavioral traits like side effects, authentication needs, rate limits, or explicit read-only indication. It provides adequate but not comprehensive transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a concise introductory sentence followed by a parameter list and return statement. It is front-loaded and necessary, though the lengthy parameter list is unavoidable due to tool complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 21 parameters, no output schema, and no annotations, the description covers parameters and return type but lacks details on parameter interactions, pagination, error handling, or response structure. It is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema_description_coverage, the description effectively adds meaning by listing all 21 parameters with brief, clear explanations (e.g., 'List of tags to filter by'). It compensates for missing schema descriptions, though some parameter semantics could be deeper.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs 'Advanced search for Stack Overflow questions with many filter options,' using a specific verb and resource. This distinguishes it from simpler sibling tools like search_by_query or search_by_error.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide explicit guidance on when to use this tool versus alternatives. It lacks when-not-to-use scenarios, prerequisites, or comparisons to sibling tools, leaving the agent to infer based on parameter lists.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_questionA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so description carries full burden. It only states the basic action and parameters, lacking disclosure of behavioral traits like error handling, authentication needs, rate limits, or whether it's read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is front-loaded with the main sentence and includes a structured Args block. It is appropriately sized, though the Python docstring format is non-standard for MCP and could be condensed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description states 'Returns: str: Formatted question details' which is minimal. It does not specify what fields are included or any pagination/error behavior, leaving gaps for a tool with no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description must compensate. The Args section explains each parameter (question_id, include_comments, response_format) with types and optionality, adding meaning beyond the schema. However, it does not reiterate default values present in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states 'Get a specific Stack Overflow question by ID', which is a clear verb+resource. It distinguishes from sibling tools (advanced_search, search_by_error, etc.) which are search-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies using this tool when you have a question ID, but does not provide explicit when-to-use or when-not-to-use guidance. No mention of alternatives or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_by_errorA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description partially covers behavior by mentioning it searches for solutions and returns formatted results. However, it does not disclose rate limits, authentication, or other potential side effects. It is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with one purpose line followed by a parameter list. The Args/Returns structure is clear and properly front-loaded. Each line adds information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 10 parameters, no output schema, and no annotations, the description adequately covers input semantics and return format. It is complete for a search tool, though it could mention result behavior (e.g., sorting).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description carries full burden. It provides clear, meaningful descriptions for all 10 parameters (e.g., 'The error message to search for', 'Programming language'), adding value beyond the schema's bare type definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search Stack Overflow for solutions to an error message', which is a specific verb+resource combination. It distinguishes from sibling tools like search_by_query (general) and get_question (single question).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus siblings like advanced_search or analyze_stack_trace. There is no context on preferred use cases or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.