Server Quality Checklist

Tool Scores

• compare_servicesA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what the tool does (comparative analysis with ranking and recommendation) and output behavior, but lacks details on permissions, rate limits, data freshness, or error handling. The description adds value by explaining the ranking and recommendation aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by output behavior and parameter details. Every sentence earns its place with no wasted words, and the Args section is clearly separated for quick reference.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, comparative analysis), no annotations, but with an output schema present, the description is mostly complete. It explains the purpose, output (ranking with recommendation and reasoning), and parameters well, but could benefit from more behavioral context like data sources or limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fully compensate. It provides clear semantic explanations for both parameters: 'services' as 'List of service slugs to compare (max 10)' and 'sort_by' as 'Metric to sort by' with enumerated values. This adds essential meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('compare reliability ratings', 'see which provider has the best track record') and resources ('multiple services', 'uptime, latency, and real-world reliability'). It distinguishes from sibling tools by focusing on comparative analysis rather than individual checks or reporting outcomes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('compare reliability ratings across multiple services side by side'), but does not explicitly state when not to use it or name alternatives among sibling tools (check_reliability, report_outcome). The comparative nature is implied to differentiate it from single-service checks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• report_outcomeA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It explains the tool's purpose (reporting outcomes) and community impact, but doesn't disclose important behavioral traits like whether this is a read-only operation, if it requires authentication, rate limits, or what happens after submission. The description doesn't contradict annotations (none exist), but leaves significant behavioral aspects unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It starts with the core purpose, provides community context, then lists parameters with clear explanations. Every sentence earns its place: the first establishes purpose, the second explains value, and the parameter section provides necessary details without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, 2 required) and the presence of an output schema (which means return values are documented elsewhere), the description is reasonably complete. It explains what the tool does, when to use it, and provides parameter semantics. The main gap is lack of behavioral transparency details (permissions, side effects, etc.), but with an output schema handling return values, it's mostly adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides clear semantic explanations for all 4 parameters: 'service' (Service slug that was called), 'outcome' (Result with specific values), 'latency_ms' (Response time in milliseconds), and 'error_type' (category if outcome was error). The description adds substantial meaning beyond the bare schema, though it doesn't explain the 'null' defaults for optional parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Leave a review — report the outcome of an API or MCP server call you just made.' It uses specific verbs ('Leave a review', 'report the outcome') and identifies the resource (API/MCP server calls). It distinguishes from siblings by focusing on outcome reporting rather than checking or comparing reliability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'report the outcome of an API or MCP server call you just made.' It provides context about contributing to community reliability ratings. While it doesn't explicitly name sibling tools as alternatives, it clearly defines the specific use case (reporting outcomes after calls), which implicitly distinguishes it from checking or comparing reliability.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_reliabilityA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does so well. It discloses key behavioral traits: the tool returns a trust score, operational status, trend, and known issues; explains that scores are based on independent probes and crowdsourced telemetry; and clarifies it's not vendor self-reporting. It doesn't mention rate limits or auth needs, but covers most critical aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by an analogy, then details on returns and data sources, and ends with parameter explanations. Every sentence adds value without redundancy, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, no annotations, 0% schema coverage, but an output schema present, the description is complete enough. It covers purpose, usage, behavior, and parameters thoroughly, and since an output schema exists, it doesn't need to detail return values explicitly, making it well-rounded for the context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, which it does effectively. It explains the 'service' parameter as a 'slug or partial name' with examples, and details the 'metrics' parameter as an optional list with specific metric options ('uptime', 'latency', etc.), adding meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Look up the independent reliability rating for an API or MCP server.' It specifies the verb ('look up') and resource ('reliability rating'), and distinguishes it from siblings by focusing on individual service checks rather than comparisons or reporting outcomes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'before you commit to a service' and 'see the real uptime, latency, error rate, and community experience.' It implies usage for pre-commitment evaluation but does not explicitly state when not to use it or name alternatives among siblings (e.g., compare_services for comparisons).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.