Server Quality Checklist

Tool Scores

• get_incidents_todayC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must fully disclose behavioral traits. It states 'as of snapshot' implying a read operation, but omits details about data freshness, idempotency, permissions required, rate limits, or side effects. The description is insufficient for safe agent action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (5 words) but lacks a complete sentence structure. It could be refined to a full sentence without additional length, improving readability. Currently it is a fragment.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description should clearly explain what the tool returns (e.g., incident IDs, severity, timestamps). 'Active incidents across all monitored services' is vague and does not specify the response structure. The tool has no parameters, so the description must compensate, but it fails to do so.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, and schema description coverage is 100% (empty). The description does not add parameter-specific meaning, but with no parameters, the baseline is 4. No contradiction or missing info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Active incidents across all monitored services as of snapshot.' It identifies the tool as returning current incidents, distinct from sibling tools like list_services, get_status, and check_all. However, it is a noun phrase rather than an imperative verb phrase, slightly reducing clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, when to use, or when to avoid. The agent receives no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statusB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It mentions returning a URL but does not disclose what 'full status' includes (e.g., uptime, incidents), authentication requirements, or any side effects. Behavior is minimally described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, front-loaded with key information, and contains no filler. However, it could include more detail without adding length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should clarify the return structure. It mentions a URL but not other fields of the 'full status record'. This leaves the agent uncertain about what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and the description does not explain the 'service_id' parameter (e.g., format, example values). It adds no meaning beyond the schema definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns a 'Full status record for one service' with a cited URL. This distinguishes it from siblings like list_services (listing all services) and get_incidents_today (incidents only).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when you need detailed status for a specific service, but does not explicitly guide when to use alternatives (e.g., list_services for all services, check_all for aggregate). No exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_allB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description must disclose behavioral traits. It mentions 'aggregate status counts' but does not explain what is counted, whether it is read-only, or any side effects or permissions needed. Lacks important operational details for a health summary tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, very concise with no extraneous words. However, the lack of parameter explanation makes it incomplete rather than efficiently complete.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, no annotations, and an undocumented parameter, the description provides only high-level purpose. Missing details on input, output, and behavioral contract.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'category' is a string with 0% schema description coverage. The tool description does not mention or explain the parameter, leaving its purpose and allowed values entirely unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Describes the tool as providing 'aggregate status counts' and 'quick health summary', which clearly differentiates it from sibling tools like list_services, get_status, and get_incidents_today. The verb 'aggregate' and resource 'status counts' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States the tool is for 'agents deciding whether to retry/fallback', giving clear context for when to use it. Does not specify when not to use or provide explicit alternatives, but the sibling tools imply other choices.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_servicesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only states listing and filtering but omits details like data freshness, pagination, authentication needs, or whether results are complete. This is insufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. Front-loaded with purpose, then filtering options. Efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description is too brief. It covers basic purpose but lacks detail on return format, expected output structure, or behavior for empty results. Adequate for a simple list tool but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with each parameter described in the schema. The description restates filtering capability but adds no new meaning beyond 'filter by category or status'. Baselines at 3 due to high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists monitored SaaS providers with their last-known status, and explicitly mentions filtering by category or status. The verb 'list' and resource 'monitored SaaS providers' are specific, and the distinction from siblings like get_status (likely single provider) is implied.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., get_status for a single provider, get_incidents_today for incidents). No exclusions or prerequisites provided, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.