Server Quality Checklist

Tool Scores

• get_roadmap_itemA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It adds that the output is 'full detail' and that the ID is numeric, but it does not disclose behavior such as error handling for invalid IDs or the exact response shape. This is a minimal but adequate disclosure for a simple read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that immediately states the tool's purpose and the key parameter. No unnecessary information or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, 'full detail' gives only a vague idea of the return content. Missing details like not-found behavior and response fields make it incomplete, though the tool is simple enough that the description is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the 'id' parameter as a 'Roadmap feature ID' with 100% coverage. The description adds only 'numeric', which is redundant with the integer type. No new semantic information beyond the schema is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns 'Full detail for a single roadmap feature' by ID, distinguishing it from sibling tools like search_roadmap and list_recent_changes which handle collections or searches.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies you should use this tool when you have a known numeric feature ID, but it does not explicitly mention alternatives or contexts like searching when the ID is unknown. No exclusion criteria are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_recent_changesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It discloses that results are filtered by a time window and sorted newest first, which is useful. However, it does not mention return format, pagination, authentication, or other edge-case behaviors, leaving some gaps for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no fluff. The first sentence is direct and informative, and the second adds a helpful metaphor without redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with three optional parameters and no output schema, the description adequately covers the core function and usage context. It lacks explicit details about response content, but the title and description make it clear the tool returns changed features, which is sufficient for a simple list operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so parameters are already well-documented. The description only loosely references the 'days' parameter with 'last N days' and does not add extra meaning for 'limit' or 'product'. Baseline of 3 is appropriate since the schema handles parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists features added or updated in the last N days, with a specific verb (list) and resource (recently changed roadmap features). It distinguishes itself from siblings like search_roadmap and get_roadmap_item by focusing on recency and change detection, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'This is the radar: what moved on the roadmap since you last looked' provides clear context for when to use the tool—as a periodic check for updates. However, it does not explicitly mention alternatives or when not to use it, but the context is strong enough given the sibling tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_roadmapA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds 'Returns summaries; call get_roadmap_item for the full description,' which reveals a key behavioral trait: results are condensed rather than full entries. It does not mention pagination, ordering, or rate limits, but for a search tool, this is reasonably informative.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary verb and resource, and every phrase adds value. It succinctly conveys what the tool does, how to filter, and what output to expect, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 6 optional parameters and no output schema, the description adequately covers the core behavior: searching, filtering, and returning summaries. It also points to get_roadmap_item for full details, completing the workflow. It could mention default limits or result ordering, but these are minor gaps given the simplicity of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides comprehensive descriptions for all 6 parameters, covering 100% of them. The description's mention of filtering by product, status, cloud instance, or platform merely restates the schema and adds no new semantic detail. Since schema coverage is high, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search roadmap features by free text and filter by product, status, cloud instance or platform.' This specifies a distinct verb (search), a resource (roadmap features), and the scope (free text and filters), effectively distinguishing it from sibling tools like get_roadmap_item, which retrieves full details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance for when to use an alternative: 'call get_roadmap_item for the full description.' This clarifies a specific use case where search is not sufficient. However, it does not explicitly mention when to avoid search in favor of list_recent_changes or get_azure_status, though the context makes it implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_azure_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It goes beyond a simple statement by explaining that the data is 'read live from the public status feed' and that an empty list means no active issues. This provides useful context about data freshness and result interpretation, though it stops short of detailing rate limits or exact incident definitions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, immediately identifying the tool's purpose and the meaning of the expected result. There is no filler or redundant content, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters and no output schema. The description fully covers what the tool does and what the result signifies. It is complete for its intended use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description does not need to explain parameter behavior, and indeed there is nothing to explain. No additional parameter semantics are required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: retrieving active Azure service incidents from a live public feed. The verb 'read' and resource 'active Azure service incidents' are specific, and it is easily distinguished from sibling tools (search_roadmap, get_roadmap_item, list_recent_changes) which relate to roadmaps and changes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking current Azure service health by describing the data source and the interpretation of an empty list. It does not explicitly list alternatives or exclusions, but its clarity and the lack of alternative tools for the same purpose make the intended usage evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.