Server Quality Checklist

Tool Scores

• sota_summits_nearC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It only states that the tool finds summits near a location but does not disclose if it is read-only, any side effects, rate limits, or whether it includes all summits or only specific ones (e.g., only activated). This leaves ambiguity about safety and behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. However, it is too brief and lacks additional context that would benefit usability (e.g., default behavior). Still, it is efficient and to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, no annotations, output schema exists but is not referenced), the description is incomplete. It does not explain the output format, any constraints on radius or coordinate system, or the type of summits returned. The presence of an output schema partially mitigates, but the description should at least hint at the response structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage with clear descriptions for each parameter (latitude, longitude, radius_km, limit). The description does not add any semantics beyond what the schema already conveys, such as coordinate format or radius constraints. Baseline score of 3 applies since schema is comprehensive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Find' and the resource 'SOTA summits near a geographic location'. It distinguishes from siblings like 'sota_summit_info' (single summit) and 'sota_spots' (recent spots) by specifying a location-based search. However, it could be more specific about the type of summits (e.g., all summits or only activated ones).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidance is provided. The description does not indicate when to use this tool over alternatives like 'sota_summit_info' for a specific summit or 'sota_spots' for recent activity. There is no mention of prerequisites or context for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sota_spotsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It only states it returns a live spot feed, lacking details on idempotency, rate limits, data freshness, or potential side effects. As a read operation, additional context like real-time nature or caching would be helpful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences, each serving a distinct purpose: first states the action, second explains the return value. No superfluous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 3 optional parameters and an output schema, the description is minimally adequate. However, it could mention the context of 'spots' (e.g., summit activations or contacts) or provide usage hints among siblings. The output schema covers return structure, so not a major gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each parameter having a description and default. The tool description adds no extra meaning beyond the schema, so baseline score of 3 is appropriate. No further clarification of parameter semantics is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves current and recent SOTA spots with summit details. The verb 'Get' and resource 'SOTA spots' are specific, and the name sota_spots distinguishes it from siblings like sota_alerts or sota_summit_info, though no explicit differentiation is given.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No usage guidance is provided. The description does not indicate when to use this tool over its siblings (e.g., sota_alerts for alerts, sota_summits_near for nearby summits), nor does it specify any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sota_alertsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It only states 'Get upcoming SOTA activation alerts' without disclosing behavior (e.g., read-only, rate limits, or output nature).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys the core purpose. However, it could include more detail without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 optional parameters and an output schema, the description is minimal. It lacks details on output format or constraints, but the schema covers the input adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds no extra meaning beyond the schema's parameter descriptions ('Look-ahead window in hours...', 'Filter by SOTA association...').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get', the resource 'upcoming SOTA activation alerts', and distinguishes from sibling tools like 'sota_spots' and 'sota_summit_info'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'sota_spots' or 'sota_summits_near'. Context or exclusions are missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sota_summit_infoB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose any behavioral traits such as data freshness, rate limits, authentication needs, or what constitutes 'detailed'. The tool is essentially a black box beyond its stated purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It could be slightly improved by mentioning availability or limitations, but it efficiently conveys the core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is an output schema (not shown in the provided definition), the description does not need to document return values. However, the lack of behavioral details and usage guidance leaves the tool somewhat incomplete for an agent to fully understand its capabilities.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (the single parameter has an informative description). The tool description adds minimal extra meaning beyond restating the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and the resource ('detailed summit information'), and identifies the input method ('by SOTA reference code'). It naturally distinguishes from siblings that handle alerts, spots, or nearby summits.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies through context that this tool is for retrieving summit details given a code, but it does not explicitly explain when to use this tool over siblings like sota_spots or sota_summits_near, nor does it mention any prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_version_infoA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but the description discloses the read-only nature and specifies exact return fields (service_name, service_version, spec_version). No hidden side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences: purpose, context/use case, and return fields. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and presence of output schema, the description fully explains the return values and use case, making it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. Baseline 4 applies as the description adds no parameter details needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool retrieves service version and upstream spec version, with specific fields like service_version and spec_version. Distinguishes from sibling tools (alerts, spots, summits) by focusing on version info.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states the use case: confirming fleet alignment across MCP deployments and detecting drift. Provides clear context but no exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.