Server Quality Checklist

Tool Scores

• get_forecastC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but doesn't explain key behaviors: it doesn't specify the forecast timeframe (e.g., hourly, daily), data source, accuracy, rate limits, error handling, or authentication needs. This leaves significant gaps for an agent to understand how to use it effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the main purpose stated first followed by parameter details. It avoids unnecessary fluff, but the parameter section could be more integrated into the flow rather than a separate list, slightly affecting structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 required parameters) and the presence of an output schema, the description is somewhat complete but has gaps. It covers the basic purpose and parameters but lacks behavioral details and usage guidelines. The output schema likely handles return values, so the description doesn't need to explain those, but it should still address other contextual aspects like when to use it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal parameter semantics beyond the input schema. It lists 'latitude' and 'longitude' as arguments but doesn't explain their format (e.g., decimal degrees), valid ranges, or units. Since schema description coverage is 0%, the description partially compensates by naming the parameters, but it doesn't provide enough detail to fully understand their usage, aligning with the baseline for moderate coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get weather forecast for a location.' It specifies the verb ('Get') and resource ('weather forecast') with the target ('location'). However, it doesn't differentiate from sibling tools like 'get_alerts' which might also provide weather-related information, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_alerts' or other weather-related tools. It lacks context about prerequisites, such as needing valid coordinates, and doesn't mention any exclusions or specific scenarios for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_alertsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves alerts but doesn't describe behavioral traits such as data freshness (e.g., real-time vs. cached), rate limits, error handling (e.g., for invalid state codes), or output format. The description is minimal and lacks essential context for safe and effective use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the purpose clearly, and the second provides parameter details in a structured 'Args:' format. There is no wasted text, and every sentence adds value. It efficiently communicates essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one parameter) and the presence of an output schema (which handles return values), the description is minimally complete. It covers purpose and parameter semantics adequately but lacks behavioral context (e.g., permissions, rate limits) and usage guidelines. With no annotations, it should do more to compensate, but the output schema reduces the burden slightly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains that 'state' is a 'Two-letter US state code (e.g. CA, NY)', providing crucial semantics and examples that the schema's generic string type lacks. With only one parameter, this compensation is effective, though it doesn't cover edge cases like case sensitivity or invalid codes.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get weather alerts for a US state.' It specifies the verb ('Get') and resource ('weather alerts') with geographic scope ('US state'), making it distinct from sibling tools like 'get_forecast' (which likely provides forecasts rather than alerts) and 'server_info' (which is unrelated). However, it doesn't explicitly differentiate from 'get_forecast' in terms of alert vs. forecast data, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention 'get_forecast' as an alternative for non-alert weather data or specify scenarios where alerts are preferred over forecasts. The geographic constraint ('US state') is stated, but this is part of the purpose rather than usage context. No exclusions or prerequisites are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• server_infoB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It indicates a read operation ('Get information'), which suggests non-destructive behavior, but doesn't specify details like authentication needs, rate limits, or what specific server info is returned. This is a minimal but adequate disclosure for a simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that directly states the tool's purpose without any wasted words. It is front-loaded and efficiently communicates the essential information, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is complete enough for basic understanding. However, it lacks details on the return format or specific server configuration aspects, which could be helpful for an agent, keeping it at a minimum viable level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100%, so there are no parameters to document. The description doesn't need to add parameter semantics, earning a high baseline score as it avoids redundancy and is complete in this regard.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get information') and target ('current Weather MCP server configuration'), making the purpose unambiguous. It doesn't distinguish from sibling tools like 'get_alerts' or 'get_forecast', which would require explicit comparison, so it falls short of a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_alerts' or 'get_forecast'. It implies usage for server configuration info but lacks explicit when/when-not instructions or prerequisites, leaving the agent to infer context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.