Server Quality Checklist

Tool Scores

• get_earthquake_newsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions fetching news but doesn't describe what 'news' entails (e.g., articles, alerts, data points), potential rate limits, authentication needs, or error handling. The description lacks details on return format, pagination, or data freshness, leaving significant gaps for an agent to understand the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the purpose stated clearly in the first sentence. The parameter explanations are concise and directly relevant, avoiding unnecessary details. However, the formatting with 'Args:' and bullet-like indentation could be slightly more streamlined, but it remains efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is partially complete. It covers the purpose and parameter semantics adequately but lacks behavioral details like output structure, error cases, or usage guidelines relative to siblings. Without annotations or output schema, more context on what 'news' returns would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics beyond the input schema, which has 0% description coverage. It explains that 'location' is a 'Name or identifier of the place where the earthquake(s) occurred,' clarifies timestamp formats for date parameters, and specifies default behaviors (e.g., defaults to current date if omitted). This compensates well for the schema's lack of descriptions, though it doesn't cover all nuances like valid location formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Fetch earthquake news for a given location within a specified date range.' It specifies the verb ('fetch'), resource ('earthquake news'), and scope ('location within date range'), making it easy to understand. However, it doesn't explicitly differentiate from sibling tools like 'get_warning' or 'get_weather_forecast', which might also involve location-based data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools or clarify scenarios where 'get_earthquake_news' is preferred over 'get_warning' or 'get_water_level_condition', nor does it specify prerequisites or exclusions. Usage is implied only through the purpose statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_weather_forecastB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes what the tool does (retrieve forecast) but lacks details on behavioral traits such as rate limits, authentication needs, error handling, or what happens if dates are invalid. This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the purpose clearly, followed by a structured 'Args' section. Every sentence adds value, with no wasted words, though the structure could be slightly more streamlined (e.g., integrating defaults into the main description).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 3 parameters, the description is moderately complete. It covers the purpose and parameters well but lacks details on return values, error cases, or behavioral context. For a retrieval tool with no structured support, it meets minimum viability but has clear gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful semantics beyond the schema by explaining each parameter: location_name as 'name or identifier', date_start and date_end with defaults and inclusivity. This clarifies usage effectively, though it could specify date format or location identifier examples for a higher score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Retrieve a weather forecast for a specific location within a given date range.' It specifies the verb (retrieve), resource (weather forecast), and scope (location and date range). However, it does not explicitly differentiate from sibling tools like get_earthquake_news or get_warning, which handle different types of data, so it falls short of a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention sibling tools or other contexts where this tool might be preferred or avoided. Usage is implied by the purpose but lacks explicit instructions, leaving gaps for an AI agent to infer correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_warningB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It describes the action ('retrieve') and date range filtering, but lacks critical behavioral details: it doesn't specify if this is a read-only operation, what permissions are needed, how results are returned (e.g., format, pagination), error handling, or rate limits. For a retrieval tool with zero annotation coverage, this leaves significant gaps in understanding how the tool behaves beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by a structured 'Args:' section with bullet-like explanations for each parameter. Every sentence adds value—no redundancy or fluff. The structure makes it easy to scan, with key information presented efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is partially complete. It excels in parameter semantics and purpose clarity but lacks usage guidelines and behavioral transparency. Without annotations or output schema, it should ideally cover more behavioral aspects (e.g., response format, safety). It's adequate as a minimum viable description but has clear gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning beyond the input schema, which has 0% description coverage. It explains both parameters: datetime_start as 'The earliest timestamp... (inclusive) from which to retrieve weather warnings' with format details and default behavior, and datetime_end similarly. This fully compensates for the schema's lack of descriptions, providing clear semantics, formats, and defaults that are not in the schema properties.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Retrieve general weather warnings issued within a specified date range.' This specifies the verb ('retrieve'), resource ('general weather warnings'), and scope ('within a specified date range'). It distinguishes from siblings like get_earthquake_news or get_weather_forecast by focusing on warnings rather than news, conditions, or forecasts. However, it doesn't explicitly differentiate from all siblings (e.g., water level conditions might overlap with weather warnings), so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like get_earthquake_news or get_water_level_condition, nor does it specify scenarios where weather warnings are preferred over forecasts or other data sources. The only implied usage is for retrieving weather warnings by date, but no explicit when/when-not instructions or alternatives are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_water_level_conditionA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it describes the retrieval action and parameter precedence, it lacks critical behavioral details such as whether this is a read-only operation, what authentication might be required, rate limits, error conditions, or the format of returned data. For a tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence. The parameter explanations are necessary given the schema coverage gap. However, the formatting with 'Args:' could be slightly more integrated, and some sentences could be more concise (e.g., the second sentence about precedence is clear but slightly wordy).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (2 parameters, no annotations, no output schema), the description is moderately complete. It covers purpose and parameter semantics well but lacks behavioral transparency and output details. Without annotations or output schema, the description should ideally provide more context about what the tool returns and any behavioral constraints to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning beyond the input schema, which has 0% description coverage. It explicitly defines both parameters: 'district: The name of the district within the specified state for which to retrieve flood warning conditions' and 'state: The name of the state in Malaysia for which to retrieve flood warning conditions.' This clarifies the geographic scope and relationship between parameters, fully compensating for the schema's lack of descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Retrieve the water level conditions associated with flood warnings for a specified district or state.' It specifies the verb (retrieve), resource (water level conditions), and context (flood warnings). However, it doesn't explicitly differentiate from sibling tools like 'get_warning' or 'get_weather_forecast' beyond the specific resource focus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage: 'If both district and state are provided, the district takes precedence. If district or state is not specified, use an empty string for that field.' This gives explicit guidance on parameter precedence and handling of unspecified fields. However, it doesn't mention when to use this tool versus alternatives like 'get_warning' or provide exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.