Server Quality Checklist

Tool Scores

• atlas_places_searchA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It discloses the return format (JSON with 3 demo options including rating, open_now, safety reasons) which adds value. However, it does not mention read-only nature, side effects, or auth requirements, leaving gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and well-structured, with a clear first-line purpose followed by parameter explanations and return info. Every sentence contributes value, and there is no redundant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters (1 required) and mentions a demo output, the description explains parameters and return format adequately. However, it lacks context on error handling, rate limits, and the 'demo' nature (whether it returns real data). This makes it somewhat incomplete for reliable use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains 'location' as a neutral place name and 'query' as the type of venue, adding meaning beyond the schema's titles and defaults. This helps an agent understand parameter intent, though more detail on valid values would be beneficial.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for places or food options, providing a specific verb and resource. It distinguishes from siblings like atlas_aqi_context (air quality) and atlas_weather_context (weather) by its focus on places/food search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus siblings or when not to use it. The description only states what it does, leaving the agent to infer usage context without explicit alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• atlas_aqi_contextA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It states the return format and that the tool is safety-relevant, but lacks details on performance, authentication, or error handling. Adequate for a simple read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise, with clear Args and Returns sections. No unnecessary text, front-loaded purpose, and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no annotations, the description covers input semantics and output structure comprehensively. Could note potential errors or defaults, but overall complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds value beyond the schema by specifying that location should be a neutral place name and provides examples. Since schema coverage is 0%, this compensates well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets AQI context for a location, using a specific verb and resource. It distinguishes from sibling tools like weather_context and safety_rules, which cover different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description does not mention when not to use it or provide context for selection among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• atlas_civic_signalA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It mentions returning a JSON string with a summary, which is useful. However, it does not disclose whether the tool is read-only, idempotent, or error handling behavior (e.g., if location is not found). Adequate but could be more transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise: two sentences for purpose and a structured Args/Returns block. No unnecessary words; front-loaded with the core function. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple input schema (one string parameter) and existence of an output schema, the description covers the essentials: what it returns (JSON string with summary) and the input. It is largely complete, though missing edge cases or error details. For a straightforward tool, this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage (no description for the 'location' property). The description adds 'Neutral place name', which gives meaningful guidance beyond the type 'string'. This compensates well for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets 'active civic or disruption signals' with specific examples (floods, closures, demonstrations, roadworks). It uses a specific verb+resource and distinguishes from siblings like atlas_aqi_context and atlas_weather_context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for disruption queries but does not explicitly state when to use or when not to use. No alternatives are mentioned, though sibling tools provide context. The guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• atlas_safety_rulesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It states 'retrieves', implying read-only, but does not disclose authorization needs, rate limits, or behavior on invalid input. Adequate for a simple retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences and an args/returns section. No wasted words, front-loaded with the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only one parameter and an output schema present, the description adequately covers the tool's action. It mentions the return type (JSON string with blocked and caution rules), which is sufficient given the output schema likely provides details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds meaning with 'Neutral place name' for the location parameter, clarifying that it expects a general location name rather than a specific address.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves safety rules (blocked/caution rules) for a location. The verb 'retrieves' and resource 'safety rules' are specific, and the tool is distinct from siblings like atlas_aqi_context and atlas_civic_signal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives. The purpose is implied by the name and description, but missing exclusions or when-not scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• atlas_weather_contextA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It describes return fields (condition, risk_level, etc.) but does not disclose potential issues like network dependency or fallback behavior beyond mentioning fallback_used. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise with Args and Returns sections, no unnecessary words. Front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only one parameter and an output schema, the description sufficiently covers purpose, input, and output. Minor gap in usage guidelines but overall adequate for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds 'neutral place name' with examples, providing crucial context for the location parameter beyond just type string.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'gets' and resource 'safety-relevant weather context'. Distinguishes from siblings like atlas_aqi_context (air quality) and atlas_civic_signal (civic signals) by focusing on weather.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicitly suggests use for safety-relevant weather but no explicit when-to-use or when-not-to-use. No mention of alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.