Server Quality Checklist

Tool Scores

• count_earthquakesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral disclosure. It correctly indicates the tool returns a count without event details, but does not describe the exact return format (e.g., integer or JSON) or any side effects. For a non-destructive read tool, this is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and followed by a concrete example. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (count operation) but has 11 optional parameters with low schema coverage and no output schema. The description references search_earthquakes for parameter details, which helps but assumes agent knows that sibling tool. Lacks explicit return format, making it minimally complete for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is very low (18%, only start_time/end_time have descriptions). The description references search_earthquakes for parameter details but does not add per-parameter semantics or clarify how filters combine. Given 11 parameters with minimal schema descriptions, the description should provide more context or grouping, which it does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool counts earthquakes without returning details, using a specific verb and resource. The example query reinforces its purpose, and it is distinct from sibling tools like search_earthquakes which return event details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides a use case ('how many M4+ quakes hit California this year') and states it supports the same filters as search_earthquakes, guiding the agent on when to use count vs search. However, it could more clearly exclude scenarios where event details are needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_earthquakesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the burden. It states default behavior and return format, but does not disclose rate limits, authentication needs, pagination limits (though limit parameter exists), or constraints like conflicting parameters. Adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences covering purpose, filters, location types, defaults, and output. No wasted words. Front-loaded with key action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 13 parameters and no output schema, the description covers main use cases well. It could be more explicit about parameter interdependencies (e.g., circle requires latitude+longitude+radius_km). Almost complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so description adds marginal value. It summarizes location options and defaults, but doesn't elaborate on parameter syntax beyond what schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches the USGS earthquake catalog, specifies filterable fields (time, magnitude, location), and differentiates between bounding box and circle location types. It also lists return fields. This distinguishes it well from sibling tools like count_earthquakes or felt_reports.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions defaults for time range, magnitude, and location, but lacks explicit guidance on when to choose this tool over siblings (e.g., count_earthquakes for counts, felt_reports for human reports). Usage is implied but no 'when not to use' or alternative references.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recent_significantA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description explains the USGS significance score (magnitude, felt reports, estimated impact) and mentions the threshold matches the official feed. This adds valuable behavioral context beyond a simple listing. However, it omits any mention of the response format or pagination.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and contains no unnecessary words. It earns its length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema), the description adequately covers the purpose and filtering logic. It lacks details about return format or sort order, but these are less critical for a list tool with clear parameter constraints.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters described. The description adds no new parameter-specific details beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it lists significant earthquakes from the last N days worldwide, using the USGS significance score. It clearly distinguishes from siblings like search_earthquakes or get_earthquake by focusing on a predefined 'significant' threshold.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving notable earthquakes but does not specify when to use this tool versus alternatives (e.g., search_earthquakes for custom filters, count_earthquakes for totals). No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• felt_reportsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, description carries full burden. It discloses output structure (sorted by time, felt counts, max intensity) and implies a read-only operation. No contradictions, but could mention limits or that results are limited to events with min_felt reports.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences: first describes function, second gives a concrete example. Front-loaded with verb and resource. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers core use case and output. With 6 parameters and no output schema, description explains return fields but could mention pagination or that 'limit' controls maximum results. Still adequate for query tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds overall context but does not provide additional semantics beyond the schema descriptions for each parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Find earthquakes near a location that have 'Did You Feel It' (DYFI) citizen reports' with a specific verb and resource. It distinguishes from siblings like 'search_earthquakes' by focusing on felt reports and including an example query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context with an example ('did anyone feel a quake near San Francisco this week'), but does not explicitly state when not to use or compare alternatives like 'get_earthquake' for details on a single event.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_earthquakeA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description lists returned fields but does not mention any side effects, rate limits, or authentication needs. Reasonable but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundancy. Purpose and details are front-loaded. Efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description fully informs the agent about input requirements, output contents, and tool purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers event_id 100%. Description adds an example and explains the id source (from search tools), providing value beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it gets full details for a single earthquake by USGS event id. It distinguishes from sibling tools (search, count, recent).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Mentions the id is returned by search tools, implying prior use of search. Lacks explicit when-not-to-use or alternatives, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.