Server Quality Checklist

Tool Scores

• describe_coverageB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must convey behavioral traits. It implies a read operation but does not explicitly state it is read-only, safe, or free of side effects, nor does it mention auth requirements or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that immediately conveys the tool's function with no extraneous words. It earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter, output schema present), the description lacks details about the return value, prerequisites, or how the coverage_id relates to other tools. It feels incomplete for a retrieval operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate. It names the parameter's purpose ('for a specific datacube') but does not explain the identifier format, provide examples, or clarify how to obtain the coverage_id.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieves' and the resource 'structural metadata for a specific datacube (coverage)', distinguishing it from sibling tools like list_coverages (lists coverages) and execute_wcps_query (executes queries).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as list_coverages or execute_wcps_query. It does not specify context, exclusions, or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_wcps_queryB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It mentions that binary data results are saved to a temporary file and the path is returned, and instructs to show the query and path to the user. However, it does not disclose error behavior, side effects (e.g., whether queries can modify data), permissions required, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no redundancy: first states purpose, second lists use cases, third explains binary handling and user instruction. Structure is logical and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (query execution), the description covers core functionality and binary handling but lacks output format, error behavior, and permissions. The instruction to show the query and path is helpful but not core behavioral context. Adequate but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'wcps_query' has 0% schema description coverage. The description merely calls it 'the WCPS query' without providing syntax, examples, or constraints. The sibling validate_wcps_query could serve as a reference, but the description does not connect to it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it executes a WCPS query in rasdaman and lists specific uses like subsetting, processing, aggregation, and filtering. This distinguishes it from siblings (describe_coverage, list_coverages, validate_wcps_query, wcps_query_crash_course) which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description suggests when to use the tool ('for spatio-temporal subsetting...') but does not explicitly state when not to use it or mention alternatives (e.g., using validate_wcps_query for validation). The guidance is implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_coveragesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the burden. It clarifies it lists 'available' coverages, implying a read operation. While it could explicitly state it is read-only and has no side effects, the behavior is clear for a simple list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single 7-word sentence. No wasted words. It effectively communicates the tool's purpose in the most concise form possible.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and a presumably descriptive output schema, the description is sufficient. It explains what the tool returns (list of coverages). It could mention that it returns a list, but since output schema likely handles that, this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty (0 parameters), so there is no parameter information needed. Baseline is 4 per guidelines. The description adds no parameter details, as none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Lists all available datacubes (coverages) in rasdaman.' It uses a specific verb ('Lists') and resource ('available datacubes/coverages'), and distinguishes from siblings like describe_coverage which details a single coverage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The purpose implies it's for getting a full list, but no mention of when not to use it (e.g., if you need details on a specific coverage, use describe_coverage). Minimal context for the agent to decide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_wcps_queryA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, but description fully discloses behavior: returns 'VALID' or 'INVALID SYNTAX: <error>'. It is a read-only validation with no side effects mentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with usage guidance followed by return format. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter validation tool with output schema present, description covers usage context, return values, and relationship to siblings. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter (wcps_query) with 0% schema description coverage. The description implies it is the query string but does not elaborate on format or constraints. Adequate for a simple tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates WCPS query syntax, using specific verb 'check' and resource 'query'. It distinguishes from sibling tools like execute_wcps_query which runs the query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use 'before execution', giving clear context. Does not list when not to use, but the sibling execute_wcps_query implies the alternative for running queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• wcps_query_crash_courseA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It transparently states it returns a crash course, implying a read-only, safe operation. No hidden side effects or behavioral traits are omitted, but it could add details like static nature or authentication needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose and a usage recommendation. Every sentence is valuable and concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple informational tool with no parameters and a likely self-explanatory output, the description provides adequate context: purpose and recommended timing. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and schema coverage is 100% trivially. Baseline 4 applies as description adds no parameter information, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns a crash course on WCPS queries, covering syntax, operations, and best practices. It distinguishes itself from sibling tools like execute_wcps_query or validate_wcps_query by being an educational resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description recommends checking this before executing queries, providing clear usage context. However, it does not explicitly state when not to use it or mention alternatives, though siblings are distinct.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.