Server Quality Checklist

Tool Scores

• eia_get_facet_optionsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint. The description adds the exact return format (`{route, facet_id, total, options: [...]}`) and a concrete usage example. This goes beyond the annotations and helps the agent anticipate results and next steps. No contradictions found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences plus a code block. It is front-loaded with the core purpose, then immediately provides return format and usage guidance. No filler or redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 parameters), the schema fully documents parameters, and the description explains the return format and how to apply the results. The output schema exists (though not shown), and the description fills in the key integration detail with eia_get_data. The only minor gap is not clarifying the `openWorldHint` implications, but this is a small miss given the overall clarity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed descriptions for both `route` and `facet_id`, including examples. The description does not add new parameter-level information beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'List the valid option values (ids) for one facet of a dataset.' This uses a specific verb ('list'), names the resource ('facet options'), and is distinct from sibling tools like browse_routes, list_facets, and get_data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains how to use the returned ids: 'Use the returned ids as values in the `facets` argument of eia_get_data' with an example. This gives clear context for when to use the tool. It does not explicitly exclude alternatives or mention when not to use it, but the usage scenario is well defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• eia_browse_routesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true and openWorldHint=true, but the description adds valuable behavioral details: behavior depends on node type, returns different response structures, and tolerates leading/trailing slashes and 'v2/' prefix. It does not contradict annotations. Minor deduction because it doesn't discuss rate limits or error handling, but the added context is substantive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a one-sentence purpose, a bulleted breakdown of node-type behavior, and a final usage directive. Every sentence earns its place, and the most important guidance is front-loaded. It is detailed but not bloated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present (has_output_schema=true), the description correctly avoids explaining raw return fields. It explains the two response types, how to interpret them, and how to feed results into eia_get_data. This is complete for the tool's complexity and context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single 'route' parameter, and the schema already documents the tolerated slash/prefix behavior and examples. The tool description reinforces this but doesn't add meaning beyond the schema. Baseline 3 is appropriate because the schema carries the parameter documentation burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it 'Explores the EIA dataset tree, or fetches a leaf dataset's metadata' with a specific verb and resource. It distinguishes itself from sibling tools by explicitly calling itself 'the primary discovery tool' and referencing eia_get_data, making its role in the workflow unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use guidance: 'Start with route="" to discover datasets, then drill down. This is the primary discovery tool; call it before eia_get_data...' It also explains the two node types and what the agent should do with each response, making the usage context very clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• eia_list_facetsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as read-only and open-world, so the safety profile is covered. The description adds the exact return structure ({"route", "facets"}) and clarifies that it is a discovery step in a multi-tool flow. While it doesn't discuss edge cases or failure modes, it provides useful behavioral context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences plus an inline return-type example. It's front-loaded with the core purpose, then gives a concise workflow. Every sentence earns its place; there is no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for this simple one-parameter tool. It states the purpose, provides the return format (which is also reflected in the output schema), and explains the relationship with sibling tools. Combined with the schema and annotations, an agent has everything needed to invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes the route parameter with an example and a pointer to eia_browse_routes, so schema coverage is 100%. The description itself does not add parameter-level meaning beyond what the schema already offers; it focuses on the workflow. Given the complete schema, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with a specific verb+resource: 'List the facet ids you can filter a dataset by.' This clearly distinguishes the tool from sibling tools like eia_browse_routes (listing routes), eia_get_facet_options (enumerating values for a facet), and eia_get_data (fetching data). The purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit workflow: after listing facets, pass a facet id to eia_get_facet_options, then use the values in eia_get_data. It also references eia_browse_routes for route discovery in the parameter description. This is concrete, actionable guidance on how and when to use the tool within the larger API.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• eia_get_dataA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint annotation, the description details the exact response structure (route, frequency, total, offset, has_more, next_offset, data, warnings) and explains pagination behavior via next_offset. It also discloses a behavioral quirk: incorrect data columns may yield empty rows. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured: a one-sentence summary, a compact response payload block, and a brief workflow. Every sentence adds value, including the caveat about data columns. It is appropriately sized for a 9-parameter tool with pagination and facets, and it front-loads the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, facets, frequencies, pagination), the description covers the essential context: how to obtain valid parameters via sibling tools, the response format, pagination using next_offset, and a key failure mode. The presence of an output schema description further supports completeness. No significant gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema describes all parameters well. The description adds workflow-level semantics: connecting route to data_columns, frequencies, and facet ids, and explaining that invalid data columns can cause empty results. This extra context goes beyond the schema's per-parameter descriptions, though the schema still carries the bulk of the meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Query rows from an EIA dataset with filtering, sorting, and paging,' which clearly states the verb (query), resource (EIA dataset), and operations (filtering, sorting, paging). It distinguishes this tool from siblings like eia_browse_routes (route discovery) and eia_get_facet_options (facet values) by focusing on data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The workflow section explicitly instructs using eia_browse_routes to find the route and metadata, then eia_get_facet_options for facet values, then 'call this tool.' It also warns about potential empty rows if data columns are incorrect. This provides clear context and alternatives, though it doesn't explicitly state 'do not use this tool for discovery' or list exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.