Server Quality Checklist

Tool Scores

• list_tablesA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the operation and an ordering hint, but does not explicitly confirm this is a read-only metadata operation, mention permissions, or note any rate limits or side effects. The agent must infer safety.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with the core function front-loaded and the usage hint second. There is no filler, redundancy, or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter discovery tool with an output schema, the description covers what the tool does and when to call it. It is slightly incomplete in not contrasting with sibling tools, but overall it is sufficient given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty input schema with 100% coverage. No parameter documentation is needed, so the 0-params baseline of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all tables') with specific output detail ('column names and types'). This distinguishes the tool from data-returning siblings like preview_table or query_data by focusing on schema metadata, though it does not name any alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit sequencing guidance: 'Call this first to understand what data is available before querying.' This tells the agent when to use the tool in a workflow, but it does not mention when not to use it or contrast with alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_tableA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry the behavioral burden. It conveys a read-only preview behavior and a row limit via 'first few rows', but does not disclose whether column types/headers are included or any output caps beyond the default, leaving gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two tight sentences with the primary behavior first followed by a concrete usage hint. No filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple preview tool with an output schema, the description covers purpose and a usage scenario, but remains thin on parameter semantics and alternative tools. It is adequate but not rich enough to fully guide an agent without additional inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not name or explain the rows parameter or explicitly map table_name to a tool argument. It only tangentially implies table via 'a table', which is insufficient compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action (show first few rows of a table) and its goal (understand structure). This clearly distinguishes it from siblings like list_tables, which lists table names, and query_data, which performs arbitrary queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs to use the tool after list_tables, giving a clear sequence. However, it does not mention when to prefer query_data over this tool for more complex data exploration, so it lacks explicit when-not/alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_climate_alignmentA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses the calculation logic (historical years use reported CO2, future years fall back to targets/IRP projections), the gap column formula, and case-insensitive partial-name search. This is substantial behavioral context beyond what the schema shows.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and well-organized: purpose first, then output/calculation details, then parameter conventions. Every sentence adds useful information and there is no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the essential behavioral and parameter details needed to invoke the tool correctly. It does not describe the output schema, but an output schema exists, so that is not a gap. It is slightly weaker on explicit sibling differentiation but still contextually complete for the tool itself.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the 'basis' parameter's allowed values and default, and it clarifies that 'utility_name' supports partial and case-insensitive matches. This adds real meaning over the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action and resource: 'Compare a utility's emissions to RMI's 1.5°C pathway, year by year.' It clearly distinguishes this from sibling tools by focusing on the RMI benchmark and the added gap/status columns, so an agent can identify its unique function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context, such as referencing get_emissions_trend for basis definitions, but it does not explicitly state when to use this tool versus sibling alternatives or when not to use it. Usage guidance is implied by the purpose rather than spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_utilitiesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It discloses what is returned, that capacity only appears when a state is given, that results are ordered by capacity largest first, and that name_contains is case-insensitive. It does not mention limit behavior or pagination, but for a read-only list tool the key behaviors are transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tight and front-loaded: main function first, return details second, usage guidance last. Every sentence adds useful information and there is no redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list utility with an output schema, the description is nearly complete: filters are explained, the conditional return field is described, and the intended workflow is stated. The only real gap is the undocumented limit parameter and the resulting size/behavior of the response.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains state_abbr with examples and the conditional capacity output, and explains name_contains with case-insensitive substring semantics. However, the limit parameter is not mentioned at all, leaving one of three parameters undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('List utilities') and goes beyond the name by detailing the returned fields (utility/parent names, EIA/FERC1 IDs, RMI type, capacity). It also distinguishes itself from emission/generation tools by positioning this as the lookup step before pulling that data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear use case: 'Use this to find the right utility name before pulling emissions or generation data.' It also explains the optional filtering behavior. However, it does not explicitly name sibling alternatives or state when not to use this tool, so it stops short of full when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_emissions_trendA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does so well: it discloses returned quantities with units, explains that the basis parameter selects separate row sets ('never sum across them'), and specifies that search is case-insensitive and supports partial names. It also clarifies the meaning of the 'owned_delivered' column for basis='all'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The main purpose is front-loaded and the remaining sentences each add functional value: outputs, units, basis semantics, row-set warning, and search behavior. The basis values are presented in a scannable list format that is easy for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with an output schema, this description tells the agent everything needed to call it correctly: what data comes back, units, boundary choices, a warning about summing across bases, and name matching behavior. The only notable omission is sibling comparison, which is already covered under usage guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the text must explain the parameters, and it does. 'basis' is fully documented with its three allowed values, defaults, and accounting meaning; the description also clarifies utility_name behavior via case-insensitive partial matching. No parameter is left semantically opaque.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action and object — 'Get yearly CO2 emissions and 1.5°C pathway comparison for a utility' — and enumerates the exact outputs (CO2, targets, IRP projection, RMI benchmark, generation, intensity), which makes the tool's role clear. It does not explicitly draw boundaries against siblings like get_generation_mix or get_climate_alignment, so some differentiation is left to inference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The use case is implied by the detailed output list, and the basis options tell the agent how to choose accounting scope. However, no guidance is given about when to prefer this tool over the overlapping sibling tools, and there are no when-not-to-use conditions or alternative routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_generation_mixA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does so excellently. It discloses the ownership default, the existence and risk of non-owned rows, the behavior of include_purchases, case-insensitive partial-name search, and the default year range. It even warns about the overstatement pitfall from mixing purchased supply with owned generation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and output metrics, then logically progresses through the ownership default, caveat about non-owned rows, and optional parameters. Every sentence adds necessary operational detail, and the structure makes the information easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a data-retrieval tool with an output schema. It covers defaults, edge cases, parameter behavior, and the important conceptual distinction between generation and purchased supply. Nothing needed for correct invocation is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by explaining all three parameters: utility_name supports case-insensitive partial matches, year is optional and defaults to all years 2005-2024, and include_purchases is thoroughly described with its effect on row splitting and columns. This adds substantial meaning beyond the bare schema types and defaults.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Get electricity generation breakdown by technology for a utility.' It also names the output metrics (capacity, net generation, capacity factor, CO2 emissions) and technology groups, making it clearly distinguishable from siblings like get_emissions_trend and get_climate_alignment.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear guidance on the default owned-generation scope and explains exactly when to set include_purchases=True. It does not explicitly name alternative tools or state when not to use this tool, but the context it provides is sufficient for most selection decisions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_dataA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full behavioral burden. It clearly states the operation is read-only, limits each call to one SELECT statement, reveals join-key incompatibilities, and warns about the owned/delivered row-set and generation/purchased-power mixing traps. These are meaningful behavioral disclosures beyond a generic 'run a query.'

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although long, the description is tightly organized: purpose, prerequisite, constraints, join-key reference, known traps, and an example. No sentence is wasted; the density is justified by the complexity of the database schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complex multi-table schema and existing output schema, the description is remarkably complete. It identifies all relevant tables, join keys, the table that lacks an ID, and both data-quality traps. It also delegates table/column discovery to list_tables() rather than trying to inline the full schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and the only parameter is an undocumented 'sql' string. The description fully compensates by defining acceptable syntax, query structure, join constraints, required filters, and providing a concrete example. An agent can confidently construct a correct SQL value from this guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Run a read-only SQL query against the database.' It also clarifies the SQL dialect (DuckDB), immediately distinguishing this general-purpose query tool from the specialized sibling get_* and table-inspection tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives strong usage guidance: call list_tables() first, one SELECT per call, CTEs allowed, join-key mapping per table, and filter traps to avoid double-counting. It does not explicitly compare this tool to specialized siblings like get_emissions_trend, so the 'when vs alternatives' guidance is slightly implicit rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.