Server Quality Checklist

Tool Scores

• solar_options_guideD
  1.3/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It only states the domain and 'DEMO', with no info on side effects, data sources, or limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short (5 words), which is concise, but fails to provide essential information. Conciseness without substance is not valuable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite low complexity and an output schema, the description is severely incomplete. It does not cover overview, usage context, parameter details, or return value expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and the description adds no meaning to the parameters. It does not explain what 'use_case' or 'budget_kes' are or how they affect the output.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description mentions the domain (solar options, Kenya households) but lacks a specific verb or action (e.g., 'recommends', 'lists', 'compares'). It does not distinguish from sibling tools like energy_rights_query or tariff_calculator.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. No context about prerequisites or typical scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• energy_subsidy_programsC
  2.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only mentions subject matter and 'DEMO', omitting any details about what the tool does, side effects, or requirements. This is insufficient for an agent to understand behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise but lacks structure. While it is short, it fails to convey essential information, making brevity detrimental rather than helpful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema and a simple input schema, the description provides almost no context. It does not explain what the tool returns or how to use it, leaving the agent underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The sole parameter 'county' is not described at all in either the schema or the tool description, leaving its semantics unknown.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Kenya energy subsidy and access programs. DEMO.' is vague about what the tool does. It does not specify the action (e.g., list, query, apply) and does not distinguish it from sibling tools like 'energy_rights_query' which also deals with energy topics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus its siblings. The word 'DEMO' might imply demonstration purposes but is not explicit, leaving the agent without clear usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• energy_rights_queryC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It fails to mention any behavioral traits such as output format, side effects, or required permissions. The 'DEMO' label hints at limited functionality but is ambiguous.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely short (one sentence) but under-specified. The word 'DEMO' appears unnecessary and adds noise. Conciseness should not come at the expense of completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter) and existence of an output schema, the description could be brief but still should explain the parameter and usage. It fails to be complete even for this minimal case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the sole parameter 'topic'. The schema only specifies its type as string; no meaning or expected values are provided, leaving the agent without guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Consumer rights for electricity in Kenya,' which clearly identifies the tool's domain and purpose. However, the addition of 'DEMO' introduces ambiguity about whether this is a production-grade tool, slightly reducing clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus sibling tools like 'energy_subsidy_programs' or 'kplc_connection_guide'. The description does not offer any context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kplc_connection_guideC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description includes 'DEMO', which suggests non-production behavior but is vague. No other behavioral traits like safety, side effects, or limitations are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, but it lacks structure and omits essential details. It achieves conciseness at the expense of completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two parameters, no schema descriptions, no annotations, and an output schema, the description is grossly incomplete. It fails to explain how to use the tool or what results to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not mention any parameters, and the input schema has no descriptions (0% coverage). It adds no meaning beyond the property names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a guide for new KPLC electricity connections in Kenya. This is a specific verb-resource combination that distinguishes it from sibling tools like energy_rights_query or tariff_calculator.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no indication of prerequisites or context of use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tariff_calculatorC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It states 'Estimate' implying a calculation, but provides no details on behavioral traits like data sources, accuracy, or that it's a demo version. The 'DEMO' tag suggests limited functionality but isn't explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, one sentence front-loading the purpose. The 'DEMO' could be made more informative while maintaining brevity, but overall it is efficient and not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool is a calculator with two parameters and an output schema, the description lacks details about return value format, units, or how the estimate is computed. It provides minimal context for correct invocation, especially considering the 'DEMO' nature.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should explain parameters. It does not mention 'monthly_units_kwh' or 'customer_type', leaving the agent to infer meaning solely from the schema. No added value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool estimates monthly KPLC electricity cost for a Kenya household, with a specific verb and resource. It distinguishes from sibling tools (energy rights, subsidies, connection guides, solar options). However, the 'DEMO' tag introduces some ambiguity about reliability.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description does not mention any prerequisites, limitations, or situations where other tools might be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.