Server Quality Checklist

Tool Scores

• solaredge_diagnosisB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose behavioral traits such as idempotency, side effects, authentication requirements, or rate limits. It only describes the output content, which is insufficient for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences plus a line for parameters. It is front-loaded with the key purpose ('System health:'). However, it could be more structured, e.g., by separating parameter details more clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has two optional parameters and an output schema. The description covers the main output categories and the check_type values, but lacks detail on the effect of 'include_recommendations' and does not address usage context like time ranges or prerequisites. Given the existence of an output schema, some gaps are acceptable, but overall completeness is moderate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds value for the 'check_type' parameter by listing possible values ('health|performance|comprehensive') that are not present in the input schema. However, the 'include_recommendations' parameter is not explained, and schema coverage is 0%, so the description only partially compensates for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: diagnosing system health including inverter status, firmware, data freshness, and optionally battery state of charge and health. It distinguishes from sibling tools like energy_analysis, forecast, and live_status, which focus on different aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for health diagnosis but does not explicitly state when to use this tool versus its siblings or when not to use it. No guidance on prerequisites or exclusions is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solaredge_energy_analysisA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains key metrics (ac_production vs pv_generation) and parameter details, but omits information about read-only nature, data freshness, authentication, or potential side effects. The description adds value but is incomplete for a tool with zero annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is relatively concise at 4-5 sentences, with key information front-loaded (energy balance and derived metrics). It uses line breaks to separate concepts, making it scannable. It could be slightly more structured, but it earns its sentences without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (so return values are covered), 4 parameters with 0% schema coverage, and no annotations, the description covers essential usage details including special metrics explanation. It lacks coverage of error conditions or data availability, but overall it is complete enough for a 4-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It effectively explains time_range values (today|1d|1w|1m|3m|1y|custom), start_date/end_date format (YYYY-MM-DD HH:MM:SS), and meters as a comma-separated list of specific options (Production,Consumption,SelfConsumption,FeedIn,Purchased). This adds significant meaning beyond the schema, though it doesn't detail each meter's effect.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides 'Energy balance and derived metrics (self-consumption rate, self-sufficiency) over a period.' It distinguishes from sibling tools (diagnosis, forecast, live_status) by focusing on historical analysis. The purpose is specific and understandable, though the verb 'analyze' is implicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides detailed guidance on parameter usage, including time_range values, date formatting for 'custom', and the optional meters parameter. However, it does not explicitly state when to use this tool versus siblings, nor does it mention when not to use it or provide alternative tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solaredge_forecastA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must disclose behavior. It states the heuristic nature and lack of weather data, which is informative. However, it does not cover other behavioral aspects like data source specifics, accuracy, or potential failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two short sentences. The key information is front-loaded: the purpose and method. Every part adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema, the description provides essential purpose and a parameter hint. However, it lacks guidance on when to use this tool vs siblings and does not explain the effect of lookback_days on the forecast.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning for forecast_type by listing possible values, but completely ignores lookback_days, leaving its semantics unclear beyond the schema's type and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides a heuristic production estimate using historical daily average without weather data, and specifies the forecast_type parameter. This distinguishes it from siblings by emphasizing the simplicity and lack of weather integration.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through 'heuristic' and 'no weather data', but does not explicitly state when to use this tool over siblings for diagnosis, analysis, or live status. No alternative tools or exclusions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• solaredge_live_statusA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description provides detailed behavioral context, clarifying that energy_today_wh is AC production and pv_generation_today_wh includes DC-coupled battery energy. With no annotations, this fully informs the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no fluff. The first sentence provides an overview, the second clarifies a technical distinction. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema and no parameters, the description covers all necessary context: what data is returned and how to interpret key metrics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so schema coverage is 100%. The description appropriately adds no parameter information, meeting the baseline for a zero-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns current power, energy flow, battery charge, and production metrics. It distinguishes itself from siblings like diagnosis and analysis by focusing on live instantaneous status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for live monitoring, but does not explicitly state when to use vs. alternatives. However, the context and sibling names make it clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.