Server Quality Checklist

Tool Scores

• get_downtime_kpiC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description fully bears the burden of behavioral transparency. The description only states it computes metrics, with no mention of side effects, permissions, rate limits, or data requirements. This is insufficient for a tool that presumably queries or processes data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short with the main sentence front-loaded. The 'Args:' section adds format info but is somewhat redundant given the schema. Overall, it is concise, though the parameter list could be omitted from the description without loss.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so return values need not be explained. However, the description lacks details on input assumptions (e.g., need for 'production gaps' data), constraints on the time range, or how it differs from other KPI tools. It is adequate but could be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists the two parameters and specifies they are ISO 8601 strings, adding format context. However, it does not explain what the parameters represent (e.g., the time range for metrics computation) beyond the schema titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool computes downtime and availability metrics based on production gaps, clearly indicating the verb and resource. However, it does not specify which exact metrics are computed, leaving some ambiguity but still distinct from sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives like get_downtime_alarms_analysis or get_kpi_summary. No context or exclusions are provided, leaving the agent with no decision support.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_downtime_alarms_analysisB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It only states the tool identifies alarms during downtime, but does not disclose whether it is read-only, performance implications, authorization needs, or any constraints. The behavioral traits are inadequately covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one sentence for purpose and two lines for parameter format. No unnecessary words. The structure is front-loaded with the core purpose followed by parameter specifications.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While an output schema exists (reducing need to describe returns), the description lacks contextual details like prerequisites, data freshness, interpretation guidance, or how this tool relates to alarm monitoring workflows. For a diagnostic tool with zero annotations, this is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add meaning. It specifies that start_time and end_time are ISO 8601 strings, which adds format info beyond the schema's 'string' type. However, it does not explain the meaning of the time range (e.g., alignment with downtime periods) or provide examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it identifies alarms active during downtime periods for root cause diagnosis. The verb 'identifies' and resource 'alarms' are specific, and it clearly distinguishes from sibling tools which focus on KPIs (downtime_kpi, productivity_kpi, quality_kpi).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for downtime alarm analysis but provides no explicit when-to-use or when-not-to-use guidance. No alternatives are mentioned, though siblings are different in nature.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_kpi_summaryB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states it bundles KPIs, without mentioning auth needs, data scope, performance implications, or any side effects. This is insufficient for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very short and front-loaded, with no wasted words. It efficiently conveys the tool's purpose and parameter format. However, it could include more context without becoming verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (two parameters, no nested objects) and the presence of an output schema, the description covers the basic function. However, it lacks guidance on usage context and does not compensate for missing annotations. It is adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must add meaning. It provides format ('ISO 8601 string') which adds value beyond the bare schema, but does not explain the expected time range, relationship between start and end, or constraints. The parameter names are self-explanatory, limiting the need for extensive semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool bundles three specific KPIs (Productivity, Quality, Downtime) into a single summary, using a clear verb+resource structure. It distinguishes itself from sibling tools by consolidating what they do individually.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining a combined KPI view, but lacks explicit guidance on when to use this tool versus the individual KPI siblings. No conditions, prerequisites, or alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_productivity_kpiB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations and the description does not disclose any behavioral traits such as read-only, auth needs, or rate limits. The description only states the tool computes metrics, which is insufficient for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise docstring-style description with parameter list. Could be more front-loaded, but no superfluous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema exists, so return values are covered. However, the description lacks details on what 'productivity metrics' includes and any limitations on the time range.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 0%, the description provides ISO 8601 format examples for start_time and end_time, adding meaningful guidance beyond the schema's bare 'string' type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it computes productivity metrics for a time range, distinguishing it from sibling tools like get_downtime_kpi or get_quality_kpi.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs alternatives like get_downtime_kpi, or what constraints apply (e.g., maximum time range).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_quality_kpiA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It states metrics are 'computed', implying a read-only operation, but does not confirm safety, data freshness, or any constraints like time range limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: two sentences for purpose then parameter list. Front-loaded with intent, no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple KPI tool. Covers parameters and basic output metrics, but missing details on aggregation level, error handling for invalid ranges, and description of return structure (though output schema exists).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds specific format information for both parameters (ISO 8601 strings), which goes beyond the schema's type-only definition. However, it does not explain acceptable date ranges or timezone handling.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool computes quality metrics (Yield %, Defect Rate %) for a given time range. The specific metrics differentiate it from sibling KPI tools (downtime, productivity, summary).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Does not provide any guidance on when to use this tool versus alternatives. No mention of prerequisites, exclusions, or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.