Server Quality Checklist

Tool Scores

• optimize_energy_scheduleC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states the core behavior (delaying non-urgent jobs) but omits important details such as whether the job_queue is permanently modified, what happens to urgent jobs, how 'non-urgent' is determined, or whether any side effects (e.g., missed deadlines) may occur. The behavior is only partially transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundant words. It effectively communicates the high-level purpose in an efficient manner, earning its place despite being brief.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no annotations, and no output schema, the description is too thin. It fails to provide necessary context about the function's return value, error handling, side effects, or how the parameters interact. The description is not complete enough for an agent to confidently invoke the tool without additional information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameter meanings. It indirectly references 'energy prices' and 'off-peak hours', which relate to current_energy_price and threshold_price, and 'non-urgent jobs' to job_queue, but it does not clarify the exact semantics or relationships between parameters. For example, whether threshold_price is a maximum or minimum threshold is ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('checks', 'delays') and names the resource (energy schedule, jobs). It clearly indicates the tool's role in reducing electricity costs by shifting non-urgent jobs to off-peak hours. While it doesn't explicitly contrast with sibling tools, the action and context are distinct enough given the sibling names.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. It implies usage for cost savings but does not mention scenarios where this tool should not be used, nor does it reference sibling tools or other optimization strategies. The context is only implied by the phrase 'to save electricity costs.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• predict_maintenance_windowC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. The verb 'predicts' implies a read-only analysis, but the description does not explicitly state that no data is modified, nor does it describe output format, error conditions, or any side effects. This lack of explicit transparency is a significant gap for a tool with no annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence of 11 words, with no redundant or filler content. It is well-structured and front-loaded with the core purpose. Every word contributes to the meaning, making it highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, no annotations, and incomplete parameter coverage, the description does not provide enough context for a complete understanding. It does not explain what the prediction output looks like (e.g., probability, time window, binary outcome) or the meaning/usage of parameters. The tool is simple, but the description leaves key details unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description fails to mention or explain any of the two required parameters (machine_id, sensor_history_hours). The phrase 'live sensor vibration from database' hints at sensor data but does not map to the parameters. The description provides no context for what these parameters mean or how they are used.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Predicts machine failure') and specifies the data source ('live sensor vibration from database'). This specific verb+resource combination clearly distinguishes it from the sibling tools, which handle sensor normalization, flow rerouting, energy scheduling, parameter adjustment, and compliance auditing. Although the tool name references 'maintenance window' and the description says 'machine failure', the intent is clear and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no explicit guidance on when to use this tool versus alternatives. It does not describe any preconditions, exclusions, or alternative tool references. The usage is implied by the purpose statement but not elaborated, leaving the agent without clear decision context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• adjust_machine_parametersA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It adds a useful internal behavior ('Fetches current recipe internally from the database') and implies a mutation, but does not disclose side effects, reversibility, safety considerations, or prerequisites. This is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences front-loading the main action, then adding one valuable internal detail. Every word earns its place, with no redundant or vague phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This tool performs a potentially impactful adjustment and has no annotations or output schema. The description explains the core purpose but omits parameter semantics, expected return values, failure modes, permissions, and any warnings about the mutation. Given the tool's complexity, this is insufficient for full autonomous invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining parameters. It does not mention machine_id or defect_data at all. The only weak hint is 'dimension oversize' which loosely maps to defect_data.dimension, but there is no real semantic guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action (calculates and adjusts) on a specific resource (machine tool offsets) with a clear purpose (to fix quality defects like dimension oversize). It is unambiguous and easily distinguishes this tool from its unrelated siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: use when a quality defect such as dimension oversize exists. It lacks explicit exclusions or alternative-tool references, but the sibling tools are sufficiently different that this is not a significant gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reroute_node_red_flowA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'dynamically changes' but does not explain side effects (e.g., whether the change is persistent, reversible, or what happens to the original flow). It also does not discuss validation, error conditions, or permissions, leaving significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no superfluous words. It front-loads the action and provides the essential context in a compact form.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three simple string parameters and no output schema, the description captures the core purpose but omits critical context such as failure handling, whether the operation is reversible, and the potential impact on the Node-RED environment. Given that this tool modifies production configuration, more detail is needed for complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description clarifies the roles of the parameters: failed_machine (source), backup_machine (destination), and product_id (the data stream). However, it does not explain value formats, constraints, or how these relate to the Node-RED flow, so it only partially compensates for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb ('changes') and resource ('Node-RED flow configuration'), with a specific outcome ('reroute production data from a failed machine to a backup machine'). This distinguishes it from sibling tools like adjust_machine_parameters or optimize_energy_schedule, which deal with different aspects of the same domain.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case (failed machine requiring rerouting) but does not explicitly state when to use it, when not to use it, or alternatives. It lacks exclusions or prerequisites, so the guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_compliance_audit_trailA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only states what the tool does (compiles and generates a PDF) but does not mention side effects, permissions, output handling (e.g., where the file is saved), or any potential destructive actions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant wording. It efficiently conveys the core purpose and expected output without wasting words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple 2-parameter schema and no annotations or output schema, the description provides the essential purpose but lacks important contextual details such as how the PDF is returned or what server-side effects occur. It is minimally viable but leaves questions about the tool's behavior beyond the primary action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the parameters 'batch_id' or 'deviation_event'. While 'after a deviation' hints at deviation_event, there is no explicit mapping or guidance on what values to provide, so the description fails to compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: compiling sensor logs and operator actions into a PDF audit report for FDA/ISO compliance. This distinct action and resource set it apart from sibling tools, which all focus on different domains like sensor normalization or energy scheduling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'after a deviation' provides clear context for when to use the tool. However, it does not explicitly mention alternatives or exclusions, though sibling tools are unrelated, making ambiguity low.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• normalize_sensor_tagsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description alone must disclose behavior. It states the transformation and the target standard, but it does not mention whether the operation modifies the input data, returns a new list, or handles unknown sensor names. The lack of information about side effects or return behavior is a notable gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the core purpose, the second gives usage guidance. It is front-loaded, concise, and contains no redundant or irrelevant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a single array parameter and no output schema, the description covers the purpose and trigger but omits the return value or output format. Given that the agent would need to know what the tool returns after normalizing, the description is somewhat incomplete, though the simplicity of the operation keeps it at a minimum viable level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no parameter descriptions (0% coverage), and the description provides minimal semantic help. It hints that 'raw weird sensor names' are the raw_tag values, but it does not explain the structure of the 'sensors' array or the role of 'machine_id'. The description fails to compensate for the schema's lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Translates raw weird sensor names into industry standard (ISA-95) formats.' This specifies the verb ('Translates'), the resource ('raw weird sensor names'), and the output standard ('ISA-95'), which distinguishes it from sibling tools like 'optimize_energy_schedule' and 'predict_maintenance_window'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit trigger: 'Call this when new machine data arrives.' This is clear and actionable, though it does not explicitly state when NOT to use the tool or name alternative tools. Since the siblings are functionally distinct, the guidance is adequate without exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.