Server Quality Checklist

Tool Scores

• identify_machineA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses idempotent behavior ('calling again returns same mint_id with created: false') and what is returned (mint_id and wallet_address). However, it does not mention side effects, auth requirements, or rate limits. Still, it provides useful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured, front-loading the purpose, then providing examples, then idempotency, then usage guidance. It is slightly long but every part earns its place. Could be more concise without the extensive list of OEMs, but that adds clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has an output schema (mentioned in context signals), the description need not explain return values in detail. It covers core functionality, usage context, and behavior. The optional parameters are not explained, but the core is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions oem, model, serial as key triple, but does not explain the optional parameters site and metadata. The description adds some meaning for required parameters but incompletely covers all five.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Provision or retrieve a persistent on-chain identity (mint_id) for any industrial machine.' It specifies the verb (provision/retrieve), the resource (on-chain identity), and distinguishes from sibling tools by stating it should be called before normalize_telemetry or create_automation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a 'USE WHEN' section that specifies when to use the tool: when a user references a specific machine by OEM/model/serial and needs a stable handle. It also advises to call this first when a new machine is introduced. While it doesn't explicitly mention when not to use it, the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• disable_automationA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description discloses that the trigger stops evaluating incoming calls but preserves its configuration and history. It also mentions re-enabling via PATCH. However, it does not address permissions, rate limits, or the effect on existing history. The disclosure is good but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is informative with two paragraphs: first explains behavior, second provides usage examples and distinction. It is not overly verbose, but could be slightly more concise by removing the PATCH instruction (external to the tool). Overall well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter, no annotations, output schema exists), the description covers behavior, usage context, and sibling differentiation. It does not mention error states or prerequisites, but for a pause operation this is adequate. Slightly more detail on the output would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description implies a single parameter (trigger_id) by referencing 'the trigger' and the PATCH endpoint, but it does not explicitly describe the parameter or its expected format. Given 0% schema coverage, the description should compensate more; however, the purpose is clear enough that the parameter is likely the trigger ID.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool pauses an automation trigger without deletion, preserving configuration and history. It distinguishes from sibling delete_automation (permanent) and implies relationship with activate_automation. The verb 'pause' and resource 'automation trigger' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides 'USE WHEN' with examples of temporary stoppage (e.g., planned maintenance) and distinguishes from permanent deletion. This gives clear guidance on when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_webhook_historyA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses key behavioral traits: it returns up to `limit` rows (with default and max), lists the returned fields, and explains that settled_tx appears only after batch settle. It does not explicitly state it is read-only or mention permissions, but the read nature is implied by 'Show' and the context is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: the first sentence states the purpose, followed by a clear list of returned fields, and a dedicated 'USE WHEN' section. Every sentence earns its place without unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the existence of an output schema (implied by the listed fields), the description already covers the key return values and behavioral nuance (e.g., settled_tx timing). It is complete for a query tool, though it could mention pagination beyond the limit parameter or error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description compensates by explaining the purpose of `trigger_id` (the webhook trigger) and providing details for `limit`: default 10, max 200, and that it returns the most-recent rows. This adds significant meaning beyond the bare schema types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it shows webhook delivery history for a trigger, using a specific verb ('Show') and resource. It distinguishes from sibling tools, which are about automation management and machine history, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides usage scenarios with example user queries ('did the alert actually go out?') and notes that soft-deleted triggers can be queried, offering clear context. However, it lacks explicit 'when not to use' or alternative tool suggestions, which are not critical given the tool's unique purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fire_sandboxA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool makes an HTTP POST to a sandbox endpoint, hashes the response, and anchors it on Solana mainnet. It also mentions the lifetime limit of 10 fires. This is sufficient transparency for a demo tool, though it could specify authorization needs or potential costs (but the 'Free tier' note implies no cost).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: first paragraph states purpose and output, second explains the process, third gives usage guidelines. Every sentence earns its place with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage context, parameter semantics, behavior, and output. An output schema exists, and the description adequately explains the return values (echo body, tx_signature, verify_url). No gaps for this tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions. The tool description adds value by explaining how parameters are used in the request (e.g., 'POSTs {message, condition: condition_text, ts}'), which goes beyond the schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Demo the full Forge watch→fire→settle loop against a built-in sandbox endpoint.' It uses specific verbs and resources, and the context distinguishes it from sibling tools which focus on automations and queries. No tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a 'USE WHEN' section that explicitly states when to use the tool: 'a developer is evaluating Forge... without onboarding any machines or paying for the Pro tier.' It provides clear context but does not mention when not to use it or name alternatives beyond the implicit context of sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_coverageA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description must disclose behavioral traits. It explains that unknown OEMs still normalize (gate disables itself) and that passing optional oem resolves vertical and checks cross-vertical gate. This adds meaningful context, though it could mention read-only nature or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is well-structured: main purpose, then parameter explanation, then usage guidance. Sentences are efficient and informative, avoiding redundancy. Could be slightly more compact, but current length is appropriate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (handling return values), the description covers purpose, usage, parameter semantics, and behavioral notes. It lacks details on output structure, but that is expected from output schema. Overall, fairly complete for a coverage-check tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must compensate. It explains the optional oem parameter: 'see which vertical it resolves to and whether the cross-vertical gate will engage.' This adds semantic value beyond the mere schema definition, though format details are absent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states what the tool does: 'Ask Forge what it can normalize BEFORE you try' and lists outputs (OEM verticals, field families, field list). This distinguishes it from sibling tools like normalize_telemetry by emphasizing its role as a prerequisite check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly specifies when to use: 'starting a new integration, or deciding whether to call normalize_telemetry'. Also clarifies behavior for unknown OEMs (absence is a soft signal, not a hard block), providing clear guidance on decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_machine_historyA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It explains query options and notes summary mode is 'Always cheap'. However, it doesn't state that the tool is read-only or safe, which is implied but not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, well-structured with bullet points and a clear 'USE WHEN' section. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given moderate complexity (6 params, output schema exists), the description covers purpose, usage, parameters, and best practices comprehensively. No missing context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description must explain parameters. It covers from_dt, to_dt, fields, limit, and summary well (types, defaults, behavior). However, the required parameter 'mint_id' is not directly explained beyond 'identified machine', leaving ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves operational history for a machine, with each row being canonical output from normalize calls. It distinguishes from siblings like normalize_telemetry and get_coverage by focusing on historical queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'USE WHEN' section lists concrete scenarios (utilization, health trends, period comparisons) and advises starting with summary=true. This provides excellent guidance on when and how to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_on_chainA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Given no annotations, the description fully explains the tool's behavior: it anchors data, computes Merkle roots for batches, hashes payloads deterministically, and returns verification links. It does not mention authorization or rate limits, but the core behavior is well covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear headings for each mode and a 'USE WHEN' section. Every sentence is informative and necessary, with no redundancy or fluff. It is concise yet comprehensive.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with three parameters, no required fields, and an output schema, the description covers all relevant context: purpose, modes, return values (though output schema exists), usage scenarios, and user instructions. It is complete and leaves no ambiguity about when and how to use the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains all three parameters (batch, mint_id, payload) by linking them to the two modes, specifying which are required per mode, and describing the payload hashing. The mint_id parameter could use more detail (e.g., its format or origin), but overall it adds significant semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool anchors data on Solana via MINT relay, with two explicit modes (batch and single-payload). It uses a specific verb ('anchor') and resource ('data on Solana mainnet'), and it distinguishes itself from sibling tools which focus on automation and telemetry.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a 'USE WHEN' section with concrete examples and instructs to always include the verify_url. It differentiates between batch and single modes. However, it does not explicitly state when NOT to use the tool, which slightly reduces the score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_automationA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses key behaviors: returns parsed_trigger for human review, does not auto-activate, requires explicit POST to /v1/triggers, includes confirmation_required and notes. Very transparent about lifecycle.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured: first paragraph defines tool, second gives examples, third explains return and activation step, fourth gives usage guidance. No fluff, each sentence adds value. Front-loaded with key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers tool purpose, usage, parameters (partially), return format, and follow-up actions. Mentions webhooks but not authentication or permissions. With only two simple params and output schema referenced, it's mostly complete but lacks security context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage for two required params. Description compensates for 'instruction' with multiple examples, clarifying its natural language format. However, 'mint_id' is not explained; context suggests machine ID but not explicit. Overall adds significant value for instruction param.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool creates automated monitoring and actions for industrial machines using natural language. Specific verb 'create' and resource 'automation' distinguishing it from siblings like 'activate_automation' or 'delete_automation'. Examples reinforce purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'USE WHEN: a user wants to set up monitoring, alerts, or automations for machine state transitions.' Provides clear post-use steps (show parsed_trigger to user, ask confirmation). Lacks explicit when-not-to-use but implied by context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• correct_mappingA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It clearly states that corrections are recorded as corpus-improvement signals, feed an offline retrain, do not hot-patch the live corpus, and that noisy feedback cannot poison others. It also details the return object (ok, feedback_id, action).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and well-structured with a main paragraph and a bulleted parameter list. It is slightly verbose (e.g., 'so every agent interaction makes normalization better' is motivational but not essential), but still efficient overall.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool complexity (7 parameters, output schema exists), the description is remarkably complete. It covers purpose, usage context, parameter meanings, return value, and behavioral implications (offline retrain). No important gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning for 5 parameters (source_field, confirmed_canonical, original_canonical, oem, mapping_id) with clear explanations. However, two parameters (sample_value, confidence) are present in the schema but not described in the description, leaving a minor gap. Baseline for 0% schema coverage is higher, so overall it compensates well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('correct', 'Teach Forge the RIGHT canonical field') and clearly distinguishes this tool from its sibling 'normalize_telemetry' by stating it is used when that tool returned wrong or abstained. The resource (canonical field mapping) is explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-to-use conditions ('USE WHEN: you or the user can see normalize_telemetry returned the wrong canonical...') and explains the context (corrections feed an offline retrain, not live). It does not exclude alternatives but the sibling tools list shows this is unique.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_automationA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses the soft-delete behavior, recoverability (30 days), immediate cessation of evaluation, hiding from list_automations, and preservation of audit trail. It also mentions the permanent deletion parameter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat verbose but each sentence adds value (behavior, recovery, usage condition). It is well-structured and front-loads the key behavior. Minor redundancy could be trimmed, but overall effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple parameter and existence of an output schema, the description covers all essential aspects: what it does, effects, recovery, audit trail, and usage guidance. It is complete for a delete tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter (trigger_id) with 0% schema coverage, but the description makes it clear that the parameter identifies the trigger to delete. No additional validation or format details are given, but the context is sufficient for a single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it soft-deletes a trigger, explains the recovery window, and distinguishes it from permanent deletion. The verb 'soft-deletes' and resource 'trigger' are specific, and it references the sibling tool restore_automation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a dedicated 'USE WHEN' section with concrete examples and contrasts with hard delete. It clearly tells the agent when to use this tool and mentions restore_automation as an alternative for undoing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• normalize_telemetryA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It fully discloses side effects: auto-provisions machine identity, writes to forge_normalized_history, evaluates triggers, and notes asynchronous webhook firing. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is relatively long but well-structured: first sentence defines core purpose, followed by mapping examples, per-call effects in bullet-like format, and a clear 'USE WHEN' section. Could be slightly more concise, but each sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters (1 required), output schema exists, and the tool has side effects, the description is complete. It explains output fields (canonical_data, mapping_id, triggers_fired), database writes, and trigger behavior, leaving no critical gaps for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so description must compensate. It explains the 'data' dict, the optional machine_id, and the oem/model/serial combination for auto-provisioning. However, 'site' parameter is not explained. Overall, adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool translates raw machine telemetry from proprietary formats into universal canonical FCS data. It distinguishes from siblings by mentioning auto-provisioning behavior similar to identify_machine and writing history accessible via query_machine_history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'USE WHEN' section lists three specific scenarios (a, b, c) and implies when not to use (e.g., use correct_mapping for fixing mapping errors). Also notes alternative identify_machine for identity provisioning.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• activate_automationA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully explains behavior: creates live trigger, monitors telemetry, fires actions with variable interpolation on condition match, returns trigger row with id, and notes the impact on machine behavior. Examples of real-world effects are given.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-organized with sections for inputs and returns, but is slightly verbose. Could tighten some sentences without losing clarity. Still, front-loads purpose effectively.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters with nested objects and an output schema, the description covers all necessary aspects: parameter details, behavior, return value, and usage context. No gaps identified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description adds detailed explanations for all 5 parameters, including types, constraints (e.g., name ≤80 chars), structure of condition and actions, and defaults. This compensates fully for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Activate a parsed automation trigger' and the resource 'on a machine'. It distinguishes from siblings like create_automation and disable_automation, providing specific verb+resource context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Call this AFTER create_automation returns a parsed_trigger and the user explicitly confirms' and warns 'Never call this tool without explicit confirmation'. This provides clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• restore_automationA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description fully covers behavioral traits: it describes the 30-day recovery window, re-enabling side effect, return fields, and error codes (410, 409). This is transparent and comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three short paragraphs, front-loaded with the core action and constraints. Every sentence adds value: purpose, return/errors, usage guidance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one simple parameter and an output schema, the description covers purpose, behavior, errors, usage, and expected result thoroughly. It is fully self-contained and meets the needs for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter trigger_id is not explicitly described in the description, but the context ('Restore a previously soft-deleted automation trigger') clarifies its role. With 0% schema coverage, the description compensates by implying the parameter's purpose. A score of 4 is reasonable given the simplicity of the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool restores a soft-deleted automation trigger within a 30-day window. The verb 'Restore' and resource 'automation trigger' are specific, and it distinguishes from siblings like delete_automation or disable_automation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly provides a 'USE WHEN' section that tells when to use (accidental deletion, undo flow) and suggests pairing with disable_automation as an alternative. This gives clear guidance on context and exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_automationsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It details that the tool lists active triggers and returns specific fields. The behavior is fully disclosed for a read-only list operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise (about 5 lines), fronts the purpose, then lists return fields, and ends with usage guidance. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (1 parameter, no annotations, output schema present but not shown), the description covers input, output structure, and usage context completely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but description adds meaning: machine_id should be the machine's mint_id or internal_id, both resolve. This compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states 'List all active automations / triggers configured for one machine.' The verb 'List' and resource 'automations/triggers' are clear. It distinguishes from siblings like create_automation, delete_automation, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit 'USE WHEN:' section provides example user queries (e.g., 'what automations do I have') and instructs to pass the machine's mint_id or internal_id. This is excellent usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.