Server Quality Checklist

Tool Scores

• approveA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It reveals that setting approved=False rejects the action and that the tool works across sessions/devices. However, it does not mention potential side effects, error conditions (e.g., invalid approval_key), or idempotency, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences. The first sentence states the main purpose, and the second adds cross-session capability. No redundant or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the sibling tools and presence of an output schema (not shown), the description covers core functionality and cross-session behavior. It does not specify how to obtain approval_key or what 'pending' means, but these can be inferred from context. Reasonably complete for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 3 parameters with 0% description coverage. The description provides meaning for approved (set false to reject) but does not explain approval_key format or the comment parameter. It partially compensates but lacks full detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Approve (or reject) a pending agent action to resume the durable run.' It uses specific verbs (approve/reject) and identifies the resource (pending agent action). This distinguishes it from sibling tools like get_status, run_goal, and schedule_goal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (to approve/reject a pending action) and notes it works across sessions/devices. It does not explicitly mention when not to use it or compare to alternatives, but the context is clear enough for an AI agent to select it over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statusA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It discloses that the tool returns status, progress, result, and pending_approvals, and explains that pending_approvals are actions the agent paused on. No destructive behavior is implied, and it is clear that this is a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences that first state the purpose and then provide key behavioral detail about pending_approvals. No filler words, good front-loading.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one required parameter and an output schema present, the description covers the return values (status, progress, result, pending_approvals) and the relationship to the 'approve' tool. It could mention error cases but is adequate for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description does not elaborate on the 'workflow_run_id' parameter beyond its name. The parameter is self-explanatory as an ID, but the description adds no additional context or formatting guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves status, progress, result, and pending approvals for an agent run. It distinguishes from siblings: 'approve' is for approving actions, 'run_goal' and 'schedule_goal' are for execution, while this tool is for retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use: to check run status and pending approvals. It mentions that pending approvals can be acted on with the 'approve' tool, providing context for usage. However, it does not explicitly state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_goalA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses key behaviors: immediate return, durability, server-side execution, survival of disconnection. No annotations exist, so description carries full burden; lacks error handling or auth details but is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each earning its place: first defines purpose, second explains behavior and next steps. No redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (one parameter, output schema exists), the description covers the essential workflow. Could mention the output schema explicitly, but it's not missing critical information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fails to elaborate on the goal parameter beyond its name. No format, examples, or constraints provided, leaving the agent to infer valid inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool starts a durable Highway agent that pursues a goal to completion. It distinguishes from sibling tools like schedule_goal and get_status by focusing on immediate execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear guidance: returns a workflow_run_id, runs server-side, suggests using get_status for follow-up. Does not explicitly exclude alternatives or mention prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• schedule_goalA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully covers behavioral traits: it explains that the agent runs unattended, that scheduling serves as authorization, and that it uses durable cron (survives restarts, no history bloat). This goes beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: the first sentence states the core purpose. Subsequent sentences add critical details without fluff. Every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, cron scheduling), the description covers usage, parameters, behavioral nuances, and return value. It is complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explains the 'cron_expression' parameter in detail with format (5-field cron UTC), examples, and meaning. The 'goal' parameter is clear from context. Since schema coverage is 0%, the description compensates effectively, adding significant meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action: 'Schedule an agent to run `goal` repeatedly on a cron schedule (UNATTENDED).' It specifies the verb 'schedule' and resource 'goal', and the 'UNATTENDED' qualifier distinguishes it from sibling tools like 'run_goal' (which likely runs once with oversight).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context that scheduling is for unattended runs and warns that only trustable goals should be scheduled because there is no per-run approval. This implies when to use but does not explicitly compare to alternatives like 'run_goal' or 'get_status', leaving some ambiguity about when to schedule vs. run manually.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.