Server Quality Checklist

Tool Scores

• get_routeA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It states the tool returns a 'primary read-only payload', indicating a non-destructive read. However, it does not mention authentication requirements, rate limits, or any other behavioral traits (e.g., whether the data is cached in real-time).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exceptionally concise with two pithy sentences. Every word serves a purpose: the verb, examples, and return type are front-loaded. No unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter, no output schema), the description is adequate but lacks clarity on the 'primary read-only payload' structure. An agent cannot infer the return format without additional context, which would be helpful for a complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with a description for 'route_id'. The tool description adds extra value by listing example route IDs (dashboard, markets, portfolio, backtest) and noting that 'command center is under settings', which aids correct invocation beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the goal with a specific verb ('Fetch') and resource ('current state of one route'), and provides concrete examples like 'dashboard, markets, portfolio, backtest'. It distinguishes from sibling 'list_routes' by specifying it fetches one route's state, not a list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for fetching a single route's data, but does not explicitly state when to use it versus alternatives such as 'list_routes' for listing all routes or 'run_action' for mutations. No when-not or prerequisite guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• run_actionA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses refusal of certain actions and need for confirmation, but does not mention rate limits, auth, response format, or side effects. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no fluff, all sentences add value. The description is front-loaded with the main purpose and then specifics.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters and no output schema, the description covers parameter usage and safety, but does not mention return values or error handling. For an execution tool, more detail on response would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds significant meaning by linking parameters to specific use cases (body for POST, path_params for placeholders, query optional, confirm for confirmation). This goes beyond the schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Execute' and the resource 'safe terminal action by action_id'. It distinguishes from sibling tools like list_actions and list_routes by focusing on execution.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear guidance on when to pass 'body' (POST), 'path_params' (placeholders), 'query' (optional), and 'confirm' (confirmation). It also mentions what it refuses (safety-disabled, secret). However, it lacks explicit when-not-to-use or alternatives, but context is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_public_dataA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It explains the result summary (written/available/reused cache counts) but lacks details on side effects, idempotency, or rate limits. It is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. Purpose is front-loaded, providing immediate clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description fully explains what the tool does and what it returns. It is complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so schema coverage is 100%. The description adds context about the return summary, which goes beyond the schema. Baseline 3 + 1 for added value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to start a public data refresh job and return a result summary. It distinguishes from siblings by highlighting it's for public data without keys, contrasting with tools like run_action or get_route.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states this is for loading fresh public data without keys or private accounts, guiding when to use it. It does not mention when not to use alternatives, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_actionsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It reveals that actions are filtered (safe, not disabled, not secret) and mentions behavioral aspects like mutating local state and needing confirmation in the output. It could elaborate on safety classification criteria.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the core purpose and exclusions, then detailing output and usage. Every sentence provides essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one optional parameter and no output schema, the description covers what the tool does, what it excludes, what columns are returned, and how to use the result with run_action. No critical gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and description adds value by explaining the route_id parameter as an optional filter with an example ('backtest'). This goes beyond the schema's minimal description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists safe operable actions, specifies exclusions (safety-disabled and secret actions), and lists the columns shown. It distinguishes from sibling run_action by noting the action_id is used with that tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (to list actions) and provides context on what each row contains, but does not explicitly state when not to use it or provide direct alternatives beyond mentioning run_action.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_routesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, but description discloses scope (15 routes), returned fields, and suggests a read-only operation. Does not mention pagination or side effects, which is acceptable given no parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no extraneous information. Action and result are front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Sufficient for a simple tool with zero parameters. Might benefit from noting read-only nature, but not required. Output schema is absent, but description covers return fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters in input schema (100% coverage). Description adds value by explaining how the output relates to get_route, which is beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it lists 15 terminal routes with specific fields (route_id, label, category, primary endpoint). Distinguishes from sibling get_route by specifying that route_id is used with get_route.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states the purpose: list the routes. Mentions using route_id with get_route, indicating subsequent usage. No exclusions or when-not guidance, but adequate for a simple list tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• terminal_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It details the return content (milestone, goal status, etc.) and hints at dependencies (provider freshness). It does not mention side effects or authentication needs, but for a read-only status tool, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence with a front-loaded purpose and a list of return items. Every sentence earns its place; no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters, no output schema, and no annotations, the description is remarkably complete. It explains the tool's role, return content, and usage timing, fully covering what an agent needs to know.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters (0 params, 100% schema coverage), so baseline is 4. The description does not need to add parameter semantics and does not introduce confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns terminal health and a Command Center summary with specific enumerated items. It positions itself as a situational awareness entry point, effectively distinguishing its purpose from sibling tools like 'run_action' or 'get_route'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to 'Call this first before operating,' providing clear context for when to use it. It lacks explicit mention of when not to use it or specific alternatives, but the context is sufficient given the sibling tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.