Server Quality Checklist

Tool Scores

• grok_runsC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavioral traits. It only states 'List', implying a read operation, but omits details like pagination, rate limits, error handling, or whether results are return full objects or summaries. Minimal transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence, which is concise but slightly ambiguous (e.g., parenthetical '(runs)' could be misinterpreted). It lacks front-loading of critical info like the primary use case.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should explain the return value format, but it does not. It also fails to mention error scenarios or how to handle large result sets. Incomplete for a listing tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each parameter described. The description adds context that run_id selects a single run, but this is inferrable from the param description. Baseline 3 is appropriate as the description adds marginal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists 'tracked Grok bridge runs' and can fetch a single run by ID, achieving a specific verb-resource combination. However, it does not differentiate from sibling tools with similar names like 'grok_run' or 'grok_show', which could cause confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies two usage modes (list all vs. by ID) but offers no guidance on when to choose this tool over alternatives (e.g., grok_check, grok_show). No when-not-to-use or prerequisite information is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grok_showC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, and the description only says 'Show status/result'. It does not disclose whether the tool is read-only, what occurs if run_id is omitted, or any side effects. The agent lacks critical behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise but overly brief. It could include more useful detail without being verbose. Structure is not an issue but content is sparse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description should explain what the return value is or how to interpret status/results. It does not, leaving the agent uncertain about the output format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for both parameters (cwd and run_id). The description adds no additional meaning beyond the schema, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool shows status/result of a Grok bridge run. It mentions 'show' and implies a single run focus. However, it does not explicitly differentiate from siblings like grok_runs (possibly listing many runs) or grok_check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool vs alternatives such as grok_runs or grok_check. The agent receiving no context on selection criteria or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grok_stopC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose behavior. It only says 'stop' without explaining what happens to the job (e.g., cancellation, termination, idempotency), side effects, or authentication needs. The description is too terse for a mutation operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, but it is under-specified rather than concise. It omits crucial behavioral and usage details that could be included without sacrificing brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters, no output schema, and no annotations, the description should explain return values, error conditions, and how to use with sibling tools. It fails to do so, leaving the agent with insufficient context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add any additional meaning beyond the schema, which already describes the parameters adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool stops a running Grok bridge job, using the verb 'stop' and the resource 'running Grok bridge job'. This distinguishes it from siblings like 'grok_run' (run) and 'grok_runs' (list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'grok_check' or 'grok_run'. There is no mention of prerequisites or context that would help an agent decide to invoke this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grok_bridge_pathA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries the burden. It states it resolves a path, implying read-only behavior, but lacks details on side effects, authorization, or return format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with the key verb and resource. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple path resolution with no parameters, the description is mostly complete. However, no output schema and lack of details on return value or when this is needed reduce completeness slightly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0 parameters, so description doesn't need to add param info. Baseline 4 applies as per guidelines.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Resolve' and the resource 'grok-bridge.mjs path', distinguishing it from sibling tools that perform different actions like checking, running, or stopping.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs siblings, though the purpose implies it's for path resolution. Usage context is minimal but not misleading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grok_checkA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description provides useful behavioral details: it returns a specific envelope with rows, and preserves old bridge JSON for backward compatibility. However, it does not explicitly state that the tool has no side effects or is idempotent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two sentences. The first sentence covers the purpose and scope, and the second explains the return format and backward compatibility. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simple nature (one optional parameter, no output schema), the description thoroughly explains the return structure and includes specific checks (node, grok binary, auth). It covers backward compatibility and the envelope format, making it complete for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter (cwd), and the description adds no additional information about it. The schema already describes it sufficiently, so the description meets the baseline without adding value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool probes whether the Grok Build bridge and CLI are ready, specifying the exact resources (node, grok binary, auth). This distinguishes it from sibling tools like grok_run (execution) and grok_stop (stopping).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies using this tool before running operations to check readiness, but it does not explicitly state when to use it or how it differs from alternatives. No guidance on prerequisites or when not to use it is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grok_runA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses behavior for background mode (enqueue and exit), default timeouts, read-only default, and that prompt is processed via a tmpdir file. It does not cover error handling or failure modes, but is otherwise transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense and informative, with no wasted words. It could be more readable with structured bullet points, but it remains sufficiently concise and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 9 parameters, no output schema, and no annotations, the description covers purpose, usage, and key parameter behaviors. It mentions polling siblings for completion. However, it does not describe the return format (jobId JSON) in detail or cover error scenarios, leaving some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 9 parameters have schema descriptions (100% coverage). The description adds crucial usage context: background timeout limits, effect of write flag, prompt processing via tmpdir, and cwd usage for project paths. These details go beyond the schema, providing safety warnings and best practices.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs a Grok Build task via bridge, distinguishing between background and foreground modes. It references polling grok_show for completion, differentiating it from sibling tools that check or list runs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit context for when to use background vs foreground, and mentions polling grok_show for status. It does not explicitly list when not to use this tool over alternatives, but the background/foreground distinction and the mention of grok_show serve as good guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.