Server Quality Checklist

Tool Scores

• grok_session_closeC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description adequately discloses key behaviors: it closes the worker but preserves session mapping for later recovery. However, it omits details on idempotency, error conditions (e.g., calling on non-idle session), and synchronization behavior, leaving notable gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (single sentence) but lacks structure; it front-loads the condition but omits critical parameter information, sacrificing completeness for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters, no output schema, and no annotations, the description is insufficient for correct usage. The missing parameter semantics and lack of behavioral detail leave the agent underinformed for this moderate-complexity tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description provides no explanation for any of the 4 parameters (branch, project_root, codex_task_id, working_directory). The agent receives no guidance on how to populate these fields, making tool invocation guesswork.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool closes the worker process of an idle Grok session while retaining session mapping for potential recovery. This purpose is distinct from sibling tools like grok_session_ensure or grok_task_start, though it could be more explicit about differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies when to call (session idle, no longer needs background processes), providing basic usage guidance. However, it lacks explicit exclusions or alternatives among siblings, and does not clarify when not to use it (e.g., for permanent termination).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grok_session_switchB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the tool switches to an independent session and avoids reusing old context, but it does not explain what happens to the old session (e.g., suspended, closed) or any side effects. Since no annotations are provided, the description carries full burden and is insufficient for understanding behavioral impact.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, efficient and to the point. It covers the trigger and action without unnecessary words. However, it could be slightly expanded to include parameter hints without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (5 parameters, 2 required, no schema descriptions, no annotations, no output schema), the description is too brief. It does not explain parameter roles, return values, or detailed behavior, leaving significant gaps for an agent to correctly invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides no information about the parameters (codex_task_id, project_root, branch, etc.), even though schema coverage is 0%. The agent must rely solely on parameter names, which may be insufficient. The description fails to compensate for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to switch to an independent Grok session when project path, working directory, or Git branch changes. It uses a specific verb ('switch') and resource ('Grok session'), and distinguishes from siblings like grok_session_ensure (which likely creates/ensures a session) and grok_session_close (which closes one).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: when project path, working directory, or Git branch changes. It implies this is the correct tool for session switching and mentions avoiding reuse of irrelevant old context. However, it does not explicitly state when not to use it or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grok_task_statusA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that the tool returns brief status (not full logs) and recommends against polling. Missing details on error handling, side effects (likely none), and authorization.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, no wasted words. Efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Without an output schema, the description should clarify the return format and potential states. It only says 'brief status' but does not specify possible values or error responses, leaving the agent underinformed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description provides no parameter explanations. With four parameters (two required), the agent lacks guidance on values like project_root or codex_task_id.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is for checking progress, troubleshooting, or confirming Grok status. It distinguishes itself from sibling tools (grok_task_start, grok_task_cancel, etc.) by focusing on status retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to call the tool (user wants status) and advises against frequent polling. However, it does not explicitly mention alternatives or when not to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grok_session_ensureA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must bear full burden. It reveals the tool only establishes a session, not coding, but omits details like session reuse, return values, error behavior, and parameter meanings, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with condition, no wasted words. Efficient and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, no annotations, and no output schema, the description lacks explanation of parameters, return value, edge cases, and behavior under various inputs. Incomplete for robust agent usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description should explain parameters. It mentions codex_task_id, project_root, and branch implicitly, but does not cover open_observer or working_directory. Partial value, but not sufficient to fully compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool connects or creates a Grok session for a Codex task, project path, and Git branch, and explicitly notes it only establishes a session, not coding. It distinguishes from siblings like grok_task_start and grok_session_close.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States 'when tasks involve code projects, call this tool first', providing clear context. Also clarifies it does not start coding, implying other tools are for that purpose. Missing explicit alternatives, but context is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grok_task_cancelA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden. It discloses that the tool cancels the task and ends the process tree, but lacks details on potential side effects, reversibility, or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, front-loading the purpose without wasted words. It is well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of a cancel action and the presence of sibling tools, the description is largely complete. It lacks details on return values or error handling, but the context is sufficient for basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description does not explain any of the 4 parameters, even though schema description coverage is 0%. It fails to clarify the meaning of fields like codex_task_id, project_root, or working_directory, leaving the agent without necessary context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is for canceling a Grok task and ending its process tree, and provides specific trigger conditions (user request or unresponsive task). It distinguishes from sibling tools like grok_task_start and grok_task_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool (user wants to stop or task is unresponsive). However, it does not explicitly state when not to use it or mention alternatives, missing some guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• grok_task_startA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the critical blocking behavior and warns against parallel execution. It also clarifies that architecture and review are Codex's domain. Without annotations, this is useful context, though it does not cover error handling or return values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise—three sentences that each add essential information: when to use, blocking behavior, and responsibility split. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite good purpose clarity, the description lacks parameter details and does not describe the return value or outcome of the call. With 12 parameters and no output schema, an agent lacks sufficient context to use the tool correctly without additional information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 12 parameters with 0% description coverage, and the tool description does not explain any parameter's meaning or usage. The description mentions the task objective and mode only implicitly, leaving a significant gap for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to hand off tasks (code modification, command execution, testing) from Codex to Grok, and it distinguishes this tool by emphasizing its blocking behavior and the division of responsibilities (Codex handles planning and review).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use the tool (after Codex completes requirements analysis) and what not to do (avoid parallel work during the blocking call). It also clarifies that planning and review remain with Codex. However, it does not explicitly contrast with sibling tools like grok_task_status or grok_task_cancel.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.