Server Quality Checklist

Tool Scores

• autocodeB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It doesn't disclose behavioral traits like session persistence, whether this is a long-running/heavy operation (an 'autocoding' workflow with max_turns suggests significant compute), or what state it creates/destroys. It doesn't explain what happens to existing sessions or how this relates to the session_name parameter's lifecycle.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence grouping the three modes compactly. No wasted words, though the 'g3 coach-player paradigm' parenthetical adds little clarity and could be considered noise. Front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 4 parameters, 100% schema coverage, no annotations, and no output schema, the description covers the mode semantics reasonably but leaves questions open: What does successful autocoding return? How does autocode relate to autocode_turn (iteration step) and autocode_session (state container)? Understanding session lifecycle and turn mechanics is important for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description does add value for the key mode parameter by elaborating each mode's purpose ('init: create session state', 'workflow: step-by-step recipe', 'single_shot: one comprehensive prompt'). The requirements parameter's role as 'source of truth' and checklist is reinforced. max_turns and session_name semantics are adequately handled by the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description does state a verb+resource ('Start autocoding') and explains the three modes. However, it's somewhat vague about what 'autocoding' produces or involves, and it doesn't significantly distinguish itself from sibling tools like autocode_turn or autocode_session. The g3 coach-player paradigm reference is unclear jargon.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The mode descriptions hint at usage ('init: create session state', 'workflow: step-by-step recipe'), but there's no explicit guidance on when to choose this tool versus autocode_turn or autocode_session, nor when to use which mode. The guidance is implied through mode definitions rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• autocode_sessionB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'persist state to file' and 'restore state from file,' which conveys the read/write nature, but it doesn't disclose important behaviors such as whether save overwrites existing files (destructive), file format specifics, error handling, or whether load creates the session in memory only. For a tool that mutates file state, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero wasted words. The description front-loads the purpose and immediately explains both action modes. Every phrase contributes meaning, and the action=... / action=... formatting is efficient and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a dual-mode tool with nested objects and no output schema, the description covers the two action modes adequately but leaves behavioral details unstated. It doesn't explain return values/output, file format expectations, or the consequences of missing state on save. The schema covers parameters well, but the description could add more about behavior for completeness given no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 100% schema coverage, the description adds meaning by mapping actions to behaviors ('save: persist state to file', 'load: restore state from file'), complementing the schema's enum descriptions. It clarifies the conditional requirement for state (save needs it, load doesn't). The description effectively reinforces and slightly enriches schema semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb-resource pair ('Save or load an autocoding session') and explains both action modes explicitly. It distinguishes from siblings by focusing specifically on session persistence rather than autocoding execution or conversation. However, it doesn't explicitly differentiate from autocode_turn, which could also seem session-related.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the two action modes (save/load) and their purposes, which gives clear context for when each is used. However, it provides no guidance on when to use this tool versus the sibling tools (autocode, dialectic, autocode_turn), and no exclusions or when-not-to-use guidance. The relationship to these siblings is implied but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• dialecticB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It mentions that mode=thesis/antithesis/synthesis requires prior text (thesis/antithesis params) and that execute=true runs prompts through backends, which adds genuine behavioral context. However, it doesn't describe error behavior, what happens on backend failure, token/rate characteristics, or the default no-execute behavior. The description adds useful context beyond the schema but is not rich in behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the core concept, and efficiently covers modes, response_style, and phase requirements in two sentences. It avoids redundancy with the schema and earns its place by explaining conceptual relationships (mode structure, phase dependencies) that the schema doesn't capture. Slightly dense but no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 13-parameter tool with no output schema and no annotations, the description captures the core dialectical model and mode selection logic. However, it doesn't explain interaction effects between execute/backend/use_search/use_council parameters, what the return value looks like in each response_style, or the purpose of lesser-known flags like use_council. Given the tool's complexity and zero annotation coverage, more behavioral context would be needed for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all 13 parameters thoroughly. The description adds value by clarifying that thesis/antithesis params are required for specific modes and that response_style controls output shape. However, most parameter semantics are already well-covered by the schema itself, so the description's marginal contribution is modest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs dialectical reasoning with a thesis → antithesis → synthesis structure, and enumerates its modes. It distinguishes itself from sibling tools by specifying the dialectical reasoning model and individual phase capabilities, though it doesn't explicitly differentiate from the autocode siblings (which appear to be a different category of tools).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for structured dialectical analysis and explains mode selection (single_shot vs workflow vs phases), but doesn't provide when-to-use vs when-not-to-use guidance or compare against sibling tools. It does guide mode choice implicitly by explaining what each mode does, but no exclusions or contextual cues about when this tool is the right choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• autocode_turnA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It explains that the tool advances state and can optionally execute through a backend, but it doesn't disclose important side-effect behaviors: that role=advance requires approved+coach_feedback, that execute=true triggers external code execution (a potentially significant/destructive action with real side effects), or error/state-failure behavior. The backend selection fallback (auto prefers Codex MCP, then CLI) is disclosed, which helps, but the external execution side effects are not flagged.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact—three sentences covering the core loop, each role, and execution behavior. It's front-loaded with the essential purpose, then elaborates roles. No wasted words. It could arguably spell out the role-specific required params, but overall it's efficiently written.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex stateful tool with 8 parameters, nested objects (state), no output schema, and no annotations. The description covers the loop mechanics but omits important usage detail: what the returned prompt/structure looks like (no output schema), what happens when state is invalid, how coach_feedback/approved interact, and the specific backend behavior consequences. For a tool this complex with no output schema and no annotations, the description could do more to complete the picture.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema individually documents all 8 parameters. The description adds interconnect between roles and requirements (e.g., advance needing approval), but does not add meaning beyond the schema for most parameters like backend, execute, timeout. The role-specific interplay (which params apply to which role) is partially in the description but the schema's descriptions stand on their own well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly explains the tool executes 'one step in the autocoding loop' and breaks out each role (player, coach, advance) with what each does. The verb 'execute' plus the explicit per-role behaviors distinguish it well from siblings like 'autocode' (likely the fuller loop) and 'dialectic'. It doesn't name the sibling alternatives explicitly, but the role breakdown provides strong functional differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the agent when to use each role ('generate implementation prompt', 'generate verification prompt', 'advance state after coach review'), giving clear usage context per mode. It establishes the state machine transition (player→coach→advance). However, it doesn't explicitly contrast with siblings 'autocode' (the full loop) or explain when to call this step function vs the complete-loop tool, nor give exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.