Server Quality Checklist

Tool Scores

• resume_loopC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states it 'removes the block' (implying a mutation) but does not mention side effects, persistence, required session state, or failure modes. The description is too sparse to inform an agent of the tool's full behavioral implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no fluff, and it front-loads the core action. However, it is under-specified to the point of being terse, which slightly reduces its value for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is a mutation with no annotations, no output schema, and only two parameters. The description does not cover essential context like side effects, prerequisites, return values, or error conditions. It is not complete enough for safe autonomous use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50% (only 'user_input' has a description). The tool description adds no additional meaning for either parameter, leaving 'session_id' completely unexplained. Given the low coverage and lack of supplementary description, the tool fails to compensate for the missing context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's action ('Removes the block and adds human input context') and resource ('the loop'), which distinguishes it from sibling tools like 'report_blocker' and 'init_loop'. However, it could be more specific about what 'the block' and 'human input context' refer to exactly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'report_blocker' or 'init_loop'. The description does not mention any prerequisites, conditions, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_tool_suggestionsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does not mention what the tool actually does with the request, whether it returns a list of tool names, whether it requires any prior context, or if it has side effects. The phrase 'ask this tool' is vague and does not reveal the nature of the response or any limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that efficiently conveys the tool's core purpose. It is appropriately sized for a tool with no parameters, and the message is front-loaded. It could be slightly more informative about behavior, but as is, it is not verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is mostly sufficient for an agent to understand when to invoke it. However, it lacks detail on what the suggestion response looks like, which could be helpful given there is no output schema to clarify this. The context signals indicate a very simple tool, so the description is nearly complete but could be improved with a note about the expected return.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the description covers 100% of the schema (which is empty). Since there are no parameters to explain, the description is not required to add param semantics. The baseline of 4 applies because there is nothing to compensate for; the tool accepts no input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to ask for help when stuck and unsure of what other tools to use. It uses a specific verb 'ask' and resource 'tool suggestions', effectively distinguishing it from siblings like report_blocker which reports blockers, and compact_memory which manages memory.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool ('if you are stuck and don't know what other tools to use'), but does not explicitly state when not to use it or mention alternatives. It provides clear context for use but lacks exclusionary guidance, such as 'do not use if you know the next step' or 'for specific blockers, use report_blocker instead'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compact_memoryA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full disclosure responsibility. It does honestly state that it empties the Active Context, which is destructive, and explains the preservation of the summary. However, it doesn't mention any side effects like whether the summary overwrites existing Compacted History, any permissions needed, or reversibility. This is a minimal but not comprehensive disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is exactly one sentence, front-loaded with the two key actions (empties, appends) and the resources involved. There is no redundant wording or filler, making it highly efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two simple parameters and no output schema or annotations, the description is adequate but thin. It explains the core behavior but omits an explanation for session_id and any details about the Compacted History format or error conditions. It is complete enough for basic use but not robust for complex decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes only context_summary, and session_id is undocumented. The description adds value by tying context_summary to the 'AI-provided summary' of the Active Context, which clarifies its purpose. However, it provides no additional context for session_id, and at 50% schema coverage, the description only partially compensates for the missing parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's actions: it empties the Active Context and appends a summary to the Compacted History. This is a specific verb+resource combination that unambiguously differentiates from sibling tools like init_loop or log_step, which handle other loop or logging tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor does it mention any preconditions or scenarios where it should not be used. It lacks explicit context like 'use when memory needs compaction' or contrasts with siblings. The usage is only implied by the tool's name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• init_loopA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It only mentions creating a file, but does not specify whether an existing file would be overwritten, what content the file initially contains, or any side effects on the session. This ambiguity is significant for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, direct sentence that conveys the core purpose without redundancy or fluff. It is appropriately front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple with only 2 parameters and no output schema, but the description lacks important context such as what happens if the session already has an active loop, error handling, or confirmation of successful initialization. It meets the minimum viability but leaves gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes both parameters (objective and session_id) with clear descriptions. The description adds minimal extra meaning beyond mapping 'given session' to the session_id parameter. Baseline 3 is appropriate given 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a new .md loop state file to start a continuous autonomous task, using a specific verb and resource. It implicitly distinguishes from resume_loop by focusing on initialization rather than continuation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'to start a continuous, autonomous task' gives clear context for when to use this tool (initial setup). However, it does not explicitly mention alternatives like resume_loop or when not to use it, but the sibling tool naming makes the distinction obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• report_blockerA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of transparency. It discloses the state change and human intervention request, which are the key behavioral effects. It doesn't mention other potential side effects (e.g., logging), but the core impact is clear and sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that conveys the essential information without any fluff. Every word earns its place, and the structure is front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with two parameters and no output schema, the description adequately covers its purpose and condition for use. It could mention that session_id identifies the loop, but given the simplicity, the provided information is sufficient for an agent to decide when to call it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 50%: the 'reason' parameter has a description, but 'session_id' is bare. The description adds no parameter-level detail, failing to compensate for the undocumented session_id. This is a significant gap given the coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Updates state to STATUS_BLOCKED') and the purpose ('asks for human intervention when absolutely stuck'). It uses a specific verb and resource, and distinguishes itself from sibling tools like resume_loop and log_step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'when absolutely stuck' provides usage context, indicating when to invoke the tool. However, it doesn't explicitly mention when not to use it or alternatives (e.g., log_step for less severe issues), leaving room for ambiguity in edge cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• log_stepA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full transparency burden. It discloses the append behavior, the rejection condition when failed=true and self_heal_strategy is missing, and the warning behavior for oversized context. It stops short of describing return values or the exact meaning of 'too large,' but the main behavioral traits are clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, front-loaded with the core action. Every sentence adds meaningful behavioral information: append, rejection condition, and context-size warning. There is no filler or repetition of schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple logging tool with 5 parameters, a strong schema, and no output schema, the description covers the essential operation and failure/warning behaviors. It is missing only minor details like the success/error return payload or what threshold triggers the size warning, which are not strictly required given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 80%, so the schema already documents the required parameters well. The description reinforces the self_heal_strategy constraint already present in the schema ('MANDATORY if failed=true') but adds little detail about action/result formatting or session_id usage. This is the baseline 3 for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Appends') and a clear resource ('the Active Context'), making the tool's primary function immediately obvious. It also distinguishes this from loop-control siblings like 'resume_loop' and 'compact_memory' by focusing on writing a step record.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear conditional usage context: any step that failed=true must include self_heal_strategy, and oversized context produces a warning. It does not explicitly name alternatives or exclusions, but the failure-requirement and warning behavior give the agent actionable guidance on when and how to invoke the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.