Server Quality Checklist

Tool Scores

• set_goalA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavioral traits. It states the tool 'records the completion condition' and starts a loop, but does not explain what happens if set_goal is called again while a goal is active (overwrite? error?), nor does it cover side effects or required permissions. The description is too brief on behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two succinct sentences. The first states the purpose, and the second provides a usage instruction. No redundant or extraneous information. Highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of three parameters and no output schema or annotations, the description covers the core usage pattern but lacks details on edge cases (e.g., re-calling set_goal, what happens on max_turns reached, error handling). It is minimally adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, but the description adds meaningful context beyond the schema: for session_id, it explains it's generated when omitted and should be reused; for max_turns, it clarifies it is a hard turn budget. This additional detail justifies a score above the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool starts a 'run-until-done goal loop' and records a completion condition. It mentions the follow-up tool check_goal, distinguishing set_goal as the starter. However, it does not explicitly contrast with clear_goal or get_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies when to use the tool (to start a goal loop) and instructs the agent to work toward the goal and call check_goal each turn. It lacks explicit guidance on when not to use it (e.g., if a goal is already active) or other alternatives, but the provided instruction is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_goalA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries burden. It discloses return format (done=true or done=false plus gap) and expected input. Lacks details on side effects or auth, but for a non-destructive evaluation tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first establishes purpose, second provides usage and return behavior. No wasted words, efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, description fully explains return values and when to call. Parameter explanations in schema complement description. Complete for this tool's needs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers both parameters with descriptions (session_id and summary). The description reinforces usage ('concrete summary') but adds little beyond schema. Baseline 3 for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Evaluate whether the active goal is fully met' with a specific verb and resource. Distinguishes from siblings (clear_goal, get_status, set_goal) by its unique evaluation purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises 'Call at the end of every turn' and specifies what to include in summary. Does not explicitly mention alternatives, but context with siblings implies it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clear_goalA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool ends the session, distinguishes between completed and abandoned states, and mentions 'self-check fallback' as context. Adequate for a simple termination action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero wasted words. Information is front-loaded with the primary action then parameter usage. Perfectly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, and the description does not mention what the tool returns (e.g., success indication, error cases). For a simple 'end session' tool, this is a minor gap; some agents may need to know if the action succeeded. Otherwise, the description covers the essentials for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value beyond the schema by explaining the 'self-check fallback' context for completed=true and explicitly stating that omitting or passing false means abandoned. This enhances understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'End a goal session' as verb+resource. Distinct from sibling tools (check_goal, get_status, set_goal) which are about checking, retrieving, or starting goals. The description also specifies the two usage modes via the completed parameter.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear guidance on when to use completed=true (when goal verifiably met) vs other cases (abandoned). Does not explicitly contrast with sibling tools, but the distinction between ending vs checking/getting is implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_statusA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. The verb 'inspect' suggests a read-only operation, and listing specific fields provides transparency about what is returned. However, it does not explicitly state that it has no side effects or require permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the purpose and lists key details without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately informs the agent of the return fields (condition, turn count, elapsed time, last evaluation). The single optional parameter is clearly explained, making the tool usage unambiguous.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100% for session_id, and the description adds context that it defaults to the most recent active session, which goes beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'inspect' and the resource 'current goal session', listing specific fields such as condition, turn count, elapsed time, and last evaluation. It is specific enough to distinguish from sibling tools like check_goal, clear_goal, and set_goal.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for inspection of session status but does not provide explicit guidance on when to use this tool versus alternatives, nor does it mention when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.