Server Quality Checklist

Tool Scores

• agency_statusA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool as read-only, non-destructive, and idempotent. The description adds valuable behavioral details: what specific data the tool returns (last update time, update availability, agent count), which goes beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no fluff. It front-loads the purpose and efficiently conveys the key information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 params, no output schema), the description fully informs the agent of what the tool does and what to expect. It covers all necessary aspects for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the description does not need to add param meaning. The baseline for 0 params is 4, and the description effectively explains the output, compensating for the absence of an output schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'check' and the resource 'agent index status', and specifies the three pieces of information returned (last update time, update availability, agent count). This distinguishes it from sibling tools like agency_browse or agency_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (for a quick status check) but does not explicitly state alternatives or when not to use. No guidance on context or exclusions is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agency_updateA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide idempotentHint=true, but description adds context: pulling from git (with 'if applicable') and rebuilding the search index. This clarifies the exact side effect beyond the annotation flags.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, no fluff. Every word adds value: specifies action, resource, and condition ('if applicable'). Efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, no output schema, and a simple action, the description is sufficient. It covers the essential behavior and conditionality, making it complete for an agent to understand and invoke.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters in schema; schema coverage is 100%. Description adds no parameter info, but baseline for 0 parameters is 4. No need for additional parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb 'pull' and 'rebuild' on specific resources 'agent templates' and 'search index'. Distinguishes from sibling tools (browse, search, status) as an update operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance. However, the idempotentHint annotation implies it can be called repeatedly without side effects, and siblings handle other tasks. Lacks explicit alternatives or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agency_browseA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only, non-destructive, idempotent behavior. The description adds clarity on how to invoke different behaviors (no args vs division), but does not add novel behavioral traits beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, well-structured with usecase and instructions tags, and front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given low complexity (1 optional param, no output schema), the description provides complete guidance on usage and alternatives, leaving no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for the division parameter. The description restates the schema's intent without adding new semantic detail, meeting the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Browse all agent divisions and their agents.' It differentiates from sibling agency_search by recommending its use when knowing the agent type.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit instructions: 'Call with no arguments to see all divisions. Pass a division name to list its agents.' Also includes when to use agency_search instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• agency_searchA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds behavioral details like output format (matching agents with file paths and spawn template) and the spawning workflow. It does not contradict annotations and provides useful context beyond them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with <usecase> and <instructions> tags, front-loading the main purpose. Each sentence adds value, though the instructions are detailed. It is concise for the complexity involved.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description comprehensively explains the output (matching agents with file paths and spawn template) and provides full workflow instructions. Given the tool's complexity and the annotations covering safety, the description is complete enough for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with both parameters described. The description adds example values for query (e.g., 'game mechanics') and division (e.g., 'engineering'), and clarifies that query should be a task description, enhancing the schema's meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Find and launch a specialized agent for any task' and the usecase elaborates on searching agent templates by keyword, returning file paths and spawn templates. It distinguishes from siblings (agency_browse, agency_status, agency_update) by focusing on search and spawning, not browsing, status, or updates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'Call this before spawning any agency subagent' and provides step-by-step instructions on how to use it: pass task description, pick best match, spawn using the template, and return response directly. This gives clear when-to-use and how-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.