Server Quality Checklist

Tool Scores

• ask_gptA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It mentions treating output as a hypothesis, hinting at potential inaccuracy. However, it does not disclose error handling, rate limits, cost implications, or what happens with invalid model IDs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two well-structured sentences. The first states the core action and key requirements. The second provides model recommendations and a caveat. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lacks output format details (no output schema). It does not explain what the tool returns (e.g., text, JSON). For a 5-param tool with no output schema, this is a significant gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, baseline 3. The description adds value by linking model IDs to use cases (e.g., gpt-5.5 for deep reasoning) and suggesting reasoning_effort coupling, going beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool asks a GPT model via ChatGPT subscription. The verb 'ask' and resource 'GPT model' are specific. Sibling tools (check_usage, get_pattern, list_patterns) have distinct purposes, so no confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says you must choose model and write instructions yourself, and provides model recommendations with use cases. It lacks explicit when-not-to-use or alternatives, but the context is clear and practical.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_usageA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description clearly indicates a read-only query with no side effects. It doesn't specify output format, but for a simple quota check this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One succinct sentence conveying the tool's purpose without any wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple no-parameter tool, the description is complete enough. It explains what the tool does and its scope. Lacks details on output format, but output schema is not provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, and schema coverage is 100%. The description doesn't need to add parameter info; baseline for zero parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool shows remaining subscription quota for the signed-in account, using a specific verb and resource. It easily distinguishes from siblings like ask_gpt (for questions) and pattern tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. While it's unique among siblings, the description doesn't suggest best practices like checking quota before API calls.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_patternA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden. It implies a read-only operation ('Return the full text'), but does not explicitly state non-destructive behavior, permissions, or side effects. Given the simplicity, it is adequate but not exemplary.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. The main action and purpose are front-loaded, and the secondary sentence provides usage guidance. Perfectly concise for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter, no output schema, and is straightforward, the description is complete. It covers what the tool returns, how to get the input, and why to use it. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers the parameter with 100% coverage and a description. The tool description adds value by reinforcing the relationship to list_patterns and giving an example pattern name, which helps the agent understand how to select a valid name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return' and the resource 'full text of an orchestration pattern', and explicitly references sibling tool list_patterns for how to get valid names. It distinguishes from siblings by focusing on retrieval of full text vs listing or usage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells when to use the tool: 'Use it to apply the pattern when orchestrating ask_gpt calls.' It also directs the agent to see list_patterns for available names, providing clear context for correct usage, though it does not explicitly state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_patternsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses that it lists patterns and returns each pattern's name, title, summary, and when to use it. It implies a read-only operation, though not explicitly stated. Sufficient for a simple list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. First states core function, second adds usage guidance and return details. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, the description covers purpose, when to use, what to do next, and what it returns. Complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist; schema coverage is 100%. The description correctly avoids parameter details. Baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List', the resource 'available orchestration patterns', and the context 'for driving the GPT subagents'. It differentiates from siblings by mentioning subsequent use of get_pattern.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to call this before non-trivial expert work (reviews, audits, etc.) and then read the chosen pattern with get_pattern. Provides clear when-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.