Server Quality Checklist

Tool Scores

• ask_codexB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only mentions sending a prompt and file attachment. Lacks details on rate limits, cost, token limits, response format, or any side effects. Minimal behavioral disclosure for an external API call.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary action. No redundant or unnecessary words. Highly concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has 3 params, no output schema, no annotations. Description covers main action but omits what the response looks like, error handling, or how attached files are used. For an API call, more behavioral context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. Description adds minimal extra meaning: 'Optionally attach a local file' repeats schema. No new format or constraint details. Adequate but not enhanced.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sends a prompt to OpenAI (GPT-4o by default) and can attach files. The verb 'send' and resource 'prompt' are specific. It implies distinction from siblings by naming OpenAI, but does not explicitly differentiate from ask_claude or ask_perplexity. Name helps.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use scenarios: second opinion, alternative implementation, OpenAI perspective. Does not state when not to use, but siblings suggest alternatives. Adequate context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_perplexityA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description bears full burden. It discloses supported file types (text/code) and explicitly states images/PDFs are not supported due to no vision API. This adds value beyond the schema. However, it omits details on output format or error handling, which would be helpful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no wasted words. First sentence defines purpose, second clarifies file constraints, third provides use-case guidance. Front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Though schema coverage is 100% and usage guidelines are present, the description fails to mention the output format (e.g., answer with citations) or any error behavior. More critically, the file type contradiction makes the description incomplete and potentially confusing for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the tool description contradicts the schema for the 'file' parameter: schema lists images and PDFs as supported, while description says they cannot be attached. This inconsistency harms reliability and clarity, making the parameter semantics misleading.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: sending a research question to Perplexity for a web-grounded answer with citations. It distinguishes from siblings (ask_claude, ask_codex) by specifying use for current information and live web search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends use for current information, documentation lookups, and questions benefiting from live web search. While it doesn't mention when not to use or alternatives, the context is clear enough for an AI agent to choose appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_claudeA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It accurately describes the action but omits behavioral details like API cost, latency, or data handling. It does not contradict annotations as none exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no redundancy. It front-lines the core action and then adds the optional file attachment. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description should explain what the tool returns (Claude's response). It does not. It also does not mention conversation history or multi-turn capabilities. While the input is well-covered, the output is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description adds useful context: it explains the optional file parameter with supported types and an example path, and gives an example for the system prompt. This exceeds the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sends a prompt to Claude (Anthropic) and optionally attaches files. The verb 'send' and resource 'Claude' are specific, and the examples (code, text, image, PDF) clarify the scope. The purpose is unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this when you want Claude's perspective...', providing a clear usage context. However, it does not explicitly differentiate from sibling tools like ask_codex or ask_perplexity, nor does it state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• brokerA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It details parallel execution, file handling differences between AIs, and the synthesize feature (default true, role of Claude). Missing info on rate limits, cost, or error handling, but the core behaviors are adequately transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences with no fluff. First sentence defines core purpose, then file attachment, then a critical limitation, then synthesis behavior. Each sentence adds distinct value and the structure is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers main features but omits details about the return format (beyond 'synthesized comparison' or 'raw responses') and error handling (e.g., if one AI fails). With no output schema, more details on the response structure would improve completeness for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining file type restrictions for Perplexity, default values for targets and synthesize, and the effect of synthesize=true/false. This goes beyond the schema's basic property descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it sends a prompt to multiple AIs in parallel and synthesizes a comparison. It distinguishes from sibling tools (ask_claude, ask_codex, ask_perplexity) which are single-AI tools, making the purpose specific and well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use this tool (for parallel AI querying and synthesis) and provides a key exclusion (Perplexity cannot handle images/PDFs). It implicitly suggests using sibling tools for single-AI queries, though it could be more explicit about alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.