Server Quality Checklist

Tool Scores

• list_modelsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. 'List' clearly implies a read-only operation, and it discloses the specific information returned (capabilities, pricing, web search costs). It doesn't hide side effects because none are expected for a listing tool, and the description gives sufficient behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action and content, then a clear usage directive. Every word contributes value with zero redundancy. This is an exemplar of concise, structured writing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with optional filters and no output schema, the description is complete. It names the output fields (capabilities, pricing, web search costs) and the usage context. It doesn't elaborate on filtering, but the schema covers that, and the description doesn't need to explain return structure beyond what it lists.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description mentions capabilities and pricing which aligns with the capability filter, but it doesn't explicitly explain the provider or capability parameters. Since the schema has 100% coverage with descriptive parameter descriptions, the baseline of 3 is appropriate; the description adds no additional parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('List') and resource ('available AI models'), and specifies the content (capabilities, pricing, web search costs). It distinguishes itself from sibling tools like ask/search/research by being about model selection rather than executing tasks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: 'Use this to choose the right model for your task.' This establishes when to use the tool, though it doesn't explicitly mention alternatives or when not to use it. The context is clear enough for an agent to know this is the model-discovery step.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• askA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries full burden. It discloses the no-web-search limitation, inability to retrieve latest/real-time info, and default model context. It also adds transparency about model pricing and trade-offs. Missing details like rate limits or output format, but the key limitations are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, then uses compact, structured lists for model tiers. It is a bit long due to multiple pricing/use-case entries, but each item is actionable and the colon-separated format makes it scannable. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters and no output schema, the description covers purpose, limitations, and model selection thoroughly. It doesn't explain return values or error handling, but for an ask tool this is less critical. Auth/rate limits are not mentioned, but the context is sufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with good descriptions of all parameters. The description goes beyond the schema by enriching the 'model' parameter with concrete model names, pricing, and suitability (e.g., gpt-5.3-codex for code, nano for cheapest). This adds substantial value for model selection.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Ask a single AI model a question' – a specific verb and resource. It explicitly states the scope (single model, no web search, no real-time info) and distinguishes from the research tool by directing multi-perspective questions elsewhere.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use vs alternatives: 'For multiple perspectives on a question, prefer research tool with mode:ask synthesize:false.' It also maps model choices to use cases (code, mid-tier, cheaper, cheapest), giving clear guidance on when to select each.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• researchA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are supplied, so the description carries the full burden of behavioral disclosure. It reveals that the tool queries 2-4 models in parallel, that synthesize:false shows side-by-side responses with latency and cost, and that search mode includes grounding. It does not mention potential rate limits or failure modes, but for a research tool the disclosed traits are substantial and non-contradictory.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and then systematically covers usage guidance, modes, synthesize behavior, and query tips. While longer than the minimal two-sentence example, every sentence contributes meaningful information and the use of lists and colons keeps it scannable and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 8 parameters, no annotations, and no output schema, the description is comprehensive: it explains modes, parameter defaults, behavioral differences (synthesize), query best practices, and cross-references the sibling search tool. It does not detail the exact response structure, but it states that synthesize:true produces 'one comprehensive answer' and synthesize:false shows responses 'side-by-side with latency and cost,' providing sufficient context for a research tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing a baseline of 3. The description adds value by explaining the semantic difference between modes (e.g., 'mode:search (default): web research with grounding' vs 'mode:ask: multi-model Q&A without web search') and the effect of synthesize (true: merge into one comprehensive answer; false: side-by-side with cost/latency), which enriches the schema's enum and boolean descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Multi-model parallel research. Queries 2-4 AI models simultaneously, then optionally synthesizes results,' which clearly states the tool's specific action and scope. It also explicitly distinguishes itself from the sibling 'search' tool by stating 'Use instead of search when...' making the purpose and differentiation unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use this tool over alternatives: 'Use instead of search when: (1) you need higher confidence via cross-validation across sources, (2) you want diverse perspectives on a topic.' It further breaks down mode-specific use cases (e.g., 'mode:ask: multi-model Q&A without web search — for architecture decisions, trade-off analysis'), giving clear context and alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that the search engine returns recent results automatically and warns about the risks of hardcoded years. It does not mention rate limits or auth, but for a read-only search tool, the disclosure is adequate and adds value beyond schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but extremely well-structured with clear sections for query tips and model guide. Every sentence delivers actionable guidance, though the overall length makes it slightly verbose. Still, it earns its place due to density.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with no output schema, the description covers purpose, when to use vs. the research tool, query optimization strategies, and model selection with cost/performance details. It even notes the Sources section behavior for include_sources, making it comprehensive for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds substantial meaning beyond parameter descriptions: query formulation rules (no specific years, one topic per query, language selection), model cost tiers and use cases, and guidance on max_tokens and include_sources. This goes far beyond the schema's baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Web search with a single model' and immediately contrasts with the research tool for multi-model use, clearly identifying the tool's scope and distinguishing it from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to use this tool ('real-time lookups') and when to use the research tool instead ('multi-model parallel research'). It also provides extensive query construction tips and a nuanced model selection guide with cost/performance trade-offs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.