Server Quality Checklist

Tool Scores

• dump_threadsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotation already declares readOnlyHint=true, so the description only needs to add extra behavioral nuance. It does mention thread persistence and cold-reload, which is useful context beyond the annotation, but it does not clarify whether the export writes files to disk (only the schema does for markdown) or describe any side effects. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, with the primary purpose front-loaded. The second sentence is a bit of a run-on ('Use thread_ids to filter specific threads, format to choose output') but is still concise and informative. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with no required fields and no output schema, the description plus schema covers the main usage: exporting threads with optional filtering and format selection. The persistence/cold-reload note adds valuable context. Could mention the default behavior (all threads) but that is already in the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters (format and thread_ids) already documented in detail. The description merely echoes 'Use thread_ids to filter specific threads, format to choose output' without adding new semantic information. Baseline 3 applies since schema carries the load.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a specific verb and resource: 'Export conversation threads as JSON or Markdown.' It clearly identifies the tool's function and distinguishes it from sibling tools like chat and listmodels, which involve interaction and model listing respectively.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implicit usage context: 'Threads persist to disk and can be cold-reloaded after memory expiry' suggests using this tool for backup or recovery after memory loss. It does not explicitly name alternatives or exclusions, but the context is clear enough for an agent to infer when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chatA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond the readOnlyHint annotation by disclosing that prompts are routed to external AI models (privacy-relevant), that conversation memory is maintained, and that it supports file context embedding and multi-turn threads. This adds behavioral context not present in the annotation, though it doesn't mention potential rate limits or data retention specifics. No contradiction with the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with 'Multi-model AI gateway,' and each phrase adds value: provider list, memory, file context, images, multi-turn. No fluff or redundancy. The description is tight while covering all key capabilities.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a complex tool with 7 parameters, but the schema carries detailed parameter semantics, so the description only needs a high-level overview. It provides that overview without covering output format or explicit alternatives, yet the sibling names (listmodels, dump_threads) and schema guidance fill those gaps. Slightly more could be said about return behavior, but the description is sufficiently complete for an entry point.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter already having detailed descriptions (e.g., temperature recommendation, model selection guidance, continuation_id semantics). The description mentions capabilities like file context and multi-turn threads, but these are already reflected in the parameter descriptions, so it adds no net new parameter meaning. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool is a 'Multi-model AI gateway' that 'routes prompts to external AI models,' listing specific providers and capabilities like conversation memory, file context embedding, and multi-turn threads. This distinguishes it from siblings (listmodels, dump_threads) which are about listing models and dumping threads, not running prompts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: the tool is for sending prompts to external AI models. However, the description does not explicitly state when to use this tool versus alternatives like listmodels or dump_threads. The schema's model parameter mentions using listmodels when no model is provided, but that guidance is outside the description, so the description itself only offers implied context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• listmodelsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the safe-read nature is covered. The description adds useful context about what information is returned (providers, names, aliases, capabilities), but it does not disclose additional behavioral traits such as pagination or formatting.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the verb 'Shows' and packs all relevant information about the tool's output without unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with no parameters and no output schema, but the description sufficiently covers its purpose and the categories of data it returns. It could be slightly more complete by mentioning that no arguments are required, but that is implicit in the empty input schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is nothing for the description to elaborate. The baseline score of 4 applies, as no parameter information is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Shows' and clearly defines the resource: configured AI model providers, available model names, aliases, and capabilities. This clearly distinguishes it from sibling tools like chat and dump_threads.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage—you'd use this tool when you need to see configured models—but it does not provide explicit guidance on when to use it versus alternatives like chat or dump_threads. There is no direct comparison or exclusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.