Server Quality Checklist

Tool Scores

• search_modelsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, indicating safe read behavior and that results may vary. The description adds no additional behavioral traits (e.g., pagination, rate limits). Given the annotations, the description is adequate but not enhanced.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loading the action and filtering. Every sentence adds value with no redundancy or waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description could provide more detail on return format. It only says 'list models,' which implies a list but lacks specifics. Complexity is low, so it is minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all three parameters. The description only reiterates optional filtering without adding new meaning. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List tracked models' with optional filtering by provider and lifecycle state. It captures the core purpose and distinguishes it from siblings like list_retiring_models or check_model by focusing on general discovery. However, it does not explicitly differentiate from similar sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to discover what is currently available from a provider,' giving clear usage context. It does not provide when-not-to-use or alternative tools, but the guidance is sufficient for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_retiring_modelsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and open-world hints. Description adds the sorting order ('soonest first'), which is useful but minor. No mention of pagination, rate limits, or other behaviors beyond what annotations imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences that are efficient and front-loaded. First sentence states the core function and sort order; second sentence adds practical usage context. No superfluous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity, full schema coverage, and read-only annotations, the description is mostly complete. It lacks details on return format or error handling, but these are acceptable omissions for a simple list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (both parameters documented). The tool description does not add any additional meaning or context about the parameters beyond their schema definitions, so baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool lists retiring models with a time window and sort order. It doesn't explicitly compare to siblings, but 'audit a codebase or plan migration work' provides context that distinguishes from general search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this to audit a codebase or plan migration work,' providing clear use cases. However, no mention of when not to use or alternatives like 'recent_lifecycle_changes' or 'check_model'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recent_lifecycle_changesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, covering safety and dynamic data. The description adds ordering (newest first) and scope (across all tracked providers), which are useful beyond annotations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main purpose and a clear use case. Every sentence earns its place with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool lacks an output schema, but the description implies a list of lifecycle changes. Given the simplicity and annotations, it is mostly complete, though the exact fields of each change could be clarified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description does not add meaning beyond the schema: provider and sinceDays are already described thoroughly in the schema. No additional param semantics provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists lifecycle state changes across providers, newest first, and provides a concrete example use case ('what model deprecations happened recently?'). This distinguishes it from siblings like check_model (single model) and list_retiring_models (specific subset).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an explicit usage example ('Use this to answer...'), providing clear context. However, it does not specify when not to use it or mention alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_modelA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint and openWorldHint, indicating a safe read operation with dynamic results. The description adds value by disclosing that it returns safety status (safe/deprecated/retired) and migration info, and by specifying input exactness. This goes beyond the annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: purpose+output, input examples, usage guideline. Front-loaded with primary action. No unnecessary words, every sentence adds distinct value. Highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a simple tool with one param, no output schema, and annotations covering read-only/open-world, the description covers the purpose, input format, output summary (status+migration), and usage context. It is nearly complete; a minor gap is that 'what to migrate to' is mentioned but not explicitly stated as a replacement ID. Still very good.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema describes the single 'model' parameter with examples. The tool description reinforces this with additional examples ('gpt-4o', etc.) and emphasizes exact string usage. Schema coverage is 100%, so baseline is 3; the extra examples and clarity push it to 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Check') and clearly identifies the resource (LLM model id lifecycle status). It distinguishes from sibling tools like 'list_retiring_models' or 'suggest_replacement' by emphasizing it checks a single exact model id string. The examples and usage directive further clarify its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Call this before writing or changing any hardcoded model id.' It does not explicitly state when not to use it, but the sibling context and focus on a single model id imply alternatives like 'list_retiring_models' for broader checks. The guidance is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• suggest_replacementA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the behavior beyond annotations: it returns recommended replacements when available, and falls back to active models from the same provider. This adds value over the annotations (readOnlyHint, openWorldHint) and is fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that is front-loaded and contains no unnecessary words. Every part is essential.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and no output schema, the description explains the return value clearly (replacement(s) or active models). No additional context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the input parameter description is adequate. The tool description does not add further details about the parameter format or examples, so it meets the baseline but does not exceed it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (suggest replacement), the resource (deprecated/retiring model), and the fallback behavior (returning active models from the same provider). It effectively distinguishes from sibling tools like list_retiring_models and check_model.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates when to use the tool: when a model is deprecated or retiring. However, it does not explicitly state when not to use it or mention alternative tools for other scenarios, but the context is sufficiently clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.