Server Quality Checklist

Tool Scores

• get_composite_historyA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, but it only states that it returns 'historical composite index values' and defines the days parameter. It does not disclose details like time granularity, ordering, handling of invalid days, or other behavioral traits. The description mostly reiterates obvious 'get history' semantics without adding deeper context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise, leading with the core purpose, then briefly listing arguments and return. Every sentence earns its place without wasted words, and the structured Args/Returns format improves scannability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (1 parameter, output schema exists), and the description covers the main purpose and parameter. However, it lacks context about the data granularity, timezone, or how the history is structured beyond 'values', which could be useful for an AI agent. The description is adequate but not rich enough for a high score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explicitly defines the only parameter 'days' with a range (1-90) and default (30), adding meaning beyond the bare schema property. This fully clarifies the parameter's purpose and valid inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Get SIT-Composite index history / trend over time,' which clearly identifies the resource (SIT-Composite index), the action (get history/trend), and the temporal scope. It distinguishes itself from the sibling get_composite_latest by emphasizing 'history / trend over time' rather than a current value.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a use case (need historical data), but it does not explicitly state when to prefer this tool over siblings like get_composite_latest, nor does it mention any alternatives or exclusions. No direct 'use this instead of X' guidance is provided, leaving the AI to infer from the name/description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_providersA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of showing this is a read-only listing operation. The word 'List' implies a safe read, but it does not explicitly state that no mutations occur or mention any other behavioral characteristics such as ordering or pagination.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core purpose and the notable return fields. There is no padding or repetition of schema data.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a simple list-all tool with no parameters and an output schema provided, the description is mostly sufficient. It loses a point because it offers no guidance on when to use this versus a more specific provider tool, which would be helpful for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description correctly does not invent parameters, and the schema covers everything there is to know (empty properties).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('List all inference providers') and the resource it operates on. It also mentions the specific data returned (model counts and price stats), which helps differentiate it from sibling tools like get_provider and compare_providers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when you want an overview of all providers, but it does not explicitly contrast this with get_provider (for a single provider) or compare_providers (for side-by-side comparison). The guidance is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_composite_latestA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It explains the composite's meaning but does not disclose operational details such as response format, rate limits, or side effects. However, as a zero-parameter read operation, the absence of behavioral warnings is acceptable, and the output schema covers return structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences efficiently state the action and provide definitional context with no filler. The first sentence tells exactly what the tool does, and the second adds useful context about what SIT-Composite represents.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero-parameter nature and presence of an output schema, the description adequately covers purpose and context. The explanation of SIT-Composite provides meaningful domain context, and no additional details are needed for this simple read-only tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters to explain; schema coverage is 100% vacuous, and the baseline for zero-parameter tools is 4. The description adds no parameter-specific info, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves the current SIT-Composite index value and its per-tier breakdown, using the specific verb 'Get' and a named resource. The 'current' qualifier distinguishes it from sibling get_composite_history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly provide when-to-use guidance or alternatives, but the 'current' qualifier implies a contrast with historical queries. However, it doesn't explicitly mention when to use this vs sibling tools, so usage context is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_model_historyA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the return type (historical price series) and a plan-dependent window for days, which is useful. However, it does not mention rate limits, data update frequency, granularity, or error behavior. For a read-only tool this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is focused and front-loaded with the main purpose, followed by differentiation context and an Args/Returns structure. It includes a useful example and parameter hints without excessive verbosity. The only slight redundancy is the differentiation paragraph, but it adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 parameters) and has an output schema, so return value details are not required. The description covers purpose, usage context, parameter constraints, and the return type, plus a plan-dependent caveat. It is complete enough for correct invocation, though it could mention data granularity or limitations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no descriptions (0% coverage), but the description's Args section adds meaning: model_id with canonical example 'openai/gpt-5.6' and days with range (1-365, default 30) and plan-dependency. This compensates well for the schema's lack of detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the verb and resource: 'Get HISTORICAL price data / trends for one model.' It explicitly scopes to a single model and historical data, which distinguishes it from siblings like get_model (current price) and compare_providers. The differentiator against aggregators like OpenRouter is also highlighted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use by contrasting with aggregators: 'aggregators like OpenRouter expose only current price; this returns the price over time, enabling trend analysis.' This implies use case (historical trends) but does not explicitly name sibling alternatives or exclusions, so it is clear but not exhaustive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_providerA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the return behavior (provider detail with model list and pricing) and the scope of the operation. There are no side effects to mention for a 'get' operation, and the return content is adequately described.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the purpose, and uses a clear Args/Returns structure. Every sentence adds value with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter, read-only tool with an output schema, the description is complete. It specifies the input format with examples and the high-level return content. The output schema can handle detailed structure, so the description need not elaborate further.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides only the parameter name and type, with 0% coverage. The description compensates by explaining that provider_name is the provider name and giving concrete examples ('DeepInfra', 'Novita', 'Venice'), making the expected value unambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get detail for one provider') and the specific resource ('provider'), and enumerates what detail is included (models, tier breakdown, price range). It distinguishes from siblings like list_providers by explicitly scoping to a single provider.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the description: use when you need detailed information about a specific provider. However, it does not explicitly mention alternatives or when not to use this tool, leaving the decision to the agent's inference from the sibling tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_modelsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It explicitly says 'Search and list,' which implies a read-only operation, and it details the return format (models with pricing, provider, tier). It does not discuss rate limits or auth, but for a search tool these are less critical. The description provides useful behavioral context beyond just the tool's name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with an 'Args:' section followed by 'Returns:' clearly labeling each parameter and the output. Every line adds value, and the information is front-loaded with the purpose statement. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 optional parameters, no required fields, output schema indicated), the description is complete. It explains all parameters, returns, and the nature of the tool. No critical information is missing for an agent to correctly select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameter descriptions (0% coverage), but the description fully compensates by explaining each parameter: query (text search on model id/name), tier (filter with allowed values), limit (range, default), and sort (examples of sort keys). This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search and list AI inference models with current pricing.' It identifies both the action (search/list) and the resource (AI inference models), and distinguishes itself from sibling tools such as list_providers (which lists providers) and get_model (which fetches a specific model).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through its parameters and return values but does not explicitly state when to prefer this tool over alternatives. It lacks phrases like 'use get_model for a single model' or 'use list_providers for providers.' The context is clear, but no exclusions or alternative recommendations are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_providersA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does well by specifying the return shape (per-provider endpoints with pricing) and adds useful context about price divergence from OpenRouter's negotiated rate. It doesn't cover error cases or permissions, but for a read-only comparison tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and well-structured with separate Args and Returns sections. It is front-loaded with the main purpose, every sentence contributes meaningful information, and the example is immediately useful.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has only one required parameter, an output schema exists, and the description covers both input format and return semantics. The additional note about price divergence enriches understanding without going overboard, making it complete for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema description coverage is 0%, the Args section in the description supplies key semantics: 'Canonical model id' and a concrete example ('meta/muse-spark-1.1'). This goes beyond the bare schema and clarifies the expected format, which is enough to guide invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Compare') and a specific resource ('price of one model across providers'), making the tool's purpose unmistakable. It also distinguishes itself from sibling tools like get_provider and list_providers by focusing on cross-provider price comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use case is clearly implied by the purpose: use this when you need to compare pricing across providers for a specific model. However, it does not explicitly mention when not to use it or name alternative tools, so it misses the mark for a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_modelA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It explicitly lists the return fields (pricing, tier, SIT score, quality-adjusted price), making the output behavior clear. It does not mention error handling or permission requirements, but for a simple get-by-id lookup this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the primary purpose, followed by a minimal Args section and a Returns list. Every sentence provides necessary information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With a single required parameter, no annotations, and an existing output schema, the description sufficiently covers what the tool does, how to specify the model, and what to expect in return. It fits well within the sibling tool set, where get_model_history clearly covers historical data and this tool's 'current pricing' differentiates it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only defines model_id as a string with no description (0% schema coverage). The description compensates fully by explaining that it is a 'Canonical model id' and providing concrete examples for two providers, which is essential for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Get full detail + current pricing for one model by its id,' which states a specific verb, resource, and scope. It clearly distinguishes this single-model lookup from search_models, list_providers, and history/composite tools by emphasizing 'one model' and 'current pricing.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'by its id' and the provided id examples establish that this tool is for retrieving details when the exact canonical model id is known. It implies use cases and leaves discovery to search_models, though it does not explicitly name alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.