Server Quality Checklist

Tool Scores

• recommend_modelC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It fails to state whether the operation is read-only, what the ranking is based on, how results are (possibly ordered) returned, or any side effects. Since it mentions no output or side effects, the description leaves the agent without relevant context about what to expect when invoking it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short sentence with no redundancy, front-loaded with the main verb. It could have included more essential details (like the limit default), but it is concise and to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and 0% parameter coverage, the description is the only source of behavioral and parameter context. It does not explain what the ranked output looks like, the meaning of enums, or the default limit (3). The tool is moderately complex (4 params, multiple enums), so the description is incomplete for an agent to call it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameter meanings. It maps 'budget' and 'requiresSelfHosted' to the mentioned 'orçamento' and 'requisito de hospedagem', and 'scenario' implicitly to 'cenário', but it does not explain the 'limit' parameter, the meaning of the enum values, or the default behavior. This is insufficient for 0% coverage from a parameter semantics perspective.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Orders' (sort/rank) and the resource 'models do catálogo' (models from the catalog), specifying the inputs (scenario, budget, self-hosting requirement). It distinguishes itself from siblings list_models and compare_models by naming the ranking behavior, which is different from simply listing or comparing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a use case (rank models for a scenario with budget and self-hosting constraints) but gives no explicit guidance on when to choose this tool over siblings like list_models or compare_models, nor any 'when not to use' conditions. The agent must infer usage from the verb and context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_modelsB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full burden of behavioral disclosure. It mentions 'mostra a classificação' indicating a ranking output, but it doesn't disclose how the comparison works, whether it's a read-only operation, any rate limits, or what happens if fewer than 2 or more than 4 IDs are given (though schema enforces this). The term 'mostra' implies a non-destructive read, but this is not explicitly confirmed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler, front-loading the core action (compare and show ranking). It's appropriately concise for a straightforward tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 2 params, no annotations, and no output schema, a one-line description is insufficient. An agent needs to know what 'comparison' entails, what the ranking output looks like, and any constraints or preconditions. The description provides only basic purpose, not enough for correct invocation without additional inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It says 'até quatro modelos' which adds semantic meaning to the IDs parameter (max 4, min 2), and mentions 'cenário específico' relating to the scenario parameter. However, it doesn't elaborate on what each scenario means or how IDs are specified, which leaves significant ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool compares up to four models and shows a ranking, which goes beyond just repeating the title 'Comparar modelos'. However, it doesn't explicitly name any sibling tool for differentiation, and the title is in Portuguese while the description is in Spanish, which is slightly inconsistent but not misleading.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'para um cenário específico', implying use case specificity, but it doesn't explicitly state when to use this over list_models, recommend_model, or explain_scenarios. The distinction is not explicitly made, so an agent must infer from context that this is for comparing multiple models side-by-side.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description has to carry the behavioral disclosure. It does convey a read-only inventory action over the local catalog, which is useful context, but it does not disclose filtering behavior, return format, pagination, or what happens when the catalog is empty.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  This is a short, front-loaded, single-sentence description that uses no filler and adds value over repeating the title. Every word earns its place in conveying the local catalog scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully communicates the core listing behavior, but the absence of annotations, output schema, and parameter documentation leaves meaningful gaps around the 'provider' filter and the exact shape of returned model profiles. For a simple list tool, this is only partially complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes the 'provider' parameter only as a string and the description never mentions it at all, despite 0% schema description coverage The agent has no guidance on whether 'provider' filters, which values it accepts, or what effect it has on the result.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Lista'), the target resource ('perfis de modelos'), and the scope ('catálogo local'). This differentiates it from the sibling tools, which recommend, compare, and explain rather than list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when it should be used: when an agent needs to enumerate model profiles from the local catalog. However, it does not explicitly compare alternatives or state when not to use this tool, leaving the routing decision to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_scenariosA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. The verb 'Explica' strongly implies a read-only, non-mutating operation and the content covered is stated. However, it does not disclose output format, state dependence, or any other behavioral traits beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no filler, front-loading the verb and the main object. Every word contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description states the tool's purpose and scope in a way sufficient for basic invocation. It lacks explicit usage guidance and output details, but the low complexity keeps this from being a serious gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. There are no parameter semantics to document, and the description appropriately focuses on purpose rather than inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Explica' and names concrete resources: the evaluated scenarios and the weights used by the recommendation mechanism. This clearly distinguishes it from sibling tools like list_models, recommend_model, and compare_models.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to prefer this tool over its siblings or mention any exclusion criteria. Usage is only implied: an agent wanting to inspect evaluated scenarios and weights would infer this tool, but no direct alternative guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.