Server Quality Checklist

Tool Scores

• ask_modelA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral transparency burden. It only states the intent ('Ask a question') without disclosing return format, streaming behavior, error handling, permissions, or side effects, forcing the agent to guess.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with a clear, front-loaded purpose statement followed by a compact Args list. It contains no filler or redundancy, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool, the description and Args are mostly adequate, but the absence of any mention of the return value (e.g., the model's generated response) is notable because there is no output schema. It also lacks error or availability information, leaving some ambiguity for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema has no descriptions, the description includes an Args block that clearly explains both parameters: 'model' is the model name with an example ('llama2'), and 'question' is the prompt to ask. This fully compensates for the schema gap and adds meaningful semantic detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific action ('Ask a question') and a resource ('specific Ollama model'), making the tool's purpose immediately clear. It also naturally distinguishes itself from sibling tools like list_models and show_model, which are about listing and viewing models rather than interacting with them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool versus list_models or show_model. There are no conditions, prerequisites, or exclusions, leaving the agent to infer usage solely from the tool name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• show_modelA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It conveys that this is a read-only operation ('Get') and scoped to a single model, but it does not mention what 'detailed information' includes, response format, errors, or permissions. This is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, consisting of a single clear sentence plus a minimal Args block. Every word earns its place, with no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter read tool, the description is mostly sufficient, but it lacks detail about the return value or what specific information is provided. Since there is no output schema, the description should have compensated by describing the output, which it does not.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the parameter. The only parameter 'name' is described as 'Name of the model to show information about', which adds little beyond the schema's title 'Name'. It does not specify allowed values, formats, or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get detailed information about a specific model' clearly states the action (get) and the resource (specific model), which distinguishes it from sibling tools like list_models (enumerating models) and ask_model (interactive querying). It is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when you need details about one particular model, but it does not explicitly state when to use it over alternatives or provide any exclusions. No when-not guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It conveys a read-only list operation, but adds no details about output format, ordering, performance, or side effects. For a simple list tool this is minimally adequate, but it does not go beyond the literal action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence that is front-loaded with the key action and resource. Every word contributes meaning, with no redundant content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (no parameters, no output schema), the description fully states what the tool does. It does not explain return value format, but for a list operation, the implied return is the collection of models, which is sufficient in this context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, and the schema is empty with 100% coverage. Per the rubric, the baseline for 0 params is 4. The description does not need to explain any input semantics since none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description "List all downloaded Ollama models" uses a specific verb ("list"), identifies the resource ("downloaded Ollama models"), and specifies the scope ("all"). This clearly distinguishes it from sibling tools like show_model (which likely displays a single model) and ask_model (which likely queries a model).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use case—simply list all downloaded models—but provides no explicit guidance on when to choose this over alternatives. It does not mention show_model or ask_model as alternatives or exclusions, leaving the decision to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.