Server Quality Checklist

Tool Scores

• local_embedA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It states the core behavior and the Ollama preset for the model, but it does not mention input limits, handling of empty/large texts, or error behavior. Core behavior is clear, but some operational details are absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: purpose first, then use cases, then argument details. Every sentence adds useful information, and the Args section is minimal and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a relatively simple embedding tool, the description covers what the tool does, when to use it, and both arguments. The output schema exists, so return values do not need to be documented here. It does not explicitly say the computation is local, but the tool name and 'Ollama preset' strongly imply it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no property descriptions, so the description must compensate. It explains that 'texts' are the texts to embed and that 'model' is the embedding model, notably adding the non-obvious Ollama preset context beyond the schema default. It could specify allowed model choices, but it covers both parameters adequately for a simple two-argument tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Computes embedding vectors for a list of texts.' It also distinguishes the tool from likely language-model siblings by noting it works 'without troubling a large language model,' so an agent can tell local_embed apart from local_ask and local_structured.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear use cases: similarity comparisons, duplicate detection, and rough topic sorting. It also implies when not to use a generative LLM, which is helpful for choosing between this tool and sibling tools, though it does not explicitly name alternatives or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It adds useful behavioral detail: results depend on backend reporting ('as far as the backend reports them'), and vLLM-specific behavior is disclosed (showing LoRA adapters and base models). This provides meaningful transparency beyond the simple operation name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the core purpose, followed by return-value details and usage timing. No wasted words, and every sentence contributes meaningful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, parameterless, and has an output schema, so the description need not re-explain return values. It covers what the tool lists, backend-specific behavior, caveats, and when to use it, which is complete for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty, so there is nothing to document. The baseline of 4 applies because no parameter semantics are needed; the description is not lacking anything in this area.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Lists') and resource ('models available on the local instance'), and goes beyond a simple label by enumerating the included fields: name, parameter size, quantization, and on-disk size. This clearly distinguishes it from the sibling tools, which are about asking, structuring, and embedding.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit guidance: 'Sensible before any other tool, to pick a fitting and actually present model name.' This tells the agent when to call it, although it does not explicitly mention when-not-to-use or alternative tools, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• local_structuredA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral disclosure. It explains validation against the schema, a retry mechanism up to twice with concrete violations, the final error containing the raw invalid output, and the token-budget tradeoff of setting think=true. This is unusually transparent and goes far beyond a minimal description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and dense: a one-sentence purpose, a concise mechanics paragraph, and a labeled Args section. Every sentence adds operational value, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity—multi-backend enforcement, retries, error behavior, and optional reasoning trace—the description covers nearly everything needed for correct invocation. Minor gaps: it doesn't state what happens when think=true is used on a model without the thinking capability, and it offers no explicit sibling routing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description is the sole source of parameter meaning. It defines every argument: model with an example, prompt's purpose, schema's expected structure, system as optional, and think with default, rationale, and model-capability constraint. This fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear, specific statement: it makes a local model return a result conforming to a JSON schema. It further details the backend mechanisms (Ollama 'format', vLLM 'response_format'), which clarifies what the tool does and distinguishes it from the sibling tools local_ask and local_embed.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Intended usage is strongly implied—you use this tool when you need schema-enforced JSON output from a local model—but there is no explicit guidance about when not to use it or which sibling to prefer (e.g., local_ask for free-form text). The agent must infer tool selection from context rather than being told.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• local_askA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does so thoroughly. It discloses the token-budget behavior of the think parameter, warns that a tight max_tokens can cause an empty answer, and notes that thinking is only available for models with that capability. This is valuable operational context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by concise usage guidance and a well-structured Args list. Every sentence adds meaningful information; there is no filler or redundancy. The length is justified by the number of parameters and the critical think-related warning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a 7-parameter tool with no annotations. It explains the return type, all parameters, usage boundaries, and a non-obvious behavioral trap. Combined with the presence of an output schema, nothing essential is missing for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description fully compensates by explaining every parameter: model references list_models, prompt is the actual instruction, system is role/behavior, temperature is determinism vs creativity, max_tokens is a cap, context is prepended text, and think explains the reasoning trace with its trade-off. This is exemplary parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: 'Asks a local model a question and returns the answer as text.' It also distinguishes itself from the sibling local_structured by explicitly saying to use that tool for machine-processable results. The purpose is clear, specific, and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly gives usage context: 'For free-text tasks: writing, rewriting, summarizing, explaining.' It also states the alternative: 'If a machine-processable result is needed, use local_structured instead.' This gives the agent clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.