Server Quality Checklist

Tool Scores

• list_modelsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It only states that it lists models, with no mention of output format, pagination, rate limits, or any side effects. For a read-only listing tool, this minimal disclosure is insufficient for full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that delivers the core information without any fluff. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of a zero-parameter listing tool, the description is minimally adequate. However, with no output schema or annotations, it does not describe the return format or any limitations, leaving some gaps for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline of 4 applies. The description doesn't need to add parameter details because there are none to document.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('models') with a clear scope ('available on the DeepSeek API'). It distinguishes itself from sibling tools like chat and list_harnesses by naming a different resource and action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. The description only states what it does, not in which contexts it is appropriate or when other tools might be preferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_harnessesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It usefully discloses the configuration source (HARNESS_ALLOW_HOSTS / HARNESS_CMD_*) and that commands are included, but it omits edge cases like empty lists or error behavior, leaving read-only as an inference.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no filler. Each phrase adds value: 'List', 'harness hosts', 'the bridge can drive', and the config source.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple listing tool with no parameters or output schema, the description covers key facts: what is listed and where the data comes from. It does not define the return format, but the purpose is clear enough for tool selection.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so no parameter details are needed. The description adds meaning by specifying the content (hosts and commands) and the source, which is sufficient for understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and identifies the resource as 'harness hosts' with scope 'the bridge can drive (from config)', clearly distinguishing it from sibling tools like list_models and use_harness.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives (e.g., before use_harness or in contrast to list_models). The description states only what the tool does, with no explicit use cases or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• use_harnessA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It does disclose a key security constraint (HARNESS_ALLOW_HOSTS) and explains the capability of driving local agents. But it omits important behavioral traits: it doesn't state that the prompt executes arbitrary commands (potential side effects), what the return value contains (stdout/stderr), or whether the call blocks until completion. This is a risky execution tool, and the description should be more candid about its effects and result format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: the first defines the action, the second expands on capabilities, and the third provides a crucial security constraint. It is front-loaded with the main verb and avoids redundancy or filler. This is a model of concise, informative writing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an execution tool of moderate complexity (4 params, no output schema), the description covers the primary purpose and a necessary prerequisite but leaves gaps: it doesn't describe the response format or error behavior, which is essential when no output schema exists. It also doesn't point to list_harnesses as a way to discover allowed hosts. Thus, while useful, it falls short of being fully self-sufficient for an agent to invoke it correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minimal extra meaning: it ties the 'host' param to the HARNESS_ALLOW_HOSTS allowlist and clarifies that 'prompt' is the task for the harness. It does not elaborate on 'cwd' or 'timeout_ms' beyond what the schema already states, so no significant enhancement is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Run a prompt in a headless host CLI' with concrete examples (claude -p, codex exec, opencode run). It also clarifies its scope by stating it lets the model drive local agents, skills, plugins, and MCP tools, which distinguishes it from siblings like chat (interactive) and list_harnesses (listing). No ambiguity remains about the tool's core function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use this tool: whenever the model needs to execute a prompt in a headless CLI harness. It also provides a prerequisite (host must be in HARNESS_ALLOW_HOSTS), which implicitly guides the agent to verify host legitimacy. However, it does not explicitly contrast with sibling tools like chat or list_harnesses, nor state when not to use it, so it misses the 'when-not' element for a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chatA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so the description carries the full burden. It discloses the core behavior (sends request, returns reply text) and the message format. It doesn't mention side effects like API costs or rate limits, but for a stateless generation tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main action and result. No redundancy or wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with well-documented params. It lacks an output schema, but the description states the return type ('assistant's reply text'), which suffices. No obvious missing context given its limited scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds only a minor nuance ('OpenAI-style' message list) beyond the schema, but does not meaningfully explain parameters further.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Send a chat completion request to DeepSeek') and clearly distinguishes this tool from siblings like list_models and list_harnesses. Mentioning OpenAI-style messages and the reply text further clarifies its unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The context is clear: this tool is for generating chat completions, while siblings manage models/harnesses. However, there is no explicit 'when not to use' or alternative comparison, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.