Server Quality Checklist

Tool Scores

• runA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states that the tool returns a JSON report, passes args straight through to `memtrust run`, and that backends without credential env vars are reported as skipped rather than errors. These are useful behavioral traits that go beyond a simple command wrapper. It could mention side effects, but for a benchmark runner this is adequately transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the main action stated first. It includes a helpful example and a behavioral note. The inclusion of the real `memtrust --help` output adds some extra length, but it is brief and provides relevant context. No sentence is wasted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple single-parameter tool, the description covers the essential aspects: what it does, how the parameter works, and the handling of missing credentials. An output schema exists, so return values are already specified elsewhere. The description is complete enough for the agent to invoke the tool correctly with minimal ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage for the `args` parameter, so the description must compensate. It does so explicitly: '`args` are passed straight through to `memtrust run`' with an example. This fully explains the meaning and format of the parameter, which is the only one. This is exemplary compensation for a poorly documented schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Run the memtrust agent-memory benchmark harness and return the resulting JSON report.' It specifies the verb (run) and the resource (the benchmark harness), making the purpose unambiguous. Even without sibling tools, the description is specific and actionable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: to run the memtrust benchmark harness. It also explains how the args are passed through and gives an example, which serves as implicit usage guidance. While there are no exclusions or alternative tools to differentiate from, the provided context is sufficient for the agent to decide when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.