Server Quality Checklist

Tool Scores

• inboxA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, but the description discloses key behavioral details: it reads addressed messages, sorts oldest first, and only shows unread ones by default. It also specifies the exact return shape, which adds transparency beyond the bare schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main action, and no redundant wording. Every clause is informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers what the tool does, how to filter, ordering, and return format, which is sufficient given the simple schema and lack of an output schema. It does not discuss edge cases or pagination, but none are indicated as relevant.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning to both parameters: it explains 'agent' as the addressee and defines 'unread_only' as hiding already-acknowledged messages. Since schema coverage is 0%, this compensation is valuable, though 'agent' could use more detail about valid values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Read'), resource ('messages addressed to agent'), and ordering ('oldest first'). This differentiates it from sibling tools like send, ack, and ask_* which are not read-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternatives are mentioned. The description implies use for reading unacknowledged messages but does not compare to history or other read-like siblings, so guidance is absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_claude_asyncA
  4.1/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and delivers richly: background execution, immediate return, mailbox delivery from 'claude', JSON-encoded body, label support, fire-and-forget semantics, non-persistence, and restart loss are all disclosed. It even notes from_agent must be a known agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and grouped logically into delivery, polling, and caveats. It is longer than strictly necessary, but each sentence contributes meaningful operational detail, so no content is wasted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The response flow is well covered: what comes back, how to poll, and how to match concurrent results. However, with no output schema and no annotations, the missing parameter-level semantics for most arguments leaves the description incomplete for a 7-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters, but it only clarifies label (matching results) and from_agent (must be known). The five other parameters (model, write, effort, prompt, workdir) are left largely unexplained, though prompt's meaning is implied by 'Claude task'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Dispatch a Claude task in the background; returns immediately,' which names a specific verb, resource, and key behavioral distinction from the synchronous ask_claude sibling. This clearly differentiates the tool from ask_claude, ask_hermes, and ask_codex.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explains when to use this async variant: 'Runs the same ask_claude in a background thread' and instructs polling via inbox(agent=from_agent). It does not explicitly name alternatives for exclusion, but the contrast with ask_claude is evident, and the delivery mechanism is well specified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• historyA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does well: it discloses ordering (newest first), the filtering behavior for the agent parameter, and the exact return shape ('{messages, count}'). This goes beyond a minimal statement by revealing the output structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and every sentence adds value. There is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool, the description covers purpose, ordering, filtering, and return format. It omits explicit explanation of the 'limit' parameter's effect, but the schema provides a default, keeping the description reasonably complete. It does not need to explain return values in more detail in the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explains the 'agent' parameter's semantics (filters to sender/recipient), but it does not explain 'limit'. Since schema description coverage is 0%, the description only partially compensates for the missing parameter documentation, leaving 'limit' to be inferred from its name and default value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as a feed of recent messages ordered newest first, explicitly labeling it a 'visibility / audit feed.' This distinguishes it from siblings like 'inbox' by implying a broader historical view rather than just received messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'visibility / audit feed' provides clear context for when to use this tool, and the agent filter is described. However, it does not explicitly name alternative tools or state when not to use it, so it stops short of full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ackA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It clearly explains idempotency, the conditional success ('only if a still-unread message... existed'), and the exact return value. This is thorough for a simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the purpose, followed by a crucial behavioral note. Every word earns its place; there is no redundancy or unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema. The description adequately covers functionality, return semantics, and idempotency, making it complete for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema offers only the parameter name and type. The description adds that the message must be unread and refers to 'that id', but it does not explain where the id comes from or any constraints beyond the schema's integer type. It adds minimal semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Mark a message read') on a specific resource (a message identified by id) and the resulting effect (removes from unread inbox). It distinguishes itself from sibling tools like 'send' or 'inbox' by focusing on the acknowledgment operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: you should call this when you have an unread message id and want to mark it read. However, it does not explicitly discuss alternatives, prerequisites, or exclusions (e.g., already-read messages), so guidance is only implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_codex_asyncA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does an excellent job: it discloses that execution is in a background thread, results are delivered via mailbox, the message body is JSON-encoded, and the task is fire-and-forget with a restart caveat. This goes beyond what schema/annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense paragraph but well-structured, with the main purpose front-loaded. Every sentence adds useful information (background execution, mailbox delivery, label matching, persistence caveat) without fluff. Minor loss for not being more scannable, but overall efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 params, async, mailbox interaction) and lack of annotations/output schema, the description covers the essential context: how to retrieve results, label usage, audience requirement, and persistence behavior. It does not explain the full return payload structure or error handling, but it is sufficiently complete for an agent to invoke and poll the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains 'label' and 'from_agent' explicitly, and implies 'prompt' as the task input. However, it does not describe 'model', 'write', 'effort', or 'workdir', relying on the phrase 'same ask_codex' to cover them. This is partial compensation but not complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it dispatches a Codex task in the background and returns immediately, distinguishing it from synchronous siblings like ask_codex. It also explains the delivery mechanism via mailbox, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explains how to use the tool (poll inbox with inbox(agent=from_agent)) and provides a key prerequisite (from_agent must be known). It also advises using label to match results for concurrent dispatches. However, it does not explicitly contrast with ask_codex_async versus ask_codex or other async variants, though the context implies background use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_hermesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the disclosure burden. It reveals that the tool spawns a one-shot `hermes chat -q`, is slower/heavier than the async mailbox, and returns `{"ok", "reply"}` or `{"ok": false, "error"}`. This covers key behavioral traits, though it omits details like timeout or prompt handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose, and every sentence adds useful information (what, when, return shape). No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter synchronous tool with no output schema, the description is complete: it explains the operation, the use case, the performance tradeoff, and the exact return format. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The phrase 'Ask ... a question' indicates that the single `prompt` parameter is the question text, but no further formatting, length, or constraints are provided. This is sufficient for a simple string parameter but not richly detailed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Ask the Hermes agent (MrAnderson) a question and wait for its reply.' It also distinguishes itself from the async mailbox by describing the synchronous one-shot nature, which differentiates it from sibling async tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states 'use it when you need an answer NOW' and contrasts this with the 'async mailbox', which is slower/heavier by comparison. This provides clear when-to-use guidance and a named alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_claudeA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description fully discloses safety and execution traits: default denial of Edit/Write/NotebookEdit, inclusion of inspection tools, `write=True` bypassing permissions and requiring HARDLINE_ALLOW_WRITE, advisory-mode neutralization/fail-closed auth checks, and the hang-to-timeout risk. This goes far beyond annotations (none provided) and clearly informs the agent of side effects and safeguards.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single opening sentence delivers purpose; remaining paragraphs are dense, information-rich parameter explanations. The long run-on style with em-dashes and parentheticals makes it slightly harder to scan, but no sentence is superfluous.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter tool with no output schema, the description covers what responses include (ok/reply plus metadata for optioned calls), when the tool hangs, env-var gating, mode-specific behaviors, and compatibility defaults. It is sufficiently complete for an agent to invoke safely, though exact return structure is left unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description compensates by explaining every non-obvious parameter: `model` pinning/default, `effort` values and default omission, `workdir` targeting and rejection, `write` opt-in semantics and prerequisites, and `mode` distinctions. Even `prompt` is inferable from the tool's purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Ask Claude Code a question and wait for its reply'—a specific verb, target resource, and synchronous behavior. The phrase 'wait for its reply' and underlying `claude -p` reference distinguish it from async siblings like ask_claude_async.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides detailed conditions: default/no-options behavior, when `write=True` is allowed/refused, and advisory-mode constraints. However, it never explicitly names alternative sibling tools or says 'use this when X vs async,' leaving comparison mostly implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_codexA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden, and it excels: it discloses ephemerality, model-default behavior, advisory mode's auth preflight and isolated environment, write-mode sandbox with the HARDLINE_ALLOW_WRITE guard, read-only fallback, and JSONL telemetry limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but front-loaded with a one-sentence summary, then systematically covers operational details. Every sentence adds value, especially the security-relevant write and advisory mode constraints, with no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter tool with no output schema, the description covers most behavioral and parameter nuances well. However, it does not describe the reply format, error behavior, or timeout semantics, which would be needed for full completeness without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters, and it does: mode (advisory vs default), model (omission passes no --model flag), workdir (repository target in default, rejected in advisory), write (requires workdir, env var, read-only otherwise), and effort (enables JSONL telemetry).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: "Ask Codex a question and wait for its reply." It then clarifies the underlying mechanism ("Spawns an ephemeral codex exec"), which helps distinguish this tool from sibling models like ask_claude/ask_hermes and from the async variant ask_codex_async.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use default vs advisory mode and explains writable vs read-only behavior. However, it does not explicitly mention alternatives or when-not-to-use, though "wait for its reply" implicitly contrasts with ask_codex_async.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sendA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully reveals side effects: persistence, delivery behavior, agent validation, and return formats. It even names native mechanisms (hermes chat / codex exec / claude -p), giving a complete behavioral picture.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise at three sentences, front-loaded with the core action, then followed by behavioral details and return format. Every sentence adds necessary value with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description covers validation rules, return values, persistence behavior, delivery mechanism, and error handling. It is fully sufficient for an agent to decide and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no parameter descriptions (0% coverage), but the description compensates by explaining allowed agent names, the deliver flag's effect, and the response structure. The message parameter is self-evident, so all non-obvious parameters are well covered.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the specific action: 'Send a message from one agent to another.' It distinguishes itself from sibling ask_* tools by highlighting the durable mailbox persistence, making clear this is for persistent inter-agent messaging rather than one-off queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool: it always persists to a mailbox, and optionally delivers a one-shot notice. However, it does not explicitly mention alternatives or exclusions (e.g., when to use ask_claude instead), so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.