Server Quality Checklist

Tool Scores

• durable_task_cancelA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden and does provide meaningful detail: cancellation is cooperative and only takes effect at the next checkpoint, not immediately. It does not cover idempotency or what happens if the task is already finished, but the core behavioral nuance is well disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, each earning its place. The essential action is stated first, and the critical checkpoint caveat follows immediately. There is no filler or repetition of the tool name beyond the initial statement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool this is reasonably adequate, but it still leaves gaps: how the taskId should be obtained, whether cancellation is idempotent, and what the result or state after cancellation looks like. With no output schema and no annotations, a bit more context would make it fully self-sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one required parameter, taskId, with zero description coverage. The tool description does not mention the parameter at all, so it adds no meaning beyond the schema's bare type and property name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action (cooperatively cancel) and a specific resource (a running durable task). This clearly distinguishes it from sibling tools like durable_task_get and durable_task_respond, so an agent can recognize its purpose without inspecting schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it: to cancel a running task that supports cooperative cancellation. The checkpoint caveat provides relevant usage context, but there is no explicit guidance about when not to use it, prerequisites, or alternatives such as getting task state first.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• durable_task_respondA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits, but it only states the target condition. It does not explain side effects of delivering input, whether the task resumes or changes state, idempotency, error behavior, or permission requirements. For a mutating action, this is a meaningful gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one efficient sentence that front-loads the verb and resource, with the key condition in parentheses. Every word earns its place and there is no wasted phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool with no output schema or annotations, the description is minimally adequate but incomplete. It omits what happens after delivery, whether the task resumes, and how an agent should handle non-input_required tasks. The title adds slight context but does not fill these gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it only implies that 'response' is the asked-for input and 'taskId' identifies the paused task. It gives no detail on response format, constraints, or the relationship between taskId and the input_required state, leaving the agent to infer parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses a specific verb+resource pair ('Deliver input') and names the exact target condition ('paused task (status input_required)'). This clearly distinguishes it from siblings like durable_task_get and durable_task_cancel, which perform different operations on similar resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an implicit when-to-use condition: the task must have status input_required. It does not explicitly say when not to use it or name alternatives, but the status condition is specific enough for an agent to select this tool for paused tasks awaiting input.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_transfer_commitA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It does disclose the core verification and move behavior, but it omits side effects such as invalidating the handle, deleting the staged file, or what happens on size/hash mismatch.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one compact sentence with no filler. It front-loads the verification behavior and states the final move action efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter finalization tool, the description covers the main action, but it lacks explicit sequencing context and parameter meaning. Without annotations or an output schema, it is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the only parameter, 'handle', is not explained anywhere in the description. The agent must infer that handle identifies the upload session to finalize. The description should compensate for the empty schema but does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action sequence: verify size and sha256, then move the staged file to its destination. This clearly distinguishes it from siblings like file_transfer_write and file_transfer_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The term 'staged file' and title 'Finalize an upload session' imply this is the terminal step after writing, but the description never explicitly says to use it after file_transfer_write or provides when-not-to-use guidance. Usage is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate-reportA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It discloses meaningful runtime traits: simulated latency (~2s per section), background execution, restart safety, and cooperative cancellation. It does not mention return values or auth, but the core execution model is well communicated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence front-loads the most decision-relevant fact (simulated slow job, per-section cost), and the second sentence lists behavioral capabilities. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema and the description does not state what the call returns (e.g., a job ID) or how to poll or cancel it via the durable_task_* siblings. For an async-job tool, this leaves an incomplete operational picture; an agent would not know how to complete the workflow from invocation through cancellation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 50%: 'topic' has a description, while 'sections' does not. The description adds behavioral meaning to 'sections' via '~2s per section,' which helps the agent understand performance implications. However, it does not explain what a section is, how topic shapes the report, or the meaning of the default value, so it only partially compensates for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Simulates a slow job') and a resource ('~2s per section'), with the title supplying the report-generation context. It clearly communicates this is a simulation/demo of an async job, which is distinct from a generic report generator. However, it does not explicitly state 'generates a market report' and leans on the title for that framing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for demonstrating background execution, progress polling, restart-safe state, and cooperative cancellation, which gives an agent a sense of when to use it. It does not explicitly name alternatives or say when not to use it, nor does it point to the durable_task_* siblings for polling/cancellation. The usage context is present but underdeveloped.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_transfer_readA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the pagination pattern and the sha256 in the final response, but does not clarify whether offset advances automatically, what happens on invalid offsets or EOF boundaries, or prerequisites like an active session. Moderate transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with essential information front-loaded: the core action first, then the pagination and hash detail. Zero fluff, every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description explains the response contract adequately: data is base64, pagination via nextOffset, eof termination, and sha256 at the end. It does not cover error conditions or session prerequisites, but for a chunked reader this is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67% and already describes defaults for length and offset. The description does not add any parameter-specific guidance beyond the schema; it focuses on output behavior. Baseline 3 is appropriate since the schema handles most parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states a specific action (fetch a byte range as base64) and resource (download session). Distinguishes from siblings like file_transfer_write (write) and file_transfer_status (status) by describing the read operation and its pagination.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides usage mechanics (repeat with nextOffset until eof) but does not explicitly state when to use this tool vs alternatives like file_transfer_status or file_transfer_open. The context is implied (within a download session) but not contrasted with siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_transfer_writeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and does add valuable behavior: it reveals that a wrong offset produces a repairable error with the correct cursor rather than failing the transfer. It also implies state mutation through 'append', though it does not detail broader side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two focused sentences deliver the core operation and a key failure-mode detail. Every sentence earns its place; no filler or redundant restating.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter no-annotation tool it covers the operation, the input encoding, and a non-obvious error behavior. It falters only on defining the session handle and how the cursor is obtained, which an agent would need to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, and the description compensates somewhat by explaining offset semantics ('expected offset' and cursor recovery). However, the required 'handle' parameter is left unexplained, and 'data' only repeats what the schema already says.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Append') and resource ('base64 chunk at the session's expected offset'), making the operation unmistakable. It also distinguishes from sibling tools like file_transfer_open and file_transfer_commit by signaling this is the chunk-append step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use during an ongoing upload session via 'session's expected offset', but it does not explicitly state when to use this tool vs file_transfer_open/file_transfer_commit, nor any prerequisites such as opening a session first.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_transfer_statusA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and it does reveal the key behavioral outcome: it reports direction, status, bytes received so far, and destination. The title 'Inspect' further implies a read-only operation. It does not mention error behavior or side effects, but for a status-reporting tool this is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short sentence that immediately lists the reported information and then adds the practical restart use case. There is no filler, and the most important content is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema, the description names the return content (direction, status, bytes, destination) and the scenario where it matters. It could be more explicit about status meanings or whether the handle survives a restart, but for the tool's low complexity, this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'handle' has 0% schema description coverage, and the description never mentions 'handle' or explains what it refers to. Agents must infer from the tool name and siblings that 'handle' identifies a transfer session, which is a significant semantic gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Report') and identifies the resource (a transfer session) with the specific fields it returns: direction, status, bytes received, and destination. This clearly distinguishes it from sibling tools like file_transfer_write or file_transfer_commit, which are operations rather than inspection.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear context for when to use it: 'useful after a restart.' This implies using it to resume or verify a transfer session rather than starting or writing. It does not explicitly name alternatives or when-not-to-use cases, but the context is sufficient for typical agent routing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• durable_task_getA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden, and it does well: it reveals that calls are repeated, that the operation is a poll, and that terminal states are 'completed' or 'failed'. It does not discuss rate limits or invalid task behaviors, but the read-only polling behavior is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the action and followed by the polling instruction. Every word contributes; there is no filler, repetition, or schema duplication.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter, low-complexity polling tool, the description is sufficient: it identifies the object, the parameter context, and the termination condition. It does not specify the exact response payload, but the absence of an output schema and the simple status check make that a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is no schema description and the body text never explicitly mentions taskId, but the phrase 'task created by this server' clarifies that the sole parameter refers to a server-issued task identifier. The property name and type are self-explanatory, though the description could have added where the ID comes from.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Poll') and a concrete resource ('a long-running task created by this server'), and the title names the resource as 'durable task status'. This makes the tool clearly distinct from siblings like durable_task_cancel and durable_task_respond without needing to inspect them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states when to use the tool (for long-running tasks created by this server) and gives an explicit polling pattern ('Repeat until status is completed or failed'). It does not explicitly contrast with sibling tools, but the get/cancel/respond naming makes the intended use obvious; no exclusion cases are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• file_transfer_openA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden, and it discloses a key behavioral trait: 'Sessions survive server restarts — resume at the reported nextOffset.' It also clarifies the upload/download protocol. It would be stronger with cleanup or session-identity details, but the disclosed persistence is genuinely useful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences with no fluff; the most important scoping ('chunked upload or download session') is front-loaded, and the protocol details are organized by direction. Every sentence adds information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The lifecycle (open → write/read → commit/EOF) is described and persistence is noted, but there is no output schema and the description does not specify what open returns beyond a reported nextOffset—no session/handle identifier is mentioned. It also omits any cleanup/cancel path, so an agent may not have everything needed to invoke this correctly in all cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 75%, and the description compensates by mapping direction to concrete workflows: upload starts at offset 0 and ends with commit, download reads until eof. It also clarifies that resumption uses a reported nextOffset. Size and sha256 remain at schema level, but their schema descriptions are already adequate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Open') and names the resource: a chunked upload or download session. It differentiates itself from siblings by situating the tool at the start of a clear workflow: upload via file_transfer_write then file_transfer_commit, download via file_transfer_read until eof.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives concrete usage context for both directions, explicitly naming sibling tools in sequence and telling the agent to start at offset 0 or read until eof. It does not explicitly state when not to use this tool or mention alternatives like file_transfer_status, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.