Server Quality Checklist

Tool Scores

• list-sessionsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It indicates a read-only listing operation via 'list' and adds context about 'recent' sessions and account scope, but does not disclose return format, pagination behavior beyond schema, or any potential side effects. This is adequate for a simple list tool but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the verb and resource, followed by the optional filters. There is no filler or redundancy, making it maximally concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core action and filters, and the schema documents all parameters. However, with no output schema, it does not describe the return shape or ordering behavior beyond 'recent'. This is sufficient for a simple list but leaves some gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for all four parameters (limit, offset, status, packageId). The description adds minimal semantic value by summarizing the filters for package and status, but does not explain limit/offset beyond what the schema already provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists sessions on the account, with a specific verb ('List') and resource ('sessions'). It differentiates from siblings like get-session (singular retrieval) and list-packages (different resource) by focusing on session listing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing sessions with optional filters, but does not explicitly mention when to prefer this over get-session or other siblings. No exclusions or alternative guidance is provided, so usage context is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-packageA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavioral context. It describes the return content (full details, fields, documents) but does not explicitly state that the operation is read-only, mention potential errors, or note any prerequisites. 'Get' implies read-only but this is not explicitly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that immediately states the action and scope. It is concise and well-structured, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter retrieval tool with no output schema, the description adequately covers the primary return value (full details including fields and documents). However, it does not mention error handling, authentication, or how this tool relates to list-packages, leaving some room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully describes the single parameter 'packageId' with 'The numeric ID of the package to fetch.' The description adds no additional parameter semantics beyond referring to a specific package, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool gets full details for a specific document package, including its fields and documents. This clearly differentiates it from sibling tools like list-packages, which would list packages rather than fetch one in detail.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for retrieving a single package by ID, but it does not explicitly mention when to use it over list-packages or provide any when-not-to-use guidance. The context is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create-sessionA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the return values (session token and URL) but doesn't mention side effects, error conditions, or whether it's idempotent. Basic transparency but leaves some behavioral aspects undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence that is front-loaded with the action and efficiently conveys the purpose and return value without excess words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and moderate complexity (prefill object), but the description covers the key context: it returns a URL for the recipient. While it doesn't detail error handling or parameter specifics, the high schema coverage and clear return description make it sufficiently complete for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds no extra parameter semantics beyond what the schema already provides, which is acceptable given full coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Start a new interview session') and the resource ('document package'), and distinguishes it from sibling tools like get-session and list-sessions by emphasizing creation and the returned URL.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when you need to start a new session and obtain a URL to send to a recipient. It doesn't explicitly name alternatives or exclusions, but the context is unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate-pdfA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosure. It mentions that the tool finalises the session and returns a download URL, which is useful, but it doesn't disclose whether the operation is reversible or any side effects beyond finalisation. Thus, moderate transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose, and each sentence adds value without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description covers the trigger condition, the action, and the return value, which is adequate for an agent to invoke correctly. It doesn't elaborate on error cases, but they are not essential given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a clear description for the one parameter 'token'. The description adds no further parameter details, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('trigger PDF generation') and resource ('a session'), and it distinguishes itself from sibling tools which are all listing/getting/creating sessions or packages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit usage guidance with the condition 'once the session status is in_progress or after filling answers programmatically', which tells the agent when to invoke it. No alternative tool is mentioned, but siblings are clearly different, so there's no exclusion needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get-sessionA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the status enums and frames the tool as a polling mechanism, implying read-only behavior. It doesn't explicitly state non-mutation, but the context strongly suggests a safe status check.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence names the action and resource; the second provides enums and a usage tip. Perfectly front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter status tool, this description is complete: it explains statuses, polling, and the token's origin. No output schema exists, so the description's information about status values suffices for the agent to interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully documents the token parameter as the session token from create-session. The description only restates 'by its token' without adding format or constraint details, so it does not elevate beyond the schema baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks a session's status and details by token, enumerating the three possible statuses. It distinguishes itself from list-sessions by focusing on a single session, and from get-package by being session-specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Poll this tool to check for completion,' providing a clear when-to-use directive. It doesn't mention alternatives or exclusions, but the polling context sufficiently guides the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list-packagesA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It mentions the return fields (IDs, names, descriptions) but does not explicitly state that this is a read-only operation or that it returns all packages without filtering. The list verb implies non-mutating behavior, adding some context but not full transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and resource, and includes only essential information. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and a simple return value, the description fully covers purpose, output, and usage context. It appropriately situates the tool in the workflow relative to create-session.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema is fully complete. The description adds no parameter semantics because there are none to describe. Baseline of 4 applies for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all document packages configured on the account, using a specific verb and resource. It distinguishes from siblings by focusing on package discovery rather than session or PDF operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises to use this tool first before creating a session, providing clear context. It does not name alternatives or exclusions, but the guidance is sufficient for a simple list operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.