Server Quality Checklist

Tool Scores

• list_templatesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds context about template purpose but doesn't disclose behavioral traits like ordering, pagination, or caching, so it adds minimal value beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary action, and no extraneous information. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one optional parameter and annotations present, the description is mostly complete. However, it could mention ordering or pagination since no output schema exists, leaving a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the parameter description fully documented inline. The tool description does not add any additional parameter meaning beyond what the schema already provides, meeting the baseline expectation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb-resource pair ('List ... templates') and adds context about what templates are, distinguishing it from sibling tools like list_envelopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates when to use (when needing available templates) but lacks explicit guidance on when not to use or how it compares to other list tools like list_envelopes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_envelopeA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true and destructiveHint=false. Description adds specific return fields (signers, documents, status, etc.) beyond annotations, providing useful behavioral context without contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with purpose, no extraneous words. Efficiently conveys scope and return contents.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read tool with one param and no output schema, description fully covers the return value (details on signers, docs, status, etc.). No missing context needed for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter 'id' with schema description 'Envelope ID' (100% coverage). Description adds no extra parameter info beyond schema, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool retrieves full details of a specific envelope, listing categories like signers, documents, status, security level, and blockchain info. Distinct from siblings like list_envelopes (list vs detail) and analyze_document (different purpose).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicit usage is clear: for full details on one envelope. No explicit comparison to list_envelopes or other siblings, nor when not to use it. Lacks explicit guidance for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• upload_documentA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are minimal (readOnlyHint false, destructiveHint false). The description adds value by detailing the process: reads from disk, requests pre-signed S3 URL, streams bytes, returns metadata. This goes beyond the structured fields. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with the main purpose front-loaded. It is concise but could be more structured (e.g., separate bullet for supported formats). No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (3 params, no output schema, minimal annotations), the description covers usage, process, supported formats, and size limit. It is complete enough for an agent to invoke correctly, though it could mention error scenarios or exact metadata returned.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds minimal extra meaning: for contentType it lists recognized extensions, but overall the schema already documents the parameters well. No significant improvement beyond schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Upload a local file to SignaTrust and return a document ID suitable for passing to create_envelope', specifying the verb (upload), resource (local file to SignaTrust), and the intended downstream usage. It distinguishes from sibling tools like analyze_document and create_envelope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use (to get a document for an envelope), supported formats (PDF, DOCX, images), and max size enforcement. However, it lacks explicit guidance on when not to use or alternatives if the file is not local.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_envelopesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so no need to restate safety. Description adds return content details (signers, documents, blockchain status) beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: purpose and return info. No fluff, efficient structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description summarizes return fields. Parameter coverage is full. Could mention default limit, but schema already does.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description only mentions 'optional status filter and pagination' without adding new parameter semantics beyond the schema's already clear descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'list signature envelopes' with optional status filter and pagination, distinguishing it from sibling tools like get_envelope (single) and list_templates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing multiple envelopes with optional filtering, but does not explicitly state when to use alternatives such as get_envelope for a single envelope.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_documentA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and non-destructive behavior. The description adds value by revealing the use of Google Gemini, the structure of the returned report (risk assessment, flagged clauses, sentiment), and the plan gating mechanism. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first covers purpose and output, the second covers usage constraints. Every sentence is essential, no redundancy, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description fully explains the return value. It covers input, output, and plan restrictions. The single-parameter tool is well-documented with no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with the single parameter 'envelopeId' described as 'Envelope ID to analyze'. The description does not add further parameter details beyond what the schema provides, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the action ('Run AI contract analysis'), the resource ('completed envelope's document'), and the output ('structured report covering risk assessment, flagged clauses, overall sentiment'). It clearly distinguishes this tool from siblings like 'get_envelope' or 'list_envelopes' by specifying analysis functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on plan requirements ('Plan-gated: free accounts receive a 403; upgrade to Pro Lite or above to use this') and error handling ('Surface the 403 message to the user rather than retrying'). While it does not directly compare to siblings, it implies use for document analysis rather than envelope management.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_blockchainA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly and non-destructive behavior. The description adds rich behavioral details: how verification works via composite hash, hash chain, and that it makes the envelope independently verifiable. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences efficiently convey purpose, return values, and significance. No wasted words; front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one parameter, no output schema, and provided annotations, the description fully covers what an agent needs: purpose, inputs, outputs (returns composite hash, file hash, tx ID, explorer URL), and the value proposition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers the single parameter with 100% description. The tool description does not add parameter-specific details beyond the schema, but contextualizes the purpose adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool verifies a completed envelope's Solana anchor, specifying what it does and what it returns. It is distinct from sibling tools that create, list, or analyze envelopes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance is provided. The description implies use after envelope completion, but lacks alternative tool recommendations or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_envelopeA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=false and destructiveHint=false. The description adds that signers are notified via delivery method and explains the legal weight of each security level. It also mentions plan inclusion. However, it does not disclose the return value (e.g., envelope ID) or potential asynchronous behavior, but overall it is transparent about core behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph of about 100 words. It front-loads the purpose, then lists requirements, then provides security level usage. No redundant sentences; every sentence adds essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters (including nested signers), 100% schema coverage, and no output schema, the description covers the main points: what the tool does, required inputs, how to supply documents/templates, and security level guidance. It does not explain the return value or error scenarios, but given the schema's completeness, it is sufficient for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed parameter descriptions. The description adds value by explaining security levels in a use-case context and tying documentIds to upload_document. It reinforces the requirements and provides optional guidance (e.g., max length for name, but schema already has it). This goes beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Create and send a new envelope for signing.' It specifies the verb (create and send), the resource (envelope), and distinguishes from sibling tools like upload_document, get_envelope, list_envelopes, and void_envelope by being the only creation tool. No ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states requirements (name, at least one signer, documents or templateId), how to supply documents from upload_document or templates, and provides detailed guidance on when to use each security level (STANDARD for routine, VERIFIED for employment/vendor/healthcare, CERTIFIED for real estate/high-value/regulatory). This helps the agent select the appropriate parameters.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• void_envelopeA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the destructiveHint annotation, the description discloses the envelope status change to VOIDED, signer notifications, audit trail recording, and failure conditions. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each serving a distinct purpose: what it does, when to use, and what happens after. No filler, front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description covers the action, side effects, preconditions, failure modes, and post-void options. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds significant meaning to the 'reason' parameter: it explains its inclusion in cancellation notices, audit events, and webhooks, and recommends providing one. The 'id' parameter is also implicitly explained in context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Void (cancel) an in-progress envelope,' clearly stating the action and resource. There is no sibling tool with similar purpose (e.g., cancel_envelope), so it stands alone.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States when to use ('when the sender needs to cancel a contract that has already been sent'), what prerequisites to check ('use get_envelope first to check status'), and notes failure cases for already COMPLETED or VOIDED. Also suggests a post-void deletion option.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.