Server Quality Checklist

Tool Scores

• list_templatesC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure but provides almost none. 'Lists the templates' implies a read-only operation but doesn't specify whether this returns all templates, paginated results, requires authentication, has rate limits, or what format the output takes. For a tool with zero annotation coverage, this minimal description leaves critical behavioral traits undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is maximally concise at just three words ('Lists the templates.'). It's front-loaded with the core action and resource, with zero wasted words or redundant information. For such a simple tool, this brevity is appropriate and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema) but lack of annotations, the description is incomplete. It doesn't explain what 'lists' means operationally—whether it returns all templates, supports filtering, requires permissions, or what the output structure looks like. For even a simple list operation, more context about behavior would be helpful for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage (empty schema), so the description doesn't need to compensate for parameter documentation. The baseline for this situation is 4, as there are no parameters to explain beyond what the schema already provides (none). The description doesn't add parameter semantics, but none are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Lists the templates' is a tautology that essentially restates the tool name 'list_templates' without adding meaningful specificity. While it correctly identifies the verb ('Lists') and resource ('templates'), it doesn't distinguish this tool from sibling tools like 'query_template' or provide any additional context about what kind of templates or what listing entails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives. With sibling tools like 'query_template' (likely for specific template lookup) and 'list_recent_contracts' (for a different resource type), the description offers no indication of whether this is for browsing all templates, filtered templates, or how it differs from query operations. The agent receives no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_template_collaboratorsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a read operation ('Returns'), implying it's non-destructive, but doesn't cover other important aspects like authentication requirements, rate limits, error conditions, pagination, or what happens if the template_id is invalid. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose. It avoids unnecessary words and gets straight to the point, though it could be slightly more structured by explicitly mentioning the parameter. Every part of the sentence contributes meaningful information about what the tool does.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (1 parameter, no annotations, no output schema), the description is incomplete. It explains the return data but omits critical context: the required parameter 'template_id' is undocumented, behavioral traits like error handling are missing, and there's no output schema to clarify the return structure. For a tool that retrieves data based on an input, this leaves too many unknowns for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the schema provides no parameter documentation. The description doesn't mention the 'template_id' parameter at all, leaving it completely undocumented. With 1 required parameter and no explanation in the description, the agent lacks context on what this parameter represents or how to use it, failing to compensate for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns the list of template collaborators' with specific details about what information is included (GUID, name, email, HTTPS link). It uses a specific verb ('Returns') and resource ('template collaborators'), but doesn't explicitly distinguish it from sibling tools like 'list_templates' or 'query_template' beyond the focus on collaborators.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites (e.g., needing a valid template_id), when not to use it, or how it differs from sibling tools like 'query_template' which might also retrieve template-related data. The agent must infer usage from the tool name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_contractC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While it mentions draft/instant sending modes and automatic handling of signature fields, it lacks critical information about permissions required, rate limits, error conditions, response format, or what happens when contracts expire. For a complex creation tool with 18 parameters, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise at 3 sentences but could be more front-loaded. The first sentence establishes purpose, but the following sentences delve into implementation details (signature fields, placeholders) that might be better placed after core functionality. Some redundancy exists between the description and schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex contract creation tool with 18 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what the tool returns, error conditions, authentication requirements, or how it differs from related template operations. The schema does heavy lifting, but the description should provide more contextual guidance for proper usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 18 parameters thoroughly. The description adds minimal value beyond the schema, mentioning placeholder fields and signer fields but not providing additional context about parameter interactions or usage patterns. Baseline 3 is appropriate given the comprehensive schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a new contract and distinguishes between draft and instant sending modes. It specifies the resource (contract) and key capabilities (draft creation, placeholder customization, signer field handling). However, it doesn't explicitly differentiate from sibling tools like 'create_template' or 'query_contract'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions when to use draft vs instant sending but provides no guidance on when to use this tool versus alternatives like 'create_template' or 'update_template'. There's no mention of prerequisites, dependencies, or exclusions relative to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_template_collaboratorC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that providing an email 'triggers an invitation email', which adds some context beyond the basic action. However, it lacks critical details: whether this is a read-only or mutating operation, what permissions are required, if the link expires, or what happens on failure. For a tool that likely modifies access permissions, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the primary action ('creates a HTTPS link') and adds conditional behavior ('sends an invitation email if an email is provided'). There is no wasted text, and every part earns its place by clarifying the tool's functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (a tool that likely modifies access and sends emails), lack of annotations, no output schema, and incomplete parameter documentation, the description is inadequate. It doesn't cover behavioral aspects like error handling, response format, or side effects, leaving the agent with significant gaps in understanding how to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% (2 out of 3 parameters have descriptions). The description adds minimal value: it echoes that email 'triggers an invitation email', which is already in the schema's email description. It doesn't explain 'template_id' (the required parameter) or 'name' beyond what the schema provides. With moderate schema coverage, the baseline is 3, as the description doesn't significantly compensate for gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('creates a HTTPS link', 'sends an invitation email') and identifies the resource ('contract template'). It distinguishes from siblings like 'create_template' or 'update_template' by focusing on collaboration rather than template creation or modification. However, it doesn't explicitly mention 'collaborator' in the description despite the tool name, which slightly reduces specificity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing template), exclusions (e.g., not for removing collaborators), or compare it to sibling tools like 'remove_template_collaborator' or 'list_template_collaborators'. The agent must infer usage from the description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_templateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action is a deletion but doesn't cover critical aspects like whether it's irreversible, requires specific permissions, affects related data, or has rate limits. This leaves significant gaps for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description is insufficient. It lacks details on behavioral traits (e.g., irreversibility, permissions), output format, or error handling, leaving the agent with incomplete context for safe and effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'template_id' documented as a 'GUID of the template to be deleted'. The description adds no additional meaning beyond this, so it meets the baseline for high schema coverage without compensating value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Deletes') and resource ('a contract template'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'delete_contract' or 'remove_template_collaborator', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'delete_contract' or 'remove_template_collaborator'. The description lacks context about prerequisites (e.g., whether the template must be unused) or exclusions, offering minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_templateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description states it 'responds with' details, implying a read-only operation, but doesn't clarify if it's safe, requires authentication, has rate limits, or what happens on errors. For a tool with no annotations, this leaves significant behavioral gaps, as it doesn't address permissions, side effects, or response handling beyond listing fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single run-on sentence that lists multiple fields without clear structuring. While it efficiently enumerates what the tool returns, it could be more front-loaded and organized (e.g., separating core details from lists). It's not overly verbose, but the lack of structure reduces readability and could be improved for better agent comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and a simple input schema with one parameter, the description provides basic completeness by listing return fields. However, it doesn't cover behavioral aspects like error handling or authentication needs, and without an output schema, it doesn't detail the structure of returned lists (e.g., Placeholder fields). For a read tool with minimal structured data, this is adequate but has clear gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'template_id' documented as a 'GUID of the template'. The description doesn't add any additional meaning about the parameter beyond what the schema provides—it doesn't explain format examples, validation rules, or where to obtain the GUID. Given the high schema coverage, the baseline score of 3 is appropriate, as the description doesn't compensate with extra insights.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to respond with template details including specific fields like template_id, title, labels, created_at, and lists of Placeholder and Signer fields, plus document_elements content. It specifies the verb 'responds with' and resource 'template details', making the purpose clear. However, it doesn't explicitly distinguish this from sibling tools like 'list_templates' or 'query_contract', which would require more differentiation for a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when this tool is appropriate compared to 'list_templates' (which likely lists multiple templates) or 'query_contract' (which might query contracts instead of templates). There's no indication of prerequisites, context, or exclusions, leaving the agent with insufficient usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_template_collaboratorC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'removes' implies a destructive mutation, it doesn't specify whether this requires admin permissions, if the action is reversible, what happens on success/failure, or any rate limits. For a mutation tool with zero annotation coverage, this leaves critical behavioral traits undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no wasted words, making it highly concise and front-loaded. Every word ('Removes the template collaborator') contributes essential information without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a destructive mutation tool with no annotations and no output schema, the description is incomplete. It lacks details on permissions, side effects, error handling, or return values, which are crucial for safe and effective use. The high schema coverage helps with parameters, but overall context is insufficient for a mutation operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters clearly documented as GUIDs for the template and collaborator. The description adds no additional semantic context beyond what the schema provides, such as format examples or relationship details. With high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('removes') and the resource ('template collaborator'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'delete_template' or 'withdraw_contract', which also involve removal operations, so it doesn't fully distinguish itself from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing to list collaborators first), exclusions, or comparisons to siblings like 'delete_template' (which removes the entire template) or 'list_template_collaborators' (which might be used to identify collaborators to remove).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_templateC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is an update operation, implying mutation, but doesn't describe permissions required, whether changes are reversible, how partial updates are handled, or what the response looks like. For a mutation tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose with zero wasted words. It directly communicates what the tool does without unnecessary elaboration or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficiently complete. It doesn't address behavioral aspects like error conditions, response format, or side effects. The complexity of the document_elements parameter (with multiple nested object types) suggests the tool has significant behavioral nuance that isn't captured in the minimal description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description mentions the same three parameters (title, labels, content) but adds no additional semantic context beyond what's in the schema. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Updates') and the resource ('contract template'), specifying what can be updated ('title, labels or the content'). It distinguishes from siblings like create_template (creation) and delete_template (deletion), but doesn't explicitly differentiate from other update operations that might exist.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing template ID), constraints (e.g., what happens if only partial fields are provided), or relationships with sibling tools like query_template (to check current state) or create_template (for initial creation).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• withdraw_contractC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states the action without disclosing behavioral traits. It doesn't clarify if withdrawal is reversible, requires specific permissions, affects contract state, or has side effects, which is inadequate for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words, making it highly concise and front-loaded. It directly states the tool's purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficient. It lacks details on behavior, outcomes, error conditions, or how it differs from siblings, leaving significant gaps in understanding for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'contract_id' parameter as a GUID. The description adds no additional meaning beyond what the schema provides, such as format examples or context, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('withdraws') and resource ('a sent contract'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'delete_contract' or explain what 'withdraw' means versus deletion, leaving room for improvement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'delete_contract' or 'query_contract', nor does it mention prerequisites like contract status or user permissions. The description lacks context for effective tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_templateB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool creates something, implying a write operation, but doesn't mention permissions required, rate limits, whether the operation is idempotent, or what happens on failure. It hints at template inheritance ('labels are inherited by contracts') but lacks details on response format or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary details. It avoids redundancy and wastes no words, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, nested document_elements structure) and lack of annotations or output schema, the description is minimally adequate. It states what the tool does but omits critical context like behavioral traits, error handling, and response format. For a creation tool with intricate input, more guidance would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no additional parameter semantics beyond what's in the schema—it doesn't explain parameter interactions, default behaviors, or examples. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Creates a reusable contract template for contracts to be based on.' It specifies the verb ('creates') and resource ('reusable contract template'), and distinguishes it from sibling tools like 'create_contract' by focusing on templates rather than contracts. However, it doesn't explicitly differentiate from 'update_template' beyond the creation aspect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., permissions needed), when to choose this over 'update_template' or 'create_contract', or any constraints on usage. The agent must infer usage from the purpose alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_recent_contractsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but only states it returns details of the latest 100 contracts. It doesn't disclose behavioral traits like pagination, rate limits, authentication needs, or what 'details' include, leaving significant gaps for an agent to understand the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste, clearly front-loading the purpose. It's appropriately sized for a tool with no parameters and straightforward functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It doesn't explain what 'details' include in the return values, and with siblings like 'query_contract' available, more context on usage and output is needed for an agent to effectively use this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description adds context by specifying 'latest 100 contracts', which provides semantic meaning beyond the empty schema, earning a baseline score above 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Returns') and resource ('contracts'), specifying it provides details of the latest 100 contracts. It distinguishes from siblings like 'query_contract' by implying a bulk retrieval without filtering parameters, though it doesn't explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving recent contracts without filtering, suggesting it's for bulk overviews rather than specific queries. However, it doesn't explicitly state when to use this versus siblings like 'query_contract' or 'list_templates', leaving some ambiguity.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_contractA
  3.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively communicates key behavioral traits: it's a read operation (implied by 'responds with'), discloses that 'signer events are included only for recent contracts', and explicitly mentions 'rate limiting'. However, it doesn't cover other potential behaviors like error conditions, authentication requirements, or pagination.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured as a single sentence that lists all return fields and includes important behavioral notes. It's appropriately sized for the tool's complexity, though it could be slightly more front-loaded by stating the core purpose more clearly before listing fields.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read operation with no annotations and no output schema, the description provides reasonable coverage of what the tool returns and some behavioral constraints. However, it doesn't explain the return format structure, error handling, or provide complete context about the 'recent contracts' limitation. Given the tool's moderate complexity, the description is adequate but has clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100% with the single parameter 'contract_id' well-documented in the schema itself. The description doesn't add any parameter-specific information beyond what's already in the schema description. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Responds with the contract details' followed by a specific list of fields returned. It uses the verb 'responds with' and identifies the resource as 'contract details' with specific attributes. However, it doesn't explicitly differentiate from sibling tools like 'list_recent_contracts' or 'query_template' beyond the scope of a single contract query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context through the mention of 'recent contracts' and 'rate limiting', but doesn't provide explicit guidance on when to use this tool versus alternatives like 'list_recent_contracts' or 'query_template'. The input schema description mentions 'draft contracts can't be queried, only sent contracts', which provides some usage constraint, but this isn't in the main description text.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_contractA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds important context about the precondition (test/draft contracts only), which is valuable beyond just stating the action. However, it doesn't cover other behavioral aspects like permissions needed, whether deletion is reversible, error handling, or what happens to associated data.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just two sentences that directly convey the action and key constraint. Every word earns its place, and the critical information is front-loaded. There's no wasted language or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive operation with no annotations and no output schema, the description provides the essential precondition but lacks completeness. It doesn't explain what 'deletes' entails (e.g., permanent removal?), error responses, or return values. Given the complexity of deletion and absence of structured data, more behavioral context would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'contract_id' fully documented in the schema as a 'GUID of the contract to be deleted.' The description doesn't add any additional parameter information beyond what the schema provides, so it meets the baseline for high schema coverage without adding extra value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Deletes') and resource ('a contract'), making the purpose immediately understandable. It distinguishes from siblings like 'withdraw_contract' by specifying deletion rather than withdrawal. However, it doesn't explicitly differentiate from 'delete_template' in terms of resource type beyond the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool: only for test or draft contracts. This implicitly suggests alternatives like 'withdraw_contract' for non-deletable contracts. However, it doesn't explicitly name alternatives or state when NOT to use it beyond the condition mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.