Server Quality Checklist

Tool Scores

• get_case_activityA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnly, idempotent, non-destructive. Description adds detail: returns envelope with items and currentEngagementPhase, describes items as chronological events, and notes null phase when no active engagement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences: first defines purpose and content, second details return structure. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Explains response structure in absence of output schema. Lacks pagination or limit info but covers key items and phase field.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter (caseId) with full schema description (format uuid, explanation). Description does not add extra meaning beyond schema; baseline 3 for 100% coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool fetches a chronological timeline with specific event types (status changes, partner actions, communications, payments), distinguishing it from siblings like get_case_payments or get_case_messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies use for full timeline but offers no explicit guidance on when to choose this over other case-related tools (e.g., get_case, get_case_payments) or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_case_contract_statusA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, destructiveHint, idempotentHint. The description adds that signing URLs are returned for outstanding documents, providing extra behavioral context beyond annotations. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with key action, no redundant words. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only tool with one parameter, the description covers the main purpose and hints at return value (signing URLs). No output schema exists, so the description compensates adequately. Could mention that it returns a list of contracts with status, but not required.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameter semantics are fully defined by the schema. The description does not add any additional meaning or usage details for the caseId parameter beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('check') and resources ('contracts') with concrete examples (debt collection agreement, power of attorney) and mentions output (signing URLs). It clearly distinguishes from sibling tools like get_case or get_case_payments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking contract signing status but does not explicitly state when to use this tool over alternatives, nor provides exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_case_paymentsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds value by specifying the scope ('money recovered so far'), implying a cumulative historical view. It does not contradict annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the essential purpose without any unnecessary words. Every word earns its place, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required parameter, no output schema, clear purpose), the description is mostly complete. It explains the action and the input, though the return format (e.g., list of payment objects) is not described, which is acceptable without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides 100% description coverage for the only parameter (caseId). The tool description does not add any additional semantic context beyond what the schema offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List'), identifies the resource ('every payment recorded on a case'), and clarifies the scope ('money recovered so far'). It clearly distinguishes this tool from siblings like get_case or get_case_activity, which address different aspects of a case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly states the tool's purpose but does not provide explicit guidance on when to use it versus alternative sibling tools (e.g., get_case_activity might also list payments). No 'when not to use' or comparison is offered.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_case_messagesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, destructiveHint. The description adds specific behavioral context: the tool reads a chat conversation, lists message fields (senderName, role, sentAt, message), and references a glossary for role meanings, going beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that are well-structured and front-loaded with the tool's purpose. Every sentence adds value, no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one required param, no output schema, no nested objects), the description sufficiently explains the return fields and points to additional resources for role interpretation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a single parameter caseId described as 'Debitura case ID (GUID)'. The description does not add further meaning about this parameter beyond what the schema already provides, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Read' and resource 'chat conversation on a case', clearly distinguishing it from sibling tools like get_case or send_case_message.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does but does not explicitly state when to use it versus alternatives, nor does it mention exclusions or prerequisites. Usage is implied but not guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_caseA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, establishing a safe read operation. The description adds that the tool returns 'full detail' and specifies lookup methods, but these contextual additions are moderate. The lack of output schema leaves uncertainty about return format, though the description is consistent with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences front-load the purpose and then provide usage instructions with zero redundancy. Every word earns its place, making it highly efficient for an AI agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a simple retrieval tool with good annotations. It covers lookup methods and the constraint. However, it lacks details on error handling (e.g., what if no case matches) and does not enumerate return fields (no output schema). Given the tool's simplicity and annotation richness, the gaps are minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with descriptions for each parameter. The description adds semantic value by grouping the three parameters as three lookup options and mandating that exactly one must be provided, which clarifies the mutual exclusivity constraint beyond schema details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Fetch one collection case in full detail', specifying the verb (fetch), resource (collection case), and scope (one case, full detail). It distinguishes itself from sibling tools like 'list_cases' (multiple cases) and 'get_case_payments' (sub-aspect) by emphasizing a single case retrieval with complete information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states that the case can be looked up by three identifiers and that exactly one must be provided. This gives clear usage guidance. However, it does not explicitly indicate when to use alternatives (e.g., 'get_case_contract_status' for status only), missing a direct comparison to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_casesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint, idempotentHint, and destructiveHint, which the description does not contradict. The description adds value by noting that results include test cases marked with `isTestCase`, and it lists lifecycle values and sortable fields, providing behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections and front-loading of the main purpose. While it is somewhat lengthy, every sentence adds value and it avoids redundancy with schema annotations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description thoroughly explains the return format (compact case summaries including specific fields), pagination, filtering, sorting, and the special case of test cases. It is fully sufficient for an agent to understand what the tool offers.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description largely reiterates information already present in the schema descriptions for parameters (e.g., sort format, enum values). It does not add significant new meaning, earning a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists debt collection cases with pagination, status filtering, and sorting, and distinguishes from `get_case` for full detail. The verb 'list' and specific resource 'cases' make the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly directs users to `get_case` for full detail on a specific case, providing a clear alternative. It does not elaborate on when not to use this tool versus other siblings, but the context of listing versus detail retrieval is sufficiently clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_team_membersA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, which describe the tool's safety. The description adds the context of 'creditor's Debitura account' but does not disclose additional behavioral traits beyond what annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the core function, and the second provides usage guidance. No redundant information; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description explains the tool's role in a workflow (resolving sender/owner) and includes the account context. It does not explain return structure, but that is acceptable without an output schema. Slightly above baseline due to practical context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the input schema already documents both parameters (page, pageSize) with descriptions. The description adds no extra meaning about parameters, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action (list team members) and the specific resource (creditor's Debitura account). It distinguishes from sibling tools by linking to specific use cases like resolving a sender for send_case_message or a case owner for create_case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises when to use the tool: 'Use this to resolve a valid sender (userId or email) before calling send_case_message, or a case owner for create_case.' This provides clear context and prevents misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• upload_case_fileA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations mark destructiveHint=true, and the description adds concrete behavioral details: max 25 MB, allowed extensions, and base64 encoding requirement. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: purpose, constraints, encoding requirement. Front-loaded with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers input constraints well but does not describe return value or success/error behavior. Given no output schema, this is a minor gap; overall sufficient for an upload tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all 6 parameters. The description adds value by specifying size and extension limits not present in the schema, and clarifying the base64 encoding requirement.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Attach' and resource 'document to a case', with specific examples of document types (invoice copy, contract, etc.). This clearly distinguishes it from sibling tools like list_case_files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for uploading files to cases with size/extension constraints, but does not explicitly state when to use it versus alternatives (e.g., no mention of when not to use it or comparison with send_case_message for document attachment).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_case_filesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint and idempotentHint. The description adds useful context about the time-limited SAS URL and downloadUrlExpiresAt, aiding the agent in managing cached URLs. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no fluff. The first sentence describes what the tool returns, the second provides critical behavioral context, and the third points to the sibling tool. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one parameter and no output schema, the description fully covers the return fields, the time-limited behavior, and alternatives. No gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for the single parameter caseId. The description adds no additional meaning beyond what the schema provides, meeting the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list) and resource (documents attached to a case), lists the returned fields, and distinguishes from sibling upload_case_file. The purpose is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Directs users to use upload_case_file for attaching documents, providing a clear alternative. However, it does not explicitly specify when not to use this tool beyond not uploading.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_case_tasksA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. Description adds useful context: the tool returns open tasks (default status), and clarifies that account-level tasks are excluded. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short paragraphs, each serving a distinct purpose: primary function, important exclusion, and cross-reference. No unnecessary text, highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (3 params, all described, strong annotations), the description is fully complete. It covers purpose, usage, exclusions, and references sibling, with no gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters meaningfully. The description mentions 'open tasks' implying the default status but adds minimal extra value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists open tasks for a specific case. Explicitly distinguishes from sibling list_tasks by noting it is scoped to a single case, providing specific verb+resource+scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use ('when you're already working a specific case') and when not to (account-level tasks not included, use list_tasks). Also points to list_tasks for more details on the task model.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_account_summaryA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already provide readOnlyHint, idempotentHint, destructiveHint. The description adds behavioral context beyond annotations, such as that counts include test cases and how to access the isTestCase flag via list_cases. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no unnecessary words. It efficiently conveys purpose, usage, and a behavioral note, earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, no output schema, and the tool's simplicity, the description fully covers what the tool does, what it returns, and a caveat about test cases. It is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so schema coverage is 100%. The description naturally doesn't need to add parameter info; baseline for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool returns a count of cases per lifecycle stage for the creditor's account, using specific verbs and resources. It distinguishes itself from sibling tools like list_cases by noting it is for a quick portfolio overview without listing all cases.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use the tool ('quick portfolio overview without listing all cases') and provides an alternative (list_cases for seeing the isTestCase flag). This gives clear context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tasksA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond the annotations (readOnlyHint=true, idempotentHint=true, destructiveHint=false) by disclosing that tasks auto-resolve, that it is a live work queue (not a log), and that tasks may disappear between calls. It also explains the 'solutionUrl' and 'action' field behavior, providing valuable behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise yet comprehensive, with each sentence serving a distinct purpose: stating the tool's function, guiding tool selection, explaining auto-resolution, describing output fields, and listing types. No unnecessary words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the high schema coverage, clear annotations, and the description's thorough coverage of usage, behavior, and output semantics (solutionUrl, action, task types), the tool is fully specified for an AI agent to select and invoke correctly. The lack of an output schema is compensated by the description's details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the description does not need to add parameter details. The description focuses on usage and behavior but does not add any new information about the parameters beyond what the schema already provides. Therefore, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list' and resource 'tasks' with the scope 'across your whole account'. It distinguishes from the sibling tool 'get_case_tasks' by specifying to use that for scoping to one case, making the purpose highly specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly provides when to use this tool vs. the alternative 'get_case_tasks', and gives detailed guidance on how to handle tasks based on the presence of an 'action' field, including when to call 'send_case_message' instead of using the 'solutionUrl'. This covers both when-to-use and when-not-to-use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• pingA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds value by stating the tool shows the creditor account, which is a behavioral detail not in annotations. However, it doesn't elaborate on the response format or potential errors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose, and contains no extraneous information. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple connectivity test tool with no parameters and no output schema, the description is complete. It explains what it does, when to use it, and what information it reveals (creditor account).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so baseline is 4. The description does not need to explain parameters, and it correctly omits any parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Verify the connection to Debitura and show which creditor account the API key belongs to.' This distinguishes it from sibling tools (e.g., get_case, list_cases) which focus on specific case operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends calling this tool first to confirm integration setup: 'Call this first to confirm the integration is set up correctly.' This provides clear guidance on when to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_caseA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnly=true, destructive=false, idempotent=true. The description adds 'Nothing is persisted,' which aligns and further clarifies that no side effects occur. It also details what the tool returns, adding valuable behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose and key behavior, and contains no extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description adequately explains what the tool returns (eligibility, partner, pricing, contracts). With full parameter documentation in the schema and clear behavioral notes, the description is complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description does not add new meaning beyond what is in the schema, meeting the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a dry-run for a collection case before creation, returning eligibility, partner, pricing, and contracts. It distinguishes itself from the sibling create_case by explicitly recommending to call it before.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'ALWAYS call this before create_case and show the user the pricing and requirements,' providing clear when-to-use guidance and what to do with the output.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_caseA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate destructiveHint=true, but the description adds crucial context: it's a legal/financial action with contractual fees, idempotent behavior, and details on 422 responses (business rejection with signing URLs). No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-organized with clear sections: action description, required workflow (numbered steps), idempotency note, error handling. Every sentence adds meaningful information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description covers behavior (idempotent, error details) and required context (preview_case, user confirmation). Could mention what a successful response contains (e.g., case ID), but overall sufficient for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3; description adds value by referencing sibling tools for parameter population (list_team_members for assignedUserEmail, recommend creditorReference) and emphasizing the required preview_case workflow, though it doesn't detail each parameter beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Submit a debt collection case to Debitura' with a specific verb and resource, and distinguishes from sibling tools like preview_case by outlining the required workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides a mandatory 3-step workflow (preview_case, user confirmation, then this tool) and warns never to call without explicit user confirmation, effectively guiding when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_case_messageA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate destructiveHint=true, and description adds details: partner is notified by email, message attributed to team member. No contradiction. Provides valuable behavioral context beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no fluff. Front-loaded with action and followed by important restrictions. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given full schema coverage and annotations, the description adds necessary behavioral details (notification, attribution, sender requirement). Complete for an agent to correctly invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds meaning beyond schema: explains that a sender is required (though schema lists as optional), and tells where to obtain sender info (list_team_members). Schema coverage is 100%, but description enhances usability.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Send a chat message') and the resource ('on a case to the collection partner'). It is distinct from sibling tools, which are primarily read-only or other operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context on when to use (communicate with partner), prerequisite (sender from list_team_members), and how to handle ambiguity (ask user). Lacks explicit exclusion but is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.