smokeball-mcp

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.1.0
Disambiguation4/5
Most tools target distinct entities and actions, but the split between patch/update for matters and leads could confuse an agent. Also, similar verbs like 'remove' vs 'delete' are used interchangeably, though descriptions clarify differences.
Naming Consistency4/5
Tools largely follow a verb_noun pattern (create_, get_, list_, update_, delete_), but there are inconsistencies: 'remove_' instead of 'delete_' for tags and relationships, and occasional non-parallel verbs like 'archive_matter' vs 'unarchive_matter'.
Tool Count2/5
With 189 tools, the server is excessively large for a typical MCP server. While the domain (legal practice management) is broad, many operations could be consolidated or parameterized. This number overwhelms agents and reduces usability.
Completeness3/5
The tool surface covers CRUD for most entities (contacts, matters, tasks, fees, etc.) and includes many auxiliary features (tags, roles, layouts, webhooks). However, there are notable gaps: no search tools for matters or contacts (only list with pagination), and batch operations are limited.
Average 2.9/5 across 189 of 189 tools scored. Lowest: 1.5/5.
See the Tool Scores section below for per-tool breakdowns.
- 1 of 2 community issues answered or closed in the last 6 months
- 47 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description offers no behavioral details like side effects, reversibility, permissions, or rate limits. Full burden on description, which is absent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise but under-specified. Efficiency is undermined by lack of necessary information for tool selection.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, description provides no context about effects, return values, or success criteria. Incomplete for agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and description does not mention parameters. No explanation of matter_id or reason fields beyond schema structure.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose2/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description restates the tool name as 'Archive a matter.' without elaboration. It does not clarify what 'archive' means or how it differs from siblings like delete_matter or close_matter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs. alternatives such as unarchive_matter or delete_matter. No prerequisites or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must carry the full burden. It only implies a mutation ('Update') and gives a string format hint for completed_str. It does not disclose whether updates are incremental, overwriting, or trigger side effects (e.g., audit logs). Minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise (2 sentences) but the first sentence is a tautology ('Update a task'). The second sentence is useful but too sparse. Could be restructured to list all updatable fields without adding much length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 5-parameter tool with no schema descriptions and no annotations, the description fails to cover the full scope of updates. Although an output schema exists (not shown), the description does not mention what the tool returns or any additional behavioral context. Poor completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It explains the format for completed_str but ignores name, notes, due_date, and task_id. Adds value for only one of five parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose2/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Update a task' which is a tautology with the name, and then focuses only on the completed_str parameter. It fails to convey that the tool can update multiple fields (name, notes, due_date). Given sibling tools like update_subtask, update_activity_code, etc., it does not distinguish itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., create_task, patch versions). No context on prerequisites, idempotency, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only says 'Create' without specifying side effects like whether duplicate relations are allowed, if it overwrites existing relations, or required permissions. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence and concise, but it is too brief, omitting important details. It could be expanded with a second sentence without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (not shown), the description does not mention what is returned upon creation (e.g., the relation ID). For a create tool with no annotations, this is inadequate. It lacks context about behavior, return values, and usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Description does not explain the parameters. While 'contact_id' and 'related_contact_id' are somewhat self-explanatory, 'relation_type' with a default empty string is not clarified. Given 0% schema coverage, the description should add value but fails to do so.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Create a relationship between two contacts' which clearly identifies the verb (create) and resource (relationship between contacts). It is distinct from sibling tools like 'update_contact_relation' and 'delete_contact_relation', though it could be more specific about what constitutes a relationship.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as updating or deleting relations. There is no mention of prerequisites (e.g., contacts must exist) or when to use this versus other relationship tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description must disclose behavioral traits. It only says 'Delete a contact by ID.' but omits whether deletion is permanent, reversible, cascading, or requires permissions. No mention of side effects or limitations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Very short but lacks necessary detail. Under-specification, not optimal conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a delete mutation with existing output schema, description fails to explain return value, success/error conditions, and impact. Extremely incomplete given complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. Description adds no explanation for 'contact_id' beyond 'by ID'. Does not clarify how to obtain the ID or any constraints on the value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
States verb 'Delete', resource 'contact', and method 'by ID'. Clearly distinguishes from sibling delete operations on different resources like delete_contact_relation, delete_event, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., archive, deactivate). No prerequisites or conditions for deletion mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It only states 'Delete a subtask,' implying a destructive action, but does not explain permanence, permissions needed, cascading effects, or success/failure behavior. This is insufficient for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (3 words), but it sacrifices necessary context. It is under-informative for a tool that involves deletion of a subtask with two required parameters. Conciseness should not come at the expense of clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the presence of an output schema (not shown), the description fails to provide essential context for a mutation tool. It does not describe return values, success criteria, error scenarios, or any side effects. The description is completely inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain what task_id or subtask_id represent or how they relate. The description adds no meaning beyond the parameter names, which are generic. This is a critical gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a subtask' clearly states the action (delete) and the resource (subtask). It identifies the specific resource but does not differentiate from other delete tools like delete_task or update_subtask. The purpose is clear but lacks sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as delete_task or update_subtask. There is no mention of prerequisites, conditions, or when not to use it. The description is purely functional.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. 'Update a staff member's details' implies mutation but omits important details like partial vs full update, required permissions, or side effects. Minimal and insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but overly brief. It lacks structure and fails to provide organized information, making it less useful than a well-structured paragraph.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 5 parameters and an output schema, the description is severely incomplete. It does not explain return values, required fields, or how to effectively use the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description adds no meaning to the parameters. It does not explain what 'role', 'email', 'first_name', etc. represent or any constraints beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (update) and resource (staff member's details). However, it does not differentiate from sibling tools like update_contact or update_firm, though the tool name itself provides some distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., update_contact), nor any prerequisites or conditions. The description is silent on usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It does not disclose any behavioral traits such as idempotency, error handling, or side effects. For a mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, which is concise but at the expense of necessary details. It is not front-loaded with critical information beyond the action. It earns its place but barely.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (4 params, no annotations, no param descriptions), the description is severely incomplete. It does not explain what a contact relationship is, what relation_type values are, or the relationship between role and matter. The presence of an output schema does not excuse the lack of context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% with 4 parameters, and the description adds no information about their meaning or usage. The parameter 'relation_type' has a default but is not explained. The description fails to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Add', the resource 'a contact relationship', and the context 'to a role on a matter'. It distinguishes from siblings like 'add_contact_tags' and 'add_role_to_matter' by specifying the relationship addition to a role. However, it does not clarify what a 'contact relationship' is in this domain.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'remove_relationship_from_role' or 'update_relationship'. There is no mention of prerequisites or context like requiring existing role or contact.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, and the description does not disclose any behavioral traits such as whether the tool checks for duplicate codes, what permissions are needed, or if it returns the created object.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence) with no unnecessary words, but it sacrifices completeness for brevity; it is structurally minimal but efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description lacks critical context about usage, constraints, and return behavior, making it insufficient for an agent to use the tool reliably.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain any of the three parameters (code, rate, description), leaving the agent with no additional meaning beyond parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and the resource ('a billing activity code'), which is specific and distinct from other create tools for different entities like contacts or matters.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., update_activity_code, list_activity_codes) or any prerequisites for creation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions 'register' implying a creation side effect, but fails to disclose permissions required, idempotency, or consequences of duplicate names. This is insufficient for safe autonomous invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, but it omits essential details. While not verbose, it sacrifices informativeness for brevity. It is adequately structured but under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (not shown), the description offers no information about return values or side effects. Combined with missing annotations and parameter semantics, the tool is insufficiently described for reliable use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, meaning the description adds no information about the three parameters (url, name, description). With no parameter descriptions in the schema either, the agent has no semantic guidance beyond parameter names and types. This is a critical gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Register a new plugin integration' clearly states the action (register) and the resource (plugin integration). It is specific enough to differentiate from other create tools for different entities, though it does not highlight unique aspects of plugin creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like subscribe_to_plugin or update_plugin. The description lacks context about prerequisites or scenarios where this tool is appropriate, leaving the agent without decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, and the description provides no behavioral details beyond the fact that it creates. Missing info on permissions, side effects, or return value structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely short but under-specified; lacks important detail. This is not concise but rather incomplete.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not mention what the tool returns or any constraints. For a creation tool with required parameters, crucial context is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the tool description adds no parameter explanations. The description does not clarify what 'first_name', 'last_name', 'email', or 'role' mean in context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
States verb and resource ('Create a new staff member'), but does not differentiate from sibling tools like create_user or create_contact. Lacks specificity about what constitutes a 'staff member' in this context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., create_user, create_contact). No prerequisites or contextual hints provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description implies user creation via invitation but does not disclose consequences (e.g., duplicate email handling, permission requirements, email sending behavior).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, appropriately concise but lacks essential details. Every word is earned, but missing value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 parameters and an output schema (not provided), the description is too minimal. No mention of return values, error states, or relation to invitation flow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. Description does not elaborate on any of the three parameters (email, first_name, last_name), relying solely on schema property names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Create a new user (invite to the firm)', specifying action and resource. It distinguishes from sibling tools like create_contact or create_staff_member, though not explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., create_staff_member, resend_user_invitation). Does not mention prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It does not disclose that the tool is read-only, uses pagination via limit/offset, or what the response contains. The brief statement adds minimal behavioral context beyond the name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short (one sentence), which is concise, but it sacrifices important details. It is front-loaded with the core action but could be more informative without losing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and 0% parameter coverage, the description is insufficient. It does not explain what activity history includes, how pagination works, or any other contextual details, though the presence of an output schema reduces the need for return value documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides no explanation of the three parameters (matter_id, limit, offset). It fails to add meaning beyond the schema, leaving an agent unable to understand parameter roles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves 'folder activity history for a matter,' using a specific verb and resource. It distinguishes the action but does not differentiate from sibling tools like get_file_history, which also fetches history.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description does not mention prerequisites, context, or why one would choose get_folder_history over other history tools (e.g., get_file_history).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states 'Get tags on a matter,' implying a read operation, but fails to mention if the operation is safe, if it requires special permissions, if tags are returned in a particular format, or any side effects. The description is too minimal to provide adequate transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (5 words), which is efficient but comes at the cost of completeness. It conveys the core purpose without any fluff, but for a tool with a single required parameter and no annotations, it should include more context. It is not unnecessarily verbose, but it is arguably too terse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has one required parameter, no annotations, and an output schema (though details not shown), the description fails to mention what the output looks like (e.g., returns a list of tags, each with id and name). It also lacks error handling or special cases. For a simple getter, this is still incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage for parameters, meaning the schema itself provides no semantic context beyond the parameter name. The description adds nothing about the lone parameter 'matter_id', such as its format, source, or relationship to the matter. This leaves the agent to guess, which is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Get) and resource (tags on a matter), making the tool's purpose immediately understandable. However, it does not elaborate on what 'tags' entail or differentiate from very similar sibling tools like 'get_contact_tags', though the resource distinction is evident.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., when to use 'add_matter_tags' or 'remove_matter_tags'). There is no mention of prerequisites, context of use, or any conditions that would make this the appropriate choice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral disclosure. It does not mention side effects, permissions, pagination, or error conditions, which is insufficient for a retrieval tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at one sentence, but it lacks necessary detail. It is not overly verbose, but it does not earn its place because it omits important context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter, list output) and the presence of an output schema, the description is still incomplete. It fails to cover behavioral aspects like pagination or limits, which are likely relevant.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter 'task_id' with 0% schema description coverage, and the description does not elaborate on its meaning, format, or how to obtain it. This leaves the agent with no additional information beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'subtasks', specifying the context 'for a task'. It is unambiguous, but it could better distinguish from the sibling tool 'get_subtask' by indicating it returns a list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'get_subtask' or 'list_tasks'. There is no mention of prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations present. The description does not disclose behavioral traits like read-only nature, pagination behavior, or any potential side effects. It only states the action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, but it lacks structural elements like front-loading key information. It is adequate but not optimized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema and the tool's complexity, the description is incomplete. It does not mention pagination, filtering, or what activity codes represent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the 'limit' and 'offset' parameters or their roles in pagination. The defaults are not mentioned.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (list) and resource (billing activity codes configured for this firm). It distinguishes from siblings like get_activity_code (single) and mutations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives such as get_activity_code for a single code. The description does not provide context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full responsibility. It fails to disclose read-only nature, authentication requirements, pagination behavior, error handling, or performance characteristics, which are critical for an agent to invoke correctly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
At only four words, the description is too terse. While it is front-loaded, it sacrifices valuable information and fails to earn its brevity with substantive detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Though an output schema exists (reducing the need to explain return values), the tool lacks explanation of pagination, ordering, or what constitutes 'files attached to a matter'. Given the 0% schema coverage and missing parameter details, the description is incomplete for reliable agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description adds no meaning to the three parameters ('limit', 'offset', 'matter_id'). The agent receives no explanation of their purpose, defaults, or constraints beyond the schema's type declarations.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and resource 'files attached to a matter', making the tool's purpose apparent. It faintly distinguishes from siblings like 'add_file_to_matter' and 'get_file', but does not elaborate on scope or behavior beyond the basic function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With many sibling tools (e.g., 'get_file', 'add_file_to_matter'), explicit usage context is absent, leaving the agent to infer appropriateness.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must disclose behavioral traits. It only states it lists memos but omits that it is a read operation, how pagination works (limit/offset defaults), whether sorting is applied, or any rate limits or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but lacks necessary detail. It front-loads the verb and object, but at the cost of completeness. While efficient, it could include more without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description need not explain return values, but it does not provide context for tool usage: no mention of required matter context, default pagination behavior, error conditions (e.g., invalid matter_id), or that only memos on that matter are listed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description adds no parameter details. It does not explain the role of 'matter_id' (required), nor the meaning of 'limit' and 'offset' (pagination). The description provides zero value beyond the schema types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'memos on a matter', with clarification that memos are notes. It distinguishes from siblings like 'get_memo' and 'create_memo' by indicating it lists all memos for a matter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For instance, there is no mention that 'get_memo' should be used for a single memo or that 'create_memo' is for adding memos. The description lacks context for usage decisions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It merely states the action but does not disclose side effects, permissions required, whether the operation is reversible, or what happens if the firm is already subscribed. This is insufficient for safe tool invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, which is efficient, but it lacks structure. It does not break down information such as parameter details, usage notes, or output expectations, making it less helpful than it could be.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and related sibling tools, the description is very incomplete. It does not explain what subscribing means, what a plugin is in this context, or what the output represents. The description fails to provide sufficient context for an agent unfamiliar with the domain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter 'plugin_id' with 0% description coverage. The tool description does not mention this parameter at all, so it adds no meaning beyond the schema. The agent receives no help in understanding what 'plugin_id' represents or how to obtain it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Subscribe this firm to a plugin' clearly states the verb (subscribe) and resource (this firm to a plugin), making the tool's action understandable. However, it does not differentiate from sibling tools like 'unsubscribe_from_plugin' or 'create_plugin', which could lead to confusion if an agent must choose between them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as 'unsubscribe_from_plugin' or 'create_plugin'. There is no mention of prerequisites, context, or conditions under which subscription is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden but only states the action. It does not disclose behavioral traits like required permissions, reversibility, or side effects (e.g., removing access).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one sentence), but it sacrifices necessary detail. It is front-loaded but incomplete.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (1 required param), the description omits critical context such as which firm is affected, whether the action is reversible, and what the output represents. The existence of an output schema does not compensate for these gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description adds no information about the 'plugin_id' parameter, leaving the agent to infer meaning solely from the parameter name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Unsubscribe') and the resource ('plugin'), and distinguishes it from sibling tools like 'subscribe_to_plugin'. However, it does not specify which firm is being unsubscribed (e.g., the current firm).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as 'delete_plugin' or 'subscribe_to_plugin'. Lacks context on prerequisites or consequences.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It only says 'Update an activity code' without disclosing whether it performs partial or full replacement, authorization needs, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise but lacks necessary detail. It is not front-loaded with key info beyond the obvious.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and minimal description, it fails to provide complete context. Even with output schema present, the description omits important behavioral details like update semantics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain any parameters. The schema has defaults and one required field, but the description adds no meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and resource 'activity code', distinguishing it from create/delete activity code siblings. However, it doesn't differentiate from other update tools like update_task.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No mention of context, prerequisites, or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden but only states 'Update', implying mutation. It does not disclose whether fields are overwritten or merged, what permissions are needed, or what the output contains. The existence of an output schema is not addressed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, making it concise, but it is under-specified and lacks essential details. It does not earn its place as it provides minimal value beyond the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, return values are partially covered, but the description does not explain the update behavior or parameter usage. For an update tool with 5 parameters, the description is too incomplete for reliable agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no information about any of the 5 parameters. Schema description coverage is 0%, so the description should compensate but fails to explain contact_id, email, phone, first_name, or last_name. This leaves the agent without context for parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'contact's details', distinguishing it from related tools like create_contact, delete_contact, and get_contact. However, it does not specify which details can be updated, relying on the schema for that information.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as create_contact or update_contact_relation. It does not mention prerequisites like the required contact_id parameter or any conditions for updating.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and description merely says 'update' without disclosing behavioral traits like idempotency, permissions, or effects of missing fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise but lacks detail needed for effective tool invocation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters with no schema descriptions and no annotations, description fails to provide sufficient context about required fields or expected behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage, and description adds no meaning to parameters like 'name', 'start_date', 'end_date', 'location'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly specifies verb 'update' and resource 'calendar event', distinguishing from siblings like 'create_event' and 'delete_event'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool instead of alternatives like 'patch_event' or prerequisites. Only generic statement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It only states 'Update a plugin's details', but fails to disclose behavioral traits like idempotency, required permissions, or side effects. For a mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (5 words), which is concise. However, it is under-specified and could include a bit more information without becoming verbose. It is efficient but not optimally informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (3 parameters, no nested objects) and the presence of an output schema, the description is incomplete. It omits parameter semantics and usage context, leaving gaps that force the agent to rely solely on the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides no parameter details. It doesn't explain that plugin_id is required, or that url and name are optional with defaults. The agent receives no guidance on parameter usage beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Update' and resource 'plugin', clearly indicating the action. It distinguishes from siblings like create_plugin and delete_plugin. However, it doesn't specify which details can be updated, which the input schema reveals as url, name, and plugin_id.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No exclusions or context about prerequisites, such as requiring an existing plugin_id. The description is too brief to aid selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It only states it creates a memo but doesn't disclose behavioral traits like whether it overwrites existing memos, permissions needed, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, very concise, but at the cost of important missing information. It is front-loaded but too sparse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (not shown) and 3 parameters, the description is insufficient. It does not mention the output, required fields, or any constraints, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description adds no parameter details. The agent must rely solely on schema titles ('Content', 'Subject', 'Matter Id'). No value added beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and resource 'memo (note)' with context 'on a matter'. It distinguishes itself from sibling tools like get_memo, update_memo, delete_memo, and list_memos_on_matter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives or prerequisites (e.g., matter must exist). The description lacks any context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only states the action with no disclosure of side effects, permissions, idempotency, or other behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise (two sentences) but under-specified; lacks structure and detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Incomplete for a creation tool with 3 parameters and no annotations; does not describe URL format, secret usage, or response behavior despite output schema existence.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%; description mentions event_type and its source but does not explain url or secret parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create a webhook subscription') and points to a sibling tool for valid event_type values. However, it does not differentiate from sibling tools like test_webhook_subscription or update_webhook_subscription.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites or exclusions provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It only states 'Delete,' implying destructiveness but no details on permissions, side effects, or reversibility. Minimal disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, very concise, but at the cost of omitting important details. It earns its place but is too brief.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists, explanation of return values is not required. However, the description does not cover success behavior, error conditions, or the destructive nature. Incomplete for a delete tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and description adds no meaning beyond field titles 'Event Id' and 'Reminder Id'. It does not explain the role of each parameter or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Delete an event reminder,' which clearly identifies the verb and resource. It distinguishes from siblings like create_event_reminder and update_event_reminder, but does not elaborate scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like delete_event or other delete tools. The description provides no context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must disclose behavioral traits. It only states the action (Get) without confirming read-only status, required permissions, or any side effects. The description adds little beyond the function name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. However, additional context could be added concisely, such as the requirement for both parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (so return values are covered), the description is too minimal for a tool with no annotations and zero parameter documentation. It does not explain the scope of 'associated' or clarify that the layout is within a matter.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description does not explain the parameters (matter_id, layout_id) or their roles. The agent has no additional meaning beyond the schema titles, which is insufficient for a 2-parameter tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'contacts associated with a layout instance', distinguishing it from sibling tools like get_contact or get_contact_tags. However, it lacks contextual specificity that would elevate it to a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as list_contacts or add_contact_to_layout. The agent must infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description lacks any behavioral details beyond the basic action. With no annotations, it should disclose what the tool returns, whether it has side effects, or any access requirements. It does not, leaving the agent uninformed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (4 words), which is efficient but at the cost of essential information. It is not structured to highlight key points, and the brevity undermines its overall usefulness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one required parameter, no annotations, but an output schema exists), the description is too minimal. It fails to explain what a lead is, what the output contains, or any additional context, making it incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should explain the parameter 'lead_id'. However, it merely says 'by ID', which adds no meaning beyond what the schema already implies. The parameter's purpose and format are not clarified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a lead by ID' clearly states the verb and resource, but it's very minimal and does not provide any additional context that distinguishes it from other similar getter tools (e.g., get_contact, get_matter). It is adequate but not exceptional.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not mention when to use this tool instead of alternatives like list_leads or patch_lead. There is no guidance on prerequisites or intended use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It does not disclose that the tool is read-only, what happens if matter_id is invalid, or any other behavioral traits like rate limits. The description only states the output type without additional context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single sentence. It is front-loaded with the key information, but it sacrifices necessary detail for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although an output schema exists, the description lacks usage guidance, behavioral notes, and parameter explanation. For a simple retrieval tool with one parameter, it covers the bare minimum but is incomplete for an agent to use confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description fails to provide any details about the 'matter_id' parameter beyond its name. No format, validation, or example is given, leaving the agent to rely solely on the schema's type declaration.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get' and the resource 'archive status and metadata for a matter'. It distinguishes itself from sibling tools like 'archive_matter' and 'patch_matter_archive' by implying a read-only operation, but does not explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'get_matter' for full matter details or 'archive_matter' to change status. The agent is left to infer context from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, and the description does not disclose any behavioral traits such as idempotency, error handling (e.g., if plugin not found), authentication requirements, or side effects. The description adds zero behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at one sentence. While there is no waste, it is overly minimal for a tool with no annotations and fails to earn its place by covering necessary details. A bit more length would improve utility.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required param, output schema exists), the description adequately states the basic purpose. However, it lacks completeness in providing usage context, parameter details, and behavioral transparency, which are important for agent guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one required parameter (plugin_id) with 0% description coverage. The description mentions 'by ID' but does not elaborate on the format, constraints, or source of the ID. It adds minimal value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a plugin by ID' clearly states the action (get) and resource (plugin) with the method (by ID). It is specific and distinguishes itself from siblings like list_plugins or get_plugin_subscription.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as list_plugins or get_plugin_subscription. There are no usage context, prerequisites, or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden of disclosing behavior. It only states 'Get', implying a read operation, but omits details like idempotency, error cases, data freshness, or what the response contains. The existence of an output schema partially compensates, but the description itself adds minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, short sentence with no unnecessary words. It is appropriately front-loaded but could be slightly expanded to include more helpful information without becoming overly verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with simple parameters and an output schema, the description is too minimal. It does not mention what the return value is (transaction details), handling of invalid IDs, or any prerequisites. The sibling context includes many related tools, but this description offers no integration or completion guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage for the two parameters ('account_id' and 'transaction_id'). The description does not explain their meaning, format, or how to obtain them. It adds no value beyond the schema, which is critical given the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a specific transaction from a bank account. It uses a specific verb ('Get') and resource ('transaction from a bank account'). The purpose is easily distinguishable from siblings like 'list_transactions' or 'create_transaction', though it does not explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., 'list_transactions'), no prerequisites, and no context about when it is appropriate. Users are left to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden for behavioral context. It does not disclose that the tool is safe (read-only), paginated (implied by limit/offset), or what the default listing includes (all tasks vs. user-specific). The output schema exists but is not described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences, but it lacks a clear structure such as front-loading the primary action or key constraints. It is underinformative for the number of parameters and sibling tools.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (3 parameters, output schema, many siblings), the description is incomplete. It does not explain the return format, pagination behavior, or how the tool fits among task-related siblings. The output schema is not leveraged in the description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only the matter_id parameter is mentioned in the description, leaving limit and offset unexplained despite their default values and role in pagination. Since schema description coverage is 0%, the description fails to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool lists tasks and can filter by matter_id, which differentiates it from other list tools for different resources. However, it does not specify what scope of tasks is returned when no filter is applied, leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus related tools like get_task, create_task, or other list operations. There is no mention of prerequisites, ordering, or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It does not disclose behavioral traits such as read-only nature, pagination behavior, rate limits, or potential errors. The minimal description leaves significant behavioral context missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, but it is too minimal. It lacks necessary detail about parameters and usage, making it under-specified rather than concise. Every sentence should earn its place, but this one does not provide enough information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (3 params, has output schema), the description should cover parameter semantics and behavioral expectations. It does not. Even with an output schema, the missing parameter descriptions and usage context make it incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not elaborate on any parameters. The meaning of limit, offset, and account_id is left entirely to the schema, which lacks descriptions. The description adds no value beyond the parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (list) and resource (transactions for a bank account). It distinguishes from sibling list tools by specifying the scope (for a bank account), which aligns with the required account_id parameter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like list_bank_accounts or other list tools. No conditions for exclusion or prerequisites are mentioned. The description provides no context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries all transparency burden. It indicates mutation via PATCH but does not explain side effects, idempotency, permissions, or error conditions, leaving significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short, but its brevity sacrifices clarity on parameter usage. It is concise but not well-structured for an agent, as it leaves ambiguity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 undocumented parameters, no annotation support, and similar sibling tools, the description is insufficient. It lacks parameter specifics, usage context, and does not leverage the output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description only adds 'Use true or false' without mapping to parameters. It does not clarify fee_id requirement or meaning of billable/billed, offering minimal help.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool toggles a fee's billable/billed state using PATCH, but does not differentiate from sibling update_fee which might also update these fields. The verb 'toggle' and mention of 'billable or billed' provide specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool instead of alternatives like update_fee or create_fee. No context about typical usage scenarios or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It only notes the HTTP method and provides a date format for one parameter. It does not explain side effects, permissions, or that only provided fields are updated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short but front-loaded with the purpose. However, the formatting of 'destruction_date: YYYY-MM-DD.' is somewhat awkward and not integrated into a clear sentence structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters (matter_id, destruction_date, retention_policy) and no schema descriptions, the description only addresses one parameter's format. It omits details about retention_policy, return values, and behavior when fields are omitted. This is incomplete for an update tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description adds meaning for destruction_date by stating the date format (YYYY-MM-DD), but does not clarify matter_id or retention_policy. This partial addition is insufficient to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates archive metadata fields on a matter using PATCH. It distinguishes from sibling tools like archive_matter and get_matter_archive by indicating a partial update operation. However, it only explicitly mentions destruction_date, not retention_policy, slightly reducing clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like archive_matter or patch_matter. It does not mention that it is for updating specific fields only, leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits like side effects, permission requirements, or state changes beyond the basic action. The description fails to compensate for missing annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, making it concise. However, it is overly terse and lacks necessary detail, sacrificing clarity for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown) and one parameter, the description is incomplete. It does not explain what happens after unarchiving, any constraints, or how the tool fits into the broader context of matter management.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'matter_id' has 0% schema description coverage, and the description adds no additional meaning or constraints. While 'matter_id' is self-explanatory, the description could clarify format or required privileges.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Unarchive a matter' uses a specific verb and resource, clearly indicating the inverse operation of 'archive_matter' among siblings. However, it lacks any additional context about what unarchiving entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. There is no mention of prerequisites, such as whether the matter must be archived first, or any conditions for invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only says 'Update', implying mutation, but provides no details about side effects, permissions, or error handling. With no annotations, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short and to the point, but it could be more informative without being verbose. It is not overly wordy, but lacks depth.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description fails to mention return values or behavior beyond the basic action. With three required parameters and no parameter descriptions, the tool is not fully explained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain any parameters. Although parameter names are somewhat self-explanatory, the meaning of relation_type and valid values are not clarified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update') and the resource ('contact relationship type'), distinguishing it from create/delete/get siblings. However, it could specify which fields are updated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like create_contact_relation or delete_contact_relation. The description lacks context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavioral traits. It only states the action (rename) but omits details such as whether the operation is reversible, requires special permissions, or affects folder contents. The presence of an output schema is noted but the description does not summarize or enhance understanding of the behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (three words) but at the expense of completeness. While it is front-loaded with the core action, it could include a brief structure like 'Renames a folder. Provide folder_id and new name.' This would maintain conciseness while adding value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is a mutation operation with an output schema (likely returning the updated folder), the description underperforms. Sibling tools include many update and patch operations, and the description does not help distinguish 'update_folder' from similar tools like 'patch_folder' or 'update_contact'. More context on what 'update' entails (only rename) is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should compensate. However, it adds no meaning beyond the parameter names: 'folder_id' and 'name'. Constraints like uniqueness, format, or length are absent, leaving the agent to infer from names only.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Rename a folder' clearly states the specific action (rename) and resource (folder), distinguishing it from the generic 'update' implied by the tool name. However, it does not explicitly differentiate from sibling tools like 'patch_folder' which may also rename folders, limiting distinctiveness.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'patch_folder' or 'create_folder'. There is no mention of preconditions (e.g., folder existence) or postconditions, leaving the agent without context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It only states 'update' without disclosing whether the operation is idempotent, what side effects occur, or any authentication requirements. Minimal transparency beyond the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with one sentence, but it sacrifices completeness for brevity. It is not front-loaded with critical information and could benefit from more structured detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of 3 parameters, no annotations, and an output schema, the description lacks contextual information about return values, error handling, or typical usage. It is incomplete for effective tool utilization.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%. The description mentions 'status' and 'notes' but does not clarify valid values, constraints, or formatting. It adds no meaning beyond the schema's type definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the verb 'Update' and the resource 'lead' with specific scoping to 'status or notes', making the purpose clear. However, it does not distinguish from the sibling tool 'patch_lead', which may also update a lead.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'patch_lead' or other update tools. The description lacks context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, and the description only says 'update' without disclosing any side effects, authorization requirements, or behavioral details. The output schema exists but is not reflected in the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with a single sentence, but it is too sparse to be helpful. It earns its place but lacks important detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of annotations and minimal description, the tool is not fully specified. While an output schema exists, the description does not cover behavioral aspects or parameter semantics, leaving gaps for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description adds no meaning to parameters. The schema lists role_id, matter_id, and contact_id with titles only; the description does not explain their purpose or usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'update' and resource 'role on a matter', which distinguishes it from siblings like add_role_to_matter or remove_role_from_matter. However, it lacks specificity on what aspects of the role can be updated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as add_role_to_matter or remove_role_from_matter. There is no mention of prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits. It states 'Create' implies mutation but does not mention permissions, consequences, idempotency, or error scenarios. The output schema exists but is not referenced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, using one sentence efficiently. It is front-loaded with the key verb and resource. However, it may be too minimal, sacrificing clarity for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 2 parameters, no annotations, and an output schema, the description is incomplete. It fails to explain return values, parameter details, or behavioral context. The agent would need additional documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should explain parameters. It does not mention 'name' or 'description' at all. The schema provides basic titles but the description adds no context about their meaning or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and resource 'authorization group', matching the tool name and differentiating from read/update/delete siblings. However, it does not elaborate on the specific kind of authorization group or distinguish from similar create tools like 'create_authorization_policy'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusion criteria. Context signals show many related create tools, but the description offers no comparative advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description does not disclose whether the request is asynchronous, the expected response, or any side effects (e.g., cost, rate limits). For a mutation tool creating a request, more detail is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (1 sentence) and avoids fluff, but it could be slightly expanded without losing conciseness to improve clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (write operation, 1 param, output schema exists), the description fails to provide adequate context about the request process, asynchronous behavior, or how the preview generation works.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'file_id' has no schema description (0% coverage), and the description adds no further meaning. It doesn't specify the format, source, or constraints of the file ID.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'request generation' and the resource 'preview (thumbnail/rendition) for a file'. It distinguishes from sibling tools like get_file_preview_info (which likely retrieves existing previews) and get_file_download_url.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as get_file_preview_info or get_file_preview_info_by_version. No mention of prerequisites or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior. It only states the creation action but does not explain side effects (e.g., fund deduction, approval workflow), mutability, or any rate limits or authorization requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (one sentence) but at the cost of omitting critical details. It is front-loaded with purpose but lacks supporting information, making it barely adequate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not mention return values or success indicators. For a creation tool, it lacks context on validation, error states, or the effect on the system's state (e.g., balance deduction).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning no parameter descriptions exist in the schema. The tool description does not elaborate on any parameters (e.g., account_id, matter_id, amount), so it fails to add meaning beyond the schema's property names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create a trust fund requisition') and provides a parenthetical clarification ('draw-down request'). It distinguishes itself from sibling tools like 'create_matter' or 'create_contact' by targeting a specific financial transaction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as other creation tools or transaction-related tools. It does not mention prerequisites (e.g., need a trust account, sufficient balance) or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description only indicates a destructive action without detailing consequences, permissions, or side effects, leaving the agent underinformed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise but lacks structure or additional context; it is neither verbose nor optimally informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of annotations and parameter coverage, the description fails to adequately inform about return values or error handling, despite an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no meaning beyond the schema; 'by ID' is already implicit from the parameter name 'code_id', and schema coverage is 0%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the resource ('activity code') with the method ('by ID'), distinguishing it from sibling tools like update_activity_code or get_activity_code.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., other delete tools) or prerequisites (e.g., needing a valid code_id).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It does not disclose behavioral traits like reversibility, side effects (e.g., cascading deletes), or required permissions. For a deletion tool, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence), which is concise but lacks sufficient detail. While brevity is positive, it sacrifices informative content for a deletion tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (not shown), the description does not mention return values or confirm success. For a simple one-parameter tool, more context about the operation's effect and result would be expected.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should compensate. It merely says 'by ID' without explaining what the ID represents or its format. The single parameter 'group_id' is not elaborated beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'authorization_group', specifying the method 'by ID'. It effectively distinguishes from sibling tools like create_authorization_group and update_authorization_group.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor are there any prerequisites or usage conditions mentioned. The description lacks context such as whether the group must be unused before deletion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It does not disclose whether deletion is permanent, requires permissions, cascades to related data, or is reversible. Only the basic action is stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at 5 words and front-loaded. However, it is under-specified; a few more words could improve clarity without losing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite low complexity (1 param, output schema exists), the description lacks completeness for a delete operation. It does not mention return values, side effects, or conditions for success/failure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one required parameter (file_id) with 0% coverage. The description only says 'by ID,' adding no meaning beyond the schema. It fails to explain the format or constraints of the ID.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a file by ID,' specifying the verb (delete), resource (file), and parameter (ID). It distinguishes from sibling tools like delete_contact or patch_file. However, it lacks nuance about what deletion entails (soft/hard delete).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as archive_matter, remove_file_from_matter, or patch_file for deactivation. The description does not mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
There are no annotations, so the description bears full responsibility for behavioral disclosure. While 'Delete' implies mutation, the description does not reveal whether the operation is irreversible, requires specific permissions, or has cascading effects. The output schema exists but is not referenced, so the agent cannot infer the response format or success/failure indicators. This is a significant gap for a deletion tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with one sentence and no redundant information. It front-loads the action and resource. However, for a delete tool, it could benefit from a brief behavioral note without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there are no annotations and only one parameter, the description should be more complete. It fails to mention that deletion is permanent, that the mapping must exist, or what the output schema contains. For a tool that can have significant side effects, this lack of context leaves the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description mentions 'by ID' but does not clarify what the mapping_id represents, how to obtain it, or its format. Since schema coverage is 0%, the description should compensate, but it merely repeats the parameter name. With only one parameter, it is slightly better than having multiple undocumented parameters, but still lacking substantive guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'a firm user mapping', with the criterion 'by ID'. This effectively distinguishes it from retrieval or update tools (e.g., get_firm_user_mapping, update_firm_user_mapping). However, it does not explain what a firm user mapping is, which slightly reduces clarity for an unfamiliar agent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like update_firm_user_mapping or get_firm_user_mapping. The agent is not told that this is for removing a mapping, while update is for modifying, and get is for retrieval. The description lacks any when-to-use or when-not-to-use context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits but only says 'Get file version history'. It does not mention pagination (though schema has limit/offset), read-only nature, error behavior, or whether it returns history for all files in the matter or a specific file. The parameter matter_id instead of file_id further confuses behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but too sparse. It lacks structure and essential details, making it minimally informative. It earns its place but could be improved by adding parameter context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists but does not compensate for missing context. The tool name suggests a file-specific operation but requires only matter_id, creating confusion. No explanation of what 'version history' entails (e.g., listing all versions of all files?). Pagination behavior is implied by schema parameters but not described.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It adds nothing about matter_id, limit, or offset. The agent has no clue what each parameter means beyond the schema's type/defaults. This is a critical gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get file version history for a matter' clearly states the action (Get), resource (file version history), and scope (for a matter). It differentiates from siblings like get_folder_history (folder vs file) and get_file (current state vs history).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., get_file for current version, get_file_download_url for downloading). No prerequisites or context provided. The agent must infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must convey behavioral traits. It only implies a read operation ('Get'), but does not discuss error handling, authentication needs, or the behavior when the item is not found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—a single sentence with no fluff. It is front-loaded and efficient, suitable for a simple retrieval tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the tool is simple and an output schema exists (so return values are covered), the description lacks parameter details and usage context. It is minimally complete but fails to add value beyond the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning to the parameters ('matter_id', 'item_id'). The tool name and parameter names are self-explanatory, but the description does not elaborate on their roles or expected formats.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get a specific item on a matter,' which is a verb-resource pair. It distinguishes itself from siblings like 'list_matter_items' (retrieves all items) and other 'get_*' tools, but does not explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., using 'list_matter_items' to fetch all items first). No prerequisites or context are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavioral traits. It states the tool gets a balance but omits details like whether it's read-only, required permissions, error conditions, or what 'protected' means. This is insufficient for an agent to understand side effects or constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, which is appropriate length, but it lacks essential detail. It is not front-loaded with critical information; it states the obvious without adding value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter) and the existence of an output schema, the description is still too minimal. It does not explain what constitutes 'protected balance', how it relates to other account operations, or what the output contains. The description leaves many gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no meaning beyond the schema. 'account_id' is not explained (e.g., how to obtain it, what format). Schema description coverage is 0%, and the description fails to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool's purpose: to retrieve the protected/reserved balance for a bank account. It uses a specific verb ('Get') and resource ('protected bank account balance'), distinguishing it from sibling tools like 'get_bank_account' (general account info) and 'protect_funds' (reserve action).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as 'get_bank_account' or 'get_bank_account_matter_balances'. It does not specify prerequisites, context, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description does not disclose behavioral traits such as required permissions, authentication needs, or what happens if ID is invalid. Minimal disclosure for a retrieval tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short, which is concise, but it lacks essential information that would justify its brevity. It is not well-structured for agent decision-making.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has one parameter, no annotations, but has an output schema, the description is insufficient. It does not cover context like return value structure or error scenarios; the output schema may partially compensate but the description leaves too much unknown.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must add meaning. It merely repeats the parameter name without explaining what a referral type ID is, how to obtain it, or its format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description 'Get a referral type by ID' clearly specifies the verb (get) and resource (referral type), and distinguishes it from sibling tools like list_referral_types which retrieves all referral types.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like list_referral_types or search tools. Does not provide context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the full burden of behavioral disclosure. It only states 'Get all contact relationships on a matter,' which is a read operation, but omits details about permissions, pagination, error handling, or the nature of the output. The existence of an output schema partially mitigates this, but the description adds minimal value beyond the tool name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. However, it is under-specified for the task, sacrificing completeness for brevity. A concise but informative description would be ideal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required param, output schema exists), the description could include more context such as the expected relationship types, any filtering, or return format. The output schema provides structure but the description alone is insufficient for an agent to fully understand the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The sole parameter matter_id has 0% schema description coverage, and the tool description provides no explanation of its meaning, format, or constraints. For a single-parameter tool, this is a critical gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'contact relationships', and the scope 'on a matter', distinguishing it from siblings like get_contact_relation (singular) or get_role_on_matter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as get_contact_relations or get_relationship_on_role. The context includes many related tools with overlapping purposes, but the description provides no differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description discloses no behavioral traits. It does not mention whether the operation is read-only, requires authentication, or returns specific fields. For a simple get operation, this is insufficient without annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at one sentence, but it is not front-loaded with key information. It is minimally adequate but lacks structure or emphasis.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values are not needed in the description. However, the description is very sparse given the tool's context among many sibling tools. It does not explain the relationship between task and subtask.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning beyond the parameter names (task_id, subtask_id). The description does not explain what a subtask is or how the IDs are used, failing to compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'specific subtask', distinguishing it from sibling tools like get_subtasks (plural) and create/update/delete subtask. It is unambiguous but lacks any additional detail about the scope or context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not indicate when to use this tool vs alternatives (e.g., get_subtasks for listing all subtasks). Usage must be inferred from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description must disclose behavioral traits. It only says 'list' and 'filter', omitting pagination behavior, default ordering, access rights, or whether it returns all expenses by default. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two succinct sentences front-load the purpose. No redundant information. However, it may be too brief, but that affects completeness more than conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite an output schema existing, the description lacks detail on pagination (limit/offset defaults and behavior), filtering semantics, and any operational context. Missing essential guidance for a listing tool with three parameters and no schema descriptions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It mentions matter_id but not limit or offset. No explanation of parameter types, defaults, or how they affect results. Insufficient compensation for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists expense entries and mentions a key filter (matter_id). It distinguishes from sibling tools like get_expense, create_expense, and list_fees, though it could be more specific about the scope (e.g., all expenses vs. user-specific).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., get_expense for a single expense, list_fees for fees). The filtering hint is present but no when-not conditions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations available, and the description does not disclose any behavioral traits beyond the basic listing operation. No mention of pagination, output format, or effect on system state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded, consisting of a single sentence. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description omits what the tool returns (a list of leads). Lacks details on sorting, filtering, or default behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema describes two parameters (limit, offset) with defaults but no descriptions. The description does not explain these parameters or their semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('leads'), and adds context 'potential new clients/matters' to clarify what leads are. This distinguishes it from get_lead (single) but not from other list tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like search or other list tools. No exclusions or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must carry the full burden. It only mentions pagination but does not disclose whether it is read-only, what the response structure is, or any side effects. For a simple list tool, it is minimally acceptable but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, brief sentence with no redundancy. However, it may be too terse considering the lack of parameter details and usage context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of many sibling list tools, the minimal description leaves agents unsure about the tool's exact behavior. It does not explain pagination behavior, ordering, or filters, and although an output schema exists, the description should still provide enough context for selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds the concept of 'offset pagination', which links to the two parameters, but it does not explain the meaning of limit or offset, their bounds, or default behavior. With 0% schema coverage, more detail is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('matters') and mentions 'offset pagination', which differentiates it from simple list operations. However, it does not specify the scope or distinguish it from sibling list tools like list_matter_items.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as get_matter or other list tools. The description lacks any contextual hints about prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden for behavioral disclosure. It only states it updates metadata, but does not mention idempotency, permissions, return value, or whether partial updates (PATCH semantics) are supported. The existence of an output schema is not leveraged in the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant words. It is front-loaded with the action and resource, though it could benefit from slightly more structure (e.g., indicating it's a patch operation).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of schema descriptions, annotations, and no output schema details, the description is insufficient. It does not specify default behavior for empty fields, the effect of folder_id, or the response format. For a mutation tool, more detail is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions are 0%, and the description only vaguely covers 'name' and 'folder' (mapped to name and folder_id). The required file_id parameter is not explained. The description adds minimal meaning beyond the bare parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'file metadata', specifying the fields 'name, folder'. This distinguishes it from other file operations like delete or get, but does not explicitly differentiate from sibling patch tools (e.g., patch_folder).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as update_matter or patch_folder. No prerequisites or exclusions are mentioned, leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits like implications of removal, prerequisites, or side effects. The description only states the action, leaving the agent unaware of constraints or consequences.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise, but it lacks structure such as examples or notes. It is not overly verbose, but the brevity sacrifices completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although an output schema exists, the description does not reference it or explain what the tool returns. Combined with missing parameter guidance, the description is insufficient for an agent to use the tool confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain what the three required IDs (matter_id, role_id, relationship_id) represent or how to obtain them. The agent receives no added meaning beyond the parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action (remove), the object (contact relationship), and the context (from a role on a matter), making the purpose clear and distinct from sibling tools like add_relationship_to_role or remove_role_from_matter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as removing the entire role or modifying the relationship. The agent lacks context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states 'Update', implying mutation, but omits details about permissions, side effects, or what happens to unspecified fields. The note on billable is helpful but insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences. The first sentence states the purpose, and the second provides specific param guidance. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and low schema coverage, the description should cover behavior, parameters, and outcomes comprehensively. It does not describe return values, error states, or the effect on existing fields, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It explains the billable parameter ('true' or 'false' to set; leave empty to skip'), but provides no information for amount, expense_id, or description, which are critical for correct usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an expense entry', which matches the tool name. However, it does not differentiate from the sibling tool 'patch_expense', which likely performs the same operation but partially.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'patch_expense' or 'create_expense'. It does not mention prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavioral traits. It only states that the tool 'adds' a contact, implying mutation, but does not specify idempotency, side effects, permissions required, or what happens if the contact is already assigned. This is insufficient for an agent to predict the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that conveys the action and purpose with no wasted words. It is appropriately sized for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters, no annotations, and an output schema, the description is too brief. It does not cover prerequisites (e.g., layout must exist on matter), return value, or edge cases. The output schema exists but is not referenced, so the agent has no idea what the tool returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. While 'matter_id', 'layout_id', and 'contact_id' are self-explanatory from names, the 'role' parameter is not explained at all. The description adds no parameter-level meaning beyond what the names imply, leaving ambiguity about the role parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Add' and the resource 'contact to a layout instance', and specifies the purpose 'for merge field population'. It distinguishes itself from siblings like 'add_layout_to_matter' and 'get_layout_contacts' by being specific about the action and target.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusions. The usage context is only implied from the purpose ('merge field population'), but there is no explicit when-to-use or when-not-to-use information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states 'Create' but provides no details on permissions, side effects (e.g., overwriting existing reminders), or rate limits. The behavioral traits are underdisclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences, containing no unnecessary words. It is well front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (3 params) and presence of an output schema, the description still falls short by not explaining two critical parameters and lacking usage context. It is not complete enough for an agent to use confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds the reminder_type options ('Notification | Email') beyond the schema's default but fails to explain event_id and minutes_before. This is partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Create a reminder for an event,' which is a specific verb and resource. It also mentions the reminder_type options. However, it does not distinguish this tool from its siblings like update_event_reminder or get_event_reminders.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as when to create a new reminder vs update an existing one. No context on prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description adds value by explaining the date format (YYYY-MM-DD), that duration_minutes is time spent, and billable defaults. However, it does not disclose side effects (e.g., does it create a ledger entry?), idempotency, or what happens on error. The output schema exists but is not described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short but unstructured: it lists three parameter explanations in a run-on manner. While it is not verbose, it could be better organized (e.g., parameter descriptions in a list) and front-load the core purpose. Missing parameter explanations reduce its efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 parameters, 4 required, 0% schema descriptions, and an output schema, the description is incomplete. It does not explain all required params, the return value (despite output schema existing), or how the fee fits into the broader matter workflow. More context is needed for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It covers date, duration_minutes, and billable (3/8), but fails to explain rate, staff_id, matter_id, description, and activity_code_id. Required params like matter_id and staff_id lack any clarification, leaving ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a fee (time entry)', which defines the tool's verb and resource. It differentiates from sibling tools like create_expense by specifying it's for time entries. However, it does not explain what a 'fee' means in context (e.g., billable time).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like create_expense or update_fee. There is no mention of prerequisites (e.g., matter must exist) or scenarios where this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral disclosure. It only states 'Delete' without explaining permanence, side effects, required permissions, or error conditions. This is insufficient for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and directly states the tool's purpose with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive tool with no annotations, the description should provide more context about irreversibility, cascading effects, or return value. It is too minimal to be complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning beyond the single parameter's name. It does not clarify event_id format, origin, or constraints, leaving the agent to infer from the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete) and the resource (calendar event) along with the key identifier (by ID). It is sufficiently specific and distinct from sibling tools like delete_event_reminder, though it does not explicitly differentiate itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., delete_event_reminder) or any preconditions. The agent receives no context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose any behavioral traits such as permanence, permissions required, or side effects. The agent is left uninformed about the consequences of deletion.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no unnecessary words. However, it is arguably too terse, bordering on under-specification.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description omits what the tool returns (e.g., success message, deleted object). Despite having an output schema, no details are given, leaving the agent uncertain about the response.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description only says 'by ID' but does not elaborate on the format, source, or constraints of the lead_id parameter. Minimal value added beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (Delete), resource (a lead), and method (by ID). It is specific and distinguishes itself from sibling tools like create_lead or get_lead.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as archiving or updating a lead. The description lacks prerequisites or context for use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states 'delete' without explaining if it is permanent, soft delete, or cascading, nor any permissions or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence with no waste, but it may be too concise at the expense of necessary details. Still, it is efficiently structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema and sibling tools, the description is incomplete. It lacks information about reversibility, return value, and when to use over similar tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description adds no meaning beyond 'by ID'. It does not explain the format, constraints, or purpose of matter_id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a matter by ID' clearly states the action (delete), the resource (matter), and the method (by ID). It is specific and distinguishes from sibling tools like archive_matter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as archive_matter or other delete operations. The description does not mention prerequisites or effects.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states the deletion operation without mentioning whether it is permanent, soft delete, cascading effects, or dependencies. This is insufficient for an agent to understand the tool's impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It is front-loaded with the action and resource. However, it might benefit from a brief behavioral note without harming conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (not shown), the description lacks information on what the output represents, error conditions, or post-deletion consequences. For such a simple tool, the description could still note that the deletion is irreversible or what permissions are needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It mentions 'by ID' but does not explain the format, source, or validation of the staff_id parameter. Minimal value added beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('delete'), the resource ('staff member'), and the method ('by ID'). It distinctly differentiates from sibling tools like create_staff_member, update_staff_member, get_staff_member, and search_staff.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., remove_user), prerequisites (e.g., permissions), or scenarios where deletion might be inappropriate. The description lacks any usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states a generic read operation. It discloses no behavioral traits such as permissions, pagination, error handling, or whether recursive contents are included. The output schema exists but the description does not leverage it to explain return structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It is front-loaded with the action. However, it may be too brief given the lack of other guidance, but conciseness is maintained.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with an output schema, the description is minimally adequate. It explains the core purpose but lacks detail on what 'contents' includes (files vs. subfolders) and edge cases. The sibling tools list is large, but the description does not address where this tool fits.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description only vaguely maps 'folder_id' and 'matter_id' to 'specific folder' and 'on a matter'. It does not explain formats, constraints, or how these parameters relate beyond the obvious. Minimal added value over schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'get the contents of a specific folder on a matter.' It specifies the resource (folder contents) and context (matter). However, it does not differentiate from sibling tool 'get_root_folder_contents', missing an opportunity to clarify scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'get_root_folder_contents' or other list tools. There is no mention of prerequisites or context for invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It only states a simple read operation but omits any details about authorization, rate limits, idempotency, or potential side effects. The tool appears to be a safe read, but the description is too minimal to confirm.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (one sentence), but it may be too brief given the lack of supporting detail. It front-loads the core purpose but sacrifices completeness for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema, so return values are likely documented elsewhere. The operation is simple (get by ID). However, the description does not mention error handling, null results, or any additional context that would help an agent handle edge cases. It meets the minimum viable standard but has clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no meaning beyond the schema. Schema description coverage is 0%, meaning the property 'invoice_id' has no documentation in the schema. The description fails to provide any context or constraints for the parameter (e.g., format, example, source).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get an invoice by ID' clearly specifies the action (get), resource (invoice), and method (by ID). It effectively distinguishes from sibling tools like 'list_invoices' and 'get_invoice_download_url'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., list_invoices or get_invoice_download_url). There is no mention of prerequisites, context, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It only states the basic purpose but fails to mention that the tool is read-only, idempotent, or what happens if the role does not exist. The output schema exists but is not referenced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no extraneous information. It is concise, though it could include a brief clarification of the return value without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the existence of an output schema (not detailed here), the description is minimally adequate. However, it does not explain the relationship between this tool and related siblings, nor does it describe the response structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no details about the two parameters beyond their names. While matter_id and role_id are self-explanatory, the description does not clarify their required formats or any constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get the contact relationship for a specific role on a matter,' which clearly indicates a retrieval operation targeting a single relationship associated with a particular role. This distinguishes it from sibling tools like get_relationships_on_matter (returns all) and add_relationship_to_role (mutation). However, the term 'contact relationship' could be more precisely defined.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidance is provided. The description does not specify when to use this tool versus alternatives such as get_relationships_on_matter or add_relationship_to_role. There is no mention of prerequisites or context for invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description does not disclose behavioral traits such as pagination, permissions, or side effects. Merely states a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words. However, it is underspecified given the context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema and a single parameter, the description is too sparse. It fails to define 'root folder' or explain how this differs from 'get_folder_contents'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description does not explain the 'matter_id' parameter beyond its presence in the schema, adding no semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get the root folder contents for a matter', using specific verb and resource. Distinguishes from sibling tool 'get_folder_contents' which likely targets a specific folder rather than the root.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool over alternatives like 'get_folder_contents'. No exclusions or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description adds no behavioral information such as authentication, idempotency, or side effects. For a simple read, minimal disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, front-loading the core purpose. Could include more detail without verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown), the description does not need to explain return values. However, for a simple get operation, it is minimally adequate but lacks context on error conditions or relationships.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description does not elaborate on the 'task_id' parameter beyond the schema's type. The parameter is self-explanatory, but the description could add format or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a task by ID' clearly states the action (get) and the resource (task) with the primary identifier. It effectively distinguishes from sibling like 'list_tasks' but lacks mention of returned data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'get_subtask' or 'list_tasks'. No context about prerequisites or use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits like permissions, side effects, or error handling. It only states the basic purpose, leaving the agent uninformed about important operational details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at one sentence. It is front-loaded and to the point, but could benefit from slight elaboration without losing efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and an output schema, the description is minimally adequate but lacks details about the expected output format or any conditions. It does not fully compensate for the missing parameter descriptions or behavioral context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the tool description does not add any meaning beyond the parameter name 'task_id'. The description does not explain the parameter's role or constraints, which is insufficient for a tool with one required parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Get' and resource 'documents attached to a task', clearly conveying the action. It distinguishes from siblings like 'get_task_document' (singular) by using 'documents' (plural), but does not explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as 'get_task_document' or 'get_file'. There is no context about prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as error handling, required permissions, or whether the operation is read-only. The simple description adds no behavioral context beyond the basic retrieval action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that contains no superfluous words. It is appropriately sized for a simple retrieval tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of many sibling tools and the need to differentiate, the description lacks completeness. It does not explain what a user is relative to other entities, nor does it address prerequisites like authentication. The output schema may cover return values, but the description itself is insufficient for full contextual understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, and the description adds no semantic meaning to the sole parameter 'user_id' beyond stating it is used to identify the user. The description does not explain the expected format or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'get' and resource 'user' by ID. It distinguishes from sibling tools like 'get_staff_member' by resource name, though without explicit differentiation. The purpose is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., 'get_staff_member' or 'search_staff'). There is no mention of prerequisites or context where this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavior. It only says what the tool does, not how it behaves (e.g., pagination via limit/offset, ordering, potential performance implications).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise but lacks important details. Could include pagination info or a brief note on what referral source types represent without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with output schema present, but description is minimal. Does not explain what a referral type is or what the output contains, though the output schema may compensate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% (parameters have no descriptions) and description does not mention parameters at all. Fails to add any meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'List' and resource 'referral source types configured for this firm', distinguishing it from its sibling 'get_referral_type' which retrieves a single type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'get_referral_type' or other list tools. Missing context for typical use cases or filtering scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits like side effects, required permissions, or whether the removal is reversible. For a mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant information, achieving maximum conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and lack of annotations, the description is too sparse. It fails to cover return value expectations, prerequisites, or error cases, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the parameters beyond their names. 'layout_id' and 'matter_id' are self-explanatory but the tool would benefit from clarifications on format or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Remove' and the resource 'a layout instance from a matter', distinguishing it from sibling tools like 'add_layout_to_matter'. However, it does not elaborate on what constitutes a 'layout instance', leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'remove_role_from_matter' or 'remove_matter_tags'. The description lacks contextual cues for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden but only states the basic function. It does not disclose side effects, permissions, rate limits, or whether the message is stored or sent immediately. Minimal behavioral insight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no redundant words. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and 0% schema coverage, the description is too sparse. It does not cover parameter details, return values (even though output schema exists), or edge cases. The tool is under-described for successful invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no meaning beyond the parameter names in the schema. Schema description coverage is 0%, and the description does not explain the purpose of optional contact_id or any formatting/constraints on message or matter_id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (send), the object (message), and the target (client via Smokeball client portal). It distinguishes itself from sibling tools like create_portal_task, which creates tasks rather than sending messages.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as create_portal_task or other communication methods. No context on prerequisites or best practices.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states 'Update'. It fails to disclose behavioral traits such as required permissions, idempotency, error handling, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no wasted words. However, it could be expanded slightly to include key details without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and low schema coverage, the description is insufficient. It does not explain that group_id is required, how name and description are used, or the effect of the update. The presence of an output schema does not compensate for the missing parameter context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description provides no additional meaning for the three parameters (group_id, name, description). The agent receives no guidance on parameter usage or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update an authorization group' clearly states the verb (Update) and resource (authorization group), distinguishing it from siblings like create_authorization_group, delete_authorization_group, get_authorization_group, and list_authorization_groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., create, delete, get). The description does not mention prerequisites, context, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description implies mutation ('update') but does not disclose resetting of fields to defaults, permissions, side effects, or validation. No annotations provided, so description carries full burden but insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. Efficiently conveys purpose and one key parameter instruction.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, so return values are covered. However, description lacks explanation of overall update behavior, handling of missing fields, and validation rules. Incomplete for a 5-parameter mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description only adds meaning for 'billable' parameter ('true/false to set; leave empty to skip'). Other parameters (rate, duration_minutes, description) lack explanation of meaning or units.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool updates a fee entry, but does not differentiate from sibling 'patch_fee' which may also update fee fields. Verb+resource is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like 'patch_fee' or 'create_fee'. Only provides specific instruction for one parameter (billable) without broader context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden, but it only says 'update', implying mutation. It lacks details on side effects, required permissions, or whether updates are partial or full. The return value is not described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence (12 words) with no wasted words. However, it is too brief for the complexity of the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 0% parameter documentation and no annotations, the description is incomplete. It does not explain the output schema (which exists) or how this update behaves. The sibling list is large but no comparison is provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description adds no information about the three parameters (email, staff_id, mapping_id). The required mapping_id is not explained, and the other two have no semantics given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'firm user mapping', and explains the mapping's purpose as linking a staff member to a system user. It distinguishes this tool from siblings like get_firm_user_mapping and delete_firm_user_mapping.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as the get or delete variants. No prerequisites or context are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Only states that fields can be updated, but does not disclose whether it's a partial or full replacement, nor any side effects or prerequisites.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with purpose, no extraneous words. Efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and ambiguous sibling tools, the description lacks essential behavioral context (e.g., partial vs full update). Incomplete for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%. Description merely repeats field names without adding meaning (e.g., allowed values for status, format constraints). Fails to compensate for schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the action 'update' on 'matter' and lists fields (name, status, description). It is clear but does not differentiate from sibling 'patch_matter', which could lead to confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'patch_matter' or 'create_matter'. Lacks context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full responsibility. It only states 'update', but fails to disclose whether it performs a full replacement or partial update, what happens if the relationship does not exist, required permissions, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. However, additional structure or bullet points could enhance readability without significant length increase.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 required parameters, no annotations, and an output schema, the description is too sparse. It does not explain the output schema or any constraints, making it incomplete for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the tool description does not explain any of the four parameters (matter_id, role_id, relationship_id, relation_type) beyond their names. This provides no added value for parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'update' and the resource 'contact relationship on a matter role', which distinguishes it from sibling tools like 'add_relationship_to_role' and 'remove_relationship_from_role'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'add_relationship_to_role' or 'remove_relationship_from_role'. The description does not include any contextual hints about preferred usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description is the sole source. It does not disclose whether adding tags is additive or replaces existing tags, what happens if a tag already exists, or any side effects. The behavioral traits are insufficiently described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, consisting of one sentence and a brief param hint. There is no waste, though it could be slightly more structured to separate the main action from parameter details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple but lacks important context such as whether the operation is idempotent, what the output schema returns, or any behavior like overwriting. For a mutation tool with no annotations, this is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description adds meaning for 'tags_csv' by noting it is comma-separated. For 'contact_id', no additional info is provided. This partial compensation justifies a middle score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Add tags to a contact' which is a specific verb and resource. It also explains the format of tags_csv. However, it does not explicitly distinguish from sibling tools like 'remove_contact_tags' or 'add_matter_tags', though the name is self-explanatory.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives, no when-not-to-use conditions, and no prerequisites mentioned. The description only states what it does without context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description only mentions 'associate a contact' but lacks details on side effects, idempotency, permissions, or error behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise but underspecified; could provide more value without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite an output schema, the description lacks context on parameter usage, prerequisites, and differentiation from sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%; only contact_id is partially described ('associate a contact to this role'), while required parameters matter_id and role_name are not explained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (Add), resource (role), and target (matter), distinguishing it from siblings like remove_role_from_matter and update_role_on_matter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs. alternatives (e.g., add_relationship_to_role) or prerequisites (e.g., matter existence, role definition).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It indicates a create operation but lacks details on side effects, required permissions, whether the policy is immediately active, or what the response contains. The existence of an output schema is not mentioned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences, front-loading the purpose. The parameter hint is appended efficiently, but the structure could be improved by including a summary of the return value or additional context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low schema coverage and no annotations, the description is incomplete. It does not explain what an authorization policy is, how it relates to authorization groups, or what the output schema provides. Additional context is needed for effective tool selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description adds meaning for 'permissions_csv' by stating it accepts 'comma-separated permission strings', but does not describe the 'name' parameter. Thus, it partially compensates for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Create an authorization policy' with a specific verb and resource. However, it does not differentiate from sibling tools like create_authorization_group or create_matter, which also create entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as update_authorization_policy or create_authorization_group. There is no mention of prerequisites, typical use cases, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral disclosure. It only states the action and parameter formats, but omits critical traits such as whether the operation is destructive, authorization requirements, idempotency, or side effects like triggering billing updates. This is insufficient for an agent to understand the tool's impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and directly states the purpose followed by parameter hints. It avoids unnecessary words and is front-loaded. However, it could be better structured by listing parameters separately, but overall it is concise and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (6 parameters, no schema descriptions, no annotations), the description is too sparse. It does not explain what an expense entry represents, the need for a matter context, or any constraints. The presence of an output schema reduces the need to describe return values, but the description still lacks essential context for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It explains three parameters (date format YYYY-MM-DD, amount as dollar value, billable as 'true'/'false' with default). However, it fails to describe the required matter_id and the optional description and activity_code_id, leaving a gap. The provided formats are helpful but incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create an expense entry', which explicitly identifies the action and resource. It provides format hints for date and amount, adding clarity beyond the name. However, it does not distinguish this tool from siblings like create_fee or create_transaction, but the resource name 'expense' is specific enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description offers no guidance on when to use this tool versus alternatives such as create_fee or patch_expense. It does not mention prerequisites like needing a matter_id, or scenarios where this tool is appropriate. The absence of usage context leaves the agent without decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description indicates it creates a matter (mutation) but does not disclose potential side effects, permissions required, or error conditions. Minimal behavioral information beyond the fact of creation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief (two sentences plus a bullet list) and front-loads the core purpose. No unnecessary words, but could be slightly more structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the required parameters but omits explanation of optional parameters (status, description). No mention of return values (output schema exists but not referenced). Lacks context on validation, error handling, or usage constraints for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains matter_type_id as from list_matter_types, client_ids_csv as comma-separated contact IDs, and number as optional. However, it does not explain the status, description, or any other optional parameters. Coverage is partial but adds meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a matter and specifies required parameters (matter_type_id and client_ids_csv). It does not explicitly distinguish from sibling tools like add_file_to_matter or archive_matter, but the purpose is evident from the name and description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., update_matter, create_lead). It mentions required parameters and that matter_type_id comes from list_matter_types, but does not specify prerequisites or constraints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must disclose behavioral traits. It merely states 'Send a notification' without mentioning authentication requirements, side effects, rate limits, or what happens when user_id is empty. This is insufficient for safe usage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very brief (one sentence), which is concise but at the expense of omitting critical information. It is not verbose, but could be more informative without significant length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters, no annotations, and an output schema, the description is too sparse. It doesn't clarify the output or behavior when user_id is omitted, leaving gaps for an AI agent to operate effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the tool description adds no parameter-level details. It does not explain the role of each parameter or default behavior for optional user_id, failing to compensate for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Send a notification') and the recipient scope ('to a user or the firm'). The resource 'notification' aligns with the tool name, and no sibling tool duplicates this functionality, making it distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites, limitations, or when not to use it, leaving the agent to infer context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It only states the basic action without disclosing behavioral traits such as permissions requirements, idempotency, error handling (e.g., invalid task_id), or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a single sentence, but it is under-specified and lacks necessary detail, making it minimally adequate rather than effectively concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although the tool has an output schema, the description does not mention return values or side effects. It omits important context such as the expectation that the subtask is linked to the specified task, leaving the description incomplete for a creation operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning the input schema provides no parameter descriptions. The tool description adds no explanation of parameters beyond their names, failing to compensate for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Create' and the resource 'subtask', and specifies the context 'under a task', which distinguishes it from sibling tools like 'create_task' and 'create_task_document'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies that the tool is for creating subtasks within an existing task, but it does not explicitly state when to use it versus alternatives like 'create_task', nor does it mention prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as side effects, permissions, or what happens upon creation. The agent is left guessing about the tool's safety and behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two short sentences. It front-loads the purpose and then adds parameter hints. No wasted words, though it could be more structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 5 parameters and an output schema (not described), the description lacks important context such as how the task relates to matters, what happens when a matter_id is provided, or the structure of the response. The agent may not understand the full implications of using this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description adds meaning for two parameters (due_date format and assigned_to_id purpose). However, four other parameters (name, notes, matter_id) are not described, leaving gaps. The description partially compensates but is not comprehensive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a task' which is a specific verb and resource. It distinguishes from sibling tools like create_subtask or create_task_document, but lacks any additional context such as what kind of task or its scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, when it is appropriate, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral disclosure. It only states the core action and a parameter hint, omitting details like idempotency, error behavior, permission requirements, or side effects (e.g., whether duplicate attachments are allowed).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded with the action. However, it is so brief that it sacrifices crucial details. It earns its keep but could be expanded slightly without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (so return details are covered), the description lacks information on when to use this tool, prerequisites, error handling, or integration with sibling tools. For a tool creating a task document, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It adds meaning only for 'file_id' (notes it comes from files API), but does not explain 'task_id' or the optional 'name' parameter. The hint is insufficient for a tool with 3 parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Attach a file to a task') with a specific verb and resource. It distinguishes from sibling tools like 'add_file_to_matter' by specifying 'to a task'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., 'add_files_to_matter_batch'), prerequisites (task must exist), or when not to use it. The single-line description lacks contextual usage advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility. It only states 'Delete' without disclosing side effects, permission requirements, or whether the operation is reversible. For a destructive action, more transparency is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise (one sentence) with no fluff, but it could be better structured to include essential usage hints. The brevity sacrifices completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter delete tool, the description combined with the output schema provides minimal completeness. However, it lacks usage guidelines and behavioral notes, making it adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The parameter 'expense_id' is named intuitively, but the description adds no extra meaning—it just says 'by ID'. It fails to specify format, source, or constraints beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete'), the resource ('an expense entry'), and the method ('by ID'). It distinguishes the tool from siblings like create_expense, update_expense, patch_expense, get_expense, and list_expenses.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites (e.g., existence of the expense), and no caution about irreversibility or dependencies. The description lacks any usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits like cascading effects, permissions, or irreversibility. It only states the basic operation, leaving the agent without insight into consequences or requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no extraneous words. It is appropriately concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema, the description is minimal. However, it lacks information about side effects, return status, or error conditions, which a delete operation typically requires for safe invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description adds no meaning to the 'folder_id' parameter beyond what the schema shows. It does not explain expected format, how to obtain the ID, or any constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a folder by ID' clearly states the action (delete) and the resource (folder), and the identifier method (by ID). It distinguishes this tool from siblings like 'update_folder' or 'create_folder'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as moving to trash or other deletion patterns. No explicit context for prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must disclose behavioral traits. It only states the basic action without mentioning irreversibility, side effects on related data, permission requirements, or rate limits. This is a significant gap for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short with no wasted words, but it may be too sparse. It conveys the core action but omits essential details that would be expected in a production tool description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description fails to provide behavioral context (e.g., whether deletion is permanent, cascading) or usage prerequisites, making it incomplete for an agent to understand the full impact of invoking this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% parameter description coverage, yet the description adds no additional meaning beyond 'by ID'. It does not clarify the ID format, source, or provide examples, so the parameter remains underspecified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Delete' and identifies the resource 'a memo' with the identifier method 'by ID'. It clearly distinguishes from sibling tools like create_memo, update_memo, get_memo, and other delete_* tools by specifying the exact action and target.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., archiving or soft-deleting), prerequisites, or exclusions. The description lacks any context about appropriate usage scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description minimally states the action but does not disclose any behavioral traits such as irreversibility, side effects, or confirmation requirements. With no annotations, the description carries the full burden but fails to provide meaningful context beyond the basic operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no superfluous words, making it concise and front-loaded. However, it is perhaps too minimal for the information needed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With one required parameter and no annotations, the description is incomplete. It does not explain what happens after deletion, any constraints, or how the output (which has an output schema) relates to the operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning beyond the parameter name. It merely restates 'by ID' without explaining the format, scope, or source of the plugin_id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Delete' and the resource 'plugin', and specifies the identifier method 'by ID'. It distinguishes itself from sibling tools like 'update_plugin' and 'create_plugin' by the verb and resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, when not to use it, or referral to other tools like 'unsubscribe_from_plugin'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavioral traits. It only states the basic deletion action, omitting critical details such as whether the deletion is permanent or reversible, cascading effects (e.g., on subtasks), required permissions, or result format. The minimal description does not adequately inform the agent about side effects or constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is highly concise (one sentence, five words). However, it sacrifices important behavioral and usage details. While front-loaded and to the point, it does not earn its place fully because it omits information that would not significantly lengthen it. A sentence like 'Permanently removes a task and its associated subtasks' would add clarity without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required parameter, no nested objects), the description should cover basics like return value (output schema exists but undocumented), permanence, and preconditions. The current description fails to provide this context, leaving the agent with insufficient information to use the tool safely and effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% (the description does not describe parameters). The only parameter, task_id, is hinted at by 'by ID' but lacks specifics such as format, length, or source. The description adds minimal value beyond the schema, failing to compensate for the lack of schema-level descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the resource ('a task'), and specifies the identifier ('by ID'). It is concise and unambiguously distinguishes this tool from sibling tools that delete other entities or perform different operations on tasks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like updating the task status or archiving. It does not mention prerequisites (e.g., task existence, permissions) or caution against unintended deletions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, and the description only states the basic operation. It does not disclose error handling, permissions required, or idempotency. The output schema is present but not referenced in the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with 6 words; appropriately sized for a simple retrieval operation. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for a basic get-by-ID operation, especially given the presence of an output schema. However, it lacks context about what an activity code is and potential failure modes.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning beyond the parameter name 'code_id'. The parameter is not elaborated, leaving ambiguity about its format or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (get) and the resource (activity code) with the specific identifier method (by ID). It distinguishes from sibling tools like list_activity_codes and create/update/delete variants.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., list_activity_codes). No usage context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It implies a read-only operation but offers no explicit behavioral details (e.g., side effects, permissions, rate limits). For a simple get, this is minimal but acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence with no redundant information. It efficiently conveys the core purpose, though could benefit from additional context without being overly verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool and the existence of an output schema, the description is incomplete. It does not explain when to use this over related tools like 'list_authorization_groups' or provide any usage context. The agent may struggle to select the correct tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'reference' has no description in the schema (0% coverage). The description mentions 'by reference' but does not clarify what constitutes a reference (e.g., ID, name). This adds minimal value beyond the parameter name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and the resource ('authorization policy'), with the method 'by reference'. It distinguishes from siblings like 'get_authorization_group' by specifying a different resource, making the tool's purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'list_authorization_policies' or 'get_authorization_group'. The description lacks context about prerequisites or suitable scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist; description only states retrieval without mentioning side effects, error handling, or read-only nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, appropriately sized for a simple get-by-ID operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Minimal description; lacks info on output (though output schema exists) and operational context; no annotations to supplement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%; description adds no meaning to 'account_id' parameter beyond its name and type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states action ('get') and resource ('bank account') with identifier 'by ID', distinguishing it from sibling tools like 'list_bank_accounts' or 'get_bank_account_matter_balances'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives; no prerequisites or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states the basic purpose. It does not disclose any behavioral traits such as read-only nature, error handling, or response format. The output schema exists but the description adds no extra context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, using only 5 words. However, it sacrifices completeness for brevity. While front-loaded, it lacks necessary details for an AI agent to fully understand its use.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description does not need to explain return values. However, for a simple retrieve-by-ID tool, it remains minimally complete. Additional context about usage scenarios or limitations would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, contact_id, is clearly named but the description does not elaborate on its format, requirements, or any constraints. Schema description coverage is 0%, so the description should have provided more details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'contact', and the retrieval method 'by ID'. It distinguishes itself from sibling tools like list_contacts by specifying individual retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance provided on when to use this tool versus alternatives (e.g., list_contacts for multiple contacts). The description lacks context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description should disclose behavioral traits. It only states the action without mentioning side effects, return format, pagination, or any constraints. The minimal description fails to compensate for missing annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, which is concise. However, it is too brief to be informative and could benefit from more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although an output schema exists (not shown), the description does not explain what 'relationships' entails or what fields are returned. For a simple retrieval tool, the description is minimally adequate but lacks context about the output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one required parameter (contact_id) with 0% description coverage. The description adds no additional meaning or format details beyond the schema, leaving the agent without guidance on valid input values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get all relationships for a contact.' clearly specifies the verb (get), resource (relationships), and scope (for a contact). It effectively distinguishes from sibling tools like 'get_contact_relation' (singular) and 'create_contact_relation'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives such as 'get_contact_relation' or 'list_contacts'. The description lacks context for appropriate use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must carry the burden. It only says 'Get an expense entry' without disclosing idempotency, permission requirements, or error behavior. Minimal behavioral info.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with one sentence, but it is under-specified. Brevity is not prioritized over informativeness; a few more details would improve without harming conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the presence of an output schema (not shown in detail), the description omits key context like what happens when expense_id is invalid or missing. Given the tool's simplicity, more completeness is expected.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description adds little: it mentions 'by ID' but does not explain the expense_id parameter's format, source, or constraints, failing to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get an expense entry by ID,' specifying the verb and resource. It distinguishes from siblings like create_expense, list_expenses, patch_expense, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as list_expenses or other get tools. The description lacks context for proper tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It does not state whether the operation is read-only, if it requires special permissions, or any side effects. The absence of such information leaves an agent without critical context for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, making it concise and front-loaded. However, it is too brief given the lack of additional elaboration in the schema or annotations. A slightly longer description would improve clarity without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the existence of an output schema, the description is insufficient. It does not explain the return value (breadcrumb format) or provide enough context for the agent to use the tool effectively, especially given the two required parameters are undocumented.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has two required parameters (folder_id, matter_id) with no descriptions. The schema description coverage is 0%, but the description does not add any explanation about these parameters, leaving the agent without understanding of what values to provide.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get the full path hierarchy (breadcrumb) to a folder.' It uses a specific verb 'Get' and identifies the resource 'folder path hierarchy'. This clearly distinguishes it from sibling tools like 'get_folder_contents' (which retrieves folder contents) and 'get_root_folder_contents'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There is no mention of when not to use it or any context that helps the agent decide between this and related tools like 'get_folder_contents' or 'get_folder_history'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral traits. It does not disclose any behavior such as authentication needs, error handling, or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence is concise, but it omits essential information. It is too minimal to be considered well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description lacks context about the tool's purpose and behavior. No annotations or additional info make it incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no meaning beyond the schema. The parameter invoice_id is not explained in terms of format or acceptable values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'download URL for an invoice PDF'. It is specific and distinguishes from sibling tools like get_file_download_url and get_invoice.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, context, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must fully convey behavioral traits. It only says 'Get', implying a read-only operation, but provides no details about idempotency, side effects, or what happens if the stage does not exist. The output schema exists but is not described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no unnecessary words. It is front-loaded with the verb and resource, making it efficient for an AI agent to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and the simplicity of a two-parameter getter, the description is too minimal. It does not explain the relationship between stage and stage set, nor does it hint at the returned data structure. A more complete description would mention that it retrieves stage details within the context of a stage set.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema defines two required parameters (stage_id, stage_set_id) with 0% schema description coverage. The description adds no semantic value, failing to explain what these IDs represent or how they relate. It does not clarify that stage_set_id scopes the stage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get' and the resource 'a specific stage within a stage set'. This verb-resource combination is precise and distinguishes it from sibling tools like 'get_stage_set' (which retrieves the entire set) and 'get_matter_stage' (which retrieves a stage for a matter).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, it does not explain the difference between this tool and 'get_stage_set' or 'list_stage_sets', nor does it mention any prerequisites or context for use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description carries full burden. It only states the action without disclosing behavioral traits like read-only nature, authentication requirements, or side effects. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no extraneous information. Efficiently conveys the core functionality.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no annotations and 2 undocumented parameters, the description is too sparse. It lacks parameter explanations and usage context, making it incomplete for agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but description adds no meaning to 'task_id' or 'document_id'. Agent must infer their purpose from context alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'Get' and resource 'specific document attached to a task', clearly distinguishing it from sibling tools like 'get_task_documents' (plural) and 'create_task_document'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives such as 'get_file' or 'get_task_documents'. No prerequisites or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so description must carry full burden. Only states pagination method but omits ordering, default sort, authentication, rate limits, or any other behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concisely one sentence, but it's too brief for a list tool with no annotations. Front-loaded, but underspecified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Low complexity, but lacks essential context like sorting, filtering, or return value summary despite output schema existing. Minimal guidance for agent invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%. Description mentions 'offset pagination' but doesn't explain limit or offset beyond their names and defaults. No added meaning for the parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List contacts with offset pagination', which is a specific verb and resource. It distinguishes from siblings like get_contact (single) and other list tools by mentioning offset pagination.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool over alternatives like get_contact or other list tools. No context on prerequisites or common use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states the basic removal action. It does not disclose any behavioral traits such as irreversibility, required permissions, or side effects beyond the removal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short with two sentences, no fluff. However, it lacks structure and does not fully describe both parameters, making it adequate but incomplete.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the two required parameters, the description only explains one. No usage context or additional behavioral details are provided. The output schema exists but is not described, leaving the tool's behavior underspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description partially compensates by explaining tag_id ('the ID of the tag to remove'), but omits any description for contact_id. With two required parameters, this is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Remove a tag from a contact,' specifying the verb 'remove' and the resource 'tag from a contact.' It effectively distinguishes from sibling tools like add_contact_tags and remove_matter_tags.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor any prerequisites or exclusions. It simply states the action without context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It does not state whether the tool is read-only, what happens with large result sets, or if any destructive side effects occur. The description is silent on these aspects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two sentences that front-load the main action. However, it could be more efficient by also briefly mentioning pagination parameters without adding length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description lacks context about pagination (limit/offset) and any filtering behavior beyond the query string. It feels incomplete for a search tool with multiple parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description adds meaning for the query parameter (name or email fragment) but does not explain limit or offset. With three parameters, only one is addressed, leaving gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it searches staff members and specifies that the query parameter is for name or email fragment. This differentiates it from siblings like get_staff_member (single record) and list_contacts (different entity).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as list_contacts or get_staff_member. The description does not mention context, prerequisites, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only indicates mutation ('Update') but fails to disclose behavioral traits such as idempotency, authorization requirements, side effects on dependent entities, or what happens if the policy does not exist. Since no annotations are present, the description carries the full burden and falls short.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences, no filler. The first sentence states the purpose, and the second provides parameter details. It could be improved by front-loading more context, but it remains efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not referenced) and the complexity of an update operation with sibling tools, the description is incomplete. It does not explain what the tool returns, whether the update is partial or full, or how errors are handled. The agent would need to infer too much.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning for the 'permissions_csv' parameter by stating it expects 'comma-separated permission strings.' However, it does not explain the 'name' or 'reference' parameters. With 0% schema description coverage, the description should compensate more comprehensively but only partially does so.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Update an authorization policy,' which clearly identifies the verb and resource. However, it does not distinguish this tool from sibling tools like create_authorization_policy or delete_authorization_group, and it lacks specificity on what constitutes an authorization policy.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, when to update versus create, or any conditions that should be met before calling.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries full burden. It does not disclose behavioral traits such as idempotency, partial updates, required user roles, or side effects. The description is too basic for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences, no wasted words. It effectively front-loads the action. However, it could be better structured by grouping billing type options.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters and an output schema, the description is insufficient. It does not explain what the output contains, error conditions, or how to interpret the allowed billing types. The tool's complexity is not fully addressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It adds allowed values for billing_type (Hourly, Fixed, Contingency), which is helpful. However, it does not explain rate (e.g., currency, unit) or matter_id format, leaving gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update billing configuration for a matter' with a specific verb and resource. It also lists allowed billing types. However, it does not differentiate from sibling tools like get_matter_billing_configuration or other update tools, limiting its distinctiveness.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There are no prerequisites mentioned (e.g., matter existence, permissions), nor any scenarios where this tool should not be used.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must bear full burden. It lacks details on side effects, idempotency, error handling, or prerequisites. For example, what happens if both title and completed_str are omitted? Behavior is unclear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences with no wasted words. The first sentence states purpose, the second gives a key constraint on a parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not explain what the tool returns or common error scenarios. The simple update operation lacks context on prerequisites (e.g., task existence) and typical usage patterns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description adds meaning for completed_str (expects 'true' or 'false'). However, it provides no guidance for 'title' parameter, leaving its use ambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and resource 'client portal task', specifying the fields 'status or title'. It includes a format hint for completed_str, helping differentiate from 'update_task' sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus siblings like 'update_task' or 'create_portal_task'. The agent is left to infer from context, which may cause confusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden of behavioral disclosure. It explains the semantics of 'completed_str' but does not address idempotency, side effects, permissions, or whether updating other fields like 'name' has any constraints. The behavior is only minimally described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at 15 words, with the first sentence stating the purpose and the second explaining a key parameter. It is front-loaded and easy to scan. However, it sacrifices completeness for brevity; additional relevant information could be included without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of 4 parameters, 0% schema coverage, and no annotations, the description is insufficiently complete. It covers only one parameter's semantics and omits behavioral traits, prerequisites, and the ability to update the name field. The presence of an output schema does not excuse the lack of input parameter guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must add meaning to parameters. It explains 'completed_str' with specific values ('true'/'false') and their effects, which adds value. However, 'task_id', 'subtask_id', and 'name' are not explained in the description, leaving those parameters with no semantic context beyond their names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update a subtask.' which matches the tool name. It identifies the action as updating and the resource as subtask. However, it does not distinguish from sibling tools like 'update_task' except by resource name, and it only highlights the 'completed_str' parameter, missing the 'name' parameter that can also be updated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'create_subtask' or 'delete_subtask'. There are no prerequisites, context, or typical scenarios mentioned, leaving the agent to infer usage solely from the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description must fully convey behavioral traits. It states 'delete' implying destruction but omits consequences like cascading deletes, permission requirements, reversibility, or error conditions (e.g., if subscription does not exist). This is insufficient for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no superfluous words. It is direct and front-loaded with the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema (not shown) which reduces the need to describe return values. However, for a delete operation, the description should mention whether it is idempotent, what happens if the subscription ID is invalid, or any side effects. The current description is minimal but functionally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has a single required parameter 'subscription_id' with 0% schema description coverage. The tool description does not elaborate on the parameter's format, source, or constraints, relying solely on the name which is generic. No added value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete) and resource (webhook subscription) along with the identifier method (by ID). It uniquely distinguishes this tool from siblings like get_webhook_subscription or test_webhook_subscription.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as update_webhook_subscription or test_webhook_subscription. The description does not mention prerequisites, contexts, or scenarios where deletion is appropriate versus other operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states the operation without disclosing behavioral traits such as access requirements, error conditions, or whether the stage is returned as a string or object. This is insufficient for an agent to understand side effects or constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, well-formed sentence with no extra words. It is immediately clear and front-loaded. Every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not shown but noted), the description is minimally adequate. However, it omits context about what constitutes a 'stage' (e.g., is it a name, ID, or object?) and potential edge cases. For a simple tool with one required param and output schema, this is borderline acceptable but not rich.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% (no parameter descriptions in schema). The description does not explain the matter_id parameter (e.g., its format, source, or relation to matter entities). The agent cannot determine what value to provide without additional context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'current stage of a matter'. It distinguishes from sibling tools like get_stage_in_set and get_stage_set, which deal with stages in a set rather than a matter's current stage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when or when not to use this tool versus alternatives. No mention of prerequisites (e.g., matter must exist) or context where other tools like get_matter might be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. 'Get' implies a read-only, non-destructive operation, which is consistent with the tool's purpose. However, it does not mention any additional behaviors like authentication requirements, rate limits, or the nature of the response (e.g., single object vs. empty result if not found).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise—single sentence with no redundant words. It front-loads the action and target. However, it could be slightly expanded with one or two more details without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has only one required parameter and an output schema (not provided in the prompt). The description is minimal but perhaps adequate for a straightforward get-by-ID operation. However, it lacks context about typical use cases, related tools, or error handling, reducing completeness for an AI agent unfamiliar with the domain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions 'by ID' but does not explain that matter_type_id is a unique identifier, how to obtain valid values (e.g., from list_matter_types), or any constraints (e.g., length, format). The description adds minimal meaning beyond the schema's field name and type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a matter type by ID.' clearly states the verb ('Get') and resource ('matter type'). It specifies the retrieval method (by ID), distinguishing it from listing operations like list_matter_types. However, it does not explicitly differentiate from other get_* tools for related entities (e.g., get_matter, get_matter_stage).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool vs. alternatives like list_matter_types (for multiple matter types) or get_matter (for matter details). It lacks any context about prerequisites, such as needing a valid matter_type_id from a previous query.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fails to disclose critical behavioral details such as permissions required, error handling (e.g., if subscription not found), idempotency, or whether the tool returns active/inactive subscriptions. Only the basic operation is stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (one sentence) but lacks necessary details for an agent to fully understand the tool's behavior. It is front-loaded but incomplete, missing parameter context and usage guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema, the description is minimally adequate. However, it does not differentiate from 'list_webhook_subscriptions' in terms of when to use which, leaving some context gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds no meaning to the 'subscription_id' parameter beyond the schema. It does not explain the format, expected length, example values, or how to obtain the ID. With 0% schema description coverage, this omission is severe.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the verb 'Get' and resource 'webhook subscription' with the discriminator 'by ID', making it distinct from siblings like 'list_webhook_subscriptions' which retrieves multiple items.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives (e.g., list_webhook_subscriptions for listing all subscriptions). The usage context is implied but lacks guidance on prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Only states 'send a test notification' without disclosing side effects, idempotency, permission requirements, or whether the subscription must be active. Lacks behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise and front-loaded. However, it is slightly too minimal given the lack of other context; a few more details would improve without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description does not explain the test behavior or expected response. Sibling tools exist but no guidance on when to use this over alternatives. Incomplete for a simple action tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage for the sole parameter 'subscription_id'. The tool description does not elaborate on this parameter, leaving its purpose and source unexplained. No added value over schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action 'Send a test notification' and target 'webhook subscription endpoint'. Distinguishes from sibling tools like create/delete/get/list/update by specifying the test nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for testing an existing subscription, but no explicit when-to-use or when-not-to-use guidance. No mention of alternatives like using get_webhook_subscription to verify existence first.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the behavioral disclosure burden. It reveals that the tool only registers (not uploads) files and that upload is handled separately. However, it fails to disclose other critical behavioral traits: idempotency, error conditions (e.g., duplicate file names), required permissions, or side effects. The minimal disclosure is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no extraneous words. It front-loads the verb and resource, and the parenthetical adds essential context efficiently. Every part earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's role in a multi-step process (file upload then registration), the description is too sparse. It omits workflow ordering, potential prerequisites, behavior on duplicate files, permission requirements, and the meaning of the output schema. The description does not sufficiently equip an agent to use the tool correctly in complex scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description is the only source for parameter meaning. However, the description does not explain any parameter: 'name', 'folder_id', or 'matter_id'. It fails to add meaning beyond the schema property names, which provide no context. For a tool with three parameters, this is a critical gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool registers a new file on a matter, with the specific verb 'register' and resource 'file'. The parenthetical '(get upload URL separately)' clarifies its role in a multi-step process, distinguishing it from file upload tools like get_file_upload_url and from other file operations like get_file or delete_file.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description hints at the prerequisite of getting an upload URL separately, implying a two-step process. However, it does not explicitly state when to use this tool versus alternatives like add_files_to_matter_batch, nor does it specify conditions or exclude cases. More direct guidance on the workflow would improve clarity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It does not disclose side effects (e.g., whether tags are appended or replaced), permission requirements, or rate limits. Minimal behavioral insight beyond mutation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two efficient sentences, front-loaded with the action. Could use more structure but no waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite low complexity, missing context on behavior (append vs replace), error conditions, and matter_id semantics. Output schema exists but description incomplete for full usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage. Description adds meaning for tags_json by specifying JSON array structure and example, but matter_id is not described. Provides partial parameter context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Add tags to a matter', using a specific verb and resource. It distinguishes from sibling tools like remove_matter_tags and add_contact_tags.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like remove_matter_tags or add_contact_tags. Lacks context on prerequisites or exclusivity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must convey behavioral traits. It only states the action, omitting details such as irreversibility, required permissions, side effects, or whether the deletion is permanent. Minimal disclosure beyond the obvious.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, no wasted words. It is front-loaded with the core action. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the moderate complexity of a delete operation, the description is insufficient. It does not mention success/failure outcomes, return values (despite an output schema existing), preconditions, or error states. The description is too bare to fully inform the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should explain parameters. It implies contact_id identifies the contact and relation_id identifies the relationship, but it doesn't clarify where to obtain these IDs, their format, or any constraints. This adds marginal value over the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Delete a relationship from a contact.' Verb 'delete' is specific, and resource 'relationship from a contact' is unambiguous. Among many sibling tools, it is clearly distinguished from create_contact_relation, update_contact_relation, and get_contact_relation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to use this tool versus alternatives like update_contact_relation, remove_relationship_from_role, or any prerequisites. The description lacks context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only indicates a read operation ('Get') and the context 'trust accounting', but does not mention permissions, safety, rate limits, or return specifics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with 10 words, front-loading the key action and scope. However, it could include essential details without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema, the description need not explain return values. The one-parameter tool is adequately described for a simple case, but missing context about prerequisite IDs and trust accounting implications makes it only adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% coverage and the description adds no meaning for the single required parameter 'account_id'. It does not explain what the ID is, how to obtain it, or its format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get', the resource 'bank account', and the scope 'per-matter balances' with the context 'trust accounting'. It effectively distinguishes from siblings like get_bank_account and get_protected_bank_account_balance.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for trust accounting per-matter balances but provides no explicit guidance on when to use this tool versus alternatives like get_bank_account. No 'when not to use' or alternative names are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It only states 'Get', implying a read-only operation, but does not confirm non-destructiveness, authorization needs, or error handling. No mention of side effects or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no superfluous words. It is front-loaded with the key action and resource. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and two simple parameters, the description is minimally adequate for a basic retrieval tool. However, it lacks context about when to use this tool over siblings like 'get_contact_relations' or 'get_relationship_on_role', and does not mention error cases or pagination (though unlikely needed for a single item).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The parameter names 'contact_id' and 'relation_id' are self-explanatory, but the description does not elaborate on their meaning beyond the schema. The description adds mild context by stating 'for a contact' and 'specific relationship'. However, schema coverage is 0%, so baseline expectation is higher; the description adds limited value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a specific relationship for a contact' clearly indicates retrieval of a single relationship resource. The verb 'get' and noun 'specific relationship' make the tool's purpose unambiguous. However, it lacks specificity about what constitutes a 'relationship' in this context, but given the siblings, it is adequately distinct from the plural 'get_contact_relations'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. Siblings like 'get_contact_relations', 'create_contact_relation', and 'update_contact_relation' exist, but no criteria for selection are given. The agent must infer from context that this tool is for a single relation by ID.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description gives no behavioral cues beyond the basic action. It does not disclose permissions required, error scenarios, or whether it returns full event details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one short sentence without filler, but it is overly minimal. While concise, it sacrifices necessary detail. It earns its place but could be expanded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is a simple retrieval with an output schema, the description adequately conveys the purpose. It is complete enough for a get-by-id operation, though it lacks error context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% coverage for event_id. The description only says 'by ID', adding no semantics beyond the parameter name. It does not specify the ID format or source.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a calendar event by ID. The verb 'Get' and resource 'calendar event' are specific, and the 'by ID' distinguishes it from list_events or create_event.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like list_events or get_event_reminders. The agent receives no context on selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior. It only states that metadata is retrieved, but omits critical context like authentication requirements, rate limits, or what constitutes 'metadata'. The agent is left guessing about side effects or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single direct sentence with no wasted words. However, it is arguably too terse; it could include additional clarifying context without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (1 required parameter, no nested objects, output schema exists) the description is minimally adequate. However, the absence of usage guidelines and behavioral details reduces completeness. The output schema likely covers return values, so that gap is partially filled.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, file_id, is described as 'by ID' but the description adds no detail about what a valid file_id looks like, how to obtain it, or constraints. Since schema description coverage is 0%, the description provides minimal extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get file metadata by ID' uses a specific verb and resource, and clearly distinguishes from sibling tools like get_file_download_url or get_file_history by specifying 'metadata'. It leaves no ambiguity about what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternative file-related tools such as get_file_download_url or get_file_history. The agent must infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description is the sole source. It does not disclose requirements (e.g., does a preview request need to exist?), side effects, or error conditions. While 'get' implies read-only, this is not explicitly stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with key information. Every word is meaningful; no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Does not explain how this tool fits with related tools like 'create_file_preview_request' and 'get_file_preview_info_by_version'. Output schema exists but is not referenced; no mention of what happens if no preview is available. Incomplete for an agent to use correctly without additional knowledge.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has one parameter 'file_id' with 0% description coverage. The description does not explain the parameter's purpose or any constraints beyond the schema. Although the parameter name is self-explanatory, the description fails to provide additional context such as format or required state.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get preview info') and the specific outputs ('status and URL') for a file. It distinguishes from sibling tools like 'create_file_preview_request' and 'get_file_preview_info_by_version' by focusing on retrieval of existing preview info.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as 'get_file_preview_info_by_version' or 'get_file_download_url'. No prerequisites like needing to first create a preview request are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states the basic read operation without disclosing any behavioral traits like permissions, side effects, rate limits, or return format details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficient at one sentence with no verbosity. However, it could include more useful information without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The presence of an output schema reduces the need to explain return values. However, the description lacks usage context, parameter guidance, and fails to differentiate from siblings adequately. It is minimally adequate for a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should explain parameters. It does not add meaning beyond the parameter names 'file_id' and 'version_id', which are self-explanatory but lack context (e.g., source of version_id).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get'), the resource ('preview info'), and the scope ('for a specific version of a file'). This distinguishes it from sibling tools like 'get_file_preview_info' and 'get_file' by explicitly mentioning version specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., 'get_file_preview_info' for current version). No context about prerequisites, such as how to obtain the version_id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It only states the basic action without mentioning that it is a read-only operation, what permissions might be required, or what happens if the ID does not exist (e.g., returns null vs error).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (5 words) and free of clutter, but it sacrifices informational value. It is appropriately concise for a simple tool, but could include a bit more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is an output schema, the description need not detail return values, but it should at least state that it returns a matter object. The description is too minimal given the existence of many similar sibling tools; it does not help the agent differentiate when to use this versus other get tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage; the parameter 'matter_id' lacks a description. The tool description repeats 'by ID' but does not add any detail about the format, expected length, or meaning of the ID, failing to add value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a matter by ID' clearly states the verb (get), resource (matter), and the selection criterion (by ID). It distinguishes from list_matters (which returns multiple matters) and other get tools that target sub-resources like get_matter_archive or get_matter_billing_configuration.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use or alternatives. The description implies usage when a matter ID is available, but does not mention that list_matters is for browsing or that other matter-specific get tools exist for different needs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It only states the basic operation, omitting details like side effects (likely none), authentication requirements, error handling, or rate limits. For a read operation, transparency is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, front-loaded with key information. No unnecessary words. However, it could be slightly more informative without harming conciseness (e.g., 'Returns the full memo object').
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema, the description does not need to detail return values. The tool is simple with one parameter, so the description adequately covers the basic function. However, it lacks context about memo structure or usage scenarios, making it minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, meaning the description adds nothing beyond the input schema. The single parameter 'memo_id' is self-explanatory but the description does not elaborate on its format, constraints, or required status. The description fails to compensate for the low schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a memo by ID' clearly states the action (get), the resource (memo), and the method (by ID). It distinguishes itself from sibling tools like list_memos_on_matter (list multiple) and create_memo (create).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., list_memos_on_matter, search) or when not to use it. An agent must infer context solely from the verb 'get'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description only implies a safe read operation, but lacks details about authorization, rate limits, or side effects. With no annotations, the description should provide more behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action, no redundant words. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description is adequate for a simple get-by-ID tool. It could mention that it returns the stage set object, but it's not required. Still appropriate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description does not explain the parameter 'stage_set_id' beyond a vague 'by ID'. No format, type hints, or examples are given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get', the resource 'stage set', and the method 'by ID'. It distinguishes from sibling tools like 'list_stage_sets' and 'get_stage_in_set'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. It does not mention any prerequisites, exclusions, or comparison to similar tools like 'list_stage_sets'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. The description says 'list all items' but omits details like pagination, filtering, sorting, or whether it is a read-only operation. This lack of transparency limits the agent's understanding.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, no filler, front-loaded with key information. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists (not shown), so return values are covered. However, the tool lacks annotations, and the description does not address behavioral aspects like idempotency or rate limits. Given the simplicity and existence of output schema, it is marginally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It adds that the matter_id is the matter on which to list items, but does not elaborate on format or constraints beyond the schema's 'string' type. The context from the tool name and single parameter makes it minimally adequate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all items (checklist items, workflow steps) on a matter,' specifying the verb and resource. It provides examples that help distinguish from siblings like 'get_matter_item' (singular) and 'list_tasks' (task-specific). However, it could be more precise about the scope of 'items'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'list_tasks' or 'list_files_on_matter.' The description implies it is for checklist items and workflow steps, but does not explicitly state usage context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden but fails to disclose behavioral traits like whether the action is destructive, reversible, or requires specific permissions. It does not explain side effects on trust account balance.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words, though it could be more informative without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description need not explain return values, but it lacks context on what 'protect' entails operationally (e.g., fund hold duration, impact on balances). Adequate but with gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description only implicitly references 'account_id', 'matter_id', and 'amount' without elaborating on constraints or the optional 'reason' parameter. It adds minimal meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Protect (reserve)' and the resource 'funds in a trust account for a matter', distinguishing it from siblings like 'unprotect_funds' and 'get_protected_bank_account_balance'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as prerequisites, restrictions, or comparison to other fund-related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility for behavioral disclosure. 'Remove a user from the firm' indicates destruction but lacks details on permissions, reversibility, cascading effects, or what 'remove' entails (e.g., deletion vs.deactivation).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at 8 words, front-loading the action. Every word serves a purpose with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description omits critical context for a destructive operation, such as result status, error cases, or implications for related data (e.g., matters, contacts). It falls short of being minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage for the only parameter 'user_id'. The tool description adds no explanation of what the parameter represents or any format constraints, leaving the agent to infer meaning from the parameter name alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Remove a user from the firm' uses a specific verb and resource, and the tool name directly conveys its action. Among sibling tools, it uniquely targets user removal, distinguishing it from other remove/delete operations on different entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, consequences, or situations where other tools (e.g., deactivation, mapping deletion) might be preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description does not disclose key behavioral traits such as whether it invalidates previous invitations, idempotency, rate limits, or authentication requirements. Since no annotations are present, the description carries the full burden, and it fails to provide sufficient context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise and front-loaded. However, it is too minimal, omitting necessary details that could be included without excessive length. Every sentence earns its place, but more information would be beneficial.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one parameter and an output schema, the description is minimally adequate but lacks details on return values and side effects. It covers the essential action but leaves gaps in behavior specification.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With schema description coverage at 0%, the tool description must compensate but does not. It only states 'Resend an invitation email to a user,' which adds no meaning beyond the parameter name 'user_id'. No format or constraints are given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'resend' combined with the resource 'invitation email' and target 'user', making the action clear. It distinguishes itself from sibling tools like 'create_user' and 'send_portal_message'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool, prerequisites (e.g., user must exist and have prior invitation), or when not to use it. There is no mention of alternatives or context-specific conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must bear the full burden of behavioral disclosure. It fails to mention permissions, side effects, reversibility, or preconditions such as sufficient protected balance. For a financial mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of a single sentence that directly communicates the tool's purpose. No unnecessary words or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (financial operation with three required parameters) and the absence of parameter descriptions and behavioral context in both schema and description, the description is incomplete. It does not clarify return values or provide enough context for safe use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage (no param descriptions), and the tool description adds no information about the three required parameters (amount, matter_id, account_id). The description fails to compensate for the schema's lack of detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Release') and the resource ('protected (reserved) funds') with the outcome ('back to available balance'). It distinguishes itself from the sibling tool 'protect_funds' by implying the opposite operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when funds need to be released back to available balance, but does not explicitly state when to use this tool versus alternatives like 'protect_funds' or any prerequisites or caveats.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral disclosure. It confirms creation but fails to mention side effects, validation rules, default values, permissions, or error behavior. The description is minimal and does not provide meaningful transparency beyond the obvious mutation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no fluff. The first sentence is slightly redundant with the tool name, but the subsequent sentences are concise and informative. Efficient for a simple create tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a create tool with 6 optional parameters and an output schema (not shown), the description covers key guidance for the two contact types but omits email and phone. It does not explain return value, though output schema may cover that. Overall, adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains contact_type and its effect on required fields (first_name/last_name vs company_name), but ignores email and phone parameters entirely. This partial coverage adds value but leaves gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a contact' and differentiates between person and company types. However, it does not distinguish from the sibling tool 'update_contact' nor specify the full scope of what is created (e.g., email, phone fields are missing from the description).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides guidance on which fields to supply based on contact_type ('person' vs 'company'), but lacks explicit context on when to use this tool versus alternatives like update_contact or create_lead. Usage is implied rather than explicitly stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden. It only states the basic action and a parameter hint, but omits side effects, required permissions, error conditions, or behavior when name conflicts occur.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences and front-loads the main purpose. However, it could be more structured without losing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters, low schema coverage, and no annotations, the description is too sparse. It does not cover error conditions or context dependencies like matter existence, despite an output schema being present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must provide parameter context. It only explains parent_folder_id behavior, leaving matter_id and name unexplained despite being required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('create a folder') and the context ('in a matter'), distinguishing it from other create tools among siblings. It also provides a specific parameter hint for parent_folder_id.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives minimal usage guidance by noting to omit parent_folder_id for root folders, but it does not explain when to use this tool versus alternatives like patch_folder or delete_folder, nor does it mention prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so description must cover behavior. It specifies that the task is visible to client and gives a date format, but doesn't mention return value, error handling, or side effects. Output schema exists but not referenced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise, no filler. But sacrifices content for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Inadequate for 5 parameters with no schema descriptions. Missing parameter meanings and behavioral context. Output schema exists but not leveraged in description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should explain parameters. It only addresses due_date format. Required parameters title and matter_id are not described.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Create' and resource 'client portal task', with a note that it is visible to the client. This distinguishes it from other task creations like create_task or create_subtask.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use or when-not-to-use guidance. Does not mention alternatives or prerequisites. Just states the basic action.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description bears full responsibility for behavioral disclosure. It only explains the sign convention for the amount parameter, but fails to mention idempotency, side effects (e.g., transaction creation), or any constraints beyond that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences. It front-loads the purpose and provides a key parameter detail without wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the existence of an output schema, the description omits important context such as the meaning of the required parameters account_id and matter_id, and the purpose of the optional transaction_type. For a tool with 5 parameters and 3 required, this is insufficient to fully understand usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It adds meaning only for the 'amount' parameter (positive/negative). The three required parameters (account_id, matter_id, amount) and two optional ones lack any semantic explanation beyond what the schema's name suggests. This is partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a bank account transaction', using a specific verb and resource. This distinguishes it from siblings like create_contact or create_matter. The additional explanation of the amount parameter further clarifies the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites, exclusions, or context. The description lacks any usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must cover behavioral traits. It only states the basic action, failing to disclose what happens if the group_id does not exist, required permissions, or any side effects. With zero annotation coverage, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with direct verb-object structure. Front-loaded and contains no superfluous words. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the return value does not need full description. However, the description omits context on uniqueness, error behavior, and permissions. For a simple retrieval tool, it is minimally adequate but leaves gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage for the only parameter. The description adds that the group is retrieved by ID, but does not explain the format, source, or validation of the group_id. Minimal added value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Get' and resource 'authorization group by ID'. It distinguishes from sibling 'list_authorization_groups' which returns multiple groups, and from 'create_authorization_group' which creates. The purpose is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'list_authorization_groups' or 'get_authorization_policy'. No exclusions or prerequisites mentioned. The description provides no context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, and the description fails to disclose behavioral traits such as read-only nature, permission requirements, or whether it returns all reminders globally or filtered. The phrase 'get all' is ambiguous.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no superfluous words. It serves its purpose concisely.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While an output schema exists (not shown), the description remains minimal. It fails to mention return structure, ordering, or scope of reminders, leaving gaps for an AI agent to infer.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not elaborate on the event_id parameter (e.g., format, required scope). The parameter is self-explanatory but the description adds no value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('all reminders for a calendar event'), distinguishing it from sibling tools like create_event_reminder and delete_event_reminder.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidance is provided. The description does not specify when to use this tool (e.g., before modifying reminders) or how it differs from alternatives like list_events or get_event.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, and the description does not disclose behavioral traits such as idempotency, error handling (e.g., behavior when fee_id not found), or required permissions. The description adds no value beyond the input schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no extraneous words. It conveys the essential information without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the existence of an output schema and the simplicity of the tool (single parameter, no nesting), the description is minimally adequate. However, it lacks details about null handling or authorization that would be helpful for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should compensate. It only restates that the parameter is an ID, which is already implied by the property name 'fee_id'. No additional meaning (format, length, example) is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get'), the resource ('a fee entry'), and the method ('by ID'). It distinguishes itself from sibling tools like list_fees, create_fee, update_fee, etc., which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, it does not explain when to use get_fee vs list_fees, or that it requires an existing fee_id. Usage context and exclusions are missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. It only states the action and parameter, omitting details on permissions, side effects, rate limits, or output format. This is insufficient for a mutation-free retrieval tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single 8-word sentence, which is concise but front-loads the action. However, it sacrifices necessary detail for brevity, leaving gaps in usage and parameter guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the tool is simple (one parameter, output schema exists), the description lacks context about the resource's semantics, typical use cases, or relationship to sibling tools. An existing output schema reduces need for return info, but missing annotations and parameter details lower completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, requiring the description to compensate. The description mentions 'by ID' but does not explain the 'mapping_id' parameter's format, constraints, or meaning beyond its name. This adds minimal value over the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get a specific firm user mapping by ID.' Verb 'Get' matches read operation, resource 'firm user mapping' is specific, and 'by ID' indicates singular retrieval. Sibling 'get_firm_user_mappings' (plural) reinforces distinct scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for retrieving a single mapping by ID but does not explicitly state when to use this tool versus alternatives like 'get_firm_user_mappings' or 'delete_firm_user_mapping'. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description fails to disclose behavioral traits such as whether the operation is safe/read-only, permission requirements, or error handling (e.g., what happens if ID does not exist).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Five-word sentence is maximally concise. No wasted words; front-loaded with purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get tool with one parameter and an output schema, the description is adequately minimal. Could mention that it returns a staff member object, but output schema likely covers that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'staff_id' has no description in schema (0% coverage), and the description only says 'by ID' without explaining format or constraints. Does not add meaningful information beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get a staff member by ID' with a specific verb and resource. It distinguishes from sibling tools like create_staff_member, update_staff_member, and search_staff.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like search_staff or list_staff. No context about prerequisites or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description should disclose behavioral traits. It only says 'list' which implies read-only, but does not mention rate limits, pagination, or other important behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and to the point, but could be structured better by mentioning the parameters or providing a note about pagination.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple operation and presence of an output schema, the description is adequate. However, it could mention that it returns a list of bank accounts.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description does not explain the parameters (limit, offset). It adds no meaning beyond what is in the schema structure.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list), resource (bank accounts), and distinguishes types (trust, operating). It is specific and differentiates from sibling tools like get_bank_account.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing all bank accounts for the firm but does not explicitly state when to use it over alternatives like get_bank_account or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description should disclose behavioral traits. It states it is a list operation (read-only) but omits details on pagination behavior (limit/offset defaults), ordering, or any constraints on the returned data. This leaves significant behavioral ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys the core purpose. No unnecessary words, but extremely brief; additional context could be added without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, the description need not detail return values. However, for a list tool with three parameters (two undocumented), the description should cover pagination semantics and any constraints. It does not, making it incomplete for reliable tool invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, meaning the schema provides no parameter descriptions. The tool description adds meaning only for matter_id ('Filter by matter_id for matter-specific fees'), but fails to explain the limit and offset parameters (e.g., maximum values, default behavior). This is insufficient to compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists fee entries, specifies they represent billable time, and mentions filtering by matter_id. This provides a specific verb-resource combination and differentiates from sibling fee-related tools like create_fee or patch_fee.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving fees, optionally filtered by matter_id. However, it does not provide explicit guidance on when to use this tool versus other list tools or fee-related actions, nor does it mention prerequisites or limits.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits like read-only, pagination, or permissions. It only states the action without clarifying safety or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words or redundancy, clearly conveying the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, output schema present), the description is minimally viable. It lacks context on what 'matter type categories' are or how they relate to other entities, but the output schema compensates for return value details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema description coverage is 100%. With no parameters to explain, the description cannot add parameter semantics, achieving a baseline of 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all matter type categories' with a specific verb and resource. It distinguishes from sibling tools like 'list_matter_types' through the resource name, though it does not explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as 'list_matter_types' or other list tools. The context signals show many siblings, but the description lacks usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only mentions 'generate the document' but does not disclose side effects, required permissions, whether it overwrites existing documents, or any rate limits. The description adds minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 11-word sentence, which is concise and front-loaded. It covers the key action and resources without superfluous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has two simple parameters and an output schema, the description provides adequate but minimal context. It does not describe the output or any return behavior, but the output schema exists to fill that gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains 'matter_id' is the matter data and 'layout_id' is from the template, providing basic context beyond the parameter names. However, it does not explain valid formats or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it 'Execute a layout merge (generate the document from template + matter data).' This specifies the verb 'merge' and the resource 'layout', distinguishing it from sibling tools like add_layout_to_matter or get_layout_design.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like add_layout_to_matter or get_layout_on_matter. The description does not mention prerequisites, exclusions, or context for use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states that the tool moves a folder, but does not disclose important details such as whether the move is recursive (subfolders and files moved), any restrictions on the target parent (e.g., cannot be a descendant), permissions required, or what happens to folder contents. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise. However, it could be slightly more detailed without losing conciseness (e.g., adding 'Use to change the parent folder of a folder'). It is front-loaded and gets the main point across immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (2 params, likely straightforward behavior), the description should at least clarify whether the move is recursive and any constraints on the target parent. With no annotations and minimal description, the tool definition is not complete enough for an AI agent to use correctly without additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% parameter description coverage, so the description must compensate. However, it only mentions moving 'to a different parent,' hinting at the 'parent_folder_id' parameter, but does not explain the 'folder_id' parameter or the meaning of an empty string default for 'parent_folder_id' (e.g., move to root?). The description adds minimal semantic value beyond what can be inferred from parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool moves a folder to a different parent. It uses a specific verb ('Move') and identifies the resource ('folder') and the change ('different parent'). This distinguishes it from sibling tools like 'update_folder' (which would modify other properties) and 'create_folder' (which creates new folders).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for moving folders, but it does not explicitly state when to use this tool versus alternatives. For example, it does not mention that 'update_folder' should be used for renaming or other modifications, nor does it provide any prerequisites or conditions for moving (e.g., permission requirements).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It implies a destructive action (removing a tag) but doesn't clarify if this permanently deletes the tag or just disassociates it, nor does it mention permissions or side effects. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences, no unnecessary words, and front-loads the core purpose. Every part is relevant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists, return values are not needed. However, for a removal operation, more context about irreversibility or confirmation would improve completeness. It meets the minimum viable standard but has gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It explains 'tag_id' as 'the ID of the tag to remove' but does not explain 'matter_id'. This leaves one required parameter undocumented, failing to fully compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action as 'Remove a tag from a matter', which is a specific verb on a specific resource. This distinguishes it from sibling tools like 'add_matter_tags' and 'remove_contact_tags'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool or when to use alternatives like 'add_matter_tags'. There are no exclusions or context about prerequisites (e.g., does the tag need to exist on the matter?).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, and the description does not disclose behavioral traits such as whether the removal is reversible, what side effects occur, or authorization requirements. For a destructive operation, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with zero waste. It front-loads the action and resource effectively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with two required parameters and an output schema. The description covers the basic purpose but lacks behavioral detail; given the simplicity, it meets a minimum threshold.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description adds no meaning to the parameters (matter_id, role_id). The description fails to compensate for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Remove a role from a matter' uses a specific verb and resource, clearly distinguishing it from sibling tools like 'add_role_to_matter' and 'remove_relationship_from_role'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs. alternatives (e.g., when to use 'update_role_on_matter' instead). The context is implied but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided. The description indicates a mutation ('update') but does not disclose any behavioral traits such as side effects, required permissions, or response behavior. For a mutation tool, more transparency is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that directly conveys the purpose. No extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (3 required parameters, all scalars) and the existence of an output schema, the description is adequate but lacks usage context and parameter clarification. A more complete description would include when to use and parameter details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description mentions 'lead time' but does not explicitly link it to the 'minutes_before' parameter. No additional meaning is added beyond the schema's property definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update an event reminder's lead time' clearly states the action (update) and the resource (event reminder's lead time), distinguishing it from sibling tools like create_event_reminder, delete_event_reminder, and get_event_reminders.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., create_event_reminder or delete_event_reminder). The description lacks context for proper tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must fully convey behavior. Only states 'create a calendar event' with no mention of side effects, permissions, or success/error behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, directly to the point. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters and no annotations, description is sparse. Output schema exists, so return not needed, but behavior and parameter details are incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%; description adds ISO 8601 format for start_date/end_date, but does not explain other 4 parameters (name, notes, location, matter_id). Partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Create a calendar event', which is a specific verb and resource. This distinguishes it from sibling create tools like create_contact or create_event_reminder.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use or not use this tool versus alternatives. Lacks context for differentiation among many sibling create tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only says 'delete' without disclosing side effects, reversibility, permission requirements, or whether it's a hard delete. The description carries the full burden but is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence of 6 words, very concise with no wasted words. However, it could include more essential information without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and an output schema, the description is adequate for basic understanding but lacks context on behavior such as idempotency, error states, or cascading effects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage for the only parameter 'fee_id'. The description says 'by ID' but does not clarify what the ID represents, how to obtain it, or any constraints, adding little value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool deletes a fee entry by ID, with a specific verb and resource that distinguishes it from other fee-related tools like create_fee, update_fee, or list_fees.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The purpose is obvious but no guidance is given on when to use delete versus alternatives like patch_fee or update_fee, nor are there prerequisites or effects mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. However, it only states 'Remove a document attachment' without clarifying whether this permanently deletes the file, requires permissions, or has side effects like unlinking only. This is insufficient for a destructive operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. However, it could be restructured to include essential behavioral details without becoming verbose, so it scores slightly below perfect.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema but the description does not mention it or the return value. Given the low complexity (2 parameters, no nested objects), the description is minimally adequate but leaves gaps in understanding the tool's effect and response. It is complete enough for a simple delete but lacks context about output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, but it adds nothing beyond the parameter names. It does not explain that 'task_id' identifies the task and 'document_id' identifies the attachment, nor does it specify expected formats. The description adds no value over the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Remove a document attachment from a task' uses a clear verb-resource combination, clearly stating what the tool does. It sufficiently distinguishes from siblings like 'delete_task' (different resource) and 'create_task_document' (opposite action).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool versus alternatives such as 'delete_file'. The context of when to remove a document attachment versus deleting the file itself is implied but not stated, leaving the agent without specific usage boundaries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral traits. It does not disclose expiration time, size limits, authentication requirements, or whether the URL is time-limited, which are critical for agent invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words, but it is under-specified, sacrificing clarity for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has one parameter and an output schema, so the description is minimally adequate. However, it could be improved by stating return value properties (e.g., URL with expiration) to fully equip the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the 'file_id' parameter beyond its name. It adds no meaning about what ID (file version ID?) is expected or how to obtain it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool obtains a pre-signed upload URL for a file version, using specific verb 'Get' and resource 'pre-signed upload URL'. It distinguishes from sibling 'get_file_download_url' which is for downloading.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for uploading file versions, but lacks explicit when-to-use or when-not-to-use guidance, and does not mention alternatives like 'get_file_download_url'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must bear the full burden. It only states 'update' without detailing mutation behavior, return values, permissions, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, front-loaded with key information, but could add a bit more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of schema descriptions and the presence of an output schema (unseen), the description leaves gaps in return values, error handling, and usage notes, making it insufficient for a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions 'content or subject,' clarifying the optional parameters, but does not describe the required memo_id or provide format constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (update) and resource (memo) along with the specific fields (content or subject), which distinguishes it from create_memo and delete_memo among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives or prerequisites like needing an existing memo_id, but it implies usage for modifying existing memos.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description is minimal but adequate for a simple read operation. No annotations are present, so the description carries full burden; however, it neither contradicts annotations nor adds behavioral details beyond the obvious get operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It is front-loaded and efficiently conveys the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get tool with one parameter and an output schema, the description is minimally complete. It does not elaborate on the return value or behavior, but the output schema likely covers return details. However, it could be enhanced with more context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and the description adds no information about the single parameter 'contact_id' beyond what the schema provides. The description does not compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get tags' and the resource 'on a contact.' It directly indicates the tool's function, distinguishing it from sibling tools like add_contact_tags and remove_contact_tags.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of context, prerequisites, or exclusions. The description is too sparse to guide tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must disclose behavioral traits. It only states the basic action, missing important details such as whether the URL expires, required permissions, or if it is a read-only operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence (8 words) with no wasted text. The description is front-loaded and efficiently conveys the essential purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and an output schema, the description is minimally adequate. It does not explain the nature of the pre-signed URL (e.g., temporary, scoped) or any error conditions, leaving gaps for a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'file_id' has no schema description (0% coverage), and the description does not add meaning beyond its name. While 'file_id' is somewhat self-explanatory, no additional context (e.g., format, source) is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('get'), the output ('pre-signed download URL'), and the resource ('file'). It distinguishes from sibling tools like 'get_file_upload_url' (upload) and 'get_file' (metadata), with 'download' specifying the operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when a downloadable link is needed, but does not explicitly state when to use this tool versus alternatives (e.g., 'get_file' or 'get_file_preview_info'), nor does it provide exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits, but it only states 'Get'—implying a read-only, idempotent operation. It does not mention permissions, error handling (e.g., not found), or response behavior beyond what the output schema provides.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no extraneous words. It is front-loaded with the core action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get tool with an output schema, the description is minimally adequate. However, it lacks details on when to use this versus sibling get tools (e.g., get_layout_design) and does not cover behavioral assumptions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description adds no details about 'matter_id' or 'layout_id' beyond their names. While the tool name and context imply these identify a matter and a layout, no format, constraints, or examples are provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('specific layout instance on a matter'). It distinguishes from sibling tools like 'list_layouts_on_matter' (which lists all) and 'get_layout_design' (which gets design metadata).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance is provided. The usage is implied from the verb 'Get' (i.e., retrieve a specific layout by ID), but no alternatives or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states the purpose without disclosing behavioral traits like idempotency, read-only nature, or side effects. For a getter, the lack of explicit read-only indication reduces transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that efficiently conveys the tool's purpose with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (not shown), the description lacks details on prerequisites or the nature of the billing configuration. It is minimally adequate but could benefit from mentioning that the matter must exist.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 0%, and the description adds no additional meaning to the 'matter_id' parameter beyond what the schema provides. No format, source, or example is given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('billing configuration') with the scope ('for a matter'), making it distinct from siblings like 'get_matter' and 'update_matter_billing_configuration'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool versus alternatives, such as 'get_matter' or 'update_matter_billing_configuration'. The usage is implied but not detailed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description implies a read-only operation via 'get', but does not disclose any behavioral traits beyond that. With no annotations, the description carries full burden but lacks details on safety, idempotence, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no waste. Every word is purposeful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a simple get operation and an output schema defining return values, the description is mostly adequate. It does not explain what a notification is or error conditions, but these are minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description mentions 'by ID', linking the sole parameter to its purpose, but adds minimal meaning beyond the parameter name 'notification_id'. Schema coverage is 0%, so description should compensate more.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a notification by ID' clearly states the action (get) and the resource (notification) with the retrieval method (by ID). It is specific but does not differentiate from other get_by_ID tools among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like create_notification or list_*. The description gives no context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral traits. It only states it requests a URL, but does not disclose authentication needs, error behavior, rate limits, or that it is a read operation. The description is minimal and leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence (12 words) that is front-loaded with the key action and context. No superfluous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is a simple URL retrieval with one parameter and an output schema exists, the description is somewhat adequate but lacks behavioral context and does not explain the return value beyond 'launch URL'. It is minimally complete for an experienced user but not for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one parameter (plugin_id) with 0% description coverage. The description does not add any meaning to the parameter (e.g., what format or where to find the plugin ID). Since coverage is low, the description should compensate but fails to do so.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Request the launch URL) and the resource (plugin). It also specifies the use case (for embedded iframe display), distinguishing it from sibling tools like get_plugin or get_plugin_subscription.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage (when you need a plugin's launch URL for iframe display) but does not explicitly state when to use it over alternatives or provide any exclusions. Among siblings, get_plugin and get_plugin_subscription exist, but no comparison is offered.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully convey behavioral traits. It only states 'Get a specific role' but does not disclose error handling, permissions needed, or what happens if the role does not exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with no redundant words. It is front-loaded and efficiently communicates the core function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple getter, the description is minimally adequate but lacks detail such as parameter formats, return value structure (output schema exists but is not referenced), and any prerequisites or constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, and the description adds no explanation of the parameters beyond their names. The agent receives no guidance on how to construct or interpret the matter_id and role_id values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('a specific role on a matter'), effectively distinguishing it from listing all roles (get_roles_on_matter) and mutation tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies that this tool is for retrieving a single role when you have its ID, but it does not explicitly state when to use it versus alternatives like get_roles_on_matter or other role-related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description bears full burden. It only states the action without disclosing any behavioral traits (e.g., read-only nature, permissions required, side effects). The verb 'list' implies read-only, but explicit disclosure is lacking.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no extraneous information, making it concise and easy to process.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description doesn't need to detail return values. However, it could mention that it returns a list of roles, and possibly note pagination. As is, it's minimally viable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds no explanation for the matter_id parameter (format, usage, or expectations). The minimal description does not help the agent understand the parameter beyond its name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('list') and the resource ('all roles assigned to a matter'), distinguishing it from sibling tools like get_role_on_matter (singular) and get_relationships_on_matter.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The usage is implied (listing roles for a matter), but there is no explicit guidance on when to use this tool versus alternatives, such as when to use get_role_on_matter for a specific role.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavioral traits. It does not state that the operation is read-only, whether it requires permissions, or any side effects. Pagination is implied via limit/offset params but not explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences front-load the purpose. No extraneous information. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists (not shown) so return values may be covered elsewhere. However, description lacks details on pagination behavior, default ordering, or error handling. Adequate for basic use but incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It only explains matter_id as a filter, leaving limit and offset unexplained. Does not add meaning beyond the schema for those parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists calendar events and can filter by matter_id. The name 'list_events' combined with the description distinguishes it from sibling tools like create_event, update_event, and get_event.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for listing events with optional matter_id filter, but no explicit guidance on when to use vs alternatives like get_event for single events, or prerequisites. No mention of when to use without filter.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must carry full burden. It does not disclose pagination behavior (limit/offset), rate limits, or error cases. Only hints at filtering, lacking detail on results structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, zero wasted words, front-loaded with the core action. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema is present so return values need not be explained, but description lacks guidance on pagination (limit/offset) and does not mention typical listing behavior like ordering or result count. Adequate but minimal.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage. Description only explains matter_id, ignoring limit and offset parameters, failing to compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'list' and resource 'invoices', with a specific filtering option (matter_id), distinguishing it from other list_* sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides basic guidance on using matter_id for filtering, but does not mention when to prefer this over get_invoice or mention pagination or ordering, and lacks exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It implicitly indicates a read operation via 'list', but does not disclose behavior like pagination, ordering, or permissions. For a simple list tool, this is adequate but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It is front-loaded with the core functionality.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one parameter and an output schema, the description covers the essential purpose. However, it lacks usage guidelines and parameter details. It is adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not add any meaning to the sole parameter matter_id beyond its name. The parameter is self-explanatory, but the description fails to compensate for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'layout instances on a matter', and clarifies they are 'document merge records'. This distinguishes it from siblings like list_layout_designs (which lists designs, not instances) and get_layout_on_matter (which gets a single layout).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as get_layout_on_matter or list_layout_designs. There is no mention of exclusions or appropriate contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It fails to mention pagination behavior despite having 'limit' and 'offset' parameters, nor does it disclose any prerequisites, permissions, or output characteristics beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence with the verb 'list' upfront. No unnecessary words, making it highly concise and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the basic purpose but lacks details on pagination behavior (given the pagination parameters) and does not differentiate from 'list_matter_type_categories' in terms of scope. With an output schema present, return values are less critical, but overall completeness is adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description does not explain the 'limit' or 'offset' parameters. The agent must infer from parameter names and defaults, which is insufficient for effective use.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'matter types', indicating a listing operation. It distinguishes itself from sibling tools like 'get_matter_type' (single item) and 'list_matter_type_categories' (different resource).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for listing all matter types, but does not provide explicit guidance on when to use this vs. alternatives like 'get_matter_type' or 'list_matter_type_categories'. No usage context or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must disclose behavioral traits. It only states the purpose and does not mention auth requirements, pagination, ordering, or whether the list is system-wide. The output schema exists but is separate from the description.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no redundant words. It is front-loaded with the key action and resource, making it efficient for quick understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description is reasonably complete for a simple listing operation. However, it could mention if the list is paginated or system-wide. The sibling tools provide context but are not referenced.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and schema description coverage is 100%, so the baseline is 3. The description adds no parameter info, but none is needed since the schema already covers it trivially.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all stage sets' with a parenthetical clarification that these are workflow stage groups. It uses a specific verb+resource pattern and implicitly distinguishes from sibling tools like 'get_stage_set' by indicating it lists all rather than retrieving a single set.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'get_stage_set' or 'list_matter_stage_mappings'. The description does not mention context, prerequisites, or scenarios where this tool would be preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It does not disclose behavioral traits such as read-only nature, rate limits, or auth requirements. Only states the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (5 words), front-loads the verb, and contains no unnecessary words. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the tool having an output schema, the description provides no context about the return structure, what a layout design is, or any related business logic. For a simple get, it is minimal but still incomplete given the lack of annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, requiring compensation. The description mentions 'by ID' but adds no detail about the design_id's format, source, or constraints. It barely adds value over the schema parameter name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a layout design by ID' clearly specifies the verb ('Get'), the resource ('layout design'), and the key parameter (by ID). It effectively distinguishes from siblings like 'list_layout_designs' and 'merge_layout'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage: when you have a specific design_id and need a single layout design. It provides clear context but does not explicitly state when not to use it or list alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It only states the action without disclosing behavioral traits such as read-only nature, pagination, or response structure. The output schema exists but is not described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of five words, with no redundancy. It is maximally concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, output schema present), the description is minimally adequate. However, it could be more helpful by clarifying the scope (e.g., firm-wide) or mentioning that it returns all mappings without filtering.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the input schema coverage is 100%. The description adds no parameter information, but since there are none, baseline 4 is appropriate. It does not provide additional context about what the mappings represent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies the verb 'List' and the resource 'matter-to-stage mappings', clearly indicating what the tool does. It distinguishes from siblings like get_matter_stage (singular) and list_stage_sets, which are conceptually different.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidance is provided. The description does not indicate when to use this tool versus alternatives like get_matter_stage or list_matters, nor does it state any prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the full burden. It only says 'list all available plugins' without disclosing pagination, filtering, permissions, or return format. Minimal behavioral insight.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no unnecessary words. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless listing tool, the description is adequate but minimal. It does not mention output schema behavior or any constraints like listing all vs. paginated results. An output schema exists but is not referenced.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so the description needs no parameter details. Baseline for 0 params is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'plugins'. It is distinct from sibling listing tools that target other entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like list_plugin_subscriptions or other listing tools. The description is a bare statement with no context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits like read-only nature or pagination. It only states 'List', implying a read operation, but lacks explicit safety details or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, clear sentence that efficiently conveys the tool's function with no redundant or missing words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential purpose but omits expected details like return format (though output schema exists), potential filtering, or pagination behavior. Adequate for such a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so the description does not need to augment schema information. Baseline for zero parameters is 4, and the description adds no unnecessary detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' and resource 'active webhook subscriptions', clearly distinguishing it from sibling tools like get_webhook_subscription (single) and create_webhook_subscription.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as get_webhook_subscription or list_webhook_event_types, nor any conditions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Only states basic mutation intent; lacks disclosure of side effects, required permissions, return behavior, or what happens to omitted fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with primary action, followed by a specific parameter hint. No unnecessary words; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Minimally adequate for a simple update tool with 3 parameters and an output schema. Lacks return value description, prerequisites, and alternative tool guidance, which are gaps given the lack of annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description adds value by clarifying that 'active' accepts 'true' or 'false'. However, no guidance is given for 'url' format or 'subscription_id' source, leaving gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Update' and the resource 'webhook subscription'. Specifies the updateable fields (URL or active state) and distinguishes from sibling tools like create_webhook_subscription and delete_webhook_subscription.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for modifying existing subscriptions, but does not explicitly state when to use versus alternatives or provide prerequisites. No exclusion criteria or contextual guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided. The description discloses that the tool creates a resource via POST and the isLead flag, but does not detail side effects, permissions, or idempotency. Output schema exists to cover return values.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. First sentence states purpose and implementation, second provides parameter guidance. Efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, the description covers input semantics reasonably but omits details about optional parameters, defaults, and prerequisites. Sufficient for basic use but not fully comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description carries the burden. It explains that matter_type_id comes from list_matter_types and client_id must be an existing contact ID. This adds significant meaning beyond the schema's empty defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a lead, which is a matter with isLead=true. It mentions the underlying API endpoint. This is specific and distinguishes from create_matter by implication, but lacks explicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides hints on parameter sources (list_matter_types, existing contact IDs), but does not explicitly state when to use this tool versus alternatives like create_matter. No when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided. The description does not mention that this is a read-only operation, nor does it disclose any behavior like pagination, limits, or sorting. For a list operation targeting a potentially large number of records, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no extra words. It is clear and front-loaded with the action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists, the description is minimally adequate. However, it does not explain the concept of 'user mappings' or what the output contains, relying on the schema. For a simple listing, this is acceptable but could be slightly improved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and schema description coverage is 100% (since there are none). Baseline is 4 for zero parameters; the description does not add or need to add parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list', the resource 'user mappings', and the scope 'for this firm'. It distinguishes from sibling 'get_firm_user_mapping' which implies singular retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this vs alternative tools. Sibling 'get_firm_user_mapping' exists but is not mentioned. No context about prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must bear the full burden of behavioral disclosure. The description only states 'List all active plugin subscriptions', but does not disclose read-only nature, idempotence, error conditions, or any constraints (e.g., rate limits, authorization needs). While 'List' implies a read operation, explicit transparency is lacking.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no extraneous words. It is well front-loaded and efficiently conveys the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, output schema exists), the description is largely complete. It covers the main purpose and scope. However, it could briefly mention that the operation is read-only or that it returns a list of all active subscriptions in no particular order.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema provides complete coverage. According to guidelines, 0 parameters yields a baseline of 4. The description adds no param info, which is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('List'), the resource ('active plugin subscriptions'), and the scope ('for this firm'). It distinguishes from sibling tools like 'list_plugins' (which lists plugins, not subscriptions) and 'get_plugin_subscription' (specific subscription).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives (e.g., 'list_plugins' or 'get_plugin_subscription'). No explicit context or exclusions are given, leaving the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only mentions toggling state with boolean strings, omitting details on side effects, idempotency, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, immediately conveys purpose and expected input format with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, so return values are covered; description is sufficient for a simple toggle tool, though it could mention partial update nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage 0%; description adds meaning that values should be 'true' or 'false', but does not clarify expense_id or combination of parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly specifies verb 'Toggle' and resource 'expense entry's billable or billed state' and distinguishes from sibling tools like 'update_expense' and 'patch_fee'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for toggling billable/billed state using 'true'/'false', but lacks explicit guidance on when not to use or alternatives like 'update_expense'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations; description only mentions PATCH method and partial update, failing to disclose permissions, idempotency, or effects on omitted fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, well-front-loaded sentence plus instruction; no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite output schema, no annotations and low schema coverage mean description should provide more context; missing prerequisites, parameter details, and result description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage; description mentions 'status or assignment' mapping to optional params, adding some meaning but lacking formats or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'partially update a lead status or assignment' and distinguishes from 'update_lead' for notes, making purpose specific and differentiating from sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly directs to use 'update_lead' for notes, implying this tool is for status and assignment; provides clear context but no further exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description is minimal and does not disclose side effects, authorization needs, or whether the operation is idempotent. For a creation tool, more behavioral context is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence of 11 words, front-loading the verb and resource with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is adequate for a simple two-parameter operation, but it lacks mention of prerequisites (e.g., existence of matter/layout design) and does not leverage the output schema to explain return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description's wording ('Add a layout design instance to a matter') implies the purpose of 'matter_id' and 'layout_design_id'. However, no additional details on constraints or format are given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Add') and clearly identifies the resource ('layout design instance') and target ('matter'), distinguishing it from sibling tools like 'add_contact_to_layout' or 'remove_layout_from_matter'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance, but the name and description imply adding a layout for document generation, which is distinct from other layout-related tools. Clear context without exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. While 'Get' implies a read operation with no side effects, it does not disclose auth requirements, rate limits, error conditions, or whether it returns cached vs. live data. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence without unnecessary words. It is front-loaded and efficient, earning its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required parameter) and the presence of an output schema, the description adequately explains the tool's purpose. However, it could mention that a valid subscription_id is required and typical return behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions 'by ID' mapping to subscription_id, but provides no format, source, or validation hints. This adds minimal value beyond the parameter name in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get'), the resource ('plugin subscription'), and the identifier ('by ID'). It effectively distinguishes this tool from siblings like 'list_plugin_subscriptions' (which lists all) and 'subscribe_to_plugin' (which creates).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you have a specific subscription ID and need its details, contrasting with sibling tools like list or create. However, it lacks explicit when-not or alternative guidance, which would elevate it to a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. 'List' implies a read operation with no side effects, but no details on output format, pagination, or ordering. Adequate for a simple list, but minimal behavioral context beyond the verb.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One short sentence, front-loaded with key information. No filler or redundant words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless list tool with output schema, the description is functional but misses the chance to mention scope differentiation (firm-wide vs matter-specific layout lists) and that output schema exists. Might confuse with sibling 'list_layouts_on_matter'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, and schema coverage is 100%, so baseline is 3. The description adds no extra parameter meaning (none needed). It neither improves nor degrades beyond baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'List' and resource 'layout designs (document templates)', clearly stating the action and object. It distinguishes from siblings like 'list_layouts_on_matter' by specifying 'available for this firm' (firm-wide scope).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like 'get_layout_design' (single) or 'list_layouts_on_matter' (matter-specific). The description implies firm-wide scope but lacks when-not or alternative suggestions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description only states it lists event types, but does not disclose any behavioral traits such as authentication needs, rate limits, or potential side effects. For a read operation, more detail on output behavior would be helpful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no fluff. Front-loaded with the action ('List') and resource ('webhook event types'). Efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and the output schema is presumably defined elsewhere, the description is complete enough for this simple listing operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so the schema coverage is 100%. The description does not add parameter information, but none is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists all available webhook event types, with a specific verb and resource. Distinguishes from sibling tools like 'list_webhook_subscriptions' which list subscriptions not event types.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No context about prerequisites or scenarios where this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description only states it adds files but does not disclose any behavioral traits such as required permissions, error handling, side effects, or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. The purpose is front-loaded, and the parameter hint is directly in the description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple batch add tool with two string parameters and an existing output schema, the description covers the essential functionality. Minor lack of info on error handling but still reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning for 'files_csv' by indicating it is comma-separated file names, but 'matter_id' receives no additional explanation beyond its name. With schema coverage at 0%, the description only partially compensates.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that this tool adds multiple files to a matter in one request, distinguishing it from the sibling 'add_file_to_matter' which likely handles single files.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says 'in one request' implying batch usage versus single-file alternative. However, no explicit 'when not to use' or prerequisite conditions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full transparency burden. It correctly indicates a write operation and specifies that only passed fields change. For a simple tool, this is adequate. However, it lacks details on authorization, rate limits, or immediate effects, which would elevate it further.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two succinct sentences with no extraneous information. The key instruction 'Only pass fields you want to change' is front-loaded. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters and no annotations, but an output schema exists, the description is minimally complete. It explains the partial update behavior but omits details like firm identification context or validation. It is adequate for a simple tool but could be more comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning no parameter descriptions in the schema. The description 'Update firm profile fields' does not add specific meaning beyond the parameter names (name, email, phone, address). While names are self-explanatory, the description should compensate more by explaining context or constraints for the parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update firm profile fields' – a specific verb and resource. It also adds the partial update nuance with 'Only pass fields you want to change,' which distinguishes it from a full replacement. This clarity helps an agent know exactly what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for partial updates but does not explicitly state when to use this tool over alternatives like 'get_firm' for reading or other 'update_*' tools for different entities. No when-not or alternative guidance is provided, only the partial update instruction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavioral traits. It states the tool retrieves data, implying a safe read operation, but does not disclose any potential side effects, required permissions, or limitations. For a simple read tool, this is minimally adequate but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that fully conveys the tool's purpose without any wasted words. It is front-loaded with the key action and resource. Every phrase earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and an output schema exists to explain return values, the description is mostly complete. However, it could briefly mention the integration context or the fact that the mapping is read-only. No major gaps are present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the schema coverage is trivially 100%. The description adds no parameter-specific information because none is needed. The baseline for zero-parameter tools is 4, and the description does not detract from that.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'Get' and clearly identifies the resource as 'integrated search field mapping', with a parenthetical clarifying what that means. It immediately conveys the tool's purpose without ambiguity. There are no sibling tools with similar names to cause confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives or when not to use it. While there are no obvious sibling tools for this function, explicit context is absent. The usage is implied by the tool's name and description but not explicitly stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It correctly indicates a partial update via PATCH, but does not mention potential side effects, permissions, or idempotency. The description is honest but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long with no fluff. It efficiently conveys purpose, usage, and alternatives.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 parameters, no annotations, but an output schema exists, the description is partially complete. It differentiates from a sibling and states the scope, but lacks details on partial update behavior, error handling, and parameter constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It mentions 'status or stage', which loosely maps to the 'status' and 'stage_id' parameters, but 'matter_id' is not described. The description adds limited semantic value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: partially updating a matter's status or stage using PATCH. It also distinguishes from the sibling tool 'update_matter', which is for name/description changes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool (for partial updates of status/stage) and when to use the alternative 'update_matter' (for name/description). This provides clear usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavior. It implies a read-only operation but does not explicitly state side effects, authentication needs, or data sensitivity. The simple get nature is generally understood, but additional clarity would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that directly conveys the tool's function with no extraneous words. It is appropriately front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, has an output schema), the description is complete. It adequately informs the agent of the action and resource without needing elaboration on return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so the description does not need to provide parameter details. The baseline for zero parameters is 4, and the description adds no unnecessary information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'this firm's profile and settings,' making the tool's purpose unambiguous. It distinguishes from siblings like 'update_firm' and other get_* tools by focusing on the firm entity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description lacks explicit guidance on when to use this tool versus alternatives, such as when a firm-wide context is needed. While the use case is implicit, no prerequisites or exclusion criteria are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist; the description is minimal and does not disclose any behavioral traits beyond listing. Could mention that it returns all groups without filtering.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, front-loaded sentence with no wasted words. Perfectly concise for a simple list operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With zero parameters and an output schema present, the description fully explains what the tool does. No additional context needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, so the description adds no param semantics, but that is acceptable. Schema coverage is 100% (empty schema). Baseline 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
A clear verb ('List') and specific resource ('authorization groups for this firm') distinguish it from sibling tools like create_authorization_group, get_authorization_group, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidance provided; no mention of when to use this vs. alternatives (e.g., get_authorization_group for a specific group). Adequate but lacks explicit direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![smokeball-mcp MCP server](https://glama.ai/mcp/servers/RosenAdvertising/smokeball-mcp/badges/card.svg)](https://glama.ai/mcp/servers/RosenAdvertising/smokeball-mcp)

Score Badge

Copy to your README.md:

[![smokeball-mcp MCP server](https://glama.ai/mcp/servers/RosenAdvertising/smokeball-mcp/badges/score.svg)](https://glama.ai/mcp/servers/RosenAdvertising/smokeball-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/RosenAdvertising/smokeball-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server