billy-mcp

by Cimalys

Overview Schema Related Servers Score Discussions

JavaScript

Local

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.1.0
Disambiguation5/5
Every tool targets a distinct resource and action (e.g., billy_create_invoice vs billy_create_bill). Even similar operations like approve_* are clearly differentiated by resource type. No overlapping or ambiguous tool purposes.
Naming Consistency5/5
All tools follow the exact pattern billy_verb_noun (e.g., billy_create_invoice, billy_list_invoices, billy_delete_invoice). Verbs and nouns are consistently used, making the API predictable.
Tool Count4/5
65 tools is high but appropriate for a full-featured accounting system covering CRUD, reports, files, bank reconciliation, and webhooks. The number is justified by the domain breadth, though on the upper end of reasonable.
Completeness4/5
Core CRUD is present for most entities (bills, invoices, contacts, products, accounts). Coverage includes files, bank reconciliation, daybook transactions, reports. Minor gaps: no update/delete for bank accounts or bank lines beyond match/unmatch.
Average 3.5/5 across 65 of 65 tools scored. Lowest: 2.3/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 4 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description must cover behavioral traits. It only says 'READ-ONLY', but fails to disclose error handling, required permissions, or response format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise (3 words) but under-specifies. While not verbose, the brevity sacrifices necessary detail, making it insufficiently informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, no annotations, minimal description. The agent has no information about return values, error states, or prerequisites, making the tool difficult to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% for the single 'id' parameter. The description adds no meaning about what the ID represents or its format, requiring the agent to infer.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Fetch one product' clearly states the action and resource. It distinguishes from sibling list and create tools. However, it lacks detail on scope (fetch by ID, as implied by schema).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus other get tools or list_products. The description only states it's read-only, which is obvious from the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'READ-ONLY' which indicates no mutation, but fails to describe the report's scope, data sources, date range interpretation, or what happens if parameters are missing or invalid.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short (two fragments) but lacks structure and substantive content. Conciseness should not come at the expense of completeness; here it fails to provide adequate information for an agent to understand the tool's full behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has three parameters, no output schema, and no annotations, the description is severely incomplete. It does not explain what the report returns, how to interpret the date parameters, or how it differs from other report tools. This leaves the agent with insufficient information for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description adds no explanations for the three parameters (fromDate, toDate, accountGroupId). While fromDate and toDate are somewhat self-explanatory, accountGroupId is not described at all, and the description provides no format or usage details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as generating a P&L (Profit and Loss) report, which is a standard financial report. The term 'P&L report' is specific and understandable. However, it does not differentiate this report from sibling report tools like billy_report_balance or billy_report_invoiced_sales.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternative report tools. It only states it is read-only, which is a general property but not usage context. There is no mention of prerequisites, typical use cases, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description includes 'READ-ONLY' which implies no side effects, but lacks additional behavioral details such as pagination behavior, data freshness, or any constraints beyond that. With no annotations provided, the description carries the full burden but falls short.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief (two sentences) and front-loaded. However, it is too minimal to be fully informative. Every sentence earns its place, but more detail is needed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, no annotations, and only 2 parameters with 0% schema coverage, the description is insufficient. It does not explain pagination, what entries contain, or how to use the tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%—the description does not explain parameters (page, pageSize). The schema provides type and constraints, but the description adds no meaning to them, failing to compensate for the lack of parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Recent activity log entries', indicating the tool lists recent action logs. The verb 'list' is implied, and it distinguishes from other list tools by specifying 'action stream' in the name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. There is no mention of prerequisites, filters, or exclusions, leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden. It labels the tool as 'READ-ONLY' but does not disclose pagination behavior, data ordering, rate limits, or what information is returned. The behavioral disclosure is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and front-loaded, with no unnecessary words. However, it is too sparse to be fully informative, trading completeness for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 5 parameters and no output schema, the description is severely lacking. It omits details on pagination, filtering, return values, and usage context, making it incomplete for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for 5 parameters, yet the description provides no explanation for any parameter (e.g., contactId, sinceDate, untilDate). It fails to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a clear verb 'List' and specifies 'bank payments', adding context that they are records of money in/out tied to invoices/bills. This distinguishes it from other list tools like billy_list_bills, but could be more explicit about the differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as billy_list_bills or billy_list_invoices. There is no mention of scenarios, prerequisites, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It states READ-ONLY, implying no side effects, but lacks details on pagination, error behavior, or performance considerations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely short, which is positive for conciseness, but the structure is a fragment. It front-loads the purpose but omits essential details, making it under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters, no output schema, and no annotations, the description is severely incomplete. The agent lacks crucial information to use the tool effectively, such as filter capabilities and response format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description provides no information about the 6 parameters (page, state, pageSize, contactId, sinceDate, untilDate). The agent cannot infer their meaning or formatting without external knowledge.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists bills/expenses and marks it as read-only, distinguishing it from mutation tools like billy_approve_bill or billy_delete_bill. However, it does not differentiate from other list tools such as billy_list_invoices, which could cause confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The READ-ONLY tag gives a hint for safe usage, but there is no explicit guidance on when to use this versus other list tools, no prerequisites, and no alternatives mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions 'READ-ONLY', which is a key behavioral trait. However, it does not disclose other behaviors like pagination, filtering via the optional contactId parameter, or whether the output is limited. With no annotations, the description carries full burden but adds little beyond the name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (two sentences). However, it lacks explanation of the parameter, which is a structural flaw. Every sentence is short but not fully informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the optional parameter and the presence of many sibling list tools, the description is insufficient. It does not explain what contact persons are, how they relate to contacts, or the effect of the contactId parameter. No output schema means the agent has no idea what the tool returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, contactId, has no description in the schema (0% coverage) and the description does not explain its purpose or effect. This is a critical gap as the parameter likely filters the list.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists contact persons, which is specific and matches the name. However, it does not explicitly distinguish from sibling tools like billy_list_contacts (contacts vs. contact persons).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool compared to alternatives such as billy_get_contact or billy_list_contacts. The 'READ-ONLY' label hints at safety but does not clarify context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description includes 'READ-ONLY', which indicates a non-destructive operation. However, with no annotations provided, the description does not disclose other behavioral traits like pagination, sorting, error behavior, or rate limits. It minimally addresses the safety profile.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short, consisting of two phrases. While concise, it lacks necessary detail, making it less useful. Every sentence is brief but fails to earn its place by providing comprehensive guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 6 optional parameters and no output schema, the description is insufficient. It does not explain the return format, pagination behavior, or how filters interact. For a list tool with many siblings, more detail is needed to enable correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 6 parameters (q, page, pageSize, countryId, isCustomer, isSupplier) with no descriptions in the schema (0% coverage). The description adds no information about these parameters, leaving the agent to guess their purpose and format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List contacts', identifying the verb (list) and resource (contacts). This distinguishes it from sibling tools like billy_get_contact (which retrieves a single contact) and billy_create_contact. However, it does not specify the scope or any filtering details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as billy_get_contact for individual lookups or other list tools. There is no mention of required context or preconditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so the description carries full burden. It only states 'READ-ONLY', indicating no mutations, but omits other behavioral traits such as pagination behavior, rate limits, authentication requirements, or what happens if no results are found. Essential context for a read operation is missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two short sentences), but conciseness sacrifices completeness. It front-loads the purpose but fails to include necessary details about parameters and return format. Every sentence earns its place, but more sentences are needed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has three parameters (with no schema descriptions), no output schema, and no annotations, the description is inadequate. It does not explain pagination, the role of 'state', or the response structure. A list tool without return format or filtering guidance is incomplete for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for the three parameters (page, state, pageSize). The description does not explain any parameter's meaning, valid values, or relationships. Without these, an agent cannot correctly use the tool. No value is added beyond the schema's property names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and the resource ('MOMS/VAT returns') and adds the behavioral note 'READ-ONLY'. However, it does not explicitly differentiate this tool from other list tools (e.g., billy_list_invoices) which also list resources. The specific resource name is unique among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus other list tools or other sales-tax-related tools like billy_get_sales_tax_return. There is no mention of prerequisites, filtering scenarios, or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description includes 'READ-ONLY', indicating safety, but no further behavioral details (e.g., pagination behavior, sorting, or limits). With no annotations, the description partially compensates but lacks comprehensive disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (two short sentences) and front-loaded. Every word earns its place, though it could benefit from slightly more detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (5 parameters, no output schema, no annotations), the description is too sparse. It does not explain how parameters like sinceDate or accountId affect the listing, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description provides no parameter explanations. The 5 parameters (page, pageSize, accountId, sinceDate, untilDate) are entirely unspecified, leaving the agent without guidance on their meaning or usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List ledger postings' clearly states the action and resource. It distinguishes from sibling list tools by specifying 'postings', which is unique among the siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus other list tools (e.g., billy_list_accounts, billy_list_bills). No exclusion criteria or alternative suggestions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for behavioral disclosure. It states 'READ-ONLY', but does not describe side effects, authorization needs, rate limits, or output format. The behavioral transparency is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two short phrases. No unnecessary words. It front-loads the key information. However, conciseness comes at the cost of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of output schema and the need to clarify the date range parameters, the description is incomplete. It does not explain what the report includes or how the date range affects results. For a report tool, this is insufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not mention the two parameters (fromDate, toDate) at all. It adds no meaning beyond the schema's bare structure, failing to explain their purpose or format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description identifies it as a 'MOMS/VAT report', indicating a sales tax report. It clearly states 'READ-ONLY', which distinguishes it from mutation tools among siblings. However, it could be more specific about the report's content (e.g., summary of tax collected).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like billy_get_sales_tax_return or other report tools. The description does not mention context or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'READ-ONLY' indicates non-destructive behavior, but no details about pagination, sorting, or output format are given. This is minimal disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise, using only two segments: the main action and a trait. Every word is functional, though it lacks a full sentence structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema, 0% parameter coverage, and three parameters, the description is too sparse. It does not clarify pagination, search, or return values, leaving the agent with incomplete information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage and three parameters (q, page, pageSize), the description adds no meaning beyond parameter names. It fails to explain how to use them or their effects.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and resource 'products', with clarification 'your sellable items/services'. It also adds 'READ-ONLY' which distinguishes from mutation tools. Among sibling list tools, this one is specific to products.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus other list tools (e.g., billy_list_accounts, billy_list_invoices) or when not to use it. The description is too brief to provide context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full responsibility for disclosing behavioral traits. It only states 'READ-ONLY', which is helpful but insufficient. It omits details about pagination, filtering, side effects, or other behaviors that the agent needs to invoke the tool safely.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two short sentences. Every word serves a purpose: the first states the core action, the second adds a safety note. No redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (3 optional params, no output schema), the description should at least mention the return format or provide parameter hints. It fails to do so, leaving the agent with insufficient contextual information for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description adds no explanation for parameters (page, pageSize, isArchived). The agent cannot infer their meaning or how they affect results. This is a critical gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List the chart of accounts' clearly specifies the action (list) and resource (chart of accounts), distinguishing it from sibling tools that create, update, or approve other resources. The addition 'READ-ONLY' further clarifies its non-mutating nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not indicate when to use this tool over alternatives (e.g., billy_list_account_groups) or specify any prerequisites or scenarios. The agent receives no guidance on selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explicitly labels the tool as READ-ONLY, which is a key behavioral trait. However, without annotations, more detail would be helpful, such as confirming no side effects or idempotency. It adds some value beyond the tool name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with no unnecessary words. It is front-loaded with the key information. However, it could be slightly more informative without losing conciseness, e.g., adding a verb.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and annotations, the description does not explain what the balance sheet contains, what the response format is, or any potential prerequisites. It is too minimal for a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter is 'date', and while the description hints at its purpose ('at a point in time'), it does not explain the expected format, required range, or whether past/future dates are allowed. With 0% schema coverage, the description should compensate more.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the resource (balance sheet) and the time constraint (at a point in time). The purpose is understandable, but it lacks a verb like 'generates' or 'retrieves'. It does not distinguish from sibling reports like billy_report_profit_loss.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternative report tools. There is no mention of prerequisites, date format expectations, or context where this report is appropriate. The usage is only implied by the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden. It discloses the mutation nature and confirm requirement, but lacks details on error handling, idempotency, or what fields are updatable. The patch object being free-form is not mentioned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (two sentences) but misses critical information about parameters and behavior. It is under-informative, not efficiently concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has a two-phase commit pattern (dry-run vs execute) and no output schema, the description does not explain the dry-run preview or define what 'id' and 'patch' represent. It is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 33% (only confirm has a description). The tool description does not add meaning for 'id' or 'patch' parameters. The confirm behavior is explained in the schema, but the overall parameter semantics are insufficiently described.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing account', which is a specific verb and resource. It distinguishes from create tools, though it does not explicitly differentiate from other update-type siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes 'WRITE — requires confirm:true' which provides usage context about the confirmation mechanism. However, it does not offer guidance on when to use this tool versus alternatives like billy_create_account or other update tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states 'READ-ONLY', which is a key behavioral trait. However, with no annotations, it fails to disclose other important aspects like error handling on missing ID, authentication needs, or return format. A 3 is appropriate for minimal but correct disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (two sentences) and front-loaded with the action. While efficient, it could be slightly improved by including the parameter requirement without becoming verbose, but overall it is appropriately sized for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and minimal description, the tool does not explain what the agent should expect as output (e.g., a contact object) or error behaviors. For a single-parameter read operation, more context is needed for the agent to use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% for the single parameter 'id', and the description does not explain it beyond the tool's purpose. The description fails to add meaning such as expected format or source of the ID, which is critical for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Fetch one contact' clearly states the specific verb and resource, effectively distinguishing it from sibling tools like billy_list_contacts (list) and billy_create_contact (create). The 'READ-ONLY' clarifies that it does not modify data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., billy_list_contacts). The minimal description does not indicate prerequisites, such as having the contact ID, nor does it explain when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description includes 'READ-ONLY', confirming non-destructive behavior. However, with no annotations provided, the description carries full burden but omits details about pagination, rate limits, or authentication requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise—only two short phrases, front-loaded with the action. No wasted words, but could be slightly expanded to include parameter guidance without losing efficiency.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having 6 parameters with enums and ranges, no output schema, and no annotations, the description is minimal. It does not explain filtering, pagination, or the meaning of 'manual daybook transactions', leaving agents underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions for any of the 6 parameters, and the tool description adds no parameter explanations. With 0% schema coverage, the description fails to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'list', the resource 'manual daybook transactions (journal entries)', and explicitly marks it as 'READ-ONLY'. This differentiates it from sibling tools like billy_list_daybooks (lists daybooks) and billy_create_daybook_transaction (creates).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like other list tools. Does not mention prerequisites, suitable contexts, or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The 'READ-ONLY' label explicitly states the tool has no destructive side effects, which is valuable given no annotations. However, it does not disclose error behavior (e.g., what happens if ID is invalid), response format, or any rate limits or auth requirements, leaving gaps for the agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at two short phrases: 'Fetch one bank line. READ-ONLY.' Every word adds value, no filler or redundancy. It is structured with a clear verb-object statement followed by a behavioral hint.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has one parameter, no output schema, and no annotations, the description is inadequate. It explains the basic action but lacks context on return values, error handling, or how this tool fits into the broader bank line workflow (e.g., matching, listing).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'id' has no description in the schema (0% coverage), and the description adds no meaning about its format, source, or constraints. The agent cannot infer what kind of ID to provide or how to obtain it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Fetch one bank line' uses a specific verb and resource, clearly indicating the tool retrieves a single bank line. It distinguishes from the sibling tool billy_list_bank_lines which lists many, and from other get tools by specifying 'bank line'. The 'READ-ONLY' label further clarifies the nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like billy_list_bank_lines, billy_match_bank_line, or other get tools. Usage is only implied by the description of fetching one bank line; no conditions, prerequisites, or prohibitions are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description only declares READ-ONLY. Does not disclose error behavior (e.g., missing bill) or response format. Minimal behavioral info.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, but could be more informative. Efficient for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple fetch tool with one parameter and no output schema, but lacks parameter guidance and behavioral details like error handling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and description adds no information about the 'id' parameter beyond its existence. No guidance on what the ID represents or where to obtain it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Fetch', resource 'one bill', and clarifies it's an expense and READ-ONLY. Distinguishes from sibling tools like billy_create_bill and billy_get_invoice.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for fetching a single bill by ID, but no explicit guidance on when to use versus alternatives like billy_list_bills or when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only adds 'READ-ONLY' beyond the name. No annotations are provided, so the description carries the full burden; it lacks details on return format, pagination, or idempotency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two short phrases with no wasted words. The core purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with no parameters and no output schema, the description adequately states the purpose but lacks explanation of the returned data or how units relate to products. Slightly below comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters and 100% coverage, so the description need not add parameter details. However, it does not explain what 'product units' means, missing an opportunity to add semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
States 'List product units' which is a specific verb and resource. Among siblings, including 'billy_list_products', it distinguishes by focusing on 'units' rather than products, but does not explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus siblings like 'billy_list_products' or others. The 'READ-ONLY' hint is present but does not provide context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It labels the tool as 'WRITE' and mentions the need for confirmation, which implicitly signals destructiveness. However, it does not elaborate on consequences (e.g., irreversibility, impact on related data) or behavior when the bill is not a draft. The input schema's confirm parameter description provides additional detail, so the description adds modest value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (one sentence) and front-loaded with the core purpose. However, it could briefly include parameter hints or context without losing conciseness. No wasted words, but slightly under-specified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description lacks crucial details: what constitutes a 'draft bill', error handling, id format, and prerequisites. A simple delete operation still benefits from mentioning common pitfalls (e.g., idempotency) or required permissions. The description is too minimal for confident invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50% (only the 'confirm' parameter has a description). The description does not explain the 'id' parameter or add any parameter semantics beyond the schema. For a tool with partial schema coverage, the description should clarify parameters like 'id' (e.g., format, source), but it fails to do so.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the specific resource 'draft bill', which immediately distinguishes this tool from sibling tools like billy_delete_contact or billy_delete_invoice. The mention of 'draft bill' specifies the scope, helping the agent select the correct tool for deleting a bill that is in draft status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description only indicates that it requires 'confirm:true' for execution, but provides no guidance on when to use this tool versus alternatives (e.g., when to use billy_update_bill instead) or when not to use it (e.g., if the bill is not a draft). There is no mention of prerequisites or contextual exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It declares read-only behavior, which is good, but lacks details on error handling or authentication requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at two short phrases, but it could be slightly more structured by mentioning the input parameter briefly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no output schema and one undocumented parameter, the description is too minimal. It fails to explain what the tool returns or the expected id format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description does not explain the meaning or format of the 'id' parameter, leaving the agent without guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Fetch') and the resource ('one MOMS/VAT return'), which is specific and distinct from the sibling list tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'READ-ONLY' label implies safety, but there is no explicit guidance on when to use this tool versus alternatives like billy_list_sales_tax_returns, nor exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Declares READ-ONLY but doesn't explain pagination, default behavior, or response format beyond the function name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Very concise with two sentences; front-loaded with purpose, but could add value without increasing length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No mention of output shape or pagination details; with sibling tools, lacks context on when to use this versus billy_get_file.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Parameters page and pageSize are named but not described in description or schema; schema coverage is 0%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'List' and resource 'files uploaded to Billy', distinguishing from upload, get, and attach siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied use case from 'List files', but no explicit guidance on when to use vs alternatives like billy_get_file or billy_list_bills.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It states READ-ONLY, indicating no data mutation, but lacks details on return format, pagination, or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, no fluff. However, it could expand slightly without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, no parameter details, and no description of how the report is generated or returned. The description is too minimal to fully inform an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% (no descriptions in schema). The description does not explain fromDate and toDate parameters, leaving their format and semantics unspecified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it reports invoiced sales by contact/product and explicitly declares READ-ONLY, distinguishing it from other report siblings like billy_report_balance or billy_report_profit_loss.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use versus other report tools or CRUD operations. The READ-ONLY label is implicit but no explicit context or exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full responsibility. It states 'READ-ONLY' which indicates no side effects, but it does not disclose behavior on missing files, error handling, rate limits, or authentication needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one sentence plus a note) and front-loaded. It is efficient but could include slightly more detail without losing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required parameter, no output schema), the description is minimally adequate. It mentions the key output ('downloadUrl') but does not list other typical metadata fields, leaving the agent to infer.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage for the single parameter 'id'. The tool description does not explicitly clarify that 'id' refers to the file's identifier or specify its format (e.g., UUID).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Fetch metadata'), the resource ('one file'), and a key output detail ('incl. downloadUrl'). It distinguishes from siblings like 'billy_list_files' and 'billy_upload_file' by focusing on a single file retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies read-only usage but does not explicitly guide when to use this tool versus alternatives like 'billy_list_files'. No exclusions or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden. It explicitly labels the tool as 'READ-ONLY', which is a key behavioral trait. However, it does not disclose any other details (e.g., pagination, caching, or data freshness).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (two phrases) and front-loaded with the action. However, it is so short that it borders on under-specification. Still, it earns its place fully.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (no parameters, no output schema, no annotations), the description is minimally adequate. It could benefit from stating the scope (e.g., all currencies in the system) or return format, but the basic purpose is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and 100% schema coverage (empty schema). The description does not need to add parameter info since there are none. Baseline 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (List) and resource (currencies), and the tool name matches. It is distinct from siblings like billy_list_accounts or billy_list_bills, but does not explicitly differentiate itself from other list tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., billy_list_accounts or billy_list_countries). The description is too brief to provide context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It notes READ-ONLY and lists filters, but fails to mention pagination behavior (page, pageSize), output format, or any rate limits or authentication details. This is insufficient for a tool with 6 parameters and no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, with two short sentences that front-load the core purpose and key parameters. Every word adds value, and there is no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters, no output schema, and moderate complexity (filtering and pagination), the description is incomplete. It omits pagination details, output format, and parameter constraints. An AI agent would need additional context to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description only partially compensates by listing 4 of 6 parameters as filters. It does not explain the format of sinceDate/untilDate, the purpose of page/pageSize, or add semantics beyond the schema's basic types. The enum for state is already present in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb 'List' and resource 'invoices', clearly differentiating from sibling tools like billy_create_invoice or billy_approve_invoice. The READ-ONLY tag further clarifies the tool's safe nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates the tool is READ-ONLY and lists available filters, providing basic guidance. However, it does not explicitly state when to use this tool versus other list tools or exclude inappropriate scenarios, leaving room for interpretation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so description carries full burden. 'READ-ONLY' explicitly declares non-destructive behavior, which is a key trait. However, no other behaviors (e.g., pagination, return format) are disclosed. Adequate for a simple list tool but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: 'List webhooks. READ-ONLY.' Every word adds value with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless read-only list, the description covers the core purpose and safety. However, it lacks any details about the output format, pagination, or what a webhook object contains, which would be helpful given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with zero parameters, so description adds nothing beyond schema. Baseline score of 3 is appropriate since schema already fully describes the input.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List webhooks' with verb and resource. 'READ-ONLY' adds context. However, it doesn't differentiate from other list tools (e.g., billy_list_bills) beyond the resource name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus siblings like billy_create_webhook or billy_delete_webhook, or when to use other list tools. Usage context is completely absent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that it is a WRITE operation and requires confirm:true, which is important behavioral context. However, no annotations are provided, and the description lacks details on side effects (e.g., idempotency, behavior on duplicate accounts) and outcome beyond the dry-run mechanism explained in the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short with one sentence, front-loading the purpose. However, it sacrifices completeness for brevity; slightly more structure (e.g., separate lines for parameter hints) would improve usability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (nested account object, write operation with dry-run, no output schema), the description is insufficient. It does not explain what the account object should contain, nor does it describe the return value or error conditions, leaving the agent under-informed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50% (confirm described, account not). The description does not add any parameter information beyond the schema, leaving the account object's structure undocumented. With a nested object, this is a significant gap that the description fails to address.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new account in the chart of accounts', which specifies the verb (create) and unique resource (account in chart of accounts), distinguishing it from siblings like billy_list_accounts (read) and billy_update_account (update).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes 'WRITE — requires confirm:true', which is a key usage condition but does not provide explicit guidance on when to use this tool versus alternatives (e.g., billy_update_account for modifications). No when-not-to-use or alternative suggestions are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses READ-ONLY behavior but does not mention authentication requirements, error handling (e.g., missing id), rate limits, or response details beyond line items. For a get-by-id tool, this is insufficient for an agent to understand potential side effects or failure modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one sentence plus a tag. Every word adds value ('Fetch one invoice including line items' and 'READ-ONLY'). There is no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-id tool with only one parameter and no output schema, the description is mostly adequate: it explains the action and that line items are included. However, it does not describe what happens on success (e.g., returns full invoice object) or on error (e.g., not found). Given the absence of annotations and output schema, more detail would be beneficial for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one required parameter 'id' with 0% description coverage. The tool description does not add any meaning to the 'id' parameter, such as format, source, or constraints. The description mentions 'including line items' but does not connect that to the parameter. For a single parameter, the description should clarify its purpose beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch one invoice including line items', which specifies the verb (fetch), resource (one invoice), and scope (including line items). It distinguishes from siblings like billy_create_invoice (mutation) and billy_list_invoices (multiple invoices). The 'READ-ONLY' tag further clarifies it is a safe read operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for reading a single invoice's details, mentioning line items. However, it does not explicitly state when not to use it (e.g., for listing multiple invoices) or provide alternatives. The 'READ-ONLY' tag gives some guidance, but lacking explicit exclusions or comparisons to siblings like billy_list_invoices or other get_* tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Explicitly notes READ-ONLY, which helps the agent know it's safe. No other behavioral traits disclosed, but no annotations provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise with no unnecessary words. Front-loaded with action and key trait.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks explanation of what a daybook is or what the output contains. With no output schema, more detail would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so the description adequately covers the input. Baseline 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists daybooks. However, no differentiation from sibling list tools like billy_list_daybook_transactions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives or any prerequisites. Simply states the action.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses the confirm safety mechanism and write nature, but omits other behaviors like return value, idempotency, or field update constraints. Adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise with only two sentences, front-loading the key verb and the critical usage constraint. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having 3 parameters including a nested object and no output schema, the description lacks detail on expected input format, output behavior, error handling, and prerequisites. The confirm pattern is described but other critical aspects are missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is low (33%), and the description only adds meaning for the confirm parameter (required for execution with dry-run preview). For id and patch, no additional insight is provided beyond the schema, leaving the agent to guess valid fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Update an existing bill', which is a specific verb+resource. It distinguishes from siblings like create or delete by name, but lacks detail on whether it's a partial or full update.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states 'WRITE — requires confirm:true', providing clear usage context. It implies a dry-run mode without confirm, guiding when to use it, though it does not explicitly contrast with sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explicitly states 'READ-ONLY', indicating no side effects, which is critical for a get operation. However, with no annotations provided, the description carries the full burden; it lacks details on permissions, rate limits, or error handling, but the read-only claim provides adequate transparency for a simple fetch.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at three words: 'Fetch one account. READ-ONLY.' It is front-loaded with the core purpose and includes a critical behavioral note. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one required param, no output schema), the description is minimally sufficient but lacks detail on return format or error conditions. It meets the basic need but does not add extra context that would be helpful for an agent, such as what account fields are returned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'id' has 0% schema description coverage, but its name is self-explanatory. The description does not elaborate on format or constraints, but the schema already defines it as a required string. Baseline score of 3 is appropriate since the parameter is straightforward.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Fetch' and the resource 'one account', making the tool's purpose unambiguous. It distinguishes itself from siblings like 'billy_get_bank_account' or 'billy_list_accounts' by specifying a single account fetch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention when-not to use it or point to siblings like 'billy_list_accounts' for broader queries.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses the confirm-for-execution pattern and dry-run behavior, which adds value beyond the schema. But missing details on what the patch object accepts and what the response returns, especially with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff. Front-loaded with purpose, immediately followed by the most critical usage detail. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Minimal for a mutation tool with nested parameter (patch) and no output schema. Lacks description of return value, permissible patch fields, or prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only the confirm parameter is described in the description, and it mostly restates the schema. The id and patch parameters are left entirely unexplained, despite low schema coverage (33%).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb+resource: 'Update an existing contact.' Distinct from create and read tools among siblings. The 'WRITE' annotation reinforces its mutating nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the confirm:true requirement, which is a crucial usage guideline. However, no guidance on when to use vs. other contact tools (e.g., create vs. update) or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It discloses 'WRITE' and the confirm requirement, but does not elaborate on consequences (e.g., irreversibility, permission needs, status changes). The input schema partially compensates by documenting the confirm parameter's dry-run behavior, but the description adds little beyond that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at 7 words, with no fluff. Every part serves a purpose: naming the action, specifying the resource, flagging the write operation, and noting the required parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the tool's simplicity, the description lacks necessary context: it does not mention prerequisites (e.g., bill must be in draft state), effects (e.g., status change), or return value. With no output schema, the agent is left guessing about the result of the operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50% (only 'confirm' has a description). The description only restates that confirm:true is required, adding no new meaning for 'id' (which lacks a schema description). The basic purpose of 'id' is implied by context, but the description does not compensate for the undocumented parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Approve a draft bill. WRITE — requires confirm:true' clearly specifies the verb (approve), the resource (draft bill), and the write nature. It distinguishes from siblings like billy_approve_invoice and billy_delete_bill by targeting a specific stage (draft) and action (approval).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states the tool's purpose ('Approve a draft bill') but provides no guidance on when to use it versus alternatives (e.g., billy_create_bill for new bills, billy_update_bill for editing). No exclusions or context for when not to use it are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states this is a WRITE operation requiring confirmation, which is transparent. However, it doesn't disclose side effects, authentication needs, or error behaviors. The input schema's confirm parameter adds detail, but the description itself is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the core action and the important WRITE/confirm constraint. No unnecessary words, but could be slightly expanded for completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a nested object parameter (bankAccount) and no output schema. The description does not explain return values, error conditions, or what constitutes a valid bankAccount object. The agent lacks sufficient context to use this tool correctly without additional knowledge.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50% (only confirm parameter has a description). The description adds no meaning beyond what the schema provides. It does not describe the bankAccount object structure, leaving the agent without guidance on required fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The name 'billy_create_bank_account' clearly indicates a bank account creation. The description specifies 'Create a bank account in Billy' and adds the WRITE label, distinguishing it from siblings like 'billy_create_account' (general account) and other non-write tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states 'requires confirm:true', alerting the agent that a second call with confirm:true is needed to execute. While it doesn't compare to alternatives, this is a clear usage constraint.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description partially discloses behavioral traits (WRITE, requires confirm). However, it omits the dry-run preview behavior when confirm is false, which is detailed only in the schema. This is a notable gap for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (8 words) and front-loaded with the core action. It could benefit from including the dry-run behavior, but it remains efficient and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with 2 parameters and no output schema, the description is adequate but lacks details like prerequisites (ownership), behavior on failure, or confirmation workflow beyond the schema. It meets minimal completeness but leaves room for improvement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 50% coverage (only confirm described). The tool description adds no parameter info beyond stating 'requires confirm:true', which mirrors the schema. The id parameter remains without explanation, so the description does not compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Remove') and resource ('a webhook'), distinguishing it from create and list webhook tools. It could be more specific about which webhook by id, but it's sufficient.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates it's a WRITE operation and requires confirm:true, which guides the agent on the destructive nature and confirmation needed. However, it doesn't explicitly state when to use this tool over alternatives, though siblings don't include another delete webhook.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The 'READ-ONLY' flag provides a behavioral hint beyond the name, but without annotations the description lacks disclosure of authentication, pagination, or return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, no fluff, front-loaded. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple no-parameter list operation with no output schema, the description is minimally sufficient but lacks detail on the nature of the countries listed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (zero parameters), so baseline is 3. The description adds no param info, but none are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List countries' with a specific verb and resource, and the name alone distinguishes it from sibling list tools like billy_list_accounts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives; it only states the action without context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly states 'WRITE — requires confirm:true' and the schema explains the dry-run behavior for confirm. However, it does not disclose if the file is copied or linked, or any other side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence plus note) and front-loaded. It is efficient but could include brief parameter explanations without significant bloat.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and 3 parameters, the description misses key context: how to get fileId, prerequisite steps, and what happens after attachment. The sibling tools (billy_list_files, billy_upload_file) hint at the workflow but are not referenced.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is low (33%: only confirm described). The tool description does not add any additional meaning for fileId or invoiceId beyond their names. It should explain where to obtain fileId (e.g., from billy_list_files) and the format of invoiceId.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Attach an uploaded file to an invoice' using a specific verb (attach) and resources (file, invoice). This distinguishes it from siblings like billy_attach_file_to_bill and billy_upload_file.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage (attaching an uploaded file to an invoice) but provides no explicit when-to-use, when-not, or alternatives. It lacks guidance on prerequisites like requiring a prior upload via billy_upload_file.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries full burden. It declares 'READ-ONLY' but does not elaborate on behavior like response structure, pagination, or hierarchical depth. For a simple list, this is adequate but lacks additional context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two short sentences front-loading the key information. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description leaves the agent without details on the return format or hierarchy structure. While the tool is simple, a brief note on what constitutes 'account-group hierarchy' would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, so the baseline is 4. The description adds no parameter details, which is acceptable since the schema covers everything.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'List the account-group hierarchy' which clearly identifies the verb (list) and resource (account-group hierarchy). It distinguishes from sibling tool billy_list_accounts which lists individual accounts rather than groups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention when not to use it or offer explicit context for selection among many list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It correctly identifies the tool as read-only and notes the default filtering behavior. However, it does not disclose other important behaviors such as pagination limits, error handling, or what happens when no results are found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two short sentences. It front-loads the main purpose, then adds critical details (read-only, required parameter, default state). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters and no output schema, the description is adequate but leaves gaps. It covers the key behavioral aspect (unmatched default) and a required parameter, but omits return format, pagination behavior, and other parameter details. For a list endpoint, it is minimally sufficient but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only 33% of parameters have schema descriptions (e.g., accountId has a helpful note). The tool description adds context for isMatched (default unmatched) but does not explain page, pageSize, sinceDate, or untilDate. Given low schema coverage, the description should provide more parameter guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists bank statement lines, specifying the resource ('bank lines') and default behavior (unmatched). It distinguishes itself from siblings like billy_get_bank_line (single line) and billy_list_bank_payments (different resource) by the resource type and scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions it's read-only, requires accountId, and defaults to unmatched lines, providing basic usage context. However, it does not explicitly guide when to use this tool versus alternatives like billy_get_bank_line or billy_list_bank_payments, nor does it mention when to set isMatched to true.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description effectively discloses that this is a mutation (WRITE) and that confirm=true is mandatory to execute. This gives essential behavioral context for safe invocation. Lacks details on failure modes or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: one sentence plus a succinct note about confirm. No redundancy or filler. Front-loaded with action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple three-parameter tool with no output schema, the description covers the core action adequately. However, it does not explain prerequisites (e.g., file must already be uploaded) or what happens on success (e.g., response, side effects), leaving minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 33% (only confirm has a description). The description adds minimal meaning beyond parameter names for billId and fileId; it implies fileId refers to an uploaded file but provides no detail about expected formats or sources.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (attach), the target (bill), and the type of file (uploaded receipt PDF). It distinguishes from sibling tool billy_attach_file_to_invoice by specifying a different resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description notes that this is a WRITE operation requiring confirm:true, which guides invocation. However, no explicit when-to-use or when-not-to-use guidance is given, nor comparisons to alternative tools like billy_upload_file or billy_attach_file_to_invoice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of behavioral disclosure. The 'READ-ONLY' label indicates no side effects, which is helpful. However, it does not disclose other traits such as pagination, authentication requirements, or rate limits. The description is adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two short sentences with no wasted words. It front-loads the core purpose and immediately adds a behavioral tag. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, no output schema), the description is largely complete. It states what the tool lists. However, it lacks specifics about the output format or scope (e.g., whether it lists all tax rates from the entire organization). Still, for a straightforward list, it meets the minimum bar.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the input schema provides complete definition. According to guidelines, baseline for 0 parameters is 4. The description adds no parameter information, which is acceptable as there is nothing to add.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List configured tax rates' uses a specific verb ('List') and resource ('tax rates'), clearly indicating the tool's function. The 'READ-ONLY' tag reinforces the read-only nature. It distinguishes itself from sibling list tools by targeting a specific entity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., other list tools like billy_list_accounts). There is no mention of context, prerequisites, or exclusion criteria, leaving the agent to infer usage from the resource name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the burden of behavioral disclosure. It explicitly states 'WRITE — requires confirm:true', indicating it is a destructive operation that requires a confirmation parameter to execute, which is clear and transparent about the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a single sentence that front-loads the core action. It is appropriately sized, though additional context on parameters could be added without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is an update mutation with no output schema, the description should explain the return value or effects of the update. It also lacks explanation of the patch object, making it incomplete for an agent to use without additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is low (33%), with only the confirm parameter having a description in the schema. The tool description does not add any meaning for the id or patch parameters, leaving their semantics unclear, especially the variable patch object.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing product', which is a specific verb (update) and resource (product). This distinguishes it from sibling tools like create, delete, get, and list products.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions 'requires confirm:true' which provides a usage guideline for execution. However, it does not explicitly state when to use this tool versus alternatives like billy_create_product, or provide any context on prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must carry the burden. It acknowledges the write nature and the confirm mechanism but does not disclose permissions, error handling, or side effects. The shape hint is useful but insufficient for full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: a single sentence plus a shape specification. Every part earns its place, with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks information about return values, which is important given no output schema. It does not explain what the tool returns (e.g., the created contact person). However, for a create tool with a simple input, the context is partially adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning by specifying the expected shape of the 'person' parameter (contactId, name, email?, phone?, isPrimary?), which the schema lacks (additionalProperties: {}). For 'confirm', the schema already provides a description. Thus, the description compensates for low schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Create') and resource ('contact person'), clearly distinguishing it from siblings like billy_create_contact. The shape details further clarify the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states that the tool is a 'WRITE' operation requiring confirm:true, providing basic usage guidance. However, it does not explicitly state when to use this tool over alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the confirm dry-run mechanism and the need for a second call to execute. It does not cover other behavioral aspects like error handling or auth, but the key behavioral trait is well explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief (one sentence plus a note on confirm), but the key information is front-loaded. The dry-run explanation is essential and well-placed. Slightly more structure could improve readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description does not mention return values or side effects. The dry-run preview is implied but not detailed. Given the complexity of updating an invoice with a nested patch object, more completeness would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only 33% of parameters have schema descriptions (confirm only). The description adds value for confirm but does not explain the 'id' or 'patch' parameters. 'patch' is a free-form object, yet the description gives no guidance on typical fields or structure.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing invoice' with a specific verb and resource. It distinguishes from sibling tools like billy_approve_invoice and billy_delete_invoice by labeling it as a WRITE operation requiring confirm:true.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the critical confirm parameter and the dry-run behavior, guiding safe usage. It implicitly distinguishes from other invoice tools, but could be more explicit about when to use update versus approve or delete.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the destructive nature ('WRITE') and the confirmation requirement, but does not elaborate on side effects, irreversibility, or permissions. Provides basic but not comprehensive transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: one sentence with essential information front-loaded. No wasted words, highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with only two parameters and no output schema, the description covers the core action and the critical confirm requirement. It does not mention return values or consequences of deletion, but given the simplicity, it is mostly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50% (only confirm has a description). The description adds the critical context that confirm:true is required to execute, going beyond the schema for that parameter. However, it does not explain the id parameter or its format, so value addition is partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'product', making the tool's purpose immediately obvious. It is distinct from sibling tools like billy_create_product or billy_update_product.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies that this is a WRITE operation requiring confirm:true to execute, giving clear usage guidance. However, it does not explicitly compare to alternatives or mention when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It declares the tool as read-only, but lacks details on return format, pagination, or rate limits. More behavioral context would be beneficial but the read-only disclosure is present.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, no redundant information. Everything is front-loaded and essential.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with no parameters and no output schema, the description covers the core purpose and behavior (read-only). It could mention that it returns all accessible tax rulesets, but the current level is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters (schema coverage 100%), so baseline 3 applies. The description adds no parameter info, which is acceptable since none exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'tax rulesets'. It distinguishes from sibling tools like billy_list_tax_rates by specifying the exact resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'READ-ONLY' label provides context that this tool is safe to use without side effects. No explicit when-not or alternatives are given, but the simple no-parameter nature makes usage clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool requires confirm:true to execute and that without it, it returns a dry-run preview. This adds behavioral context beyond the input schema. No annotations are provided, so the description carries the full burden.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, using two sentences to convey purpose, operation type, and key requirement. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with two parameters and no output schema, the description covers the essential behavior and confirm flow. It could be improved by explaining what 'associationId' represents and any post-conditions, but it is largely adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning to the 'confirm' parameter by explaining the dry-run vs. execution behavior. However, the required 'associationId' parameter is not described beyond its type, leaving its role unclear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Remove a bank-line reconciliation') and the resource ('bank line'). It is specific and distinguishes from the sibling tool 'billy_match_bank_line'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates it's a WRITE operation requiring confirm:true, implying the user must explicitly confirm. However, it does not provide guidance on when to use this tool versus alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses write operation, confirmation mechanism, format restrictions, and auto-detection of MIME. However, it omits details like file size limits, idempotency, behavior on duplicate filenames, and error handling, leaving some gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is compact (approx. 70 words) with clear structure: action, confirmation requirement, format constraints, next steps, and auto-detection. Every sentence adds essential information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of output schema, the description adequately covers the primary function, parameters, and expected return (metadata with id). Missing details on error cases and size limits are minor; overall it provides sufficient context for correct tool usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with good parameter descriptions. The description adds value by explaining the confirm parameter's two-step workflow and the purpose of the returned id, but does not significantly enhance understanding beyond the schema for other parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states upload action, resource (file to Billy), and scope (receipt/invoice PDF/image). Distinguishes from sibling attach tools by explicitly stating the next step to attach the file using billy_attach_file_to_bill or billy_attach_file_to_invoice.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions write operation with two-step confirmation (dry-run then confirm), required file formats, and conversion advice. Provides clear post-upload guidance. Lacks explicit exclusions (e.g., when file already exists), but context implies appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description bears full burden. It states the mutation nature ('WRITE') and the confirm requirement, but does not describe side effects, idempotency, error behavior, or response format. The provided shape helps but additional behavioral context is missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise, front-loaded sentences. No filler. Every sentence contributes: purpose, nature, confirm requirement, and payload shape.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose and payload shape, but omits return value (no output schema), prerequisites, or behavior for duplicate creation. For a mutation tool with nested parameters, more completeness would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema describes only the confirm parameter; the product parameter lacks schema description. The description compensates by providing a detailed shape for the product object, including fields like name, prices, and optional fields. This adds significant semantic value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a product/service', which matches the tool name and distinguishes it from siblings like billy_update_product and billy_delete_product. The verb and resource are explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description specifies it is a WRITE operation requiring confirm:true, guiding the agent to use a two-step call pattern. However, it does not explicitly contrast with alternatives (e.g., when to use update vs create), but the intent is clear from the context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool is a WRITE operation and that confirm:true is needed for actual execution, contrasting with a dry-run default. This adds essential behavioral context beyond the absent annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that cover purpose, usage warning, and parameter shape without any redundant wording.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of annotations and output schema, the description adequately covers the confirm behavior and webhook structure. However, it lacks details like possible event values or response format, which are not critical for basic invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers confirm with good detail, but the webhook object is minimally specified (additionalProperties: true). The description compensates by providing the shape '{ url, event, isActive? }', adding meaning to the nested parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Create a webhook' with a shape hint, clearly indicating the verb and resource. It distinguishes from sibling tools like billy_list_webhooks and billy_delete_webhook, but does not elaborate on the webhook's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly notes 'WRITE — requires confirm:true', guiding the agent on the necessary flag to execute the mutation. It does not provide when-not or alternatives, but there is no direct sibling for creating webhooks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description bears full transparency burden. It explicitly labels the operation as WRITE and mandates confirm:true for execution, disclosing that the tool mutates data and requires an extra confirmation step. This adequately informs the agent of the tool's behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences: first for purpose, second for key behavioral requirement. No unnecessary words, every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the two-phase execute pattern but does not describe the return format or success/error behavior. Given no output schema and moderate complexity, it lacks some contextual completeness, though the essential workflow is covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is low (25%), with only 'confirm' described. The description adds meaning by explaining the dry-run/preview behavior tied to confirm, and clarifies subjectType values ('invoice/bill/bank payment'). However, subjectId and bankLineId remain underspecified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Reconcile' and the resource 'bank line', specifying it reconciles with an invoice/bill/bank payment. This distinguishes it from sibling tools like billy_unmatch_bank_line.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage guidance: it's a WRITE operation requiring confirm:true, and implies a dry-run mode without confirm. It doesn't explicitly mention alternative tools, but the confirm requirement effectively guides when to execute vs preview.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It accurately reveals the critical two-step confirmation pattern (dry-run vs. actual execution). It does not cover other behavioral aspects like idempotency, rate limits, or error handling, but the key behavior is transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences that deliver essential information. It uses clear formatting and front-loads the purpose and critical usage note. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 parameters, no output schema) and the importance of the update operation, the description covers the main purpose and the confirmation workflow. However, it lacks details on return values, success/failure indicators, and prerequisites, leaving some gaps for a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers 50% of parameters (2 parameters, 1 required). The description adds value by listing example fields for the 'patch' object, which is otherwise a free-form object in the schema. However, it does not provide detailed descriptions for each field, so it only partially compensates for the schema's lack of detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates fields on the organization, uses a specific verb ('Update'), and references the resource ('organization'). It distinguishes from sibling tools that update other entities (e.g., billy_update_account, billy_update_bill). The purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that this is a WRITE operation requiring confirm:true, and implies a dry-run mechanism without confirm. It provides clear guidance on the confirmation workflow. However, it does not mention when to use this tool versus alternatives, such as other update tools or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It clearly communicates the mutation nature (WRITE) and the two-step confirm pattern. However, it does not disclose error behavior or reversibility.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences, no redundant information, and front-loaded with the key action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with two parameters and no output schema, the description covers the essential behavior and constraints. It lacks information about the return value, but is otherwise sufficient for an agent to decide if it's the right tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The confirm parameter description in the schema already explains its role, and the description adds no new details about parameters. The id parameter lacks description in both schema and text. Given 50% schema coverage, the description provides marginal added value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a draft invoice' clearly states the verb and resource, and further specifies the scope (draft only) and restriction (approved/paid cannot be deleted). This distinguishes it from sibling tools like billy_approve_invoice or billy_update_invoice.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly notes that it is a WRITE operation requiring confirm:true, guiding the agent on mandatory usage. It also states that approved/paid invoices cannot be deleted, but does not mention alternative tools or actions for those cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses the confirm dry-run vs. execution behavior and the email payload structure. However, it omits prerequisites, success/error responses, and other backend effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three well-structured sentences: purpose, WRITE nature with confirm, and email payload format. No redundant or extra information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters, nested object, and no output schema, the description covers the core action, the confirm safeguard, and email format. Missing potential error cases or return values, but sufficient for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is low (33%), but the description compensates by detailing the email object fields (to, cc, bcc, subject, message, attachPdf). It also clarifies the confirm parameter's role. Some fields like replyTo and country are omitted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Send an invoice by email via Billy,' specifying the verb (send) and resource (invoice by email). It distinguishes from sibling tools like billy_approve_invoice or billy_create_invoice.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the WRITE nature and the confirm:true requirement for actual execution, which guides when to use. However, it does not explicitly state when not to use or provide comparisons to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explicitly states 'WRITE — requires confirm:true,' which is a critical behavioral detail beyond the schema (where confirm is optional). This informs the agent that a second call with confirm:true is needed to execute, preventing accidental writes. No annotations are provided, so the description carries the burden well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: one for purpose and behavior, one for the shape. It is highly concise, front-loaded with the action and critical requirement, and contains no unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and a complex nested parameter, the description provides the transaction shape and explains the confirm workflow. However, it lacks details on return values, error handling, or additional constraints like required fields in lines. Still, it is largely sufficient for a create tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers only 50% of parameters (confirm with description, transaction with none). The description adds the full shape of the transaction object: daybookId, entryDate, description, and lines with fields and types. This compensates for the sparse schema and provides essential guidance for constructing the nested object.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a manual journal entry,' which is a specific verb+resource. The name billy_create_daybook_transaction reinforces this. It distinguishes from sibling tools like billy_create_bill or billy_create_invoice by specifying manual journal entries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for manual journal entries but provides no explicit guidance on when to use this tool versus alternatives like creating bills or invoices. No when-not-to-use or alternative tool names are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It declares the tool as READ-ONLY, indicating no side effects. It also mentions a limitation regarding listing. However, it does not cover authentication, rate limits, or error handling, which are minor gaps for a simple fetch operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences. The first sentence states the action and read-only nature. The second provides context about listing limitations and an alternative. Every sentence is efficient and adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one required parameter, no output schema, and no annotations, the description covers the essential purpose, read-only behavior, and listing limitation. It references an alternative for the default account. It could potentially mention the output structure, but it's not critical for a simple fetch tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is only one parameter (id, required) with 0% schema description coverage. The description adds 'by id', which is already implied by the tool name and schema. It does not specify format, length, or any additional constraints, offering minimal added value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch one bank account by id', specifying the verb (fetch), resource (bank account), and method (by id). It distinguishes from listing all bank accounts, which is not supported, and mentions an alternative for the default account.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states 'READ-ONLY' and notes that Billy v2 does not support listing all bank accounts, advising to look up the default via billy_whoami. This provides clear when-to-use, when-not, and an alternative tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description includes 'READ-ONLY', explicitly disclosing that the tool does not modify data. Since no annotations are provided, this behavioral hint is valuable. It does not cover other aspects like rate limits, but for a simple list tool, this is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at two sentences, front-loading the purpose and adding the behavioral note. Every word is meaningful with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter list tool, the description is complete. It specifies the resource and the read-only nature. No output schema or additional context is necessary, and the sibling tools do not overlap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, and the schema is empty with 100% coverage. According to guidelines, 0 parameters yields a baseline of 4. The description adds no parameter information, but it is not required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'payment-terms modes', making the purpose unambiguous. It distinguishes itself from sibling list tools by specifying a unique resource, and there are no other tools with similar names.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool versus alternatives. While the tool's purpose is clear, it does not mention contexts where it is preferred or excluded, leaving the agent to infer from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description discloses the WRITE operation, confirm requirement, and dry-run preview. It adds key behavioral context but omits details like permissions or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise with no wasted words. Purpose and key usage information are front-loaded in two short sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the critical confirm behavior and bill shape, but lacks details on return values, validation rules, or required nested fields. Adequate for a create tool with moderate complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers confirm with description; bill object is freeform. The description adds a specific shape (contactId, entryDate, currencyId, lines) that goes beyond the schema, aiding parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new bill.' and provides a specific shape, distinguishing it from siblings like billy_update_bill and billy_delete_bill.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly notes the two-step confirm process ('WRITE — requires confirm:true') and dry-run behavior, but doesn't explicitly contrast with other create tools or mention alternative contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses write operation, confirm flag behavior, and the shape of the contact object. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, both essential. First states purpose, second adds critical usage requirement and structure. No filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, required confirm flag, and contact shape. No output schema, but the description gives enough for an agent to know what to expect. Slight lack of detail on the response behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50% (confirm described, contact not). Description adds the shape fields (name, isCustomer, isSupplier, etc.) for the contact object, supplementing the schema's gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Create a new contact' — a specific verb+resource. Distinct from sibling tools like billy_create_contact_person and billy_update_contact.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Mentions 'WRITE — requires confirm:true', indicating the need for a confirmation flag. Does not explicitly state when not to use or alternatives, but provides clear context for execution.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses default state (draft), write operation, and confirmation requirement. Without annotations, the description provides necessary behavioral context. However, it could mention that creation is non-destructive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Short and front-loaded: one sentence plus a compact shape definition. Every part is useful, no waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a creation tool, but missing return value information. No output schema exists, so the description should indicate what is returned (e.g., the created invoice object).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description provides a detailed shape for the 'invoice' object (contactId, entryDate, currencyId, lines with fields), which is not present in the schema (only 'object' with additionalProperties). This compensates for low schema coverage and adds significant meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Create a new invoice (default state: draft)' with a specific verb and resource. Distinguishes itself from sibling tools that involve invoices (approve, attach, delete, etc.).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Mentions 'WRITE — requires confirm:true', instructing the agent to use confirm for execution. However, it does not explicitly state when to use this tool versus alternatives like billy_update_invoice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations, but description discloses write nature, confirm step, and deletion constraint. Could detail error behavior for invalid deletion attempts.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no fluff, key info front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers essential aspects for a delete tool with constraints. No output schema needed. Minor gap on error handling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50% (id undocumented). Description emphasizes confirm parameter but adds no details for id. Partially compensates.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Delete a contact' with the verb and resource. Differentiates from siblings like billy_get_contact or billy_update_contact.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions the confirm requirement and the constraint that contacts with invoices/bills cannot be deleted, guiding when to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It discloses the WRITE nature and the confirm:true requirement. Does not detail all side effects (e.g., data creation) but covers key behavioral trait.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences: purpose, behavioral note, shape. Front-loaded with key info. Zero wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, usage (confirm), and parameter shape. No output schema, but acceptable for a create mutation. Could mention return value (e.g., created payment ID) but not essential.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50%, but description compensates fully by specifying shape of bankPayment object including fields like entryDate, cashSide, cashAmount, etc. Adds substantial meaning beyond the schema's minimal description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Create a bank payment' and explains its use to mark invoices/bills as paid. Differentiates from sibling create tools like billy_create_bill and billy_create_invoice by specifying resource and context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions the need for confirm:true to execute, separating dry-run from actual mutation. Says 'Used to mark invoices/bills as paid' but does not explicitly contrast with alternative payment methods or other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It declares 'READ-ONLY' and states it retrieves the current organization, which sufficiently discloses the behavior for a simple read operation. Could mention more about return format, but not necessary.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence plus 'READ-ONLY' label; every word earns its place. No redundancy or unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, no-output-schema tool, the description is complete. It states the purpose, usage context, and behavioral safety (read-only). No gaps given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so baseline 4 applies. Description adds no parameter info because none needed; schema coverage is 100% trivially.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'Get' with clear resource 'current Billy organization', and adds context 'sanity check the token, see org settings'. It distinguishes from sibling read tools like billy_get_* because it targets the organization root and serves as a connectivity test.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states 'sanity check the token' which tells when to use: verifying authentication. Context is clear, though it doesn't explicitly state when not to use or provide alternative tools, which is acceptable given its unique purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses the write operation and the two-step confirmation pattern: 'WRITE — requires confirm:true. Without it, this tool returns a dry-run preview... explicit second call with confirm:true is needed to write.' This is comprehensive behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences front-loading the key purpose, state change, and critical behavioral requirement. No wasted words, every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, usage, and behavioral details adequately for a simple approval tool. It could explicitly state that the id must refer to a draft invoice, though this is implied by 'draft → approved'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds significant meaning beyond the schema, explaining the confirm parameter's dual behavior: dry-run preview without true, actual mutation with true. The schema only had a basic description for confirm; the description clarifies the execution model completely.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb 'Approve', the resource 'draft invoice', and the state change 'draft → approved'. It clearly distinguishes this tool from siblings like billy_send_invoice by stating it's required before sending or marking paid.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states 'Required before sending or marking paid', which gives clear context on when to use this tool. It implies not to use it on non-draft invoices, but does not explicitly name alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![billy-mcp MCP server](https://glama.ai/mcp/servers/Cimalys/billy-mcp/badges/card.svg)](https://glama.ai/mcp/servers/Cimalys/billy-mcp)

Score Badge

Copy to your README.md:

[![billy-mcp MCP server](https://glama.ai/mcp/servers/Cimalys/billy-mcp/badges/score.svg)](https://glama.ai/mcp/servers/Cimalys/billy-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Cimalys/billy-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server