billy-mcp

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.3.0
Disambiguation4/5
Most tools have distinct purposes, but there is potential confusion between list_daybook_transactions, list_transactions, and list_postings, which all deal with accounting entries. The descriptions help, but an agent might struggle to pick the right one.
Naming Consistency4/5
Tool names predominantly follow a verb_noun pattern (e.g., create_invoice, list_contacts). However, billy_raw_request and execute_script break the convention, and the mix of 'list_' and 'get_' for single vs. multiple items is consistent.
Tool Count3/5
40 tools is a large number for a server focused on a single accounting application. While each tool serves a specific purpose, the count is on the high end, potentially overwhelming for an agent.
Completeness4/5
The tool surface covers core accounting workflows (contacts, invoices, bills, payments, journal entries, files). Notable gaps like sales tax returns or inventory are absent, but the raw_request tool provides an escape hatch.
Average 4.1/5 across 40 of 40 tools scored. Lowest: 3.4/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 5 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false. The description confirms creation and return of the contact with ID. However, it lacks details on idempotency, error handling, or any side effects. With annotations already covering basic safety, the description adds minimal behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that immediately states the tool's purpose and output. It contains no extraneous information and is well front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 15 parameters and no output schema, the description does not explain parameter interactions (e.g., how email triggers contact person creation, default type), nor does it describe the response structure beyond mentioning the ID. The tool's complexity warrants more comprehensive context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 15 parameters are fully described in the input schema (100% coverage). The tool description does not add any additional meaning beyond the schema; it merely restates the purpose. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Create a new contact (customer or supplier) in Billy. Returns the created contact including its ID.' It uses a specific verb and resource, and distinguishes the tool from siblings like update_contact and list_contacts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as create_contact_person or update_contact. It does not mention prerequisites, exclusions, or typical scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, indicating a safe read operation. The description adds minimal context about invoice-related data but does not disclose pagination behavior or sorting. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words, front-loaded with the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should explain return value structure. It hints at names/emails but lacks specifics. Pagination details are only in schema, not explained in context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description adds no additional meaning beyond the existing parameter descriptions. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list', the resource 'contact persons', and the scope 'belonging to a Billy contact'. It distinguishes from sibling tools like list_contacts by specifying the sub-resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as list_contacts or other listing tools. There is no mention of prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate it's not read-only and not destructive; the description matches 'create'. However, it adds no additional behavioral context (e.g., permissions, reversibility, side effects) beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences front-load the action and outcome with zero wasted words. Ideal length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with 100% schema coverage and no output schema, the description adequately covers purpose and return value. Missing mention of the 'verbose' parameter's effect on response, but otherwise complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are fully documented. The description mentions 'optionally with per-currency unit prices' which aligns with the 'prices' parameter, but adds no extra meaning or usage detail beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create a product in Billy') and the optional feature ('optionally with per-currency unit prices'), and specifies the return value ('Returns the created product including its ID'). This distinguishes it from sibling tools like update_product or list_products.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., update_product). There are no prerequisites, when-not-to-use, or contextual hints for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, so description adds value by disclosing that date and matched filters are applied client-side within the fetched page, which is critical for correct usage. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core purpose, and contains no extraneous information. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 7 parameters, no output schema, and the complexity of client-side filtering across pages, the description lacks guidance on pagination, total records, and effective use of filters. It is incomplete for an agent to use correctly without additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add new meaning to parameters; the schema descriptions are already clear. The client-side filtering note is about behavior, not parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'imported bank lines', and distinguishes it from siblings like list_bank_line_matches by noting client-side filtering. It is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions client-side filtering but does not explicitly state when to use this tool versus alternatives like list_bank_line_matches or list_bank_payments, nor does it provide guidance on pagination or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotation 'readOnlyHint: true' already indicates safe read operation. The description adds that it returns invoices with paging metadata, consistent with read-only behavior. No additional behavioral traits (e.g., pagination defaults, rate limits) are disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the action and filters, the second states the return value. It is concise and front-loaded with essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 12 parameters and no output schema, the description covers key aspects: filtering capabilities and return of paging metadata. It does not mention default pagination or sorting behavior, but the schema fills in parameter details. Overall adequate for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the input schema already documents all 12 parameters. The description briefly mentions filter categories (customer, state, date range, free-text) which correspond to some parameters, but adds no extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists invoices with specified filters (customer, state, date range, free-text search). It is distinct from sibling tools like 'get_invoice' (single invoice) and 'list_bills' (bills), but does not explicitly differentiate from them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for filtering and listing invoices, but provides no explicit guidance on when to use this tool versus alternatives (e.g., when to use 'get_invoice' instead) or mentions prerequisites or limitations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, which matches the read operation. The description adds that the response includes lines, balance, and paid status, but doesn't disclose other traits like authentication requirements or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with 14 words, front-loaded with the main action and purpose. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with one parameter and no output schema, the description adequately explains what the tool does and what the response includes (lines, balance, paid status).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the schema description for 'id' is already clear. The description does not add new semantic meaning beyond the parameter name and schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Get a single Billy bill by ID' with specific verb and resource, and distinguishes from siblings like list_bills, create_bill, and approve_invoice.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., list_bills for multiple or get_invoice for invoices). The agent must infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, so the read-only nature is known. The description adds that the isApproved filter is applied client-side within the page, which is a behavioral trait beyond annotations. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with the core purpose, no wasted words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core purpose and a key behavioral detail, but lacks usage guidance, return format (no output schema), and context for parameter interactions. Adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed parameter descriptions. The description adds minimal extra meaning beyond noting the client-side behavior of isApproved. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List bank line matches (bank reconciliation groupings)', which is a specific verb and resource. It distinguishes from siblings like list_bank_lines by specifying the context (bank reconciliation groupings). The client-side filter note adds precision.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., list_bank_lines, list_accounts). It lacks context for selection, which is important given the many sibling list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true. The description's 'list' is consistent and adds no new behavioral context beyond the existing annotation, so it meets the baseline but does not enhance transparency further.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys the core purpose with no redundant information. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 8 parameters, no output schema, and good annotations, the description covers the essential purpose but lacks details on pagination, output format, and scope (only manual entries). It is minimally adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description's mention of filtering by state and date range repeats schema content. No additional parameter semantics are provided beyond what the schema already describes.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists daybook transactions (manual journal entries) in Billy, with filtering by state and entry date range. The verb 'list' and resource 'daybook transactions' are specific, and the parenthetical distinguishes it from broader transaction lists.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for manual journal entries but does not explicitly provide when-to-use or when-not-to-use guidance, nor does it mention alternatives like list_transactions for all transactions. Guidance is inferred but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnlyHint=true, and the description adds critical behavioral details: client-side application of transactionNo/voucherNo filters, silent ignoring of unbound params, and scanning up to 5000 records. This goes beyond annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences containing all essential information with no fluff. The critical constraint about client-side scanning is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description covers the main behavioral nuances (client-side filtering, scanning limit) and parameter semantics through the schema. The verbose parameter's effect is described in the schema. Missing explicit mention of return structure or pagination behavior is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all 8 parameters. The tool description adds context about client-side processing for transactionNo, voucherNo, minEntryDate, maxEntryDate, and isVoided, which is not evident from parameter descriptions alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists booked transactions with a specific verb and resource. It additionally notes client-side filtering behavior for transactionNo/voucherNo. However, it does not explicitly differentiate from sibling list tools like list_daybook_transactions, leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives such as list_daybook_transactions or search tools. The client-side scanning limit and date bounding hint is useful but does not constitute clear when-to-use or when-not-to-use instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the description's 'List' aligns. However, the description does not add behavioral context beyond the annotation, such as rate limits or authentication needs. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. Every word is informative; no waste or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main filter categories. Although pagination and sorting are not mentioned, the schema handles those. Given no output schema, the description is reasonably complete for a listing tool with comprehensive parameter descriptions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents each parameter. The description summarizes filter types but does not add meaning beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'supplier bills (expenses)', and the system 'in Billy'. It also enumerates filter types, distinguishing this tool from siblings like get_bill or list_bank_payments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for listing bills with filters, but does not explicitly state when to use it versus alternatives like get_bill or list_contacts. No when-not or exclusion criteria are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds value beyond annotations: it confirms irreversibility (matching destructiveHint=true) and reveals that the tool continues past individual failures and reports per-id results. This provides important behavioral context not captured in the annotations alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (two sentences, 26 words) with no wasted words. It front-loads the core action and includes key behavioral traits (approval, failure continuation, irreversibility) efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a batch operation with error handling and irreversibility, the description covers essential aspects: batch scope, approval requirement, failure behavior, and irreversibility. It does not detail the response format, but the mention of 'per-id results' provides sufficient guidance without an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters. The description does not add new meaning beyond what the schema provides, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'void', the resource 'daybook transactions', and highlights the batch capability. It distinguishes itself from the sibling 'void_daybook_transaction' by explicitly mentioning 'many' and 'one call', making its purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for bulk operations with 'Void many daybook transactions in one call', but does not explicitly state when to use this vs. the single-void sibling tool. The guidance is implied by the name and context, not explicit, which limits clarity for the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false and destructiveHint=false, so the description's mention of draft state adds a bit of behavioral context. But it does not disclose permissions, rate limits, or side effects beyond what annotations imply.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no redundant information. Every word is purposeful and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool complexity (10 parameters, no output schema), the description covers the core functionality well but omits return value format. However, the lack of output schema is not the description's responsibility. The description is adequate for a creation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description does not add any parameter semantics beyond what is already in the schema; it only reiterates 'coded to expense accounts' which is covered by accountId.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create'), the resource ('supplier bill'), and the context ('in Billy with one or more lines coded to expense accounts'). It distinguishes from siblings like create_invoice by specifying 'supplier bill (expense)'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides usage context by noting the default draft state and how to book immediately (state=approved). However, it does not explicitly differentiate from sibling tools like create_invoice or create_bank_payment, nor does it state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true. Description adds that the result includes 'full address and settings', which is valuable behavioral context beyond the annotation. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no wasted words. Essential information is front-loaded. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with one parameter and no output schema, the description sufficiently explains what is returned. Lacks mention of error cases or edge conditions, but low complexity reduces need.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters (id with description 'Contact ID'). Description does not add additional meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action (Get), resource (single Billy contact), method (by ID), and included data (full address and settings). Distinct from sibling tools like list_contacts (multiple) and update_contact (modify).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for retrieving a single contact with details, but does not explicitly contrast with alternatives like list_contacts or when to use other tools. No exclusion criteria or prerequisites stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description aligns with a read operation. It adds valuable context about returning per-currency prices, which is not visible in annotations or schema. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that is front-loaded with the primary action and resource, immediately followed by the unique return value. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although there is no output schema, the description mentions the key return feature (per-currency prices). For a simple get-by-ID operation, this is largely sufficient, though error handling or missing product cases are not addressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter 'id' described as 'Product ID'. The description repeats 'by ID' but adds no additional semantics or format details beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'Billy product by ID', and adds distinct feature 'plus its per-currency prices'. It distinguishes from siblings like list_products (list all) and update_product (modify).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for fetching a single product by ID, but it does not explicitly state when to use this tool versus alternatives like list_products or update_product. No exclusions or alternative suggestions are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this as read-only. The description does not add extra behavioral details beyond what annotations provide, but it does not contradict them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences that are directly to the point, no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple listing tool with good annotations, it provides enough context (purpose and use case). However, it does not explain return values since no output schema is present, but the implication is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all 3 parameters with descriptions. The tool description adds no additional parameter-specific information, so baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists daybooks in Billy and specifies the use case of finding the daybook ID for manual journal entries, distinguishing it from other list tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides a specific use case (finding daybook ID for manual journal entries) but does not explicitly state when not to use it or differentiate from other list tools like list_daybook_transactions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=true, consistent with listing. The description adds context that files are append-only via the API. The verbose parameter description also clarifies response format (compact vs full), providing extra behavioral insight beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with purpose, and contains no extraneous information. Every sentence is valuable and concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 fully described parameters, readOnlyHint annotation, and no output schema, the description adequately explains the tool's purpose and key behavior (append-only, file types). It could mention pagination behaviors but parameters already cover that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all parameters described (page, verbose, pageSize). The description does not add additional meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists files (receipts, invoices PDFs) and specifies that files are append-only via the API. It distinguishes itself from sibling tools like upload_file and attach_file by focusing on listing existing files.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for retrieving files and notes append-only behavior, but it does not explicitly state when to use this tool versus alternatives (e.g., upload_file, attach_file). The guidance is implicit rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so the description adds limited behavioral context. Mention of free-text search is helpful but does not disclose rate limits, pagination specifics, or the compact vs verbose detail (which is only in schema).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with zero waste. Front-loaded with the core action and purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with no output schema, the description is adequate. It covers the primary use case and search capability. Could mention that it returns compact records by default (but that detail is in schema).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description adds little beyond what the schema provides. 'Free-text search' maps to the 'q' parameter, but the description does not explain other parameters like 'page', 'pageSize', 'verbose', etc.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('List products'), the resource ('products in Billy'), and the use case ('find a product ID before creating invoice lines'). It distinguishes from sibling tools like 'create_product' or 'get_product'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to use the tool ('before creating invoice lines') and mentions support for free-text search. No explicit exclusion of alternatives, but the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate it's not read-only and not destructive. The description adds the partial update behavior but does not disclose return value, error handling, or other side effects beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys the core purpose and key behavioral constraint with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the parameter count and schema coverage, the description is adequate for a simple update operation. However, it could mention the return value or confirm the update success.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 14 parameters have descriptions in the schema (100% coverage). The description does not add additional meaning to individual parameters beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool updates an existing contact and specifies that only provided fields are changed, distinguishing it from create_contact and other tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'Only provided fields are changed' provides clear usage context for partial updates, and the sibling tools (e.g., create_contact, get_contact) help differentiate when to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false and destructiveHint=false, so the description's mention of 'attach' as a write operation aligns. The description adds context about allowed ownerTypes and an example, but does not disclose further behavioral traits like reversibility or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences plus a brief example. The first sentence states the purpose efficiently, and the example reinforces without excessive length. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, and the description does not explain the return value (e.g., what is returned after attachment). The verbose parameter hints at response format in the schema, but the description lacks this context. For a mutation tool, additional info on response could improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are fully described. The description adds value by directing how to get fileId (from upload_file or list_files) and provides an example that clarifies the relationship between ownerType and ownerId. This goes beyond the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description explicitly states the verb 'attach' and the resource 'uploaded file to a Billy record', and specifies the record types (daybookTransaction, invoice, bill). The example with daybookTransaction further clarifies scope. It clearly distinguishes from sibling tools like upload_file and list_files.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description indicates how to obtain the required fileId ('Get the fileId from upload_file or list_files') and provides an example illustrating ownerType and ownerId usage. It does not explicitly mention when not to use, but the context is clear for attachment tasks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate it is not read-only and not destructive, which aligns with 'Add'. The description adds the email usage context but does not disclose other behaviors like success/failure handling or constraints. With annotations covering basic safety, this is adequate but not exceptional.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first defines the action, second provides a practical reason for the email parameter. No unnecessary words, front-loaded with essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is provided, and the description does not mention return values. The verbose parameter hints at response structure, but missing information on what the tool returns (e.g., created person ID). Adequate for basic use, but could be more complete for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already described. The description adds value by explaining the email purpose ('Emails are used to mail invoices') and noting that name or email must be set. This goes beyond the schema descriptions, earning a higher score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add'), the target ('existing Billy contact'), and the data ('name and/or email'). It also provides context ('Emails are used to mail invoices'), distinguishing it from creating a new contact or listing contact persons.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly indicates use on existing contacts ('to an existing Billy contact'), providing context for when to use. It does not explicitly list alternatives or when not to use, but the context is clear. The sibling tools help differentiate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint=true, indicating a safe read operation. The description adds no new behavioral traits beyond confirming it returns a single account. With annotations carrying the safety burden, the description is adequate but not additive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that immediately states the tool's purpose. It contains no filler or redundant information, earning its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (single parameter, read-only, no output schema), the description and annotations together provide complete context for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (the 'id' parameter is described as 'Account ID'). The tool description does not add extra meaning beyond what the schema already provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a single account from the Billy chart of accounts by ID' uses a specific verb ('Get') and resource ('account'), clearly distinguishing it from sibling tools like 'list_accounts' (which returns multiple accounts) and 'get_account_balances' (different resource).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when the account ID is known, providing clear context. However, it does not explicitly state when not to use this tool or mention alternatives like 'list_accounts' for retrieving multiple accounts, leaving some room for ambiguity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true. The description adds value by detailing what is returned (lines, balance, paid status), which is beyond the annotation. No contradictions or further behavioral details (e.g., error handling) are given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, 15 words, front-loaded with verb and resource. Every word is necessary, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-ID tool with one parameter and no output schema, the description sufficiently states what is returned. No further information is required to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single parameter 'id' described as 'Invoice ID'. The description mentions 'by ID' which reinforces, but adds minimal new meaning beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a single invoice by ID, specifies the included data (lines, balance, paid status), and uses a specific verb 'Get'. This distinguishes it from sibling tools like list_invoices or get_bill.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you have a specific invoice ID but does not explicitly state when to use this over alternatives like list_invoices or get_bill. No when-not or alternative guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint false and destructiveHint true, aligning with the description's statement that approval books the invoice and cannot be reverted. The description adds the irreversibility detail, though it lacks permission or side effect info.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences efficiently convey the core purpose and key behavioral constraint (irreversibility). No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential behavioral aspect (state change and irreversibility) but does not address output format or the verbose parameter. However, given the simplicity of the tool and lack of output schema, it is mostly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters described in the schema. The description does not add meaningful parameter semantics beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Approve', the resource 'draft Billy invoice', and the specific state change from draft to approved. It distinguishes this tool from sibling tools like create_invoice or delete_invoice.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when an invoice is in draft state and needs approval. It warns about irreversibility but does not explicitly mention when not to use or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate mutability (readOnlyHint=false) but not destructiveness. The description adds value by disclosing client-side balance validation, the approval flow, safe-retry deduplication, and the default draft state. This context enriches the behavioral understanding beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, each serving a distinct purpose: core function, validation/approval, and state/dedup. It is front-loaded with the primary action, concise, and free of redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 10 parameters and no output schema, the description covers core creation, balance validation, dedup, and state. It does not explain the response format or error handling, but the verbose parameter hints at response details. The lines schema description covers currency balancing, so overall it is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-specific details beyond what the schema already provides (e.g., idempotencyKey usage is mentioned but the schema already describes it). Thus, no additional semantic value is added for parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Create a manual journal entry (daybook transaction) in Billy with balanced debit/credit lines.' It specifies the resource (daybook transaction), action (create), and key constraints (balanced lines). It distinguishes itself from sibling tools like list_daybook_transactions and void_daybook_transaction by focusing on creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance on when to use idempotencyKey for safe retries and explains the default draft state and approval option. However, it does not explicitly state when not to use this tool or mention alternatives for other journal entry types, so it lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, so the description must clarify behavior. It mentions creation defaults to draft and that approval books the invoice, which implies a mutation but not destruction. It does not disclose any side effects like irreversible number assignment or email sending. Lacks details on what happens to the contact or inventory. Adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences plus a short clarifying note. Every sentence serves a purpose: identity, default behavior, and a key semantic detail about units. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 12 parameters, no output schema, and annotations with no safety hints, the description covers the essential purpose, default state, and currency semantics. It mentions the verbose parameter in the schema but not in the description. The absence of output description is mitigated by the verbose parameter documentation. Overall, it's fairly complete given the high schema coverage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already described. The description adds one meaningful piece: 'Amounts are in the invoice currency,' which is not in the schema. It also reinforces the 'state' parameter's effect. Since schema is thorough, the added value is moderate but tangible.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Create an invoice (or credit note) in Billy with one or more lines,' clearly specifying the verb (create), resource (invoice/credit note), and key constraint (one or more lines). It distinguishes itself from sibling tools like approve_invoice (which modifies state after creation) and delete_invoice.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage guidance: 'Created as draft by default; pass state=approved to approve immediately (approval assigns the invoice number and books it).' It explains the default behavior and how to change it but does not explicitly state when not to use this tool (e.g., if you only need to approve an existing invoice, use approve_invoice instead). Sibling tool approve_invoice exists, but the description omits that alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true. The description adds the critical behavioral quirk that certain filters are applied client-side, which is not obvious from the schema. This compensates well.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. Front-loaded with purpose, then important note. Exceptionally concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, but description mentions compact vs verbose mode via the verbose param. Lacks explicit return value format, but for a list tool with schema-documented parameters, this is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover all 7 parameters 100%. The description adds only the client-side filtering context for some params, which is helpful but not a major addition. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description explicitly states the tool lists bank payments recorded in Billy and provides a use case (reconciliation). Clearly distinguishes from sibling tools like create_bank_payment or list_bank_lines.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives context for when to use (reviewing payments) and a notable limitation: server-side only supports organizationId, with client-side filtering for other params. Could be improved by explicitly stating when not to use, but the hint is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, so agent knows it's safe. Description adds that it returns a list of contacts with filtering/search capability. No contradictions; moderate additional context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose. Every sentence adds value; no waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, annotations present, and description covers purpose and usage hint. Output schema is missing but verbose parameter hints at response format. Slightly incomplete regarding response structure but adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds general info about free-text search and filtering but does not explain individual parameters beyond what schema provides. No extra value for parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists contacts (customers and suppliers) in Billy, supports free-text search and filtering. It distinguishes from siblings like get_contact and create_contact by specifying the list and find-ID use case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Use this to find a contact ID before creating invoices or bills.' It does not discuss when not to use or alternatives, but the given context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate destructiveHint=true, and the description adds the critical behavioral detail that the action is irreversible and voided entries cannot be reinstated. This aligns with and enhances the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, no wasted words, and front-loads the core purpose. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple void operation with two parameters and no output schema, the description covers purpose and key behavioral detail (irreversibility). It could mention what the response looks like, but given the minimal complexity, it is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters (id and verbose). The description does not add additional semantic meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (void), resource (daybook transaction), and system (Billy). It also distinguishes from sibling tools like create_daybook_transaction and batch_void_daybook_transactions by specifying it acts on a single transaction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (to void a transaction) but does not explicitly state when not to use or mention alternatives like batch_void_daybook_transactions for multiple voids. The irreversibility warning provides some guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint: true, and the description adds that the tool 'fetches and aggregates all matching postings server-side', disclosing its server-side processing behavior. No contradictions. It could further detail potential performance impacts, but the guidance on date bounds mitigates that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three concise sentences: purpose, example usage, and performance advice. No fluff, every sentence earns its place. Front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main use cases (single account balance and full trial balance) and performance tips. While it lacks details on return format (no output schema), the aggregation behavior is explained. For a tool with 6 optional parameters and no output schema, the context is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for all parameters. The description adds value by providing an example for accountNos and emphasizing that minEntryDate is 'strongly recommended' for speed, which enhances understanding beyond schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool computes account balances or a trial balance from postings, with examples like 'what is the balance of account 5211'. It distinguishes from sibling tools like list_accounts (which lists accounts without computing balances) and list_postings (which returns individual postings).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage scenarios: ask for a specific account balance via accountNos, or omit filters for a full trial balance. It also advises bounding with minEntryDate/maxEntryDate for performance. However, it does not explicitly compare to alternatives for similar tasks, e.g., when to use list_postings instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, and the description does not contradict that. It adds valuable behavioral context, such as the verbose parameter saving ~90% context, and hints about relevant fields (isPaymentEnabled, isBankAccount).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, with the first sentence stating the main purpose and the second providing specific usage guidance. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 8 parameters but none required. The description covers common use cases and hints at relevant return fields. While no output schema exists, the description compensates by suggesting specific fields to look for. A minor gap is not mentioning pagination limits beyond the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so baseline is 3. The description adds practical guidance beyond the schema, such as looking for 'isPaymentEnabled/isBankAccount' and explaining the verbose parameter's benefit.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List the chart of accounts in Billy' with specific use cases 'find account IDs for bill lines, product revenue accounts, and bank accounts for payments'. It distinguishes from sibling tools by focusing on this specific resource and its uses.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use the tool, such as finding account IDs for specific purposes, and suggests looking for the fields 'isPaymentEnabled/isBankAccount'. However, it does not explicitly mention when not to use it or name alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds context beyond readOnlyHint: discloses that date/void/text filters are client-side (within page) and only accountId is server-side. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each with distinct info: purpose, alternative, filtering behavior. No fluff, front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main behavior and filter distinctions no output schema but description is sufficient. Could mention pagination implications (fetch many pages).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds meaning beyond schema: describes text as 'searches LINE-LEVEL text' and client-side behavior for several params. Baseline 3 due to 100% schema coverage, but description adds useful context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists ledger postings (individual debit/credit entries) and distinguishes from get_account_balances for aggregates. Adds detail on client-side vs server-side filtering.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises using get_account_balances for balances/aggregates, and notes that accountId is the only server-side filter. Could mention other siblings but is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explicitly warns 'WARNING: this posts a real accounting transaction,' adding critical behavioral context beyond the annotations' destructiveHint: true. It also clarifies the debit/credit direction for invoices vs bills. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, starting with purpose, then behavior details, and ending with a warning. It is efficient and front-loaded, though could be slightly more structured with bullet points.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 11 parameters, full schema coverage, and no output schema, the description covers core behavior, side effects, and critical warnings. It could mention the return value (e.g., created bank payment) but is otherwise complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, the description adds value by providing extra context for key parameters, e.g., 'cashAccountId: must have isPaymentEnabled=true; see list_accounts' and 'feeAccountId required when feeAmount set.' This goes beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it records a bank payment to mark an invoice or bill as paid, specifying the association via 'invoice:<id>' or 'bill:<id>'. It differentiates from sibling tools as no other tool records bank payments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for marking payments and warns about posting a real accounting transaction, but does not explicitly list alternatives or when not to use it. The context is clear enough for an AI agent to infer appropriate use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark it as destructive (destructiveHint: true) and writable (readOnlyHint: false). The description confirms deletion and adds that it returns 'meta.deletedRecords', which is beyond annotations. However, it doesn't disclose any additional side effects or prerequisites beyond 'drafts only', which is already covered. Still, it adds value, so 4 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler. First sentence front-loads the action and key constraint. Second sentence states return value. Every word is essential.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple deletion tool with two well-described parameters and annotations, the description covers purpose, usage guidelines, constraints, and return value. No output schema exists, but the return info is provided. The description is fully adequate given the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for both parameters (id and verbose), so the schema already provides meaning. The description does not add new parameter details; it only reinforces that id must be a draft invoice. Baseline 3 is justified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Delete a Billy invoice', specifies the resource type, and adds a critical constraint 'drafts only'. It distinguishes from voiding/crediting approved invoices, making the purpose unambiguous and differentiated from sibling tools like approve_invoice.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when NOT to use ('approved invoices must be voided/credited instead') and suggests alternatives ('voided/credited'), providing clear usage boundaries. This helps the AI agent decide between this tool and others like void_daybook_transaction or create_invoice (for credit notes).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint annotation, the description reveals that ownerReference is a client-side filter within the page, and explains the verbose parameter's effect on output. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each conveying essential information. The first two sentences are clear; the third could be merged but is not excessive.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given full schema coverage and readOnlyHint, the description covers purpose, filtering, and response options. Absence of output schema is mitigated by the verbose parameter description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value by explaining that ownerReference is a client-side filter and that verbose controls response verbosity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the verb 'list' with the resource 'attachments' and clarifies they are 'file↔record links'. It distinguishes itself from sibling tools like 'attach_file'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides context on when to use the ownerReference filter ('to see what's attached to a specific record') but lacks explicit when-not-to-use or alternative tool guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already set readOnlyHint=true, so the description doesn't need to restate it. However, the description adds context about the specific data returned and its prerequisite role, which is valuable beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states what the tool does, second states when to use it. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with no output schema, the description adequately lists return fields and explains its role as a prerequisite. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema already provides 100% coverage for the single optional parameter. The description adds meaning by implying the parameter is optional and that calling without it yields default organization, aligning with 'Call this first to learn the organization ID'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool retrieves organization details and lists specific fields (name, base currency, VAT settings, etc.). It distinguishes itself from sibling tools by indicating it should be called first to obtain organization ID and base currency.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to call this tool first to learn organization ID and base currency used by other tools, providing clear guidance on when and why to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description discloses partial update behavior ('Only provided fields are changed') and replacement behavior for prices ('Providing prices replaces all existing prices'), supplementing annotations (readOnlyHint=false, destructiveHint=false) without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. Front-loaded with verb and resource. Highly concise and structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the key behavioral aspects (partial update, price replacement). Could mention that other fields are optional or that the response format varies, but not necessary given 100% schema coverage and no output schema. Minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Adds significant meaning beyond the schema: clarifies that it's a partial update and that prices array replaces all existing prices. Schema coverage is 100%, but these behavioral nuances are not captured in the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action 'Update an existing Billy product' and specifies key behaviors like partial update and price replacement. Distinct from sibling tools like create_product and other update tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied use for modifying an existing product, but no explicit mention of when not to use or alternatives like create_product for new products. Could be improved but still clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes beyond annotations (destructiveHint=true) by detailing the real accounting effects, the need for BILLY_WRITE_MODE for non-GET methods, and the silent ignoring of undocumented query params—critical behavioral context not captured in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is relatively long but well-structured with clear warnings and examples. It front-loads the main purpose and uses concise language. Minor opportunity to trim redundant cautions, but overall effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a raw request tool with no output schema, the description covers all necessary aspects: how to construct requests, handling of body, query, path, method, and response truncation. It also warns about silent failures and write mode constraints, making it fully self-contained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds significant clarity: explains the singular resource key wrapping for body, the path format, query parameter usage, method enum defaults, and the maxChars truncation behavior. This adds value beyond the schema's descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as an 'escape hatch' for Billy API v2 endpoints not covered by dedicated tools, providing specific examples (GET /currencies, /countries, etc.). It distinguishes itself from sibling tools which are all dedicated endpoints.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (uncovered endpoints), when not to (prefer dedicated tools), and provides critical usage constraints: write mode requirements, silent ignoring of query params, and real accounting effects of POST/PUT/DELETE.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate destructiveHint=true, and the description expands with rich behavioral details: every mutating call is logged server-side, mutation log always returned even on throw, dry_run simulates writes, read-only mode blocks non-GET, confirm mode requires approval. Limits (timeout, API calls, output cap) are disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is thorough but slightly long; however, it is well-structured with clear sections (purpose, scope, safety, limits, gotchas). Every sentence adds value. Minor improvement could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (scripting, 4 params, no output schema), the description comprehensively covers return values (mutation log always returned, dry_run response), safety modes, limits, and Billy-specific gotchas. It fully enables correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds meaningful context beyond the schema: the 'script' parameter's available APIs, 'dry_run' simulation behavior, default timeout and max API calls. It clarifies parameter semantics without redundancy.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs JavaScript scripts against the Billy API for batch operations, specifying the language and use cases (loops, aggregation, reconciliation). It distinguishes from sibling tools by positioning itself as an alternative to many single-tool calls.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool (for loops, aggregation, reconciliation instead of many single calls) and provides examples. It also covers safety modes (dry_run, read-only, confirm) and Billy gotchas, guiding proper usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds significant behavioral context beyond the readOnlyHint annotation. It discloses that subjectReference is enforced client-side within the fetched page, impacting efficiency expectations. It also explains the verbose parameter's default behavior and context savings (90%). No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each serving a distinct purpose: purpose, usage example, and behavioral note. No filler or redundant information. Front-loaded with the core message.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema, but the description compensates by detailing the verbose parameter's behavior. It covers pagination implicitly and addresses filtering behavior. For a list endpoint, this provides sufficient context for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. However, the description enriches the semantics: it provides concrete examples for subjectReference ('invoice:<id>' or 'bill:<id>'), clarifies client-side enforcement, and explains the verbose parameter trade-off (context savings). This goes well beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists 'balance modifiers (payment associations)' and provides a specific usage example filtering by 'subjectReference'. The verb 'List' and resource 'balance modifiers' are distinct from sibling tools like list_accounts, list_bills, etc. The purpose is unambiguous and differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear when-to-use example: 'Filter by subjectReference ... to see which payments settled a specific invoice or bill'. This explicitly ties the tool to a common use case. It does not explicitly list alternatives or when-not-to-use, but the context is sufficient for most cases. Sibling tools like list_bank_payments serve different purposes, so confusion is unlikely.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behavioral traits: file type rejection with 422 error, append-only nature (files cannot be deleted or modified). This adds context beyond annotations (destructiveHint: true) by explaining the specific mutability constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise, well-structured, and front-loaded with the core purpose and constraints. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers most aspects: purpose, accepted types, outcome, limitations. However, lacks detailed description of return value structure (e.g., exactly which fields in compact vs full response). Minor gap given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but description adds value by explaining the purpose of verbose (saves ~90% context) and fileName override. This goes beyond schema descriptions, meriting a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (upload a local file), accepted file types, and expected outcome (get file ID). Distinguishes from sibling attach_file by noting that this tool is for uploading while attach_file links to a record.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly specifies when to use (upload files) and what file types are accepted/rejected. Provides behavioral guidance: files are append-only and cannot be modified or deleted via API. Points to attach_file for subsequent linking.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![billy-mcp MCP server](https://glama.ai/mcp/servers/Thourum/billy-mcp/badges/card.svg)](https://glama.ai/mcp/servers/Thourum/billy-mcp)

Score Badge

Copy to your README.md:

[![billy-mcp MCP server](https://glama.ai/mcp/servers/Thourum/billy-mcp/badges/score.svg)](https://glama.ai/mcp/servers/Thourum/billy-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Thourum/billy-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server