Server Quality Checklist

Tool Scores

• invoice_listA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds beyond this by specifying the return shape ({total, invoices[]}) and including that it returns full line_items and payment state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded and relatively concise, but it combines all information in a single paragraph. The contradiction with the schema reduces trust.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 9 parameters with 100% schema coverage, no output schema, the description adequately provides return structure and usage context. The contradiction is a gap but overall completeness is good.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. However, the description contradicts the schema on limit default (says 50, schema says 20) and max (says 200, schema says 100), which will mislead an agent. It does add some explanatory context for filters like overdue_only.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search and filter invoices') and resource, and distinguishes from siblings by noting the tool's output includes full line_items and payment state, with explicit use cases for dashboards and driving invoice_remind/invoice_risk workflows.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises when to use this tool ('build dashboards or drive invoice_remind/invoice_risk workflows') and documents optional filters and pagination. It does not explicitly state when not to use it, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoice_createA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behaviors: auto-calculation of subtotal, discount_total, tax_total, total; generation of invoice_number in format INV-YYYY-NNNN; setting status to 'draft'; and returning the full invoice object. Annotations already indicate it is not read-only and not destructive, but the description adds context beyond annotations without contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise yet comprehensive: 5 sentences that front-load the purpose, cover required/optional parameters, defaults, auto-calculations, and return value. Every sentence contributes essential information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with no output schema, the description covers the return value ('full invoice object') and important details like default values and auto-calculations. It does not mention error conditions or validation failures, but given the rich schema and annotations, the completeness is high.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining parameter defaults (currency defaults to client's default_currency then USD, due_date defaults to issue_date+30 days), constraints (line_items must be non-empty), and auto-calculation behavior. This provides meaning beyond the schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Create a new invoice for an existing client.' It specifies the verb (create), resource (invoice), and includes detailed parameter requirements. The tool is distinct from siblings like invoice_send or invoice_mark_paid, which are for different actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for creating invoices but does not explicitly state when not to use it or mention alternatives. It says 'Required: client_id (UUID) and line_items' but lacks exclusions like 'Do not use this for drafts created elsewhere.' No sibling comparisons are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoice_mark_paidA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=false and destructiveHint=false, confirming it's a write operation. The description adds significant transparency by detailing side effects: updates amount_paid, amount_due, status only when balance reaches zero, and updates client payment_history. It also specifies the return value (confirmation with paid amount and remaining balance). No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose, followed by behavioral details and side effects. Every sentence adds value without redundancy or extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the medium complexity (3 parameters, no output schema, side effects), the description covers essential behavior, side effects, and return value. It could mention validation (e.g., invoice must exist and not be fully paid) but overall is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already describes each parameter. The description reiterates that amount is optional for full balance settlement, but adds little beyond the schema. It integrates parameters into the tool's flow but doesn't provide new details per parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Record a full or partial payment against an invoice.' It details the specific verbs and resources (updates amount_paid, amount_due, sets status conditionally) and distinguishes itself from sibling tools like invoice_create, invoice_list, and invoice_remind by focusing on payment recording.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives. While it describes the primary function and side effects, it lacks guidance on when not to use it (e.g., if payment already recorded) or references to alternative actions (e.g., invoice_create for new invoices).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoice_riskA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint true, indicating safe, non-destructive behavior. The description adds value by explaining the model inputs (invoice amount, client history, etc.) and output structure (risk score, factors, recommended action), which goes beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose, and each sentence adds value: purpose, model inputs, output and use case. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one param, no output schema), the description fully covers purpose, behavioral model, return values, and usage context, making it complete for agent selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter 'invoice_id', and the schema already describes it as a UUID. The description only says 'for a specific invoice', adding no additional meaning beyond the schema, so baseline score applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool predicts late-payment risk for a specific invoice on a 0-100 scale, using a specific verb and resource. It also mentions the use case (prioritizing collection effort), distinguishing it from sibling tools like invoice_list or cashflow_report.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use for prioritizing collection effort on high-value invoices, providing clear context. However, it does not mention when not to use the tool or suggest alternatives, leaving a minor gap.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• client_manageA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description fully discloses behavioral traits: safe to call repeatedly, idempotent, returns existing record on duplicate email, and describes the return structure (id, payment_history, timestamps). Annotations (idempotentHint=true, destructiveHint=false) are consistent and reinforced.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose, and every sentence adds necessary context. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 10 parameters, full schema coverage, and no output schema, the description provides sufficient context: it explains the return object, idempotency via email, and links to invoice_create. Could mention update behavior, but it's implicit in 'returned unchanged'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds value by explaining the dedup role of email and listing enum options for default_currency, but does not significantly enhance meaning for other parameters beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates or upserts a client record used by invoice_create, specifying accepted fields and return object. It distinguishes itself from sibling tools which are all invoice-related, so no confusion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context by stating it's used by invoice_create and explains idempotent behavior via email dedup. However, it does not explicitly instruct when to use this tool versus alternatives, though no direct alternative exists among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoice_demo_seedA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations set destructiveHint:false and readOnlyHint:false. Description adds that each call appends a fresh batch with unique UUIDs, making it safe and non-destructive. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences in a single paragraph, front-loaded with purpose. Detailed but not verbose. Could be slightly more structured with bullet points, but it's effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With zero parameters and no output schema, the description fully explains the tool's behavior, data generated, and integration with other tools. Mentions return values (counts and sample IDs). Complete for a seeding tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has zero parameters, so description cannot add param detail. Schema coverage is 100% (empty). Description explains what the tool does without parameters, which is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool populates the store with a demo dataset, specifying 8 clients across 5 archetypes and 25-45 invoices with line items, tax, etc. It distinguishes from sibling tools by being a seeding tool for evaluation purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this to evaluate InvoiceFlow via MCP Inspector without real credentials' and 'Safe to call multiple times'. It implies when to use (testing) and not to use (real data). No explicit when-not to use alternatives, but context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoice_sendA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond annotations (readOnlyHint=false, destructiveHint=false), the description discloses PDF generation, status change to 'sent', and conditional email attempt. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences, each adding essential information: action, always generates, conditional email, return value. No redundant text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With good annotations and schema coverage, the description covers key behaviors (PDF, status, email) and return value. Could mention handling of already-sent invoices, but overall complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents parameters. The description adds value by specifying that 'message' defaults to a summary of amount and due date, which is not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool generates and delivers an invoice PDF, marking it as 'sent'. It distinguishes from siblings like invoice_create (creates) and invoice_mark_paid (marks paid) by focusing on delivery and PDF generation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use (to send invoice) and provides context on email delivery depending on environment variable. It mentions out-of-band handling as an alternative, but does not explicitly list exclusions or alternative tools by name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• payment_reconcileA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description thoroughly discloses behavioral traits: on match, the invoice is marked paid, amounts are updated, and payment_history is recomputed. It also notes that no error is thrown on no match, and that the first invoice is reconciled when multiple match. This exceeds annotations which only indicate non-readonly and non-destructive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, front-loaded with purpose, then matching rule, then effects, then return behavior. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers matching logic, effects, return behavior, and touches on edge cases (multiple matches). However, it does not explain the fate of 'reference' and 'payment_method' parameters (e.g., stored or ignored), and lacks detail on validation errors.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with descriptions. The description adds beyond schema by specifying the tolerance (within one cent) and case-insensitivity for email, providing additional context not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool matches an incoming external payment to an open invoice, using specific verbs and resources. It distinguishes itself from siblings like invoice_mark_paid by focusing on reconciliation from external sources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the matching rule and behavior when multiple invoices match, but does not explicitly state when to use this tool versus alternatives like invoice_mark_paid. The context of external payments vs manual marking is implied but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cashflow_reportA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint: true, idempotentHint: true, and destructiveHint: false. The description adds behavioral details like null handling for avg_days_to_payment and forecasting logic for projected_income_30d, which go beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured paragraph that front-loads the main purpose and lists return values concisely without extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description fully enumerates all return fields (period, totals, rates, breakdowns) and includes handling for null values and forecast logic, making it complete for a zero-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters (schema coverage 100% with empty properties), so baseline is 4. The description correctly states 'Takes no input,' which is sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Generate a portfolio-wide cash flow summary across all invoices' with specific verb ('Generate') and resource ('cash flow summary'), and the listed return fields distinguish it from sibling tools like client_manage or invoice_list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Takes no input — always reports on the current full dataset. Ideal for dashboards and monthly close,' providing clear context and when-to-use guidance without needing alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• invoice_remindA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses full behavioral details: increments fields, changes status, returns a message, and safely refuses on paid invoices. No contradiction with annotations (readOnlyHint=false, destructiveHint=false).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words, front-loaded with the core action. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Completeness is high: describes return value (generated message), side effects, and safe refusal. No output schema, but description compensates fully. No gaps for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Adds meaning beyond the schema: explains that message is optional with a default friendly reminder referencing invoice details, and invoice_id is a UUID. Schema coverage is 100%, but description enriches understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool records a payment reminder, specifying the side effects (increments reminder_count, sets last_reminder_at, advances status) and distinguishes it from invoice_send by noting it does not actually send email.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says to use invoice_send for actual delivery and that it refuses to remind on paid invoices, providing clear when-to-use and when-not-to-use guidance. This distinguishes it from related tools like invoice_send and invoice_mark_paid.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.