Server Quality Checklist

Tool Scores

• record_transactionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It states the debit/credit balance requirement, but does not explain side effects, whether transactions are immutable, concurrency behavior, or error handling. The lack of annotation coverage makes this a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with three short sentences and a clear list of keys for entries. It front-loads the core purpose and includes the critical balancing constraint without excess wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, and the schema provides minimal type info with no descriptions. The description touches on the essenpurpose and one constraint, but does not cover side effects, validation errors, default memo behavior, or interaction with reconciliation. Given the complexity of a financial transaction tool, more guidance is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for undocumented parameters. It explains the 'entries' structure (keys: account, amount_cents, side) and the balancing rule, but provides no details on 'transaction_id' or 'memo' beyond their names. This partial coverage leaves some parameters under-specified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool records a transaction in a ledger, using the verb 'Record' with the resource 'transaction'. It distinguishes from siblings by focusing on the ledger entry creation, but does not explicitly contrast with reconciliation or balance queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for recording financial entries but does not provide explicit when-to-use or when-not-to-use guidance. It mentions the double-entry requirement (debits equal credits), which is a key usage constraint, but no alternatives are named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_accountB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. The description states it creates an account, implying a write operation, but doesn't disclose any side effects, permissions required, or whether the operation is reversible. It also doesn't mention what happens if an account with the same name exists, or any other behavioral traits. This is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with two sentences. The first sentence states the purpose, and the second provides a critical constraint. It's front-loaded and efficient, with no wasted words. However, it could be slightly more structured by separating the constraint into a clearer format, but it's still well within acceptable conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that there is no output schema and no annotations, the description should provide more context. It explains the account_type constraint but doesn't describe what the tool returns, any side effects, or error conditions. For a creation tool with two parameters, it's minimally adequate but lacks completeness in terms of behavioral and return-value context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The description adds meaning to 'account_type' by listing valid values, which is helpful. However, it doesn't add any semantics for the 'name' parameter beyond what the schema provides (a string). The description partially compensates for the lack of schema descriptions but doesn't fully cover both parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Create a new account in the ledger.' It uses a specific verb ('create') and resource ('account in the ledger'), which distinguishes it from siblings like 'record_transaction' and 'get_balance'. However, it doesn't explicitly differentiate from potential sibling tools that might also create entities, but given the sibling list, it's clear enough.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear constraint on the 'account_type' parameter, which is a usage guideline. However, it doesn't explicitly state when to use this tool versus alternatives, nor does it mention any prerequisites or context. The sibling tools are different enough that the purpose is clear, but no explicit when/when-not guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_balanceB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the operation and unit but does not mention error behavior, permission requirements, or whether the tool is purely read-only. The lack of any caveats or context makes it under-transparent for a tool with zero annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, perfectly front-loaded, and every word adds value (operation, resource, unit). There is no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one string parameter, no output schema, and no annotations, the description is adequate but thin. It covers the core purpose and return unit, but leaves out likely failure conditions, expected input format, and any behavioral caveats. It is minimally viable but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain the parameter meaning. It only says 'an account' without clarifying how account_name is formatted, validated, or what values are accepted. This adds little beyond the schema's own parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Get the balance of an account' and adds the unit of measure 'in cents'. It clearly distinguishes from siblings like record_transaction or reconcile, as it is a read-only balance query.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies a read-only balance retrieval but provides no explicit when-to-use guidance or mention of alternatives. For a simple getter this is minimally acceptable, but it does not address edge cases or distinguish from sibling operations like reconcile.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reconcileA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral burden. 'Check' implies a non-mutating read-only operation and 'across the ledger' gives scope, but the description does not disclose return behavior or what happens when equality is not satisfied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one focused sentence with no filler. The verb and object are front-loaded, making the purpose immediately clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description captures the core function well. It could add return-value details, such as whether it returns a boolean or a report, but it is adequate for a simple reconciliation check.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the input schema is empty. There is no parameter ambiguity for the description to compensate for, so the baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Check that total debits equal total credits') against a clear resource ('the ledger'). It is distinct from sibling tools like get_balance, create_account, and record_transaction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The use case is implied: an agent would call this when verifying ledger balance. However, it does not explicitly say when to prefer reconcile over get_balance or other alternatives, nor does it provide when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.