Server Quality Checklist

Tool Scores

• diagnose_write_contextC
  2.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states 'Read-only,' which is a clear behavioral disclosure that the tool has no side effects. This is valuable given the absence of annotations. However, it does not mention authentication requirements, error behavior, or what the diagnostic output contains. It provides only a minimal behavioral trait and some scope, but the burden is higher without annotations, so a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no fluff. It is front-loaded with 'Read-only' and states the three diagnostic domains compactly. While it could be more informative, the structure is efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This tool has five parameters, no output schema, and no annotations. The description does not explain the purpose of each parameter, what the diagnostic output looks like, or any prerequisites. It is far too sparse to allow an agent to correctly invoke the tool. A comprehensive description would need to cover parameter roles, expected response, and usage context, all of which are missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description bears the full responsibility for explaining parameters. It mentions 'wallet ownership,' 'browser write config,' and 'category-v2 runtime mapping,' which could loosely relate to walletId, token, and category fields, but it does not explicitly define any parameter. The token parameter is entirely unaddressed. The description provides no meaningful semantics for the five parameters, so the agent cannot determine what values to supply.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Read-only diagnostics for wallet ownership, browser write config, and category-v2 runtime mapping.' It names specific resources and the diagnostic nature, which distinguishes it from simple getters like get_wallets or get_categories. It does not explicitly contrast with sibling tools, but the purpose is unambiguous and not a tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description offers no guidance on when to use this tool versus alternatives. It does not state conditions under which diagnostics are needed, nor does it mention when to avoid using it. While 'diagnostics' implies a troubleshooting use-case, there is no explicit context or exclusion, leaving the agent to guess.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_user_infoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full disclosure burden. It hints that authentication is required ('authenticated profile') but does not state what happens when no valid token exists, how the token override interacts with the default auth context, or what the returned profile contains. For an operation with zero annotation coverage this is a gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no fluff. It's efficiently written, though its brevity is partly why behavioral details are missing — the compactness is good, the content is thin.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple tool — one optional parameter, no output schema, no nested objects — so a minimal description is nearly sufficient. But for a read operation it would benefit from stating the fallback behavior when no token override is supplied and whether unauthenticated calls error out. Adequate but lean.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% on the single token parameter ('Optional access token override'), so the schema already documents it. The tool description adds no parameter-level meaning beyond what the schema conveys, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb+resource (retrieve the authenticated profile) that distinguishes it from the data-fetching siblings (get_wallets, get_categories, get_transactions). However, it doesn't explicitly differentiate from login, which also concerns authentication — the boundary between 'fetch the profile' and 'authenticate' is implied rather than stated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to call this tool or when to prefer it over siblings. Given that the token parameter is described as an 'override,' the description never states that prior authentication (via login) is a prerequisite, nor does it mention any condition under which this should or shouldn't be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transactionsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'fetch', which implies read-only, but does not explicitly state side-effect-free behavior, authentication requirements, pagination, or return format. The agent lacks critical context about what happens when this tool is called.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero redundancy. It is appropriately front-loaded with the core action and scope, and every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters, no output schema, and no annotations, the description is too sparse. It fails to mention return format (e.g., list vs single object), pagination, error handling, or whether the token parameter is ever required. An agent would lack sufficient information to correctly interpret the response.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are already documented in the schema. The description adds minimal extra meaning ('between two dates' clarifies the date range, 'for a wallet' reinforces walletId), but does not provide new details beyond what the schema already contains.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Fetch transactions for a wallet between two dates' clearly states the verb (fetch), resource (transactions), and scope (wallet + date range). It distinguishes itself from siblings like get_wallets and get_categories, which target different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives like preview_transaction or add_transaction. The description simply states what it does, leaving the agent to infer when it is appropriate without explicit context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_walletsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of explaining behavior. It says 'List' which implies a read, but it doesn't explicitly state it's non-destructive, doesn't mention response structure, pagination, rate limits, or any side effects. The

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, no fluff, but it's so short it doesn't convey much. For a simple tool it's acceptable, but there's room for additional clarifiers. However, front-loading is good.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, the description covers the core purpose. However, no mention of return format, pagination, or any edge cases. Given no annotations and no output schema, a bit more would help, but the simplicity might justify a 3.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers the single optional token parameter with a description. The tool description adds nothing about the parameter's usage or purpose beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('wallets'), and adds a clear scope ('accessible to the authenticated user'), distinguishing it from other list-style tools that operate on different resources. This leaves no ambiguity about what the tool returns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives like get_transactions or get_categories. It doesn't mention any prerequisites (e.g., authentication token), nor does it explain when not to use it. The description is a bare statement with no usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_categoriesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Retrieve' indicates a read operation and the wallet scope is clear, but the description does not disclose response shape, error behavior, authentication requirements, or whether default categories are returned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single front-loaded sentence with no filler. Every word earns its place and the key scoping constraint is stated directly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with complete schema coverage, the description is minimally sufficient: an agent knows what to call and with what scope. However, the absence of annotations and output schema leaves some gaps around return format and behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds only the contextual link 'for a specific wallet' to walletId, while token is already documented in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Retrieve') and a clear resource ('categories') for a specific wallet, so the purpose is obvious and distinct from sibling tools like get_wallets or get_transactions. It does not explicitly contrast itself with a sibling, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the wording: an agent would use this when it needs categories for a wallet. However, there is no explicit when-to-use guidance, no exclusions, and no mention of alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• preview_transactionB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations supplied, the description carries full responsibility for disclosing behavior. It does well by explicitly stating that no write occurs, which is the most important behavioral trait for an AI agent to know. It also indicates that category resolution and duplicate detection are performed, adding meaningful behavioral context beyond argument names.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single efficient sentence: an action, a result, and a safety constraint, all without redundancy. It front-loads the core behavior and does not waste tokens on restating the name or title.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 8 parameters, no output schema, no annotations, and minimal schema descriptions, so the description must carry substantial weight. It covers the no-write safety message and category resolution, but omits the expected return value, behavior when a duplicate is found, error handling, authentication via token, and how to choose categoryId versus categoryName. An agent would be under-served when attempting to correctly invoke this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 13%, and the schema contains no descriptions for 7 of 8 parameters. The description's mention of category resolution relates to categoryId/categoryName, but it does not explain amount, date, note, walletId, token, or runtimeCategoryId. The agent must infer their meanings from names and types alone, which is risky.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states two concrete behaviors: resolving a category and checking for an exact duplicate, while explicitly noting that nothing is written. It gives the tool an identity beyond its title, though it stops short of explicitly naming how this differs from sibling tools like add_transaction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'without writing anything' implies this is a read-only/preflight variant of add_transaction, giving some contextual guidance. However, it never explicitly says when to prefer this tool over add_transaction, diagnose_write_context, or get_categories, nor does it mention prerequisites like being logged in.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_transactionA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behaviors: category ownership validation, duplicate blocking, and the confirmation requirement. It does not mention side effects like balance updates or failure modes, but the disclosed behaviors are significant.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, with the core action front-loaded and supporting details following. It is concise, avoids redundancy, and is well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives enough context to understand the operation's nature but omits information about the return value or what happens on success/failure. Given that there is no output schema and the description does not mention these aspects, it is not fully complete for an agent to anticipate results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides descriptions for about 73% of parameters, and the tool description does not add extra meaning to the undocumented ones (e.g., token, with, allowDuplicate). The description relies on the schema, which covers most but not all parameters, so the score is moderate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (create a transaction) and adds context about validation and duplicate blocking. However, it does not explicitly mention the resource (wallet) or differentiate from sibling tools, though the core purpose is evident.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It specifies that explicit confirmation is required and that duplicates are blocked by default, which provides some usage guidance. However, it does not indicate when to use this tool versus preview_transaction or get_transactions, nor does it mention that this is the write operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the core behavior—authenticating and returning an access token—but omits token lifetime, session effects, or error behavior. This is adequate for a simple login but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One compact sentence with no filler; the authentication action and return value are front-loaded. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is sufficient for a simple two-parameter login call, but without annotations or an output schema it leaves the token's subsequent use unspecified and does not describe the response shape. These are meaningful gaps for an agent needing to chain this with sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are fully described in the schema (100% coverage), so the description adds no parameter-level detail beyond what the schema already provides. The baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Authenticate') and resource ('Money Lover'), and clearly distinguishes itself from the sibling data-operation tools by being the only authentication tool. It also specifies the outcome: returning an access token.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys that login is for obtaining an access token, which implies it should precede authenticated calls, but it never explicitly says 'call this before other tools' or mentions alternatives/exclusions. Usage context is therefore implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.