Server Quality Checklist

Tool Scores

• get_transactionsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral transparency burden. It discloses the read-only nature via 'Fetch' and explains the default recent window behavior, which is useful context. However, it does not cover pagination, auth requirements, or other edge cases, leaving it adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, consisting of three short sentences with the key action front-loaded. Every sentence earns its place: the action, the timestamp format, and the default behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description does a reasonable job explaining the core behavior and the optional date range. It lacks details about the return format and edge cases, but for a four-parameter tool with a clear schema, it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters, so the baseline is 3. The description adds value by clarifying that dates are Unix timestamps and by explaining that omitting the date range makes the bridge return its default recent window, which goes beyond the schema's per-parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches transactions for connected accounts, using the specific verb 'Fetch' and a clear resource. The sibling tools (list_accounts, connection_status, claim_setup_token) are about accounts, status, and setup, making the transaction resource distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the agent the tool is for retrieving transactions and gives context about the date range, but it does not explicitly mention when to prefer this tool over alternatives or provide exclusions. Usage is implied by the purpose, but clear when-to-use guidance is absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• claim_setup_tokenA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It discloses that the token is one-time, that the exchange yields a permanent Access URL, and that the URL is stored. It also warns of the single-claim constraint. Yet it omits possible side effects (e.g., overwriting existing stored URLs) and error behavior, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no unnecessary words. The first sentence conveys the action, source, and result; the second delivers a critical limitation. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with no output schema, the description covers the source of the token, the transformation, storage, and the single-use constraint. It lacks explicit mention of the function's return value (e.g., whether it returns the permanent Access URL) and error cases, but these are not critical for basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully describes the setupToken parameter. The tool description adds context by stating the token is Base64 and originates from 'bridge.simplefin.org/simplefin/create', plus underlines its one-time nature. This enhances understanding beyond the schema's baseline description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action: 'Exchange a one-time Base64 setup token... for a permanent Access URL and store it.' This clearly differentiates from sibling tools like connection_status, list_accounts, and get_transactions, which deal with status and data retrieval rather than initial setup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Run this once to connect' provides clear usage timing, indicating this is the initial setup step. Additionally, the note 'a setup token can only be claimed a single time' warns against re-use. However, it does not explicitly name alternative tools or conditions under which this tool should not be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_accountsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It adds useful behavioral context: 'Fast', 'balances-only mode', and 'no transactions'. However, it does not disclose other behavioral aspects such as authentication requirements, read-only nature, or possible pagination, though for a simple list tool this is a moderate gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, front-loaded with the primary purpose. Every sentence provides distinct value: the first states what it does, the second adds scope and performance. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with no parameters and no output schema, the description conveys the core output (accounts and balances), the exclusion (transactions), and a performance trait. It could specify the exact return structure or ordering, but given the low complexity, the description is sufficiently complete for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and schema coverage is 100% (trivially). The description adds no parameter details, but none are needed. The baseline for a zero-parameter tool is 4, as there is no missing semantics to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('accounts') with clear scope ('all connected accounts with current balances'). It also explicitly excludes transactions, which differentiates it from the sibling tool get_transactions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool: for account balances, not transactions. The phrase 'no transactions' serves as an exclusion, and the 'balances-only mode' hints at the lightweight use case. However, it does not explicitly name alternatives like get_transactions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• connection_statusA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a key behavioral trait: 'without revealing credentials,' which is important for trust and safety. It also implies a read-only status check, but does not detail response formats or edge cases. Still, it adds meaningful behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the purpose and includes a valuable security note. Every word earns its place, with no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although there is no output schema or annotations, the description adequately conveys the core behavior for a simple status-check tool: it reports configuration status and avoids leaking credentials. It could mention the exact output format or that no external calls are made, but for this low-complexity tool, the description is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description is not required to elaborate on parameter meanings. According to the rubric, 0 params yields a baseline score of 4, and the description adds no unnecessary parameter info, which is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Report whether an Access URL is configured.' It uses a specific verb ('Report') and resource ('Access URL') and distinguishes itself from siblings like claim_setup_token and list_accounts by focusing on status rather than setup or data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: to check if an Access URL is configured either via environment variables or a prior claim. However, it does not explicitly mention alternative tools or exclusions, but the context is sufficient for basic usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.