Server Quality Checklist

Tool Scores

• finary_organizationsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'List' and mentions 'authenticated user', which implies a read-only operation but does not state side effects, rate limits, response format, or any required authentication details. The lack of explicit behavioral context is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no redundant words. It is perfectly concise and front-loads the core action, making it efficient for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema), the description is minimal but lacks important context. It does not explain pagination, what an 'organization' entails, or how this relates to sibling tools. For a tool with no annotations, more behavioral and situational details are needed for complete understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is trivially 100%. The description adds nothing about parameters because none exist, but the baseline for zero-parameter tools is a 4, and the tool meets this baseline as there is nothing to clarify.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the function: 'List Finary organizations' with the scope 'available to the authenticated user.' The verb 'list' is specific and the resource is clear. It doesn't explicitly differentiate from sibling tools, but the name and context make the intent obvious, so it earns a 4.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like finary_portfolio or finary_transactions. An agent is left to infer usage from the name and generic phrasing. There are no exclusions or specific scenarios mentioned, so this dimension is weakly addressed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finary_transactionsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states 'read', which implies a non-mutating operation, but it does not disclose pagination semantics, filter behavior, response structure, or any side effects. For a tool with 9 parameters, this is insufficient behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, which is concise but arguably too sparse for a tool with 9 parameters. It front-loads the core purpose, but lacks structure to organize the various optional filters, making it under-specified rather than appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 9 parameters, no output schema, and no annotations, the description is incomplete for an agent to call the tool correctly. It does not explain date formats, pagination behavior, query syntax, or how filters interact, leaving too much to guesswork.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only generically mentions 'optional filters' and names account types, which maps loosely to the 'account_type' parameter, but it does not explain any of the other parameters (page, query, marked, account, perpage, dates, institution). The added value over the raw schema is minimal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'read', the resource 'transactions', and scopes to specific account types (checking, investment, credit). This distinguishes it from sibling tools that deal with user, org, connections, and portfolio data, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this is the tool for transaction data, and since no sibling tool covers transactions, the usage context is evident. It does not explicitly mention alternatives, but given the sibling set, the division of responsibility is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finary_portfolioB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the verb 'Read', implicitly indicating a non-mutating operation, but it does not mention permissions, rate limits, return format, or any side effects. The list of asset types adds domain context but not behavioral transparency. This is a significant gap for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence with no unnecessary words. The verb is front-loaded, and the enumeration of asset types is compact and informative. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema and no annotations, the description should explain what the tool returns and any other usage context, but it does not. It omits any mention of the return structure, error handling, or the org_id parameter, which is essential to know when calling the tool. The enumeration of asset types covers only part of the invocation requirements, leaving the definition incomplete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. It explicitly lists the valid values for asset_type, which is useful and goes beyond the schema (where no descriptions exist). However, it completely ignores org_id, leaving its purpose and usage undocumented. The partial compensation leads to a middling score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Read') applied to a well-defined resource ('portfolio class'), and enumerates all valid asset types (investments, securities, cryptos, etc.), making the tool's purpose unambiguous. It naturally distinguishes itself from sibling tools by naming a resource category not covered by any sibling (e.g., me, organizations, transactions).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for reading portfolio holdings of a given asset type, but it provides no explicit guidance on when to choose this tool over siblings. Since siblings are clearly different domains (me, organizations, connections, transactions), the context is somewhat clear, but the description does not explicitly state scenarios or exclusions, so it remains at the 'implied' level.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finary_institution_connectionsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full burden of behavioral disclosure, but it only states the action. It does not mention read-only nature, response format, pagination, or any caveats about connection status semantics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, concise sentence with no filler. The key action and object are front-loaded, making it easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter listing tool, the description is reasonably complete. However, without an output schema or any mention of response fields (e.g., what 'status' values look like), an agent lacks some context about the expected result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the baseline is 4. The description correctly adds no unnecessary parameter detail, and there is nothing to compensate for since the schema is trivially complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a clear resource ('connected financial institutions') along with a relevant detail ('connection status'). It clearly distinguishes this tool from siblings like finary_portfolio or finary_transactions by focusing on institution connections.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no indication of when to use this tool versus alternatives, nor any exclusions or prerequisites. An agent is left to infer the use case from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finary_meA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It explicitly states 'Does not modify Finary', which is a key behavioral trait for a read operation. It does not mention return format or errors, but for a simple profile read this is acceptable. Adding 'Read' reinforces the non-destructive nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the core action and a clear non-modification statement. No fluff or redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only tool, the description covers the essential purpose and side-effect behavior. It does not explicitly state return data format (no output schema exists), but an agent can reasonably infer it returns the profile object. Minor gap, but adequate for this simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the baseline is 4. The description does not need to explain parameters because none exist. Schema coverage is 100% vacuously, and no parameter documentation is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Read' with a specific resource, 'authenticated Finary profile', and explicitly states it does not modify. This distinguishes it from sibling tools like finary_transactions or finary_portfolio without ambiguity. The purpose is clear and specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates the tool retrieves the user's own Finary profile, which sets clear context for when it applies. However, it does not explicitly mention alternatives or conditions for when not to use it, though the resource is so distinct that usage is implied. Missing explicit routing to siblings, but adequate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.