Server Quality Checklist

Tool Scores

• lookup_companyA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It does disclose input normalization ('Spaces and dashes are fine — they are stripped') and the list of returned fields, which is useful. However, it omits potential error conditions (e.g., NIP not found on the whitelist), rate limits, or any explicit statement that this is a read-only operation, leaving gaps in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two short sentences explaining the main purpose and one argument explanation. It is front-loaded with the core action and contains no redundant or filler words, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single parameter, read-only lookup) and the presence of an output schema, the description covers the essentials well—purpose, result fields, and input format. The only notable gap is the lack of explicit guidance on when to use this tool over sibling check_vat_status, but this is a minor omission for such a straightforward operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only provides the parameter name and type, while the description adds critical semantics: it specifies that the NIP must be a 10-digit Polish NIP and that spaces/dashes are stripped. This fully compensates for the absent schema description, providing an agent with all necessary input details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Look up a Polish company by NIP'), identifies the authoritative source ('Ministry of Finance VAT whitelist'), and enumerates the returned data (company name, VAT status, address, registration date, bank accounts). This differentiates it from sibling tools like check_vat_status by implying a full profile rather than a single status check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides contextual information about the target (Polish company, NIP) but does not explicitly explain when to use this tool versus the sibling check_vat_status or other alternatives. Usage is implied rather than stated, so some ambiguity remains for an agent deciding between tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_vat_statusA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the return format (yes/no/exempt answer plus company name) and describes the check as 'definitive', but it doesn't mention error handling for invalid NIPs, whether any side effects occur, or data source reliability. The description provides some transparency but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, immediately answering 'what does this do?'. The return behavior, use case, and argument format are each explained in a short, purposeful way without any wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the tool's simplicity, the description covers purpose, return values, a concrete use case, and parameter format. An output schema exists, and the description already clarifies the return structure, making it complete for agent decision-making.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description fully compensates for the single parameter 'nip' by specifying the format ('10-digit Polish NIP') and allowed formatting ('Spaces and dashes are fine'). This adds practical value beyond the schema's bare string type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('check') and resource ('NIP registered as an active VAT payer in Poland'). It distinguishes itself from siblings like lookup_company and get_all_rates by focusing specifically on VAT status verification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context for when to use the tool ('useful for fast agent decisions such as 'can I trust this invoice?''). While it doesn't explicitly name alternatives or exclusions, the use case is specific enough to guide selection among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_all_ratesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses a key behavioral nuance: NBP publishes rates only on banking days, so weekend dates return the most recent available table and the effective date is shown. This goes beyond the schema and helps the agent interpret results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences plus a compact Args block. It front-loads the core purpose and only adds necessary detail about the parameter. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one optional parameter, output schema available), the description is highly complete. It covers purpose, parameter semantics, and a real-world edge case. It does not list every currency or error handling, but that exceeds what's needed for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only a 'date' parameter with zero description coverage, so the description must compensate fully. It does: explains the ISO format, marks it optional, and describes what happens with weekend dates. This gives the agent everything needed to invoke the tool correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get all current PLN exchange rates from the National Bank of Poland' with specific scope (Table A, list of currencies). It differentiates from siblings like get_currency_rate, which likely returns a single rate, by emphasizing 'all' rates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides useful context (optional historical date, weekend behavior) but does not explicitly mention when to use this tool versus alternatives like get_currency_rate or convert_currency. The usage guidance is implied by the 'all' phrasing, not stated directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_currencyA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description bears full responsibility for disclosing behavior. It transparently states the use of NBP mid-market rates, the PLN pivot for foreign-to-foreign conversions, and the optional historical date. It does not mention potential edge cases like rate unavailability for certain currencies or errors, but it covers the core operational details beyond what a schema would show.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with a clear single-sentence purpose, followed by a concise Args list. Every sentence adds necessary information, with no filler or repetition. The structure is easy to scan and immediately usable by an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core functionality, parameter meanings, and the pivot mechanism. It has an output schema (though not shown), so return values are presumably handled there. Missing are notes on error handling or unsupported currency codes, but given the tool's moderate complexity, the description is sufficiently complete for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, and it does. Each parameter (amount, from_currency, to_currency, date) is explained with examples for currencies and an explicit format for date. This adds all the semantic meaning missing from the schema, which only has titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the tool's function: converting an amount between PLN and another currency or between two foreign currencies using NBP mid-market rates. It distinguishes this from sibling tools like get_currency_rate by emphasizing conversion of an amount rather than just rate lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys when to use this tool (for conversions) and explains the PLN pivot mechanism, which is essential for understanding how conversions work. It does not explicitly name alternatives or exclusions, but the sibling tool names (get_currency_rate, get_all_rates) imply those are for rate-only tasks, providing implicit differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_currency_rateA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses a meaningful behavioral detail: 'Tries Table A first and falls back to Table B for exotic currencies.' This explains why different currencies may be resolved via different NBP tables. It does not cover failure modes or error handling, but the disclosed fallback logic adds important context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the primary function in the first sentence, followed by a compact and clear Args block. There is no fluff, and every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 parameters, 1 required), and the description covers the data source (NBP), the currency pair (vs PLN), the fallback behavior, and parameter formats. Since an output schema exists, return value details are not needed. Minor gaps like error cases are not covered, but the core usage is fully specified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions for its parameters (0% coverage). The description compensates fully by defining currency_code as an ISO 4217 code with examples ('EUR', 'USD', 'GBP') and date as an optional ISO date (YYYY-MM-DD) for historical rates, providing formats and semantics beyond the raw field names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get the official NBP mid-market exchange rate for a single currency against PLN.' It uses a specific verb and resource, and the phrase 'single currency' differentiates it from the sibling tool 'get_all_rates'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates the tool is specifically for retrieving a single exchange rate against PLN and optionally for historical dates. It provides clear context but does not explicitly rule out alternatives like 'convert_currency' or state when not to use it, though the single-currency framing implicitly guides selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.