Server Quality Checklist

Tool Scores

• get_historical_ratesB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It adds useful context: it specifies the API key requirement (ALLRATES_API_KEY) and describes the data granularity for each period (e.g., hourly for 1d, daily for 7d/30d, weekly for 1y), which goes beyond the input schema. However, it does not cover other behavioral aspects such as rate limits, error handling, response format, or whether it's a read-only operation, leaving gaps for a tool with authentication needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences: the first states the purpose and period details, and the second specifies the API key requirement. It is front-loaded with key information and avoids unnecessary words, though it could be slightly more structured (e.g., separating period details into a list). Every sentence adds value, making it efficient but not perfectly optimized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, authentication requirement, no output schema), the description is moderately complete. It covers the purpose, period options with granularity, and authentication need, but lacks details on output format, error cases, or sibling tool differentiation. Without annotations or an output schema, more behavioral context would be beneficial, but it meets a minimum viable level for basic understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear documentation for source, target, and period parameters. The description adds minimal value beyond the schema: it mentions 'currency pair' which aligns with source/target, and lists the period options with data granularity details, but does not provide additional syntax, format, or usage examples. Given the high schema coverage, a baseline score of 3 is appropriate as the description compensates slightly but not significantly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get historical exchange-rate data points for a currency pair over a period.' It specifies the verb ('Get'), resource ('historical exchange-rate data points'), and scope ('currency pair over a period'), which distinguishes it from siblings like get_exchange_rate (likely current rates) and list_currencies (likely currency metadata). However, it doesn't explicitly differentiate from get_rates_authenticated, which might also provide historical data, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context by listing available periods (1d, 7d, 30d, 1y) and stating it requires an AllRatesToday API key, which suggests when to use it (for historical data with specific timeframes and authentication). However, it lacks explicit guidance on when to use this tool versus alternatives like get_rates_authenticated or get_exchange_rate, and does not mention any exclusions or prerequisites beyond the API key.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_rates_authenticatedA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context: authentication requirements (AllRatesToday API key), rate limit characteristics ('higher limits'), and input format for targets ('comma-separated'). However, it doesn't describe the return format, error conditions, or what 'higher limits' specifically means compared to other tools.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded with the most important information. Every sentence earns its place: the first states the core purpose and key differentiators, the second covers authentication requirements, and the third provides a concrete example of parameter usage. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides adequate but incomplete context. It covers authentication, rate limits, and multi-target format, but doesn't describe the return values, error handling, or how it differs from sibling tools beyond 'higher limits.' For a tool with 4 parameters and authentication requirements, more behavioral context would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema: it clarifies the 'target' parameter accepts 'comma-separated targets like EUR,GBP,JPY' which is implied but not explicitly stated in the schema description. This meets the baseline of 3 when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get rates with higher limits and multi-target support.' It specifies the action ('Get rates') and key capabilities (higher limits, multi-target support). However, it doesn't explicitly differentiate from sibling tools like 'get_exchange_rate' or 'get_historical_rates' beyond mentioning higher limits and multi-target support.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when higher limits and multi-target support are needed, and when an AllRatesToday API key is available. It mentions 'comma-separated targets' as a specific usage pattern. However, it doesn't explicitly state when NOT to use it or name alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_exchange_rateA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses key behavioral traits: returns a 'single rate number' (output format), 'No API key required' (authentication needs), and 'current mid-market' (rate type). It doesn't mention rate limits, error conditions, or data freshness, but provides useful operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences that each earn their place: states purpose, specifies return format, and discloses authentication requirement. No wasted words, front-loaded with core functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter read operation with no output schema, the description provides good coverage: purpose, return format, and authentication context. It could mention data source or refresh frequency for completeness, but covers the essentials well given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema (both parameters are ISO 4217 codes). Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get'), resource ('current mid-market exchange rate'), and scope ('between two currencies'). It distinguishes from sibling tools by specifying it returns a 'single rate number' (vs. historical rates or authenticated rates) and 'No API key required' (vs. get_rates_authenticated).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('Get the current mid-market exchange rate') and implicitly contrasts with siblings through 'No API key required' (vs. get_rates_authenticated) and 'single rate number' (vs. historical data). However, it doesn't explicitly state when NOT to use this tool or name alternatives directly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_currenciesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively adds context beyond the input schema by specifying that no API key is needed (implying no authentication requirements) and that data is cached for 24 hours upstream (indicating potential rate limits or freshness considerations). However, it does not detail response format or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, consisting of two sentences that efficiently convey the tool's purpose, key features (no API key, caching), and scope. Every sentence adds value without redundancy, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no output schema, no annotations), the description is largely complete, covering purpose, authentication, and caching. However, without an output schema, it could benefit from briefly mentioning the return format (e.g., list of objects with code, name, symbol) to fully guide usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately does not discuss parameters, maintaining focus on the tool's purpose and behavior, which aligns with the baseline expectation for tools with no parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List all supported currencies') and resource ('currencies'), with explicit details on what information is included ('code, name, and symbol'). It effectively distinguishes from sibling tools like get_exchange_rate or get_historical_rates, which focus on rates rather than listing currencies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by stating 'No API key required' and 'Cached 24h upstream', which helps determine when to use this tool (e.g., for basic currency info without authentication). However, it does not explicitly mention when not to use it or name alternatives among siblings, such as get_rates_authenticated for authenticated rate data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.