Server Quality Checklist

Tool Scores

• get_interest_ratesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It explains what data is returned (ECB policy rates) and mentions historical data capabilities, but doesn't disclose behavioral traits like rate limits, authentication requirements, data freshness, or error conditions. The description adds value but doesn't fully compensate for the lack of annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with purpose first, then details, then concrete examples. Each sentence adds value, though the examples section could be slightly more concise. The front-loading of the core purpose is effective.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a data retrieval tool with no annotations and no output schema, the description provides adequate context about what data is returned and sample use cases. However, it doesn't describe the return format, data structure, or potential limitations. Given the 4 parameters and lack of output schema, more completeness would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 4 parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema descriptions. It implies date range and filtering capabilities through the examples, but doesn't provide additional semantic context about parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get key ECB interest rates' with specific resources identified (MRO, Deposit Facility, Marginal Lending Facility). It distinguishes from siblings by focusing specifically on ECB policy rates rather than exchange rates, inflation, or other economic data tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about what questions this tool answers (ECB rate queries), but doesn't explicitly state when to use this versus alternatives like 'get_yield_curve' or 'get_exchange_rates'. The examples help clarify the scope but don't provide explicit exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_money_supplyA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses what data is returned ('outstanding amounts or annual growth rates') and provides definitions of M1/M2/M3 aggregates. However, it doesn't mention behavioral aspects like rate limits, authentication requirements, data freshness, or error conditions that would be important for an agent to know.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It starts with the core purpose, explains return options, defines key terms (M1/M2/M3), and provides concrete examples - every sentence adds value. The information is front-loaded with the most important details first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter tool with no annotations and no output schema, the description provides adequate context about what the tool does and examples of use cases. However, it lacks information about return format structure, error handling, and other behavioral aspects that would help an agent use it effectively, especially given the absence of output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description adds some value by explaining what M1/M2/M3 represent and providing example use cases, but doesn't add parameter-specific semantics beyond what's already in the comprehensive schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get euro area monetary aggregates (M1, M2, M3) from the ECB.' It specifies the verb ('Get'), resource ('monetary aggregates'), source ('ECB'), and distinguishes it from siblings by focusing on money supply data rather than exchange rates, inflation, or other economic indicators.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context through examples of questions the tool answers, indicating it should be used for queries about M1/M2/M3 money supply levels or growth rates. However, it doesn't explicitly state when NOT to use this tool or mention alternatives among the sibling tools for related economic data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_inflationA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It adequately describes what data is returned (HICP inflation data) and the source (ECB/Eurostat), but lacks information about rate limits, authentication requirements, data freshness, error conditions, or response format. The description doesn't contradict any annotations (none exist).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It begins with the core purpose, adds details about data formats and scope, then provides concrete usage examples. Every sentence serves a purpose, though the example section could be slightly more concise. The information is front-loaded with the most important details first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a data retrieval tool with 5 parameters and no output schema, the description provides adequate context about what data is available and example use cases. However, without annotations covering behavioral aspects and no output schema describing the return format, there are significant gaps in understanding how to interpret results or handle potential issues.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing comprehensive parameter documentation. The description adds minimal value beyond the schema, mentioning 'annual rate of change (%) or index level' and 'euro area or individual countries' which are already covered in the schema's enum and descriptions. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('Get') and resource ('HICP inflation data from the ECB'), distinguishing it from siblings like get_exchange_rates or get_interest_rates. It identifies this as 'the official eurozone inflation measure' and specifies what data is returned (Harmonised Index of Consumer Prices).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance through three concrete example questions that demonstrate when to use this tool: for current eurozone inflation rates, country-specific inflation over time, and cross-country comparisons. This clearly differentiates it from sibling tools that handle currency conversion, interest rates, or dataset searches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_yield_curveA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that data is 'updated daily' and specifies available maturities and issuer types, adding useful context. However, it does not mention potential rate limits, error conditions, or authentication needs, leaving gaps in behavioral understanding for a data retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with core functionality, followed by details and examples. Each sentence adds value without redundancy, such as stating data source, maturity range, update frequency, and illustrative use cases, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description does a good job covering the tool's purpose, parameters, and usage context. It could improve by detailing return format or error handling, but it adequately informs an agent for a data query tool with clear parameters and examples.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value by specifying 'Available maturities range from 3 months to 30 years' and clarifying the data source and rating context, which enhances understanding beyond the schema's parameter descriptions, justifying a score above baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves 'euro area government bond yield curve data from the ECB' and specifies it returns 'spot rates from the ECB's AAA-rated government bond yield curve.' This distinguishes it from sibling tools like get_exchange_rates or get_inflation by focusing on bond yields, with specific examples reinforcing the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through example questions like 'What is the current 10-year euro area yield?' but does not explicitly state when to use this tool versus alternatives such as get_interest_rates or search_datasets. It provides context for bond yield queries but lacks direct guidance on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• explain_datasetA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes what the tool returns ('dimensions, valid values, example series key') and its scope ('all 100+ ECB datasets'), but lacks details on error handling, rate limits, or authentication requirements. The behavioral disclosure is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose in the first sentence. The bulleted examples are helpful but slightly verbose. Overall, it's efficient with little waste, though it could be slightly more streamlined.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (1 parameter, no output schema, no annotations), the description is mostly complete. It covers purpose, usage, and output content well, but lacks details on behavioral aspects like errors or limits. The absence of an output schema is partially compensated by describing return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'dataset_id' well-documented in the schema. The description adds minimal value beyond the schema by mentioning example IDs (e.g., 'EXR', 'FM') and referencing search_datasets for discovery, but doesn't provide additional syntax or format details. Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('explain the structure', 'understand what a dataset contains') and resources ('ECB dataset', 'dimensions, valid values, example queries'). It distinguishes from sibling tools by focusing on dataset metadata rather than data retrieval or conversion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('Use this after search_datasets to understand what a dataset contains') and provides clear alternatives by mentioning sibling tools in examples (e.g., 'EXR dataset' relates to get_exchange_rates). It also clarifies scope ('Works for all 100+ ECB datasets, not just the ones with dedicated tools').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_exchange_ratesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and adds valuable behavioral context beyond what the schema provides. It discloses that rates are published at '16:00 CET each business day' (timing constraint) and mentions the scope ('~40 currencies'), which helps set expectations about availability and freshness. No contradictions exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It starts with the core purpose, adds important behavioral context, then provides concrete example questions that illustrate usage without wasting words. Every sentence earns its place by adding value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only data retrieval tool with no output schema, the description provides good context about data source, timing, and scope. The example questions effectively demonstrate the tool's capabilities. However, without annotations or output schema, it could benefit from mentioning the return format or data structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 5 parameters thoroughly. The description doesn't add any parameter-specific semantics beyond what's in the schema descriptions (e.g., it doesn't explain date format nuances or the '+' syntax further). Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('Get') and resource ('EUR exchange rates from the European Central Bank'), distinguishing it from siblings like convert_currency (which likely performs conversion) and other economic data tools. It specifies the data source (ECB) and type of rates (reference rates for ~40 currencies).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool through example questions that demonstrate appropriate use cases (e.g., 'What is the current EUR/USD exchange rate?'). However, it doesn't explicitly state when NOT to use it or mention alternatives like convert_currency for actual currency conversion versus just getting rates.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_datasetsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly describes the search functionality and return format (matching dataset IDs and names), but doesn't mention limitations like result count, pagination, search scope, or error conditions. The behavioral information provided is adequate but lacks depth about operational constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with clear paragraph separation: purpose statement, usage guidance, and concrete examples. Every sentence adds value - no redundant information. It's front-loaded with the core functionality and appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (search operation with 1 parameter), no annotations, and no output schema, the description provides good contextual coverage. It explains the tool's role in the workflow, distinguishes it from siblings, and gives usage examples. However, it doesn't describe the output format in detail (just 'matching dataset IDs and names'), which would be helpful since there's no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already fully documents the single 'query' parameter. The description doesn't add any parameter-specific information beyond what's in the schema, maintaining the baseline score of 3 for adequate but not enhanced parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('Search') and resource ('ECB's 100+ statistical datasets'), and distinguishes it from siblings by explaining it returns dataset IDs and names for discovery. It explicitly differentiates from data retrieval tools like get_exchange_rates and metadata tools like explain_dataset.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('to discover what data the ECB publishes') versus alternatives ('use explain_dataset to learn its structure, or use the specific data tools... for common queries'). It includes three concrete example questions that demonstrate appropriate usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_currencyA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively explains key behaviors: the tool uses ECB reference rates, supports EUR-based conversions and triangulation, works with latest or historical rates, and handles specific date queries. However, it doesn't mention error conditions, rate limits, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and concise. It starts with a clear purpose statement, provides operational details in a compact format, and includes helpful examples that demonstrate real-world usage without unnecessary elaboration. Every sentence serves a clear purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a conversion tool with no annotations and no output schema, the description provides strong context about the ECB source, triangulation method, and date handling. The examples effectively illustrate usage patterns. However, without an output schema, some information about return format would be helpful, though the examples partially compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description adds some context about currency code examples and the date parameter's default behavior, but doesn't provide significant additional semantic value beyond what's in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('convert an amount between currencies') and resources ('using ECB reference rates'). It distinguishes itself from sibling tools like 'get_exchange_rates' by focusing on conversion rather than rate retrieval, and explicitly mentions the triangulation method for cross-currency conversions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool through the examples section, which shows it answers conversion questions. It also implicitly distinguishes from siblings by not being for retrieving datasets, exchange rates, or other financial data, making it clear this is for conversion operations specifically.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.