Server Quality Checklist

Tool Scores

• get_tableB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full behavioral burden. It successfully discloses the return format ('JSON string with formatted table data') and important constraints (names 'must match series_codes length'), but fails to declare safety properties (read-only status), rate limits, or what 'formatted' specifically entails.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description follows a clean docstring structure with front-loaded purpose ('Get a formatted table...') followed by Args and Returns sections. Despite the parameter documentation being lengthy, it is necessary given the schema lacks descriptions. No sentences appear wasted or redundant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers basic operation, parameters, and return values, which suffices for simple usage. However, it lacks context on the BCRP domain, does not differentiate functionality from get_data, and provides no safety/authorization guidance expected for a data retrieval tool with no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage, the Args section effectively compensates by documenting all three parameters: series_codes (identifying them as BCRP codes), names (with the length constraint), and period (with specific format examples 'YYYY-MM/YYYY-MM' or 'YYYY'). This provides the semantic meaning missing from the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a 'formatted table with custom column names,' specifying both the action (Get) and distinguishing feature (custom column names) that differentiates it from sibling tools like plot_chart and search_series. However, it omits the domain context (BCRP) from the main sentence, only mentioning it in the Args section.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus siblings like get_data (likely for raw data retrieval) or plot_chart. There are no 'when to use' or 'when not to use' statements, forcing the agent to infer suitability based solely on the 'formatted table' keyword.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_dataA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It successfully documents the return format (JSON string with array of records) and date range behavior (format specifications and null handling). However, it lacks information on safety (idempotency), performance (rate limits), or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The docstring-style format (Args/Returns) is structured and readable. Information is efficiently presented with minimal redundancy, though the formal structure is slightly less front-loaded than pure prose. Every sentence provides necessary detail about parameters or return values.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-parameter read operation with an output schema, the description adequately covers the input semantics despite zero schema coverage. It explains the BCRP domain context, parameter formats, and return structure. It could benefit from mentioning data frequency or timezone handling, but covers the essentials.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Excellent compensation for 0% schema description coverage. The description provides concrete examples for series_codes (e.g., 'PN01652XM') and detailed format specifications for the period parameter ('YYYY-MM/YYYY-MM' or single 'YYYY-MM'), plus null behavior. This adds critical semantic meaning absent from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Fetch), resource (time series data), and scope (specific BCRP series codes). However, it does not explicitly differentiate from sibling tools like 'search_series' (which likely finds codes) or 'get_table' (which likely returns tabular rather than time-series format).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. Given siblings include 'search_series' and 'get_table', the description should clarify that this requires known series codes and returns structured time-series data rather than search results or visualizations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• plot_chartA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It successfully communicates the side effect (saves a PNG file to disk) and return value (file path), but fails to mention other critical behavioral aspects like whether it overwrites existing files, required directory permissions, or error handling when series codes are invalid.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The structure is well-organized with purpose front-loaded, followed by return value and parameter details. The 'Args:' format is readable and efficient. Minor deduction for the vague adjective 'professional' which doesn't add technical clarity, and the description could slightly condense the optional parameter notes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters and file output, the description is nearly complete. It documents all parameters and discloses the PNG output behavior even though an output schema exists (which relieves some descriptive burden). Minor gap: it doesn't specify the chart type (line, bar, etc.) or default file naming behavior when output_path is omitted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage, the description excellently compensates by documenting all 5 parameters in the Args section with clear semantics, data types (List, string), and specific format guidance (e.g., 'YYYY-MM/YYYY-MM' for the period parameter). This provides essential value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Generate[s] a professional chart for BCRP series data,' providing a specific verb, resource type, and domain context. It implicitly distinguishes itself from siblings like get_data and get_table by focusing on visualization rather than raw data retrieval, though it could explicitly mention 'visualization' or 'PNG image' in the primary sentence.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to select this tool versus alternatives like get_data or get_table. While mentioning 'Returns the path to the saved PNG file' hints at the output format difference, there is no 'when-to-use' or 'when-not-to-use' guidance, nor any mention of prerequisites like valid BCRP series codes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_seriesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and discloses key behavioral traits: 'deterministic search with fuzzy matching' explains the algorithm, and 'ambiguity error if multiple matches are equally scored' documents error conditions. It also specifies the return structure containing 'codigo_serie and confidence'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The Args/Returns structure is slightly formal but efficiently organizes information. Every section serves a purpose: the opening defines scope, the algorithm sentence explains behavior, and the Returns section documents output despite the presence of an output schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter search tool with an existing output schema, the description is complete. It covers the search mechanism, error behaviors, return format, and parameter semantics without unnecessary verbosity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, requiring the description to compensate. It successfully does so by defining the query parameter as a 'Search term' and providing concrete Spanish-language examples ('tipo de cambio', 'inflacion', 'PBI') that clarify expected input format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search'), resource ('BCRP economic indicators'), and mechanism ('by keyword'), clearly distinguishing it from siblings like get_data (retrieval) and plot_chart (visualization).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The examples ('tipo de cambio', 'inflacion') provide implied usage context for finding series codes, but there is no explicit guidance on when to use this versus get_data directly or workflow prerequisites (e.g., 'use this first to obtain codigo_serie').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.