Server Quality Checklist

Tool Scores

• get_series_infoA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. The verb 'Get' suggests read-only behavior and the listed metadata fields indicate what is returned, but it does not disclose potential side effects, authentication needs, rate limits, or error behavior. It is not misleading but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely compact: a one-sentence purpose statement followed by a minimal Args block. Every word earns its place, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (1 parameter) and the existence of an output schema, the description sufficiently covers purpose and parameter semantics. It could be slightly more complete by noting that search_series should be used if the series ID is unknown, but overall it's adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. The Args section clearly defines 'series_id' with concrete examples ('UNRATE' or 'GDPC1'), which adds meaning far beyond the bare schema title 'Series Id'.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves metadata for a specific FRED series, listing example metadata fields (title, units, frequency, range, notes). This specific verb+resource phrasing distinguishes it from siblings like search_series (searching) and get_observations (retrieving data points).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you already know a specific series ID, but it does not explicitly state when to use this tool versus search_series or get_observations, nor does it mention any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_observationsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for disclosing behavior. It explains the compact return format, the point-count cap, the '.' sentinel for missing values, and parameter behaviors (e.g., units, frequency, sort_order). It does not cover error cases or rate limits, but given the output schema exists, the return metadata structure is not needed here. Substantial behavioral disclosure is present, though a few edge behaviors are omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured, starting with a one-sentence purpose, followed by a concise overview of behavior and caps, then a bulleted list of arguments. Every sentence contributes useful information, and the formatting makes it easy to scan. It is appropriately compact for the complexity of the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 7 parameters, no annotations, and no schema descriptions, the description covers all the essential invocation details: required series_id, date handling, transformations, aggregation, sorting, and limits. It also notes the result-size cap and missing value representation. It does not mention potential prerequisites like API keys or rate limits, but those are not implied by the tool's context. The presence of an output schema leaves return format details to the schema. Overall, it is nearly complete, with minor room for adding explicit exclusions or prerequisites.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description includes an 'Args' section that thoroughly explains every parameter: series_id with example, start/end dates with format and inclusivity, units with possible values, frequency with possible values, sort_order, and limit. This fully compensates for the lack of schema descriptions, adding significant meaning beyond the raw property types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource: 'Get the observations (actual time-series values) for a FRED series.' It clearly distinguishes this from sibling tools like search_series (searching) and get_series_info (metadata) by indicating this tool retrieves time-series data points. The purpose is unambiguous and action-oriented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool, such as retrieving observations over a date range with optional transformations. It gives practical guidance on handling result-size caps (narrow date range or use frequency aggregation), but it does not explicitly name alternatives or state when not to use this tool relative to siblings. This is clear context without explicit exclusions, meriting a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_seriesA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the search-and-return behavior, ordering, and the practical mapping from natural language to series_id. It stops short of describing the exact return structure, but the output schema likely covers that. This is solid for a non-destructive search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a lead sentence, a usage context sentence, a result-ordering note, and a clear Args list. Every sentence adds value, no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple two-parameter input, an output schema that likely defines return values, and clear guidance on purpose and usage, the description is complete. It gives the agent all needed context to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description fully compensates. It explains query as free-text search with examples, and limit as max number of series with a range (1-50). This adds meaning well beyond the bare schema properties.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches FRED for data series matching a query, with a specific verb ('Search'), resource ('FRED'), and scope ('economic data series'). It also distinguishes itself from siblings by indicating its role in converting natural language to a series_id for get_observations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this first' and names the downstream consumer (get_observations), providing clear when-to-use guidance. It also mentions ordering by popularity, which helps set expectations. No exclusions needed for a search tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.