Server Quality Checklist

Tool Scores

• get_seriesB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It explains parameters and return type (observation series), but does not disclose behavior like pagination, error handling, rate limits, or the effect of the limit parameter on results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with a clear structure: core purpose, then parameter details. No unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists (context shows has_output_schema=true), so return values are covered. However, missing details about limit parameter behavior and potential data truncation limit completeness. Also no mention of how the series data is structured (e.g., frequency, units).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must explain all parameters. It covers series_id (with examples), start/end (ISO format), but omits the limit parameter. This leaves ambiguity about its purpose and default behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it returns observation series for a FRED economic data series, with examples of series IDs. However, it does not explicitly differentiate from siblings like get_series_latest or search_series.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that series_id is required and start/end are optional, but it lacks guidance on when to use this tool versus alternatives like get_series_latest or search_series. No when-not or context for selecting this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_seriesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully convey behavioral traits. It mentions returning 'most popular matches' (sorting) and includes frequency, units, and observation range. However, it does not explain pagination, error handling, or how the 'limit' parameter affects results. The description adds value beyond the schema but is not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no extraneous information. It front-loads the core action and purpose, then adds value by specifying the output purpose ('so an agent can pick the right series_id'). Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given two parameters and the existence of an output schema (not shown), the description adequately covers the tool's purpose and main output. It could include more on how search results are ordered or pagination, but overall it is fairly complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must clarify parameters. It explains that 'query' is a keyword search, but does not describe the 'limit' parameter beyond its schema presence. For a tool with 2 parameters, this is partial but sufficient for basic understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Search' and resource 'FRED catalog for series'. It clearly states the output: 'most popular matches with their frequency, units, and observation range', which helps distinguish from siblings like 'get_series' that retrieve a single series by ID.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for finding a series_id when you don't know it, but it doesn't explicitly state when not to use or compare to alternatives like 'get_series' or 'get_series_latest'. The phrase 'so an agent can pick the right series_id' gives context, but lacks explicit exclusion guidelines.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_release_calendarA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavioral traits. It indicates a read operation (return) but does not mention any potential side effects, auth requirements, or rate limits. For a simple calendar lookup, this is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with two sentences: one stating the primary function and one adding context. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one parameter and an output schema present, the description is largely complete. It could briefly mention the output format or typical release examples, but it covers the essential use case adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate. It mentions 'days' but only repeats the parameter name ('next *days* days') without explaining its meaning, format, or constraints beyond the schema's default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Return), the resource (upcoming FRED data releases), and the scope (next *days* days). It distinguishes itself from siblings like get_series or search_series by being a calendar tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit context for when to use the tool (macro overlay, event risk flagging) and distinguishes from siblings implicitly. However, it does not mention when not to use it or provide alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• health_checkA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. Mentions it doesn't call FRED, but does not describe what the probe does, what it returns, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: two short sentences with no waste. Front-loaded with purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and output schema exists, description covers the essential purpose but lacks detail on what the health check entails.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, schema coverage 100%. Description adds no further parameter info, but baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it is a local health probe, and 'Never calls FRED' distinguishes it from sibling tools that might call external services. The verb 'probe' and resource 'health' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance. The phrase 'Never calls FRED' implies it is safe to call frequently, but no alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_series_latestA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It mentions 'cheaper' as a behavioral trait but does not disclose error handling, rate limits, or prerequisites like series existence. The presence of an output schema reduces the need to explain return format, but more context on behavior is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. First sentence states purpose, second provides usage guidance. Front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema present), the description covers the core purpose and usage. However, it omits minor but helpful details like error handling or validation, but overall is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, yet the description adds no extra meaning to the single parameter series_id beyond its name. While the usage implies it's a FRED series identifier, no format or examples are given, making it less helpful for an agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the single most-recent observation for a FRED series, using specific verb 'Return' and resource. It also distinguishes itself from the sibling tool get_series by noting it's cheaper.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Cheaper than get_series when an agent only needs the current value of a macro indicator', providing a clear when-to-use and when-not-to-use directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_server_infoA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description effectively discloses the key behavioral trait: it is a local operation with no FRED calls, which is sufficient for this simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences that are front-loaded and contain no unnecessary words, efficiently conveying purpose and a key differentiator.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema, the description fully covers what the tool does, its context (local metadata), and its distinct behavior (no FRED calls).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the baseline score of 4 applies; the description adds value by specifying the tool's scope beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves local server metadata and explicitly distinguishes itself by noting 'Never calls FRED,' which separates it from siblings that likely rely on FRED.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for local metadata without external calls but does not explicitly state when to use this tool versus alternatives like get_series or search_series.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.