Server Quality Checklist

Tool Scores

• fred_searchA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes the output (series IDs, titles, metadata) but does not disclose if the operation is read-only, requires authentication, or has rate limits. The lack of contradictions keeps it at a minimum.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, efficiently covering purpose and usage context. It is front-loaded with key information and contains no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 10 parameters and no output schema, the description lacks details on pagination, ordering, filtering logic, or exact metadata returned. The schema covers parameters individually, but overall context is only adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so each parameter has a description in the schema. The tool description adds no new meaning beyond the schema, only vaguely mentioning 'keywords, tags, or filters.' Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for FRED data series by keywords, tags, or filters and returns matching series with IDs, titles, and metadata. It implies distinguishing from siblings (browsing or getting specific series) but does not explicitly differentiate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'Use this to find specific series when you know what you're looking for,' giving a usage context. However, it does not mention when not to use it (e.g., for browsing categories or retrieving full series) or suggest alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fred_get_seriesA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It correctly indicates a read operation ('Retrieve data') and lists supported transformations, but it does not disclose potential errors (e.g., invalid series ID), rate limits, or any side effects. The description is adequate but lacks depth on behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—two sentences with no redundancy. It front-loads the core action and then lists key capabilities efficiently. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 11 parameters and no output schema. The description covers purpose and supported features but does not explain the return format (e.g., observations with dates and values). Given the complexity, more detail on what the response contains would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with detailed descriptions for all 11 parameters. The description adds no specific parameter-level meaning beyond the schema, only a high-level overview. Per rules, a baseline of 3 is appropriate given the rich schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Retrieve data' and the resource 'any FRED series by its ID', directly indicating the tool's function. It also mentions supported features like transformations, frequency changes, and date ranges, which distinguishes it from siblings fred_browse and fred_search that handle browsing and searching, not data retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving data for a known series ID, but it does not explicitly guide when to use this tool versus its siblings. No direct 'when to use' or 'when not to use' advice is provided, leaving the agent to infer the context from the description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fred_browseA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It doesn't explicitly state that the tool is read-only or idempotent, though browsing operations are inherently safe. The description could benefit from mentioning that it does not modify data, but the current text is adequate for standard browsing behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the purpose and lists options efficiently. Every word earns its place; no extraneous information. Ideal for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description covers the main use cases and parameter dependencies. However, it does not describe the return format (e.g., 'returns a list of nodes' or 'paginated results'). Adding a brief note about the output would make it fully complete for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds value by explaining the mapping of browse_type to different browsing modes and implying which parameters are relevant (e.g., category_id for category_series). This helps the agent understand parameter combinations beyond the individual descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Browse' and resource 'FRED's complete catalog'. It distinguishes from sibling tools (fred_get_series for data retrieval, fred_search for text search) by focusing on catalog navigation via categories, releases, and sources. The browse_type enum provides specific options, making purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for each browse_type option (e.g., 'use browse_type='categories' to explore the category tree'), but does not explicitly state when to prefer this tool over fred_search or fred_get_series. While the usage is clear for browsing actions, exclusions could be added for completeness.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.