Server Quality Checklist

Tool Scores

• get_archiveB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that the tool returns 'Full NYT archive API response (unformatted)', which hints at the raw, unprocessed nature of the output, but fails to address critical aspects such as rate limits, authentication requirements, error handling, or data format specifics. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose stated first, followed by parameter details and return information. Each sentence adds value without redundancy. However, minor improvements could include briefer phrasing or bullet points for parameters, but overall it is efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, no output schema), the description is adequate but incomplete. It covers the purpose and parameters well but lacks details on output structure, error cases, and usage context relative to siblings. Without an output schema, more explanation of the 'unformatted' response would enhance completeness, making it minimally viable with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics beyond the input schema, which has 0% coverage. It explains that 'year' and 'month' parameters are for specifying the archive period, provides default values (current year/month), and clarifies valid ranges (month 1-12). This compensates well for the lack of schema descriptions, though it could detail constraints like year limits or handling of invalid inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get New York Times articles from a specific month and year archive.' It specifies the verb ('Get'), resource ('New York Times articles'), and scope ('from a specific month and year archive'), but does not explicitly differentiate it from sibling tools like 'get_latest_news' or 'search_articles' in terms of when to use each, which prevents a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention sibling tools such as 'get_latest_news' for recent articles or 'search_articles' for keyword-based searches, nor does it specify use cases like historical research versus current news. This lack of comparative context results in minimal guidance for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_articlesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions pagination (page parameter) and returns formatted articles, but lacks details on rate limits, authentication needs, result limits, error handling, or whether this is a read-only operation. The description doesn't contradict annotations (none exist), but provides minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with a clear opening sentence followed by Args/Returns sections. Every sentence adds value, though the opening could be slightly more specific about what makes this search unique compared to siblings.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 5 parameters, no annotations, and no output schema, the description covers parameters well but lacks behavioral context (rate limits, auth) and doesn't explain the return format beyond listing fields. It's adequate but has clear gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by documenting all 5 parameters with clear semantics: query purpose, sort options with default, date formats, and pagination details. It adds significant value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches New York Times articles with specific criteria (query, date range, etc.), which is a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'get_latest_news' or 'get_archive', which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_latest_news' or 'get_archive'. It mentions search criteria but doesn't specify use cases, exclusions, or comparisons with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_latest_newsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'real-time news feed' and describes the return format, which adds some context. However, it lacks details on rate limits, authentication needs, data freshness, or potential errors (e.g., invalid sections). For a tool with no annotations, this is a significant gap in behavioral transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It starts with a clear purpose statement, followed by organized sections for 'Args' and 'Returns' with bullet-like formatting. Each sentence adds value: the purpose sets context, parameter details are essential given low schema coverage, and return format clarifies output. There is no wasted text or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, 0% schema coverage, and no output schema, the description does a decent job but has gaps. It fully documents parameters and return format, which is crucial. However, it lacks behavioral context (e.g., rate limits, errors) and usage guidelines versus siblings. For a tool with this complexity and lack of structured data, it's adequate but incomplete, meeting minimum viability with clear room for improvement.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It provides detailed semantics for all four parameters: 'limit' (number of items, default 20), 'offset' (pagination offset, default 0), 'source' (news source options and default), and 'section' (news section, default, example, and reference for available names). This adds substantial meaning beyond the bare schema, effectively documenting parameter usage and constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the latest news items from the NYT news wire (real-time news feed).' It specifies the verb ('Get'), resource ('latest news items'), and source ('NYT news wire'), making the action and target explicit. However, it doesn't explicitly differentiate from siblings like 'get_most_popular' or 'search_articles', which likely serve different purposes (e.g., popularity-based vs. search-based retrieval).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions a 'real-time news feed' but doesn't clarify how this differs from siblings like 'get_archive' (historical news) or 'get_most_popular' (popular articles). There are no explicit when-to-use or when-not-to-use instructions, leaving the agent to infer usage based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_most_popularA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the return format but doesn't cover important aspects like rate limits, authentication needs, error handling, or whether this is a read-only operation. For a tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections for Args and Returns, making it easy to parse. It's appropriately sized with no redundant information, though it could be slightly more concise by integrating the resource reference more seamlessly. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is partially complete. It covers parameters well and specifies the return format, but lacks behavioral details like error handling or rate limits. Without annotations or output schema, it should do more to be fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning beyond the input schema, which has 0% description coverage. It explains the semantics of both parameters: 'type' as popularity type with options and default, and 'time_period' as time period in days with options and default. This fully compensates for the lack of schema descriptions, providing clear parameter context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the most popular New York Times articles.' It specifies the resource (NYT articles) and the verb (get), but it doesn't explicitly differentiate from siblings like 'get_latest_news' or 'search_articles' in terms of popularity vs. recency or search criteria. The purpose is clear but lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying the type of popularity and time period, but it doesn't explicitly state when to use this tool versus alternatives like 'get_latest_news' or 'search_articles'. There's no guidance on prerequisites or exclusions, leaving usage context somewhat implied rather than clearly defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bestseller_listA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions pagination via 'offset' and references an external resource for list names, but lacks details on rate limits, authentication needs, error handling, or the structure of the 'unformatted' response. For a tool with no annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement followed by Args and Returns sections. It uses bullet-like formatting efficiently, though the 'Returns' section could be more descriptive. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides basic purpose and parameter info but lacks details on response format, error cases, or operational constraints. It is minimally viable for a simple retrieval tool but incomplete for robust agent use, as it doesn't fully address the complexity implied by external references and pagination.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful context for both parameters: 'list' is explained with examples and a reference for available names, and 'offset' is clarified as a pagination offset with default values. This goes beyond the basic schema, though it could detail format constraints more explicitly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('New York Times bestseller lists'), distinguishing it from sibling tools like get_archive or get_latest_news. It precisely identifies what data is retrieved without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying the type of data retrieved (bestseller lists), but does not explicitly state when to use this tool versus alternatives like get_most_popular or search_articles. No guidance is provided on prerequisites or exclusions, leaving context somewhat open-ended.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.