Server Quality Checklist

Tool Scores

• hn_categoryA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey behavior. It states that the tool returns a list of story dicts, implying a read-only operation, but it does not explicitly confirm this or mention rate limits, authentication, or any other behavioral constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using a clear three-line structure: purpose, Args, Returns. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (2 simple parameters, no required fields), the description covers the core functionality and parameter details. The presence of an output schema (not shown) reduces the need to describe return structure, but the description could still benefit from mentioning error cases or context about the source.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description explicitly lists valid enum values for category and the allowed range for limit (1-30), adding meaningful context beyond the input schema which only shows types and defaults. This fully compensates for the 0% schema coverage indicated in context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves stories from a specific HN category, with a list of valid categories. However, it does not distinguish itself from the sibling tool hn_top_stories, which might be interpreted as similar.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like hn_search or hn_top_stories. No when-not conditions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hn_get_commentsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It specifies that comments are top-level and in ranking order, and outlines the return structure, but does not disclose error behavior, rate limits, or what happens with invalid story_id.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a clear front-loaded purpose, followed by structured Args and Returns sections. Every sentence is informative with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has two parameters and an output schema described in the Returns section, the description covers the core functionality well. It lacks details on error handling or edge cases, but is adequate for a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description provides clear meaning for both parameters: story_id is the HN item id and limit is the number of comments (1-30, default 10). This adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets top-level comments for a story in HN ranking order, using specific verb and resource, and distinguishes from sibling tools like hn_get_story.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like hn_get_story or hn_search. There is no mention of context, prerequisites, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hn_top_storiesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, description carries full burden. It states returned fields but doesn't disclose read-only nature, potential API rate limits, or error cases. Adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two clear sections (Args, Returns) with no fluff. Every sentence adds value. Appropriate length for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers parameters, defaults, return fields (with output schema support). Missing error handling or preconditions, but sufficient for a straightforward read tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description fully compensates. It explains limit (1-30) and hours (window back from now), adding constraints and defaults beyond the schema's empty descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves top Hacker News stories from a time window, ranked by score. It distinguishes from siblings like hn_search (search) and hn_get_story (specific story), making intent unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool over alternatives (e.g., hn_search, hn_category). The description only explains functionality without context on preferred use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hn_get_storyA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must convey behavior. It states the tool returns a story dict with specific fields and an error dict if not found. This covers basic behavior for a read-only operation, but lacks details like authentication or rate limits, though not critical for this simple public API.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise, front-loaded with purpose, followed by clear Args and Returns sections. No wasted words; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-story retrieval tool with one parameter and no output schema, the description adequately explains input, output format (including error case), and purpose. No additional context is necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must explain parameters. It states 'story_id: The HN item id.' This provides some context beyond the schema's type and title, but is minimal. It compensates partially but could be more specific.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves full metadata for a single HN story by ID, using a specific verb ('Get') and resource ('story'), and distinguishes it from siblings like hn_search or hn_get_comments.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied: when you have a story ID and want metadata. However, no explicit when-to-use or when-not-to-use guidance is given, and alternatives are not mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hn_searchA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses return format (dict with hits and total_hits), hit structure (id, kind, title, etc.), sort defaults, limit range, and that it's full-text search via Algolia. Does not mention rate limits or idempotency, but search is inherently read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Structured as docstring with Args and Returns sections. Each sentence adds value, though the return format list is somewhat lengthy but necessary. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description thoroughly explains return values and hits structure. Parameters are fully explained. Covers defaults and constraints. Lacks error handling or edge cases, but sufficient for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds meaning beyond schema: explains query as search string, sort options with default 'relevance', limit range 1-50 and default 20. It compensates well for lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Full-text search across HN stories and comments via Algolia,' specifying the verb (search), resource (HN stories and comments), and method (Algolia). It distinguishes from sibling tools like hn_get_story and hn_top_stories, which are specific retrieval tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains parameters but does not explicitly state when to use this tool versus alternatives. It implies usage for broad search, but lacks explicit when-not or comparison to siblings, reducing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.