Server Quality Checklist

Tool Scores

• get_articleA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses a non-obvious behavioral trait: refusal is identical whether the article does not exist or the user lacks read permission, which prevents information leakage. It does not mention error return format or permissions, but the security behavior is valuable and goes beyond a bare 'read' statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: the first states the core purpose and input source, the second states a key behavioral detail. Every word earns its place, and the verb is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with two params and no output schema, the description adequately explains what it does and an important error behavior. It does not describe the response format or permission requirements, but these are not critical given the tool's simplicity and the input provenance is covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description repeats that repo and slug come from search_articles, which adds slight context beyond the schema's 'as returned by search' but does not provide new meaning. No additional parameter semantics are given.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Read') and resource ('one article in full'), and clearly distinguishes from siblings by indicating it operates on a repository and slug returned by search_articles. This unambiguously differentiates it from search_articles (search) and list_repos (list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage after search_articles by specifying the inputs are what search_articles returned, giving clear context for when to invoke this tool. However, it does not explicitly state when not to use it or name alternatives, so it stops short of full guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_reposA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses a key behavioral trait: the list is restricted to readable repositories and explicitly excludes non-readable ones. It also mentions included data (article counts). No mention of error behavior or pagination, but for a simple read-only list, this is reasonable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose and scope. No redundant text; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main return value (repositories and article counts) and explicitly states a limitation (no non-readable repos). Given the tool's simplicity and lack of parameters, this is sufficient, though it could optionally mention output format or pagination.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is trivially complete. Per the rubric, 0 params baseline is 4. The description doesn't need to add parameter information because there are none.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: lists repositories readable by the token, with article counts. It distinguishes from sibling tools (search_articles, get_article) which focus on articles rather than repos. Scope is explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context: use when you need to know which repositories the token can read and their article counts. It does not explicitly mention alternatives or exclusions, but the sibling names make the distinction obvious. No wrong-usage guidance, but also no explicit 'when-not-to-use'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_articlesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It discloses ranking logic (titles > summaries > headings > prose), strict word matching, and permission filtering (returns only entitled articles). It does not cover rate limits or pagination, but the mentioned behaviors are useful and not contradicting any annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, front-loaded with the main purpose. Each sentence adds essential information: scope and verb, ranking behavior, and matching/permission constraints. There is no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with no output schema, the description adequately explains scope, ranking, and filtering. However, it does not describe the return format (e.g., list of article titles, summaries) or pagination behavior, which would be helpful for agents. The sibling tools and parameter schema help, but a slightly richer description would be more complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds semantic value by explaining the query parameter's behavior (every word must appear, adding words narrows results), which goes beyond the schema's simple 'What to look for'. It doesn't add details for repo or limit but meaningfully enhances the query parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches how-to documentation across all readable repositories, using the specific verb 'Search' and identifying the resource. It distinguishes this from siblings like get_article (which likely retrieves a specific article) and list_repos (which lists repositories).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use it ('across every repository this token may read') and explains matching semantics (all query words must appear), which helps in selecting it. It doesn't explicitly mention exclusions or alternatives, but the scope and behavior are clear enough. A stronger statement contrasting with get_article would merit a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.