Server Quality Checklist

Tool Scores

• list_tagsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint true and idempotentHint true, so description need not repeat. Description adds article counts behavior but no further traits. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero wasted words. Front-loaded with purpose, efficient structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Combined with output schema and annotations, description provides sufficient context: specifies corpus, indicates output includes counts, and links to sibling. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with description, and the tool description adds no extra parameter details beyond what schema provides. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists all topic tags with article counts, specifying the corpus ('Sovereign AI Blog') and linking to sibling tool. Verb+resource+scope are explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly suggests using before calling search_blog with a tag filter, providing clear use context. No exclusion or when-not-to, but sufficient guidance for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_articleA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and idempotentHint true. The description adds the error behavior (returns article with error field on missing slug) and the return content structure (Markdown body plus metadata). This provides additional useful behavioral context beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: two sentences. The first states the primary purpose, the second covers return type and error case. No redundant or extra information. Every word contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple nature of the tool (one parameter, no nested objects, output schema exists), the description adequately covers retrieval and error handling. Missing discussion of authentication or rate limits, but these are not critical for this read-only, idempotent tool. Overall complete enough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a detailed parameter description. The tool description does not add new information about the slug parameter beyond what the schema already provides. Baseline score of 3 is appropriate for this case.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'retrieve', the resource 'blog article', and the key parameter 'slug'. It distinguishes from sibling tools like 'search_blog' by focusing on full content retrieval. The purpose is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates when to use the tool (to get full article content by slug). It implicitly guides usage after search_blog via the parameter description. However, it lacks explicit when-not or alternative tool references, which would elevate it to 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• diagnose_sglangA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint and idempotentHint. The description adds that it is 'pure pattern-matching, no inference, no external calls', and details the output structure (critical issues, warnings, recommended config). This goes beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded, and every sentence adds value. Two paragraphs with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 optional parameters and an output schema (not shown but noted), the description explains purpose, behavior, and output structure comprehensively. It also states when to use (with any available config info) and the 'no inputs' case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and parameter descriptions are adequate. The description reinforces that parameters are optional and how default values affect behavior (skip check), which adds value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates an SGLang configuration for a specific hardware (NVIDIA DGX Spark). It distinguishes itself from sibling tools (get_article, list_tags, search_blog) which are unrelated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that all parameters are optional, to supply only what you have, and that with no inputs it returns a recommended config and 'unknown' verdict. It does not explicitly discuss when not to use, but the sibling tools are unrelated so context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_blogA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond annotations (readOnlyHint, idempotentHint) by detailing the TF-IDF ranking mechanism, a scoring threshold (0.001), determinism for a given knowledge base snapshot, and the behavior matrix. It clearly states the tool is pure read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-line summary, followed by a behavior matrix table. Every sentence adds value, and there is no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the moderate complexity, full schema coverage, and presence of an output schema, the description covers all necessary behavioral aspects (filtering, sorting, default behaviors). It could optionally mention result format or pagination, but the output schema presumably handles that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema coverage, baseline is 3. The description adds significant value by explaining how query, sort, and tag interact (e.g., empty query defaults to date_desc, non-empty query uses relevance, etc.), which is not captured in the parameter descriptions alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool searches the Sovereign AI Blog for articles matching a natural language query, with optional tag filtering and sorting. It distinguishes itself from sibling tools like list_tags (which discovers tags) and get_article (which retrieves a specific article), and the unrelated diagnose_sglang.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a 'Behaviour matrix' that explains how different parameter combinations behave, giving implicit usage guidance. It also explicitly suggests using list_tags to discover available tags. However, it does not directly compare when to use this tool versus get_article or diagnose_sglang.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.