Server Quality Checklist

Tool Scores

• list_certifying_bodiesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description confirms the tool is read-only and lists 217 bodies with exam counts and optional filtering. However, it does not specify output format or pagination, which are minor gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The first sentence covers the core purpose and scope, the second provides usage guidance and filter details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main functionality and filter option, but lacks explicit mention of return fields (e.g., body name and count). Given the tool's simplicity, this is a minor omission.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The parameter 'contains' is described in the schema with a substring filter example. The description reiterates this and adds no additional semantic meaning beyond the schema, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool lists certifying bodies with exam counts, provides example queries, and mentions optional filtering, clearly distinguishing it from sibling tools that deal with exams themselves.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use-case examples ('what certification providers/vendors are covered', 'how many AWS/Microsoft/Cisco certs') and mentions the optional substring filter, providing clear guidance on when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_examsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must cover behavioral traits. It discloses that each output row links a free practice exam and that input accepts multiple identifier types. However, it does not discuss auth needs, rate limits, or any destructive actions, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first sentence states core purpose and what is compared; second sentence adds usage examples and input variations. No redundant or vague wording. Information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple comparison tool with one parameter and no output schema, the description covers purpose, input types, output feature (practice exam links), and typical user intents. Lacks explicit return format but sufficient for agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the sole parameter. The tool description adds value beyond the schema by explaining the comparison context and the metrics displayed, and by providing usage examples that clarify the parameter's purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'compare', resource 'certification exams', and lists specific metrics (price, duration, passing score, question count, domain count). Distinguishes from sibling tools like get_exam_blueprint (single exam details) and search_exams (search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit use cases (e.g., 'X vs Y', 'which is harder/cheaper') and acceptable input types (ids, codes, names). Does not explicitly state when not to use or mention alternatives, but the examples effectively guide the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_examsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes input and output: returns exams with code, body, question/domain counts, and practice link. No annotations provided, but the description adequately captures the tool's behavior as a search.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, well-structured paragraph that front-loads the main action and includes examples. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description lists returned fields. Parameter descriptions are complete. Could mention pagination or empty results, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All 4 parameters have schema descriptions (100% coverage). The tool description adds context but does not significantly enhance parameter meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool searches certification exams by name, code, body, or vendor. Includes example queries and distinguishes from siblings like compare_exams or list_certifying_bodies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to call this tool whenever a user mentions a certification or exam, with concrete examples. Does not explicitly mention when not to use it, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_exam_blueprintA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; the description carries the full burden. Discloses the comprehensive data it returns (domain breakdown, weights, passing score, etc.) and mentions a free practice-exam link, which is sufficient for a read-only tool. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the return content, then usage examples. Every part is necessary and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one parameter and no output schema, the description covers key aspects: what it returns, acceptable inputs, and a practice exam link. Minor omission like error handling is acceptable for this simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with a description for 'exam'. The description adds value by reiterating accepted formats and providing concrete usage examples ('exam_id, exam_code, or certification name'), enhancing understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it retrieves the full published blueprint for one certification exam, listing specific details. Distinguishes from siblings by focusing on a single exam and contrasting with compare_exams, list_certifying_bodies, and search_exams.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit examples of when to call (e.g., 'what's on the X exam') and what inputs it accepts. Lacks explicit when-not-to-use or alternative sibling tool references, but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.