Server Quality Checklist

Tool Scores

• ch_cyber_search_advisoriesB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions return fields (severity, affected products, CVE references) but does not disclose pagination, error handling, authentication needs, or behavior for empty results. This is insufficient for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, no fluff. First sentence states purpose, second lists outputs. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (3 params, no nested objects, no output schema), the description covers core functionality and return format. Missing details like pagination or error handling are minor given the context. It is mostly complete for agent usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%. The description adds value by providing example queries in German (e.g., 'kritische Schwachstelle'), which helps the agent formulate queries. However, it does not elaborate on the 'limit' or 'severity' parameters beyond what the schema provides, so overall added value is modest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches NCSC-CH security advisories and warnings, with a specific verb 'search' and resource. It distinguishes from sibling tools like 'ch_cyber_get_advisory' (single advisory) and 'ch_cyber_search_guidance' (different resource).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusions. It does not mention when to prefer this over 'ch_cyber_get_advisory' or other siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ch_cyber_get_guidanceA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description only states the core function. It does not disclose whether the operation is read-only, what happens on invalid reference, rate limits, or any side effects. The agent is left with minimal behavioral information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 9 words, front-loaded with the action and resource. Every word contributes meaning without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one parameter and no output schema, the description is adequate but missing details on the return format or response structure. It minimally explains what the tool does but leaves the agent guessing about the output.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with a basic description. The tool's description adds value by providing example formats (e.g., 'ISB-2023-01', 'NCSC-CH-TLP-2024-01') and reinforces the reference-based lookup, exceeding what the schema alone offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('NCSC-CH guidance document'), and the key parameter (by 'reference'). It distinguishes from sibling tools like 'ch_cyber_search_guidance' by focusing on a specific document retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when the user has a specific reference, but it does not explicitly state when to use this tool vs. alternatives like searching. No exclusion criteria or alternate tool recommendations are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ch_cyber_get_advisoryA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It states the tool retrieves an advisory by reference, but does not disclose behavior on missing references, required permissions, or any side effects. Adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that is front-loaded with the verb and resource. No wasted words, efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one parameter and no output schema, the description is largely adequate. It could optionally mention the return format or typical response, but it's not strictly necessary given the simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% but only provides generic description. The description adds an example format ('NCSC-CH-2024-0001'), which adds concrete meaning beyond the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (get), resource (security advisory), and method (by reference) with an example format. This distinguishes it from siblings like search_advisories which would be for searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus siblings such as ch_cyber_search_advisories or ch_cyber_get_guidance. It does not provide when-not or alternative scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ch_cyber_list_frameworksA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. It states the scope (covered in this MCP) and mentions specific inclusions (ISB, ISG, advisory series). However, it does not disclose behaviors like whether the list is static or dynamic, or any auth requirements. Acceptable for a simple read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence, front-loaded with key action and resource. No extraneous information, every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters or output schema, description is reasonably complete. It lists what is included, which suffices for a simple list tool. Could hint at how to use the framework names elsewhere, but not necessary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist (input schema empty). Description adds value by listing the types of frameworks, enhancing understanding beyond the schema. Baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description uses specific verb 'List' and resource 'NCSC-CH frameworks and standard series'. It clearly distinguishes from sibling tools which deal with guidance and advisories, not listing frameworks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when or when-not to use. The purpose is implied from sibling context (listing vs. searching/getting), but no direct guidance on alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ch_cyber_search_guidanceA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It states the tool returns documents with specified fields, implying a read-only operation. However, it doesn't disclose behaviors like pagination, sorting, performance considerations, or authentication requirements. For a search tool, the description is adequately transparent but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that front-load the core purpose and expand on coverage and return fields. Every sentence adds value; no wasted words. The structure is clear and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is nearly complete for a search tool: it specifies what is searched, what is returned, and mentions the document types. However, it lacks details like default ordering, result count limits, and whether the search is full-text (implied). Given no output schema, the description provides a good overview but misses minor behavioral context like pagination or query syntax.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all parameters have descriptions). The description adds context about the overall search scope but doesn't provide additional semantics for individual parameters beyond what the schema already states. For example, the query parameter's language hint is already in the schema. The description is not necessary for parameter understanding, so a baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it performs full-text search across NCSC-CH guidelines and technical reports, listing specific document types (ISB standards, NCSC-CH recommendations, etc.) and the fields returned (reference, title, series, summary). This distinguishes it from sibling tools like ch_cyber_search_advisories (which searches advisories) and ch_cyber_get_guidance (which retrieves a single document).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use this to search for guidance documents. It covers a broad range of document types, but it doesn't explicitly state when to use this tool versus alternatives like ch_cyber_search_advisories. The purpose alone provides sufficient context for an agent to select it when searching guidelines, but explicit when-to-use guidance would improve the score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ch_cyber_aboutA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It correctly indicates a read operation returning metadata, but does not disclose any potential side effects, rate limits, or authorization requirements. The description is truthful but not deeply informative.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no fluff, efficiently conveying the tool's purpose and output. It is well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, no output schema, and simple purpose, the description fully covers what an agent needs to know: what the tool does and what it returns. Sibling tool names further clarify its distinct role.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters (0 params, schema coverage 100%). The description adds value by specifying the contents of the returned metadata (version, data source, coverage, tool list), which is helpful beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns metadata about the MCP server, listing version, data source, coverage, and tool list. This is distinct from sibling tools like ch_cyber_get_guidance or ch_cyber_search_advisories, which deal with content rather than server information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for initial discovery or troubleshooting, but does not explicitly state when to prefer this tool over alternatives or when not to use it. No exclusions or conditions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.