Server Quality Checklist

Tool Scores

• se_cyber_list_frameworksA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It only states it lists frameworks, without disclosing whether it is read-only, requires authentication, or any other behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence that efficiently communicates the tool's purpose without extra words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description states what is listed but does not specify the output format, fields, or pagination, leaving some gaps given no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the description does not need to add parameter details. Baseline 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists all BSI frameworks and standard series, with specific examples, distinguishing it from sibling tools that focus on guidance and advisories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternatives guidance is provided; usage is implied as a straightforward listing tool, but lack of exclusions or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• se_cyber_get_advisoryA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Only states it gets an advisory; lacks details on behavior like error handling, data freshness, or whether it performs external calls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, well-formed sentence of 12 words. No redundancy or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple getter with one parameter and no output schema, the description covers the essential: what it returns and how to specify it. Could mention that the reference must match a specific format, but adequate for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 100% coverage with description of 'reference'. Description adds concrete example ('BSI-CB-K24-0001'), providing clarity beyond schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'Get', resource 'specific BSI security advisory', and identification method 'by reference' with an example. Distinguished from sibling se_cyber_search_advisories which is for searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies use when a specific reference is known, but does not explicitly contrast with alternatives like se_cyber_search_advisories for when reference is unknown.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• se_cyber_search_advisoriesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description implies a read-only operation by stating 'returns advisories', but with no annotations provided, it does not explicitly confirm no side effects, no auth requirements, or rate limits. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first states the core action, second adds key details. No extraneous information, well front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 3 parameters (1 required, no output schema), the description sufficiently covers what is returned (severity, affected products, CVE references). Could mention pagination or default limit, but schema already covers limit defaults.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%. The description adds value beyond the schema by providing example queries ('kritische Schwachstelle', 'Ransomware', 'VPN') and noting that CVE references are returned 'where available', which clarifies optionality not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches BSI security advisories and alerts, and lists return fields (severity, affected products, CVE references). It distinguishes from sibling tools like se_cyber_search_guidance which searches guidance instead.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives (e.g., se_cyber_search_guidance or se_cyber_get_advisory). No mention of prerequisites or when to avoid.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• se_cyber_search_guidanceA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It mentions full-text search and returned fields (reference, title, series, summary) but omits pagination, ordering, or rate limits. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no fluff. Lists covered content and return fields efficiently. Front-loaded with purpose. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and 5 parameters, the description covers purpose, scope, and return fields. Missing details on pagination or result sorting. Overall sufficient for a search tool, but could enhance with limit/offset info.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers all 5 parameters with descriptions (100% coverage). The description adds context about document types (TR series, IT-Grundschutz, etc.) but does not significantly enhance parameter meaning beyond the schema. Baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool performs full-text search across BSI guidelines and technical reports, listing specific series. The verb 'search' combined with the resource distinguishes it from siblings like se_cyber_get_guidance for fetching a single document.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when searching BSI guidelines but does not explicitly state when to use this tool vs siblings like se_cyber_search_advisories or se_cyber_get_guidance. No information on prerequisites or when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• se_cyber_aboutA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so description carries full burden. It accurately states the tool returns metadata with specific fields, implying a read-only, non-destructive operation. No hidden side effects or requirements are omitted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence with no extraneous words. Every piece of information is essential.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, no output schema, and a simple purpose, the description fully covers what the tool does and returns. Complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema is empty (no parameters), so schema coverage is 100%. Baseline score of 3 is appropriate as there are no parameters to describe.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Return' and the resource 'metadata about this MCP server', listing specific items (version, data source, coverage, tool list). It distinguishes from sibling tools which focus on guidance, advisories, and frameworks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when or when not to use this tool versus siblings. However, the purpose implies use for server introspection, which is clear enough for most agents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• se_cyber_get_guidanceA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It states it retrieves a document by reference, but does not explain what happens if the reference is invalid, whether the document content is returned fully or partially, or any authentication requirements. For a simple retrieval tool, this is adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently conveys the tool's purpose and input format without any extraneous words. It is front-loaded with the key action and resource.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a simple interface with one parameter and no output schema. However, the description does not explain what the output (the guidance document) contains—e.g., title, version, content, metadata. Given the lack of annotations and output schema, more context about the return value would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a description for the 'reference' parameter. The tool description adds value by providing concrete examples of valid references, which clarifies the expected format beyond the schema's generic 'MSB document reference' description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('specific BSI guidance document'), and the method ('by reference'). It provides concrete examples of valid references, making the tool's purpose unambiguous and distinguishing it from sibling tools like se_cyber_search_guidance.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that the tool should be used when the user knows the exact reference for a BSI guidance document. While it doesn't explicitly mention when not to use it or alternatives, the purpose is clear enough in context with the sibling tools. It could be improved by stating 'for searching, use se_cyber_search_guidance instead'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.