Server Quality Checklist

Tool Scores

• helixar_triage_alertB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals key behavioral traits: the tool generates a narrative (implying creation/processing), hard-caps severity at 'high' (a constraint), and outputs kill-chain stages. However, it lacks details on error handling, rate limits, authentication needs, or what 'triage' entails operationally. The description adds some value but leaves significant gaps for a tool with mutation-like behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, dense sentence that efficiently packs key information: purpose, parameters, and a behavioral constraint. It's front-loaded with the core function. However, it could be slightly more structured (e.g., separating parameter explanations) and omits some useful details, keeping it from a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (security triage tool with 2 parameters, no annotations, and no output schema), the description is moderately complete. It covers the basic purpose and parameters but lacks details on output structure, error cases, or integration context. For a tool that likely returns structured analysis, the absence of output schema means the description should do more to explain results, but it only hints at outputs (kill-chain stage, narrative).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'payload' and 'format' parameters, explaining that format choices are 'executive, technical, or brief' and defaulting to 'technical'. However, it doesn't explain what the 'payload' parameter should contain (e.g., structure, content type) or provide any additional semantics beyond the enum values. The description adds minimal value over the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Triage a Vigil / ATP detection payload into a kill-chain stage... with a Claude-generated narrative'. It specifies the verb ('triage'), resource ('Vigil / ATP detection payload'), and output components (kill-chain stage, narrative format). However, it doesn't explicitly differentiate from sibling tools like 'helixar_hdp_validate' or 'helixar_inspect_mcp', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by mentioning 'Vigil / ATP detection payload' and narrative format choices, suggesting it's for security analysis scenarios. However, it provides no explicit guidance on when to use this tool versus the sibling tools (validate or inspect), nor does it mention any prerequisites or exclusions. The guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• helixar_hdp_validateA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses key behavioral traits: it validates against a specific IETF draft, surfaces specific violation types, and cites sources in outputs. However, it lacks details on error handling, performance characteristics, or authentication requirements that would be helpful for a validation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that efficiently convey core functionality and output characteristics. It's front-loaded with the main purpose, though could be slightly more structured for a complex validation tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (nested objects, no output schema, no annotations), the description provides good purpose clarity but lacks parameter guidance and detailed behavioral context. It's adequate for understanding what the tool does but incomplete for proper usage without consulting external documentation about the HDP delegation format.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage and 2 parameters (one complex nested object), the description provides no parameter information. It doesn't explain what the 'chain' object should contain or what 'strict' mode does, leaving significant gaps beyond what the bare schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('validate') and resource ('HDP delegation chain') with explicit reference to the IETF draft. It distinguishes from sibling tools by focusing on validation rather than inspection or alert triage, and provides concrete examples of what it surfaces (scope escalations, depth violations, etc.).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when validation of an HDP delegation chain is needed, but provides no explicit guidance on when to use this tool versus the sibling tools (helixar_inspect_mcp, helixar_triage_alert). There's no mention of prerequisites, alternatives, or exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• helixar_inspect_mcpA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does well by explaining authentication requirements (quick mode is authless, deep mode requires api_key) and cost implications (quick mode is free). However, it doesn't mention rate limits, error handling, or what happens when scanning fails. For a security scanning tool with no annotation coverage, more behavioral context would be helpful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly front-loaded and concise. The first sentence establishes the core functionality, and the second sentence efficiently explains the two operational modes with their key differences. Every word earns its place with no wasted text or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (security scanning with multiple modes), no annotations, and no output schema, the description does a reasonable job but has gaps. It explains the scanning purpose and mode differences well, but doesn't describe the return format (risk score structure, findings format, security brief details) or error conditions. For a tool with no output schema, more information about return values would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 25% (only the 'target' parameter has a description), so the description must compensate. It adds significant value by explaining the 'mode' parameter's semantics (quick vs deep modes with rule counts and authentication differences) and implying the 'api_key' parameter's purpose for deep mode. However, it doesn't explain the 'context' parameter at all, leaving one parameter undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('scan', 'returns') and resources ('MCP server', 'Helixar's Sentinel detection rules'). It distinguishes itself from sibling tools by focusing on security scanning rather than validation or alert triage. The description explicitly mentions what the tool does: scanning against detection rules and returning risk scores, findings, and security briefs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use different modes: 'quick mode is free + authless (top 8 rules)' and 'deep mode runs all 26 rules with an api_key.' This gives practical guidance on mode selection based on authentication and rule coverage. However, it doesn't explicitly mention when to use this tool versus the sibling tools (helixar_hdp_validate, helixar_triage_alert), which would be needed for a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.