Server Quality Checklist

Tool Scores

• resolve_pipelineA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavior disclosure. It states what the tool returns (name, priority, transformations, dynamic sources) and that resolve_sources triggers additional resolution. However, it does not disclose whether the tool is read-only, required permissions, error behavior, or if it modifies state. The description is adequate but missing these traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences with zero wasted words. The main purpose is stated first, followed by the optional behavior. Every sentence adds value, and the structure is front-loaded for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description covers the main purpose and the optional parameter. However, it does not describe the return format, which would be helpful since there is no output schema. It also omits error scenarios or prerequisites. The description is adequate but not fully complete for a tool with no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents both parameters thoroughly. The description adds context by integrating resolve_sources into the overall behavior, but it does not provide additional meaning beyond the schema's own descriptions. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool resolves a processing pipeline (builtin name or YAML file path) and reports its properties like name, priority, transformations, and dynamic sources. It differentiates between resolving with and without the resolve_sources flag, and the verb 'resolve' paired with the resource 'pipeline' is specific. Among siblings like list_builtin_pipelines, this tool focuses on a single pipeline, so purpose is distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus siblings like list_builtin_pipelines or parse_condition. It does not provide when-not-to-use scenarios or prerequisites. While the second sentence explains the resolve_sources option, it lacks context on choosing between this tool and others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• author_adsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the main behavior (reporting and scaffolding a template) and implies it is a non-destructive reporting tool, but it does not clarify whether any file writes occur, how the 'active config' is determined, or what happens on input errors. This is adequate but not rich in behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the primary action ('Report') and efficiently packs the key output details without any filler. Every phrase earns its place, making it concise and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a reporting tool with no output schema, the description explains the return values (ADS sections, missing sections, scaffolded template) and input options. It lacks information about error handling, output formatting, or how the active config is resolved, but these are secondary for a report tool. The core completeness is solid, though not exhaustive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so per the rubric, the baseline is 3. The description repeats the parameter names ('yaml' and 'path') but adds no additional meaning beyond what the schema already provides. It doesn't explain mutual exclusivity, directory support, or resolution behavior, which the schema already covers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Report each detection rule's ADS sections, the required sections it is missing under the active config, and a scaffolded `rsigma.ads.*` template to complete.' This uses a specific verb ('Report') with a resource ('detection rule's ADS sections') and details the exact output, which distinguishes it from sibling tools like `lint_rules` or `convert_rules`.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating what it does and that it accepts inline YAML or a file path, but it does not explicitly mention when to use this tool versus alternatives, nor does it provide exclusions or prerequisites. The context is clear enough for an agent to infer when it's appropriate, but there is no direct comparison to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_conditionA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It correctly states the output (parse tree or error), but does not disclose whether the operation is idempotent, side-effect free, or if any authentication is required. For a simple parse operation, this is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise and front-loaded with the core purpose. No unnecessary words or repetitions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one simple string parameter and no output schema, the description covers the input, output, and an example. It is missing some details about the structure of the parse tree or error, but these are minor for a tool that returns JSON. Mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds an example ('e.g. `selection and not filter`') which clarifies the expected format beyond the schema's type definition. This adds value for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Parse', the resource 'Sigma condition expression', and the output 'parse-tree as JSON' or 'structured parse error'. It includes an example, and the tool is well-differentiated from siblings like parse_rule which handles full rules.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like parse_rule. The description only implies usage for parsing condition expressions, but does not provide context on when it is appropriate or when another tool should be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lint_rulesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral transparency. It mentions returning findings and auto-fix availability but does not explicitly state that the tool is read-only or does not modify files. This leaves some ambiguity about side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and every word adds value. No unnecessary filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema), the description covers input methods and return fields. It does not explain prerequisites or output format beyond listed fields, but that is adequate for a linting tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description adds marginal value. It reinforces that path can be a file or directory and that yaml is inline, but these details are already in the schema. No additional semantic nuance is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lints Sigma rules against the specification, specifying the verb 'lint' and the resource 'Sigma rules'. It distinguishes itself from siblings like parse_rule or convert_rules by focusing on validation and returning findings with specific fields.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to use the tool: accepts inline yaml, a file path, or a directory path. However, it does not explicitly state when not to use it or mention alternatives, leaving usage guidance slightly incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_fieldsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It discloses pipeline application and input options, but does not mention return format, pagination, performance, or any read-only hint. Adequate but could be more transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. Critical information is front-loaded. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters, 0 required, and no output schema, the description covers the main behavior, input modes, and pipeline purpose. It is complete enough for a listing tool, though return format could be hinted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining why pipelines are applied ('so the field names match what the engine evaluates') and clarifies that path and yaml are mutually exclusive input methods.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and resource ('event fields referenced by Sigma rules'), and adds detail about provenance. It clearly distinguishes from sibling tools (e.g., list_builtin_pipelines, lint_rules) which do different things.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (e.g., when you want to inspect field usage after applying pipelines) but does not explicitly state when to use this tool versus alternatives or mention any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_rulesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explains the validation process and the optional resolve_sources behavior, but with no annotations, it fails to disclose whether the tool has side effects, requires specific permissions, or returns specific output. It does not state that validation is read-only or what happens on failure, leaving some behavioral uncertainty.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the core purpose and followed by concise parameter guidance. Every clause earns its place, with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a validation tool with no output schema and no annotations, the description covers the main inputs, the process, and optional behaviors. It lacks explicit mention of return values or error handling, but the core context is sufficiently complete for an agent to select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides 100% coverage for all four parameters with descriptive texts. The description reiterates the yaml/path mutual exclusivity and briefly defines resolve_sources, but adds minimal value beyond the schema, which already explains each parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly and specifically states the tool's purpose: 'Validate that Sigma rules parse and compile cleanly' and details the validation process (parse, build detection engine, check correlation references). This distinguishes it from siblings like lint_rules, parse_rule, and convert_rules, which have different focuses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on what the tool does and how to invoke it (accepts inline yaml or path, optional pipelines and resolve_sources). It does not explicitly state when to use it versus alternatives or provide exclusions, but the purpose is well-defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• fix_rulesA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states that unsafe fixes are never auto-applied, that write persists changes to disk, and that formatting is preserved. This adequately discloses key behavioral traits without contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loaded with the main action, and provides necessary details efficiently. It avoids unnecessary words but could be slightly more organized with bullet points or clearer separation of return value and optional behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains return values (fixed YAML and counts). It covers all parameters and safety constraints. It lacks mention of error handling or validation, but overall it is sufficiently complete for a 4-parameter tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description reiterates constraints already in the schema (e.g., write:true only with path). It adds no new parameter-level information beyond summarizing the purpose, which is adequate but not exceptional.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool applies safe auto-fixes to Sigma YAML, listing examples like lowercase keys and typos. It distinguishes from sibling tools by focusing on fixing rather than listing, parsing, or converting rules.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that unsafe fixes are never applied and that write:true requires a file path. It also mentions optional lint_rules to restrict fixes, providing context on when to use parameters. However, it does not explicitly contrast with alternatives like lint_rules or convert_rules.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_rulesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the two conversion modes (native vs delegated), the effect of skip_unsupported, and the role of the target parameter. This adds behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first states the core purpose, the second lists parameters and key conditions. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 parameters, no annotations, and no output schema, the description provides sufficient context on conversion modes and parameter roles. It does not describe the output format, but the core functionality is well-covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Since schema_description_coverage is 100%, the baseline is 3. The description paraphrases the schema's parameter information without adding significant new meaning beyond listing the parameters and noting mutual exclusivity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Convert Sigma rules to backend-native queries' with a specific verb and resource, and distinguishes between native and delegated targets, differentiating it from sibling tools like parse_rule or fix_rules.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use native targets vs delegated sigma-cli mode, including the prerequisite --allow-sigma-cli flag. It does not explicitly mention when not to use the tool or cite alternatives, but the guidance on target selection is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• evaluate_eventsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It goes beyond the obvious by explaining the engine selection behavior: 'Detection-only rules use the stateless engine; collections with correlations use the stateful correlation engine.' This is valuable behavioral context. It also mentions optional `pipelines` and `match_detail`, hinting at configurability. It doesn't mention side effects or return format, but given it's a pure evaluation tool, the key behavioral trait is covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: first sentence states purpose, second explains engine behavior, third covers inputs and options. It is front-loaded with the most important information and contains zero filler. This is an exemplary level of conciseness for a tool with 9 parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (9 parameters, mutual exclusions, stateful/stateless engines) and the absence of an output schema, the description is quite complete. It covers the core input mechanisms, the engine dichotomy, and optional settings. However, it doesn't outline the expected return structure beyond 'matches,' which could be ambiguous for a tool with no output schema. A bit more detail on response format would make it fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description repeats the main input methods (yaml/path, events/events_path) but adds no new semantic detail beyond what the schema already provides. It does group the parameters into logical pairs, which is helpful, but it doesn't compensate for any missing schema information because none is missing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Evaluate JSON events against Sigma rules and return matches.' The verb 'evaluate' and resources 'JSON events' and 'Sigma rules' precisely define the operation. It distinguishes itself from sibling tools like lint_rules or validate_rules, which focus on static analysis, whereas this tool executes rules against events.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to use the tool ('Rules via inline `yaml` or `path`; events via an inline `events` JSON array or an NDJSON `events_path`') and explains when different engines apply (detection-only vs stateful correlation). It doesn't explicitly name alternative tools for exclusion, but the strong purpose clarity makes the intended use implied. This earns a 4 rather than a 5 because there is no explicit 'when not to use' guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_builtin_pipelinesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states 'list', indicating a read-only operation, which is consistent. It does not detail sorting, pagination, or other behaviors, but for a straightforward list, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that immediately conveys the purpose and key output traits (priority and shape). No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description indicates what the output contains (pipelines with priority and shape). It is adequately complete for a simple list tool, though more detail could be added.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the schema coverage is 100%. The description does not need to add parameter semantics. According to the guidelines, 0 parameters defaults to baseline 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists builtin processing pipelines and specifies the output includes priority and shape. It distinguishes from sibling list tools like list_backends and list_fields that target different resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for listing builtin pipelines, but does not explicitly state when to use it versus alternatives like resolve_pipeline or list_backends. No exclusions or contextual advice is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reverse_convertA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the result is 'a reviewable skeleton,' specifies the supported Lucene subset in detail, and clearly reports error behavior for 'Boosting, fuzzy/proximity, and non-numeric ranges.' This adds meaningful behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three dense, information-packed sentences with no redundancy or filler. It front-loads the purpose and efficiently covers scope, dialect details, metadata requirements, and error conditions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description covers the essential aspects: purpose, dialect scope, required metadata inputs, and error conditions. The only minor gap is that it does not specify the exact return structure beyond 'draft Sigma rule (YAML),' but the description is otherwise sufficient for operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters already have individual descriptions. The tool description adds value by explaining why metadata parameters are needed (query carries no metadata) and provides detailed syntax for the dialect parameter, going beyond the schema's brief field labels.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Reverse-convert a SIEM query into a draft Sigma rule (YAML).' It is specific in verb and resource and distinguishes itself from siblings like convert_rules by explicitly indicating the reverse direction (SIEM query → Sigma) and the YAML output format.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides useful context by noting that 'a query carries no metadata' and instructs to pass title/id/level/status and logsource fields. However, it does not explicitly state when to use this tool versus alternatives like convert_rules or parse_rule, nor does it provide exclusions or when-not-to-use conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_backendsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses a specific conditional behavior (sigma-cli targets appended when server runs with --allow-sigma-cli), but does not mention read-only nature, authentication needs, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. First sentence covers primary purpose, second adds an important conditional. Efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters and no output schema, description provides enough context: what is listed (backends with formats and methods) and a conditional behavior. Could be slightly more complete by noting it is a read-only listing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so schema coverage is 100%. Description adds meaning beyond schema by detailing that the list includes output formats and correlation methods, plus the sigma-cli conditional.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'list' and resource 'conversion backends', specifying the output includes 'output formats and correlation methods'. It distinguishes from sibling list tools like list_builtin_pipelines and list_fields by targeting backends.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for discovering available backends, but lacks explicit guidance on when to use this tool versus alternatives like list_builtin_pipelines or list_fields. No when-not or context exclusions provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• parse_ruleA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes return of AST or parse errors and support for multi-document. No annotations, so description carries burden; however, does not explicitly state read-only or idempotent nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first covers purpose and output, second covers parameters. No extraneous information, front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple parse tool with two parameters; explains inputs, outputs, and error handling. Without output schema, could detail return format more, but sufficient for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, description adds mutual exclusivity and path resolution behavior beyond schema descriptions, providing useful context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states verb 'parse', resource 'Sigma YAML', and output 'structured AST as JSON' or parse errors. Differentiates from siblings like parse_condition and convert_rules.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Specifies input modes (inline yaml or file path) but does not explicitly state when to use versus alternatives. Context from sibling tools implies differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tune_rulesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool proposes (not mutates) a rule, includes verification ('closed before/after verification'), enforces a hard constraint ('No proposal may suppress a true positive'), and outlines the result components (filter YAML, field rationale, clusters, coverage). It doesn't mention auth or rate limits, but the non-destructive 'propose' nature is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loads the primary purpose, and packs essential behavioral and output details without redundancy. Every clause adds value, and the structure is easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 14 parameters and no output schema, this description is remarkably complete. It covers inputs, source selection, optional transformation, output contents, and a critical invariant. The schema covers the parameter details, so the description needs only focus on high-level behavior and constraints, which it does thoroughly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds meaningful context for key parameters by explaining the sources (inline `yaml` or `path`), target selection (`rule`), and the optional role of `pipelines`. This goes beyond the schema's field descriptions, particularly in clarifying the relationship between `yaml` and `path`.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Propose') and resource ('spec-native Sigma filter rule') with explicit inputs (false-positive and true-positive JSON event arrays). It distinguishes this tool from siblings like `validate_rules` or `convert_rules` by focusing on filter rule generation from FP/TP data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for how the tool is used (via inline yaml or path confined to --rules-dir, optional pipelines, target selection with rule) and implies the use case of tuning rules against event arrays. It doesn't explicitly mention when not to use it or name alternatives, but the context is strong enough for an agent to infer applicability.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.