Skip to main content
Glama

Validate manual SEI CERT findings

validate_findings
Read-onlyIdempotent

Checks manually written review findings against CERT rule candidates, verbatim quote evidence, and file/line bounds, rejecting invalid or unverifiable ones with reasons before formatting.

Instructions

Accept or reject findings written from a build_review_packet review, with reasons. A finding passes only if its rule_id exists (C++ rules are rejected on .c files), its quote is a passage of 20+ characters found in that rule's text or examples (case, whitespace, curly quotes and markdown are normalised), it has a file, a line >= 1 and a message, and the profile does not disable the rule. With packet, the line must also fall inside a hunk whose candidate_rules include the rule. Use on every manual finding before format_findings; check_files and check_diff output needs no validation. Pure computation on the arguments; reads no files.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "findings": "<findings>"
    },
    "name": "validate_findings"
  }
}

Input Schema

NameRequiredDescriptionDefault
packetNoThe build_review_packet result (or just {hunks}) to enforce hunk and candidate-rule bounds. Strongly recommended.
profileNoReview profile: 'default' (plain SEI CERT), 'ceph' (Ceph severities, suppressions and idioms) or an absolute path to a profile .yaml file.default
findingsYesFindings to check; an empty list is valid.

Output Schema

NameRequiredDescriptionDefault
summaryYes{accepted, rejected} counts.
acceptedYesGrounded findings in the check_files finding shape plus quote and fix; pass to format_findings.
rejectedYes{finding, reasons[]} for each rejected finding.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed9 schema fields changedv0.1.2
    • addedInput schema / $defs
      Added value: +{
      +  "FindingIn": {
      +    "additionalProperties": true,
      +    "description": "A finding written from a build_review_packet hunk.",
      +    "properties": {
      +      "confidence": {
      +        "anyOf": [
      +          {
      +            "maximum": 1,
      +            "minimum": 0,
      +            "type": "number"
      +          },
      +          {
      +            "type": "null"
      +          }
      +        ],
      +        "default": null,
      +        "description": "Defaults to 0.5.",
      +        "title": "Confidence"
      +      },
      +      "file": {
      +        "description": "Path exactly as in the hunk's `file` field.",
      +        "title": "File",
      +        "type": "string"
      +      },
      +      "fix": {
      +        "anyOf": [
      +          {
      +            "type": "string"
      +          },
      +          {
      +            "type": "null"
      +          }
      +        ],
      +        "default": null,
      +        "description": "Compliant replacement code.",
      +        "title": "Fix"
      +      },
      +      "line": {
      +        "description": "Line inside that hunk's start_line..end_line.",
      +        "minimum": 1,
      +        "title": "Line",
      +        "type": "integer"
      +      },
      +      "message": {
      +        "description": "What is wrong in this code and why it violates the rule.",
      +        "title": "Message",
      +        "type": "string"
      +      },
      +      "quote": {
      +        "description": "Verbatim passage (>= 20 characters) copied from that rule's text or examples.",
      +        "title": "Quote",
      +        "type": "string"
      +      },
      +      "rule_id": {
      +        "description": "A rule_id from the hunk's candidate_rules, e.g. EXP34-C.",
      +        "title": "Rule Id",
      +        "type": "string"
      +      },
      +      "severity": {
      +        "anyOf": [
      +          {
      +            "enum": [
      +              "error",
      +              "warning",
      +              "note"
      +            ],
      +            "type": "string"
      +          },
      +          {
      +            "type": "null"
      +          }
      +        ],
      +        "default": null,
      +        "description": "Overrides the rule's default severity.",
      +        "title": "Severity"
      +      }
      +    },
      +    "required": [
      +      "rule_id",
      +      "file",
      +      "line",
      +      "message",
      +      "quote"
      +    ],
      +    "title": "FindingIn",
      +    "type": "object"
      +  },
      +  "PacketIn": {
      +    "additionalProperties": true,
      +    "properties": {
      +      "hunks": {
      +        "description": "The packet's hunks; only file, start_line, end_line and candidate_rules[].rule_id are read.",
      +        "items": {
      +          "additionalProperties": true,
      +          "type": "object"
      +        },
      +        "title": "Hunks",
      +        "type": "array"
      +      }
      +    },
      +    "required": [
      +      "hunks"
      +    ],
      +    "title": "PacketIn",
      +    "type": "object"
      +  }
      +}
    • addedInput schema / properties / findings
      Added value: +{
      +  "description": "Findings to check; an empty list is valid.",
      +  "items": {
      +    "$ref": "#/$defs/FindingIn"
      +  },
      +  "title": "Findings",
      +  "type": "array"
      +}
    • removedInput schema / properties / findings_json
      Removed value: -{
      -  "title": "Findings Json",
      -  "type": "string"
      -}
    • addedInput schema / properties / packet
      Added value: +{
      +  "anyOf": [
      +    {
      +      "$ref": "#/$defs/PacketIn"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ],
      +  "default": null,
      +  "description": "The build_review_packet result (or just {hunks}) to enforce hunk and candidate-rule bounds. Strongly recommended."
      +}
    • removedInput schema / properties / packet_json
      Removed value: -{
      -  "anyOf": [
      -    {
      -      "type": "string"
      -    },
      -    {
      -      "type": "null"
      -    }
      -  ],
      -  "default": null,
      -  "title": "Packet Json"
      -}
    • addedInput schema / properties / profile / description
      Added value: +"Review profile: 'default' (plain SEI CERT), 'ceph' (Ceph severities, suppressions and idioms) or an absolute path to a profile .yaml file."
    • addedInput schema / properties / profile / examples
      Added value: +[
      +  "default",
      +  "ceph"
      +]
    • changedInput schema / required
      Previous value: -[
      -  "findings_json"
      -]New value: +[
      +  "findings"
      +]
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
      +  "additionalProperties": true,
      +  "properties": {
      +    "accepted": {
      +      "description": "Grounded findings in the check_files finding shape plus quote and fix; pass to format_findings.",
      +      "items": {
      +        "additionalProperties": true,
      +        "type": "object"
      +      },
      +      "title": "Accepted",
      +      "type": "array"
      +    },
      +    "rejected": {
      +      "description": "{finding, reasons[]} for each rejected finding.",
      +      "items": {
      +        "additionalProperties": true,
      +        "type": "object"
      +      },
      +      "title": "Rejected",
      +      "type": "array"
      +    },
      +    "summary": {
      +      "additionalProperties": {
      +        "type": "integer"
      +      },
      +      "description": "{accepted, rejected} counts.",
      +      "title": "Summary",
      +      "type": "object"
      +    }
      +  },
      +  "required": [
      +    "accepted",
      +    "rejected",
      +    "summary"
      +  ],
      +  "title": "Validation",
      +  "type": "object"
      +}
  2. First observedv0.1.0

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnly/idempotent annotations by enumerating the exact pass conditions: rule_id existence, C++ rules rejected on .c files, quote length of 20+ characters, normalization of case/whitespace/curly quotes/markdown, required file/line/message, and profile-based rule suppression. It also discloses the extra packet constraint (line must fall in a hunk whose candidate_rules include the rule) and confirms 'Pure computation on the arguments; reads no files.'

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the action, then the pass criteria, then the workflow guidance and the 'reads no files' guarantee — a sensible ordering with no filler. The middle criteria sentence is long and list-dense, but each clause carries distinct validation semantics that cannot be dropped without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a pure-computation validator with an output schema, rich annotations, and fully documented parameters, the description covers the remaining burden: pass/fail rules, normalization behavior, profile effects, and the packet-based bound. Nothing an agent needs to call it correctly or interpret a rejection is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description still adds meaning by explaining that `profile` can disable rules, that `packet` is 'strongly recommended' and enforces hunk and candidate-rule bounds, and that `findings` are the manual ones requiring validation. It does not detail severity/confidence/fix parameters, but those are fully documented in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Accept or reject findings written from a build_review_packet review') and immediately defines the accept/reject criteria, so an agent knows exactly what the tool decides. It is clearly distinguishable from siblings like format_findings and check_files, which are named in the text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says 'Use on every manual finding before format_findings' and carves out the exclusion: 'check_files and check_diff output needs no validation.' Both the when and the when-not are stated along with the adjacent tool in the workflow.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.