Skip to main content
Glama

Summarize SEI CERT checker coverage

get_rule_coverage
Read-onlyIdempotent

Counts SEI CERT rules per detection tier (deterministic, partial, LLM-only) to reveal how much review static analysis can carry and where manual review is needed.

Instructions

Count SEI CERT rules per detection tier: deterministic (a rule-specific checker exists), partial (only general checkers map) and llm-only (no checker; manual review needed), both for all upstream checker mappings and for the pinned reference toolchain (clang-tidy and Clang Static Analyzer 19.1.7, cppcheck 2.17.1). Use to judge how much of a review static analysis can carry. Computed from bundled data; it does not probe the analyzers installed here (check_files reports those in tools[]). For one rule use get_rule; to list a tier's rules with titles use list_rules with coverage.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "get_rule_coverage"
  }
}

Input Schema

NameRequiredDescriptionDefault
kindNoCount only rules or only recommendations.any
langNoCount only SEI CERT C (c) or C++ (cpp) rules.any
include_idsNoAdd rules_all_mappings and rules_runnable: rule ID lists per tier (~10 KB).

Output Schema

NameRequiredDescriptionDefault
totalYesRules counted after the lang/kind filters.
sourcesYesUpstream mapping sources and commits.
toolchainYesPinned analyzer versions the tiers were computed for.
definitionsYesMeaning of deterministic, partial and llm-only.
all_mappingsYesCounts per tier using every upstream checker mapping.
runnable_toolchainYesCounts per tier using only checkers available in the pinned toolchain.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv0.1.2

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, closed-world and non-destructive, so the safety profile is covered. The description still adds real behavioral context beyond them: results are computed from bundled data rather than probing local installations, and the pinned toolchain versions are spelled out. It does not discuss cost or the ~10 KB payload of include_ids (that lives only in the schema), which keeps it short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core verb and resource, then tier definitions, then usage framing, then sibling routing — a logical order with no filler sentences. It is dense and long-ish for a counting tool, but every clause carries information; slight compression of the tier glosses would help.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return shape need not be explained. The description supplies the tier definitions an agent needs to read that output, the data provenance caveat, and the boundary with check_files and list_rules — everything required to call it correctly with an accurate mental model.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and both enums are documented there, so the schema already carries parameter meaning. The description adds the conceptual distinction between the two counting bases, which informs interpretation of results, but says nothing extra about kind, lang, or include_ids. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise verb and resource ('Count SEI CERT rules per detection tier') and defines each tier inline (deterministic, partial, llm-only), plus the two counting bases (all upstream mappings vs. pinned reference toolchain). An agent can distinguish this aggregate-counting tool from get_rule and list_rules purely from the text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit purpose ('Use to judge how much of a review static analysis can carry') and routes to named alternatives with their selection conditions: get_rule for a single rule, list_rules with `coverage` to enumerate a tier's rules. It also pre-empts a likely wrong assumption by saying this does not probe installed analyzers and that check_files reports those in tools[].

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.