Skip to main content
Glama
engineering-audit

engineering-audit

by rodlunt

engineering-audit

CI Latest release Licence Python 3.10+ Checked with ruff and mypy

TL;DR: engineering-audit turns the AI coding assistant you already use (Claude Code, Codex CLI, Gemini CLI) into an engineering-practice auditor. It has two audit modes:

  1. Full repository audit: your assistant sweeps a whole repository against a pack of sourced engineering rules and produces a self-contained HTML report. Every finding says what is wrong, why it matters and how to fix it, with the citation behind the claim attached, and can be filed as a GitHub issue or copied out for pasting anywhere.

  2. Inline decision-time checks: one-line triggers in your assistant's context make it load the relevant rules at the moment you are making a matching decision (designing a schema, cutting a branch, shaping an API). No report, just the right rules at the right moment.

Starting a new project? The optional Engineering Grill uses the same rules before you build. Your assistant interviews you, works out which domains matter, and records the decisions and checks the project will need, without expecting you to understand the framework first.

Three complete rule domains ship in this repository, ready to run: data modelling, testing strategy and presenting data, 54 rules with their full source citations, in examples/taster-rules/. You can run a real audit right now with no sign-up; the full sixteen-domain, 260-rule pack is available on request (Rules access).

Need a walk through? There is a free interactive walkthrough → that takes you from a bare Windows, Mac or Linux PC to a finished audit of your own repository. It rehearses every step below in a simulated terminal, with the wrong turns coached, before you do them for real. No terminal experience assumed.

Save me the chit chat: show me how to install it →

How to use

From nothing to a first audit in five steps. The taster rules ship in this repository, so none of this needs access to the full pack.

Never touched a terminal? Rehearse these five steps first in the free interactive walkthrough linked at the top: it takes a non-technical founder from a bare Windows, Mac or Linux PC to a finished audit, in a simulated terminal with the wrong turns coached.

Step 1: pick your assistant

The tool works through the assistant you already drive: Claude Code or OpenAI Codex CLI (both proven end to end; standalone audits recorded 2026-08-09 and 2026-08-10, with a further Codex run on Linux 2026-08-19), or Gemini CLI (documented, not yet exercised end to end; see the support matrix). GitHub Copilot is not supported.

Step 2: dependencies

Dependency

Why you need it

Check it

uv

runs the MCP server via uvx straight from a pinned release tag, and installs its own Python (3.10+) if the machine lacks one

uvx --version

git

clones this repository for the taster rules, and any other rules pack

git --version

your assistant's CLI

drives the audit and hosts the MCP registration

claude --version (or codex, gemini)

GitHub CLI gh, optional

only if the assistant should file findings as GitHub issues; filing from the report page instead needs only a PAT in your browser

gh auth status

There is no pip install and no npm anywhere: uvx fetches and runs the tagged release directly.

Step 3: get the rules onto disk

Clone this repository. The taster pack (the three complete domains named in the TL;DR) is a working rules directory at examples/taster-rules/:

git clone https://github.com/rodlunt/engineering-audit
cd engineering-audit

Have the full pack instead? Registration below is identical, just point --rules-dir at that clone's domains/ directory. See Rules access for how to ask.

Step 4: register the tool with your assistant

Every command below is pinned to the current release tag (@v0.15.0) rather than the moving main branch: an unpinned git dependency resolves to whatever main holds at install time and silently moves on later cache refreshes. Find the latest tag on the Releases page. Updating to a newer tag is a deliberate act with its own command, below, not something that happens on a git pull: the pin lives in your assistant's MCP registration, not in this repository.

Paths beginning with /path/to/ are placeholders. Replace them with the real absolute path on your computer; run pwd inside a folder when you need to see its absolute path.

Claude Code

Register the server (swap in the taster path from Step 3, or your full-pack path):

claude mcp add engineering-audit --scope user -- uvx --from git+https://github.com/rodlunt/engineering-audit@v0.15.0 \
    engineering-audit-mcp --rules-dir /path/to/engineering-audit/examples/taster-rules

--scope user registers it for every repository. Without it claude mcp add defaults to local scope, which registers the server for the current directory alone and leaves it unavailable everywhere else. That failure is silent and lands later (issue #245): a skill run in any other project reports list_domains unavailable with nothing pointing at scope as the cause. If that happens, run claude mcp list in the project where it failed; if engineering-audit is missing there but present in the directory you installed from, it was registered without --scope user. Fix it with claude mcp remove engineering-audit followed by the add command above.

To update to a newer tag, remove first and re-add. claude mcp add refuses to overwrite an existing name (MCP server engineering-audit already exists in user config), so changing the tag on its own is not enough:

claude mcp remove engineering-audit
claude mcp add engineering-audit --scope user -- uvx --from git+https://github.com/rodlunt/engineering-audit@v0.15.0 \
    engineering-audit-mcp --rules-dir /path/to/engineering-audit/examples/taster-rules

The change only takes effect in a new session. The one you are in keeps the server it started with, so verify after restarting rather than before:

claude mcp list | grep engineering-audit    # must show the tag you just set

begin_run's response states tool_version too. If it names the old version, the session predates the re-registration and the run is not exercising the build you think it is.

Install the audit skill (gives you a natural-language entry point: "audit this repo"):

cd /path/to/engineering-audit
scripts/install-skills.sh audit

It copies rather than symlinks, so the installed skill does not change under you when this checkout switches branch. scripts/install-skills.sh --check reports when a copy is stale.

Full details: integrations/claude-code/.

OpenAI Codex CLI

Register the server (verified against codex-cli 0.114.0):

codex mcp add engineering-audit \
    --env ENGINEERING_AUDIT_RULES_DIR=/path/to/engineering-audit/examples/taster-rules \
    -- uvx --from git+https://github.com/rodlunt/engineering-audit@v0.15.0 engineering-audit-mcp

Inline mode: generate the trigger fragment and append it to your repo's AGENTS.md (or ~/.codex/AGENTS.md for all repos):

uvx --from git+https://github.com/rodlunt/engineering-audit@v0.15.0 engineering-audit-fragments \
    --rules-dir /path/to/engineering-audit/examples/taster-rules --out-dir .
cat AGENTS-fragment.md >> AGENTS.md

Standalone audit: in a codex session, ask it to read AUDIT.md from this repository and run the audit. Headless notes and caveats: integrations/codex/.

Gemini CLI

There is no packaged extension. Gemini CLI resolves an extension only from a repository root, which would put a manifest, a GEMINI.md and a commands/ directory in the root of a tool that also serves Claude Code and Codex, and none of it was ever exercised against a real Gemini CLI. Registering the server by hand is one paste and has no such cost.

Add the server to ~/.gemini/settings.json (or a project-level .gemini/settings.json), swapping in the taster path from Step 3:

{
  "mcpServers": {
    "engineering-audit": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/rodlunt/engineering-audit@v0.15.0",
        "engineering-audit-mcp"
      ],
      "env": {
        "ENGINEERING_AUDIT_RULES_DIR": "/path/to/engineering-audit/examples/taster-rules"
      }
    }
  }
}

Then start gemini in the repository you want audited and paste:

Audit this repository against the engineering rules. Read AUDIT.md from the
engineering-audit repository and follow it, driving the engineering-audit MCP tools
through to a rendered report.

Inline triggers work too, and need no extension: generate the fragment and merge it into whichever GEMINI.md tier you want it to apply to.

uvx --from git+https://github.com/rodlunt/engineering-audit@v0.15.0 engineering-audit-fragments \
    --rules-dir /path/to/engineering-audit/examples/taster-rules --out-dir .
cat GEMINI-fragment.md >> GEMINI.md

Gemini support is documented, untested: Gemini CLI was not available to exercise any of it. Check gemini --help before an unattended run. Details and caveats: integrations/gemini/.

Optional: plan before you build

Install Engineering Grill after registering the MCP server if you want a guided, plain-English planning conversation before code is written. It reads the domains from the connected rules pack, so the taster pack gives a three-domain grill and the full rules pack gives the complete framework.

Headless / CI

Skip the interactive configuration page by pointing ENGINEERING_AUDIT_CONFIG at a saved configuration JSON (shape documented in AUDIT.md); get_config then returns immediately. Example driver, Claude Code:

claude -p "Read AUDIT.md at <path> and audit this repository via the engineering-audit \
MCP tools." --mcp-config mcp.json --allowedTools "mcp__engineering-audit__*,Read,Glob,Grep"

Every run also checks this repository's tags for a newer release, on by default (see Security for what that discloses). On an air-gapped machine, or anywhere the network cost or the call itself is unwanted, pass --no-update-check to engineering-audit-mcp or set the ENGINEERING_AUDIT_NO_UPDATE_CHECK environment variable.

What keeps the staleness checks working

Two checks tell you whether what you are running is current: one for the tool, one for the rules pack. Neither ever guesses. A check that could not run reports could-not-check, which is a distinct state from current, so a stale build is never reported as fine. What that honesty does not do is tell you when a check has gone blind, and both go blind on install shapes that look perfectly ordinary.

Check

Attached when

Blind when

What you lose

Tool build

installed from a git URL, which is what every command in Step 4 does

installed from a downloaded archive or a plain wheel, or run from a local or editable checkout

nothing warns you that a pin left on an old tag, or a stale uvx cache, is serving an old build

Rules pack

the rules directory is a git clone with an origin remote, which is what Step 3 produces

the rules arrived as a downloaded zip, were vendored into another repository, have no remote, or have uncommitted changes

nothing warns you that the run is judging your repository against superseded rules

Following Step 3 and Step 4 as written keeps both attached: git clone for the rules, and uvx --from git+...@<tag> for the tool. Any other shape still runs and still audits correctly. It just cannot tell you it is out of date.

The provenance rows in the report header carry the answer for the run in front of you. Both reading could-not-check at once is the combination to watch for, because that is a build of unknown age judging your repository against rules of unknown age, with nothing able to detect either.

Step 5: run your first audit

Ask your assistant to audit the repository you have open ("audit this repo against the engineering rules"), tick the domains on the configuration page that opens, wait for the sweep, then open audit-output/report.html. What to expect while it runs is in What a run looks like; what it costs in tokens is in What a full run costs.

Related MCP server: code-review-mcp

Engineering Grill, before the code exists (BETA)

Status: beta, and the label is meant literally. The interview shipped in v0.13.0 as a separate interrogate skill and was folded into Engineering Grill by issue #239: one pre-build skill, not two. Nobody has yet completed a full interactive session with it. Expect it to change.

An audit sweeps the rules over a repository that already exists. Engineering Grill runs them the other way, as questions about work that has not started. It classifies every domain the pack returns, derives the full question set from the rules of those whose triggers genuinely fire, then puts the highest-consequence questions first, one at a time, in the Hot Seat. The rest are held, not discarded: it names the total, offers a deep dive through everything remaining, and records what was answered, what was deferred and what was never asked.

Triage is global on purpose. One domain's third-best question routinely matters more than another's first, and a per-domain ranking cannot see that. Questions carry a reversibility grade and a blast radius so they can be compared across domains at all.

The rule that shapes the whole design is that no full domain document enters the conversation with the user. That is stated as an invariant rather than an architecture: sub-agents satisfy it where the host has them, serial read-and-discard satisfies it where the host does not, which is what keeps one skill shippable on more than one assistant.

Ten of the sixteen domains are design-time by their own Load this when: statements, so most of the pack was already pointed at the moment before the code, with nothing to deliver it there.

It never starts a run. It calls list_domains and get_domain only, both of which work with no run in progress, so it works on a directory that is not a repository yet or on nothing but a description. Setup is in integrations/engineering-grill/, and the optional hook that offers it when plan mode starts is in integrations/claude-code/README.md.

What has been exercised: question derivation on three domains (d02, d01, d15) against one brief, the no-run guarantee, and the hook's failure paths.

What has never been run even once: the cross-domain triage. It is the newest part and the part everything else now depends on, so treat its output with more suspicion than the rest.

What has not: the other thirteen domains, and the interactive loop itself with a real person answering. Nobody has yet finished a full interrogation. Until they have, treat the shape of the session as unproven and the question quality as sampled rather than measured.

This is a beta of a Claude Code integration specifically. There is no Codex or Gemini equivalent, and unlike audit there is no assistant-neutral protocol document behind it: the skill file is the whole specification. That is deliberate for now and is the first thing to change if it earns its place.

Scope

This documentation states functional behaviour (what the tool does) and the security and privacy properties documented inline (how the access token is held, what telemetry you opt into, what leaves your machine and when). The two human-facing surfaces, the configuration page and the report page, target WCAG 2.2 Level AA. Applying the same proven versus documented, untested distinction used below for the assistant integrations: one criterion is proven, the rest are documented, untested. Contrast is machine-checked on every test run, tests/test_report_stylesheet.py computes contrast ratios from the CSS custom properties and asserts 4.5:1 in both the light and dark palettes. No keyboard-only pass and no screen reader pass has been recorded against either surface, and neither template currently sets an explicit minimum target size on its interactive controls, so those criteria (and WCAG 2.2's target size, success criterion 2.5.8, specifically) have not been checked by anyone. The target is real; only the contrast ratio has a recorded check behind it. Performance is explicitly out of scope for now: no throughput or latency target is stated or tested anywhere in this repository.

How it works

The tool is a local MCP server (Python, stdio). It points at a local directory of rule documents and serves them to the agent driving the audit. The agent supplies the judgement; the server supplies everything that must not depend on an LLM's memory: schema-validated finding capture (a rule the agent did not check can never be recorded as a pass), the configuration page, deterministic report rendering, source citations attached from the rules pack itself, and GitHub issue filing with an explicit confirmation step.

Two audit modes, plus an optional project-start skill:

  • Engineering Grill: before code is written, the assistant sorts the loaded domains by relevance, asks framework-backed questions and records the resulting plan. It uses only the read-only list_domains and get_domain tools and does not start an audit run.

  • Standalone audit: tick the domains to audit on a local configuration page (or supply a saved config for headless runs), the agent sweeps the repository, and you get report.html plus optional GitHub issues.

  • Inline: one-line triggers merged into your assistant's instruction context tell it to call get_domain(...) at decision moments (designing a schema, cutting a branch, shaping an API), so the rules arrive exactly when they are useful.

Support matrix

Assistant

Inline mode

Standalone audit

Claude Code

proven (in daily use via skills)

proven (recorded run 2026-08-09)

OpenAI Codex CLI

documented, untested

proven (recorded run 2026-08-10)

Gemini CLI

documented, untested

documented, untested

GitHub Copilot

unsupported

unsupported

"Proven" means a recorded end-to-end run exists. "Documented, untested" means the integration follows the assistant's official documentation, with individually verified pieces labelled in the integration README, but no full audit has been exercised on it yet. Copilot is deliberately unsupported rather than silently absent.

What a run looks like

A standalone audit is a conversation plus one browser page. From your seat:

  1. Ask for the audit ("audit this repo against the engineering rules"). The assistant gathers run metadata and starts the run.

  2. A configuration page opens in your browser (http://127.0.0.1:<port>/). Opening it is best-effort: in a remote or display-less session no tab can appear, so the assistant also prints the URL; open it yourself if nothing popped up. Tick the domains to audit, choose report-only or GitHub issue filing, and submit. Nothing proceeds until you submit: the tool never falls back to a domain selection you did not make.

  3. The assistant sweeps the repository domain by domain. This is the slow part: minutes for a small repository and a few domains, longer for a big selection. You can ask for progress; it can report which domains are recorded and which remain.

  4. Everything lands in audit-output/ inside the audited repository: report.html (the deliverable, openable in any browser; the assistant offers to open it when the run finishes) and run-state.json (the raw machine-readable results, which can re-render the same report later via engineering-audit-render). If you chose GitHub filing, the assistant previews the issues and asks before filing anything.

audit-output/ belongs to the audited repository, not to this tool. Commit it, ignore it or delete it as that repository's own conventions dictate.

What a full run costs

A full sweep is token-hungry, and the configuration page's domain tick boxes are the cost control: cost scales close to linearly with the domains you tick. Budget from these recorded runs rather than guessing, one row per host that has completed one:

Host

Tool version

Scope

Active time

Findings

Tokens

Claude Code, Fable 5 orchestrating Sonnet subagents (this repository, 2026-08-09)

0.4.0 (established from tag history, not stamped into a retained report; see docs/example-audit-cost.md)

all 16 domains, 260 rules

47 minutes end to end, sweeps running four at a time

33 (every one filed as a GitHub issue)

2,010,691 subagent tokens, roughly 100k to 170k per domain (excludes the orchestrator)

Codex CLI 0.147.0, gpt-5.6-sol at high reasoning effort (external React SPA, roughly 344 files, 2026-08-10)

0.5.1

all 16 domains of the standard pack

19 minutes 21 seconds

32 (122 rules could not be evaluated)

6,172,397 input plus output, of which 96% is cached input; 269,293 non-cached input plus output

Both rows predate v0.9.0 and v0.9.1, which added required per-finding and per-domain output that neither run had to produce; see the comparability note in docs/example-audit-cost.md before budgeting a current run from either figure.

The two token columns are not the same measurement and must not be subtracted or averaged. Codex does not fan out to one subagent per domain the way the Claude Code skill does, and that single difference drives everything: fanning out gives each subagent a small fresh context and bills mostly uncached input, while staying in one long context re-reads a large accumulated context every turn. Hence Codex's 6.17M being 96 per cent cache reads. The Claude Code figure counts per-subagent totals and excludes the orchestrator conversation entirely. Use each row within itself, for the shape of run it describes. Full per-host detail, including what each run did and did not measure, is in docs/example-audit-cost.md.

The report

Self-contained HTML, generated locally; nothing leaves your machine unless you choose to send or file it. It is a written report meant to be read top to bottom, in this order:

  • A computed headline, first thing on the page: one sentence naming what needs attention first, built from the run's own counts, with a second line saying what was set aside or could not be evaluated so a partial sweep cannot read as a clean bill of health.

  • Findings, sorted worst first rather than in the order they were recorded, each in three parts (the issue and location, why it matters, suggested fix) with the rule's citation appended automatically from the rules pack. The tool refuses to publish a finding whose rule carries no citation. Each finding also carries its domain's self-assessed confidence and whether that domain's rule text was fetched this run, so a finding from a shaky domain does not look identical to one from a solid one. The confidence never appears on its own: it ships with how many of that domain's rules could not be evaluated, out of how many, so a domain claiming high confidence over half a domain it could not check says so in the same breath. The four severity levels are defined on the page, and stated as assigned by the assistant named in the header rather than measured.

  • Issues: tick boxes to select findings, then file them to GitHub directly from the report (fine-grained PAT, used in memory only, sent only to api.github.com), or copy the selected set for pasting into an LLM or editor, or copy them one at a time. Only critical and high findings are ticked on load; nothing is hidden, and an already-filed finding shows unticked with a link to it.

  • A tool performance summary about the audit run itself, below the findings rather than above them: one table with a row per domain carrying rule verdicts, findings by severity, files inspected and skipped, confidence and fetch status, with the could-not-evaluate and not-applicable reasons in full, and an evidence boundary block naming what each domain did not read. Every number ships with its base, a domain set aside in full cannot read as a domain swept clean, an unchecked rule is never presented as a pass, and a value the tool was told rather than measured says so. Longer sections sit behind summaries that each carry their own numbers, so the detail is one click away rather than occupying the top of the page.

  • Feedback to the author: freeform text, two optional questions asking what you concluded from the report and what you would fix first, and tick-box consent over which run statistics accompany any of it. Everything here is off by default and never prefilled. Finding text never leaves your machine through this channel.

It prints sensibly too: the issue-filing section is dropped from print rather than clipped, with a line saying where to find it, and collapsed sections are expanded on paper.

A live example: docs/demo/report.html (download and open locally; GitHub does not render raw HTML in the browser). Generated from the invented demo rules pack in tests/fixture_pack, not a real audit against a real repository.

The rules

The author's rules pack covers sixteen decision domains, 260 rules in all, each rule carrying a cited source, a volatility tier and a verification date, and each domain proven against a real system before it is trusted:

#

Domain

Rules

Fires when you are...

d01

Designing a Data Model

15

modelling entities, choosing keys, constraints, normalising, writing DDL or migrations

d02

Eliciting and Specifying Requirements

16

deciding what to build, writing requirements or user stories, checking the right problem is being solved

d03

Modelling Structure and Behaviour Before Building

15

deciding what to diagram before coding, drawing or reviewing FMC/UML/SysML models

d04

Structuring Code and Applying Design Patterns

14

designing classes or modules, weighing a design pattern, choosing data structures or error handling

d05

Choosing What to Test and How Much

18

choosing test levels and coverage, weighing testing against risk, planning load or soak tests and CI gates

d06

Structuring a Repo, Branches and CI/CD

15

structuring a repository, writing CI/CD workflows, handling automation credentials, cutting releases

d07

Handling Untrusted Input and Secure Coding

16

writing code untrusted input can reach: forms, auth flows, credentials, sessions

d08

Threat Modelling and Security Risk Decision-Making

15

running a risk assessment, threat-modelling a system, prioritising vulnerabilities, justifying a control

d09

Responding When Something Breaks in Production

16

writing incident response plans, defining recovery objectives, running post-incident reviews

d10

Designing APIs and Service Contracts

14

creating or extending an HTTP API, choosing verbs and status codes, versioning or deprecating an interface

d11

Choosing Architecture and Deployment Topology

16

picking an application architecture, deciding VM/container/serverless topology, planning scaling and rollout

d12

Making an Ethical or Professional Judgement Call

17

facing pressure to cut a corner, decisions affecting users or the public, handling personal data

d13

Estimating and Pricing Work

16

scoping work before quoting, choosing estimation methods, setting contingency, defending an estimate

d14

Fault Diagnosis of a Running System

19

investigating an outage, a slow or wrong-answering service, or an intermittent bug

d15

Interface Design and Prototyping

17

laying out a screen, designing a form, writing button and error copy, deciding confirmation vs undo

d16

Presenting Data for Decisions

21

putting a number, chart or table in front of somebody who has to decide something

Try it right now: the taster pack

Three complete domains (d01, d05, d16, 54 rules with their full source citations) are published in examples/taster-rules/ as point-in-time exports from the maintained pack. They are a working rules directory, and they are what the How to use steps register by default: run a real audit before asking for anything.

Rules access

The full pack lives in a private repository with access granted per user (the maintained originals, their revision history and proving records). Ask via the rules pack access request form.

Once granted, the pack arrives the same way the taster did: a repository you clone.

git clone https://github.com/rodlunt/engineering-framework

Then swap the rules path in your Step 4 registration for the clone's domains/ directory (claude mcp remove then re-add for Claude Code; codex mcp add overwrites in place). Updating the rules afterwards is git pull in that clone. Keep it a real clone rather than a downloaded archive: the clone keeps its origin remote, so the run's rules-pack staleness check stays attached, exactly as described under "What keeps the staleness checks working".

The tooling works with any rules directory in the expected format (**Trigger:** header, ### N. Title rules, Rule id: footers with Source: fragments), so you can also write your own pack.

Development

uv sync
uv run pytest -q

CI runs the same suite on every push and pull request. Tests use an invented fixture rules pack; no private rule content exists in this repository. The renderer and configuration page are deterministic and fully testable with no LLM involved.

This is a solo-maintainer repository, and its merge gate is deliberately CI-only: every change lands via a pull request that must pass the check status; no human review requirement is configured.

Tracking issues and PRs

Up to now, the PR description has been the deliberate change record for this project: each PR body explains what changed and why, and that has been treated as sufficient in place of a separate issue-linked history. From now on, every PR that has a tracking issue links it with a Closes #N line (or Fixes #N), so the issue tracker and the merge history stay in step instead of relying on the PR description alone. Release PRs (chore(release): X.Y.Z) and housekeeping PRs with no issue behind them are the standing exception: there is nothing for either to close, so no keyword is expected on them. Nothing currently checks for the keyword on a PR that does carry a tracking issue; a CI check failing a PR with no closing keyword and no opt-out label is an option the maintainer can pick up separately, not something this policy statement adds on its own.

Eval harness

evals/ holds a deterministic scorer for audit quality: a small fictional golden repository with known planted findings and controls, and engineering-audit-eval to check a run-state.json against them. The scorer is CI-safe and has its own tests; the audit run that feeds it calls a real LLM and is run and checked by hand. See evals/README.md.

Roadmap

  • Thin CLI wrapper driving an agent CLI headlessly end to end (a manual, scripted version of this now lives in evals/README.md; a first-class wrapper command is still open).

  • Remotely served rules with revocable access.

  • Codex and Gemini support-matrix rows moving to proven once live runs are recorded.

Licence

The tooling in this repository (the MCP server, the deterministic report renderer, the configuration page, and every supporting script) is licensed under Apache-2.0. Rules packs are licensed separately and are not covered by this repository's licence; see Rules access.


Built by Rodney Lunt. If this saved you some time, you can buy me a coffee.

Available Tools

10 tools
begin_runA

Start a fresh audit run and create its output directory, or resume an interrupted one.

    assistant/model/repo_name/repo_commit/started are supplied by the
    calling agent; tool_version defaults to the installed package version
    if omitted. repo_dir is
    the path to the repository being audited, on disk; it is optional,
    but file_issues needs it to detect the GitHub repository to file
    against, unless a repo is given explicitly on that call instead.
    Calling this twice without finishing the first run (via
    render_report) is an error, since it would silently discard whatever
    domain results have already been recorded; pass replace=True to
    explicitly discard the in-progress run and start over.

    A run's progress is saved to a crash-recovery file in output_dir as it
    goes, so a server that stops mid-run (host restart, dropped
    connection, machine asleep) loses at most the domain in flight. When
    this call finds such a file for an unfinished run in output_dir it
    starts nothing and returns a description of it plus an instruction:
    run_started is False, "meta" is absent, and "resumable" says whether
    it can be continued at all. Call begin_run again with resume=True to
    continue that run (its recorded domains are kept, and the response
    lists which domains are still missing), or resume=False to discard it
    and start fresh. resume=False is the only way to overwrite saved
    results, and replace=True counts as the same explicit decision.
    Resuming a run for a DIFFERENT repository is refused outright, as is
    resuming one whose saved state cannot be read; either way, nothing is
    started and nothing is deleted until told.

    environment records the host facts the report header cannot carry, and
    its keys are a closed set: 'os' (e.g. "macOS 15.2", "Ubuntu 24.04"),
    'host_cli' (the CLI application driving this audit, e.g. "codex",
    "claude-code") and 'host_cli_version' (that CLI's version string).
    Collect them from the machine you are running on rather than guessing,
    and omit any key you cannot determine: an omitted fact and a guessed
    one are not the same thing. Any other key is refused outright, because
    this metadata is included in feedback issues filed publicly on the
    tool's own repository. Do not name the assistant, the model or the tool
    version here; all three are already fixed rows in the report header.

    The recorded metadata also stamps two provenance SHAs, best-effort:
    tool_commit (the git commit the installed tool build was made from,
    via its PEP 610 install record) and rules_pack_commit (the loaded
    rules pack directory's git HEAD, '-dirty' suffixed if it has
    uncommitted changes). Either is None when it could not be
    determined, which the report renders as "unknown" rather than
    guessing: a report must be traceable to the exact tool build and
    rules version that produced it, not just a package version number
    that can lag behind either.

    started is the caller's own claim about when the run began, taken on
    trust like everything else the calling agent asserts. This call also
    stamps meta.server_started from the server's own clock at the moment
    it runs, independent of that claim; render_report does the same for
    meta.server_finished. Neither figure is treated as more authoritative
    than the other in the rendered report: a resumed run genuinely spans a
    wall-clock gap that is not audit work, so the server's elapsed time is
    not automatically the truer duration, but an assistant-supplied
    duration that was never checked against anything is worse. The report
    states both and flags it when they diverge by more than expected,
    rather than presenting an unmeasured number as fact.

    The run also performs a best-effort tool update check, comparing
    tool_commit against the tool's latest tagged release on GitHub, and
    the rules pack against its own remote the same way. Each result
    lands in the returned meta (update_check and pack_update_check
    respectively), prefixed "current", "stale", "could-not-check" or
    "not-checked" (see engineering_audit.update_check for the exact
    strings). The calling agent MUST tell the user when either reports
    stale or could-not-check, rather than silently proceeding as if the
    installed build were confirmed current: this tool is installed via a
    pinned uvx reference, and a stale pin or cache would otherwise serve
    an old build forever with nothing to say so. This check runs
    automatically and discloses only the caller's IP address and the
    fact that this repository's tags were queried, no repository
    content, findings or paths; it can be turned off with
    --no-update-check or the ENGINEERING_AUDIT_NO_UPDATE_CHECK
    environment variable, in which case both fields read "not-checked",
    which is not something to warn the user about, since turning it off
    was their own choice.
    

Inputs: model (required), started (required), assistant (required), repo_name (required), +7 more. Returns: object

file_issuesA

Preview or file GitHub issues for every recorded finding, via the user's own gh CLI.

    Requires config.issue_mode == "github": if the user chose in-report
    delivery instead, this raises rather than filing issues nobody asked
    for. Requires at least one recorded domain result.

    confirm=False (the default) NEVER files anything and never invokes
    gh at all: it returns a preview {repo, count, titles, instruction}
    so the calling agent can show the user exactly what is about to be
    filed on their repository, and get explicit agreement, before a
    single issue goes out. Filing on someone's repo is outward-facing;
    this confirmation step is mandatory, not decorative.

    confirm=True files one issue per finding that has not already been
    filed, so retrying after a partial failure does not double-file the
    ones that succeeded. Filed issues are tracked, and returned, per
    finding under a key of the form "<rule id>#<n>" (n counting that
    rule's findings in recording order), not per rule id: a domain result
    may carry two findings for the same rule, and both of their issue
    urls have to survive. The target repository is `repo` if given,
    otherwise detected from the audited repository directory recorded
    by begin_run's repo_dir. If any issue fails to file, filing stops
    immediately and the error lists exactly which findings were filed
    (with their URLs) and which were not, so a retry knows where to
    resume.

    Each filed issue carries the "engineering-audit" label. The label is
    checked once per call and created on the target repository if it is
    missing; the response's label field reports which of present,
    created or unavailable happened. Unavailable (creation failed) files
    the issues unlabelled and says so once, in warnings, rather than
    once per issue.
    

Inputs: repo, confirm. Returns: object

get_configA

Fetch the resolved audit configuration, or report that the user has not submitted the configuration page yet.

    Requires start_config to have been called first. Every response carries
    a "status" field, and it is the only field worth branching on:

    - "configured": the configuration is resolved and is in the response's
      "config" and "selected_domain_ids". Stop calling this tool.
    - "waiting": the interactive page is up and nobody has submitted it
      yet. This is NOT a failure and NOT a configuration. Tell the user the
      audit is waiting on them at the "url" in the response, then CALL THIS
      TOOL AGAIN. Keep calling it while the status says "waiting".
    - a raised error: the run's overall deadline (timeout_s) elapsed with
      no submission. Tell the user the audit is not proceeding. Never fall
      back to a domain selection nobody chose.

    In preset mode the configuration is already known and comes back as
    "configured" on the first call.

    This tool deliberately blocks for at most a short interval per call
    (about 25 seconds) and then returns "waiting", rather than holding one
    call open for the whole of timeout_s. Hosts impose their own per-tool
    timeouts, independent of timeout_s (Codex has
    mcp_servers.<name>.tool_timeout_sec), and a call held open past one of
    those is cancelled by the host, which can take the whole MCP process
    and this run's configuration page down with it (issue #85). timeout_s
    remains
    the run's overall waiting budget and is enforced here, cumulatively,
    across however many calls it takes: it is measured from the moment the
    page opened, so polling more often does not buy the user more time, and
    polling less often does not cost them any. To keep waiting past the
    deadline, call again with a larger timeout_s; that is an explicit
    decision to extend, not a silent one.
    

Inputs: timeout_s. Returns: object

get_domainA

Return the full document text for one domain, given its id (e.g. 'd01').

    This tool serves the full rule text: it is meant for the local agent
    driving the audit, which needs the rules to apply them. Nothing else
    in this package returns rule body text.

    Because of that, this call is recorded against the run in progress: it
    is the one observable event that could have supplied the rules a
    verdict is meant to rest on. record_domain_result says so when verdicts
    arrive for a domain this was never called for, and the report names
    that domain. The claim either way is only ever that the text was
    fetched, never that it was read.

    A fetch made when no run is in progress belongs to no run and is not
    recorded: call begin_run first, then fetch each domain as you come to
    it.
    

Inputs: domain_id (required). Returns: result

list_domainsA

List every domain loaded from the rules pack, and report any files in the pack directory that were skipped because they had no Trigger line.

Inputs: no named parameters. Returns: object

record_domain_resultA

Record the audit result for one domain.

    The payload itself is pydantic-validated by DomainResult (finding and
    verdict consistency, could-not-run reason, could-not-evaluate and
    not-applicable notes, both of which are the verdict's stated reason,
    every finding's precondition, the completed domain's
    uninspected_evidence, and that every consulted_sources entry has a
    non-blank url, title and why). On top of that: the domain must be one of the domains
    selected for this run, a completed result must carry a verdict for
    every rule the domain defines, and every consulted_sources rule_id
    must be one of this domain's own rules; a completed result missing a
    verdict raises IncompleteResultError listing exactly which rule ids
    are missing, and an unattributable consulted source raises
    UnknownRuleIdError, so the agent can fix and resubmit rather than a
    skipped rule silently passing or a citation silently pointing at
    nothing. Re-recording an already-recorded domain requires
    replace=True, to guard against an accidental overwrite.

    Verdicts for a domain get_domain was never called for during this run
    are recorded, not refused, and the response says "rules_fetched": false
    and carries a warning naming what that means. The report names the
    domain too. Recording rather than refusing is deliberate: refusing
    would be trivially satisfied by fetching the text and ignoring it,
    which destroys the signal, while the verdicts and the fact that they
    were unsupported both survive this way. Tell the user when you see it.

    Two fields are refused outright rather than recorded with a warning,
    because unlike an unfetched domain there is no signal to preserve by
    letting them through: a finding without a `precondition` (issue #178)
    and a completed domain without `uninspected_evidence` (issue #179).
    Both are one sentence the auditor already knows the answer to, and in
    both cases being unable to write it is the finding. A finding whose
    precondition cannot be named belongs at not-applicable, and a domain
    that cannot say what it did not read has not established what its
    absence claims are worth. See AUDIT.md step 3 and step 4.
    

Inputs: result (required), replace. Returns: object

render_reportA

Finish the run and render its report.

    Requires a resolved configuration. Sets meta.finished to the given
    ISO timestamp, renders the deterministic HTML report (which itself
    refuses to render an incomplete run: a selected domain with no
    recorded result, or a completed result missing a rule verdict, raises
    rather than producing a report that looks clean over a gap), and
    writes both report.html and run-state.json to the run's deliverables
    directory: config.deliverables_dir if the configuration page (or a
    preset AuditConfig) named one, otherwise the run's own output_dir,
    unchanged from how every run before that choice existed behaved.
    output_dir itself is never affected by this choice; it stays the
    run's working directory for the crash-recovery progress file
    regardless of where the finished deliverables land (issue #109).
    Any issue URLs filed this run via file_issues, and any
    feedback issue filed via submit_feedback, are carried on the
    RunState itself, so the written run-state.json is self-sufficient:
    it (and its schema_version) can be handed to
    engineering-audit-render later to re-render the same report without
    this server, this run tracker, or either URL, still in memory.

    This call also stamps meta.server_finished from the server's own
    clock, alongside the caller-supplied finished. See begin_run's
    server_started for why the report keeps both this figure and the
    caller's rather than trusting either one alone.

    The finished run stays reachable for one last submit_feedback (the
    order AUDIT.md documents), which rewrites both files to carry the
    feedback issue's link. It stops being reachable at the next
    begin_run.

    Both files are written atomically, and the run's crash-recovery file
    is removed once they are on disk: from here the run-state.json is the
    record, and a later begin_run on this output directory starts clean
    rather than offering to resume a run that is already finished.

    The response also carries "rules_fetched": which domains had their
    rule text fetched this run, which recorded verdicts without it, and
    which were carried in from a saved run that never recorded it. Any
    domain in the second list is named in the report and must be named to
    the user as well: it says the verdicts for that domain were reached
    without the rules they are verdicts on.
    

Inputs: finished (required). Returns: object

run_statusA

Report progress for the current run: which selected domains have recorded results, which are still missing, and the findings count so far. Read-only over the run itself; it also carries any queued crash-recovery warning that no earlier response has reported yet.

Inputs: no named parameters. Returns: object

start_configA

Begin configuring the audit run.

    If the ENGINEERING_AUDIT_CONFIG environment variable names a path to
    a valid AuditConfig JSON file, it is loaded immediately (the
    documented headless/CI path); an invalid or unreadable file is a
    loud error, never a silently-applied default. Otherwise this starts
    the interactive localhost configuration page, opens it in the user's
    browser when one is available (best-effort; the response's
    opened_in_browser field says whether a tab actually opened), and
    returns its URL for the agent to show the user as the fallback.
    

Inputs: no named parameters. Returns: object

submit_feedbackA

Send optional run feedback to the tool author.

    Requires a resolved configuration. There is nothing to send unless
    config.feedback_text was set on the configuration page, or the
    calling agent supplies extra_text; if neither is present this
    raises rather than filing an empty, pointless issue.

    The feedback body always carries the free text plus a run-metadata
    section (tool version, rules pack, assistant, model, repository,
    timestamps), and then each telemetry section the user consented to
    on the configuration page (coverage totals, findings rollup by
    severity/domain id, self-assessment, environment, consulted sources
    by rule id/url/why, rule verdict distribution by domain and in
    total, run duration and the divergence verdict between its two
    measurements, which domains had their rule text fetched via
    get_domain, and the reader's own conclusions after reading the
    report); an unconsented section is left out entirely. Finding text
    itself is never included, only counts.

    report_conclusion and report_fix_first (issue #135) are the
    reader's own answers, in their own words, to the two questions the
    finished report's own feedback form asks: in one sentence, what did
    this report tell them about their repository, and what would they
    fix first. Pass these only if the human using this session actually
    read the finished report and dictated an answer back; never guess
    or paraphrase one on their behalf. Both are ignored unless the
    reader_conclusions section was consented to on the configuration
    page, same as every other telemetry section here.

    Files a labelled issue on the tool author's feedback repository via
    gh. If gh is unavailable or filing fails for any reason, the
    feedback is never lost: this returns a mailto fallback instead,
    with the same body, so the agent can offer to open the user's mail
    client or hand over the text to paste in manually.

    May be called either before or after render_report. Called after,
    it sends feedback for the run just finished and rewrites that run's
    report.html and run-state.json so both carry the feedback issue's
    link; the response's report_updated field says whether that rewrite
    succeeded, and a failed rewrite is reported as a warning rather than
    an error, because the issue is already filed by then and raising
    would invite a retry that double-files it.
    

Inputs: extra_text, report_fix_first, report_conclusion. Returns: object

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.6/5.0

    Scored across 10 tools

    Disambiguation5/5

    Each tool targets a distinct stage of the audit workflow: configuration, run lifecycle, domain metadata, rule text, result recording, progress reporting, report rendering, issue filing, and feedback. Even closely related pairs like list_domains/get_domain and start_config/get_config are cleanly separated by action and state. No two tools plausibly do the same job.

    Naming Consistency4/5

    Nine of ten tools follow a clear verb_noun snake_case pattern: begin_run, list_domains, get_domain, record_domain_result, render_report, file_issues, submit_feedback, start_config, get_config. The outlier is run_status, which reads as a noun phrase rather than an imperative verb+object form. It is still stylistically consistent and readable, so this is only a minor deviation.

    Tool Count5/5

    Ten tools map cleanly onto the full audit lifecycle without redundancy or bloat. Each tool earns its place, covering configuration, run start/resume/status, domain and rule inspection, result recording, report rendering, issue filing, and feedback submission. The scope is well matched to the server's purpose.

    Completeness4/5

    The tool surface covers the core audit lifecycle well: configuration, run management, domain discovery, rule text retrieval, result recording, rendering, issue filing, and feedback. The main gap is that previously recorded findings are only summarized by run_status or surfaced indirectly via file_issues and render_report; there is no direct tool to reread detailed recorded results after resuming a run. This is a workable omission rather than a dead end.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      A production-ready MCP server that provides AI assistants with comprehensive GitHub developer tooling including PR analysis, code review, changelog generation, dependency auditing, commit summarization, and refactoring suggestions.
      MIT
    • A
      license
      A
      quality
      B
      maintenance
      An MCP server that provides local code quality analysis for AI coding assistants, supporting file analysis, git diff review, and full project scanning with quality scoring.
      5
      3
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      An MCP server that provides code quality checks for AI coding assistants, including file size limits, ESLint, architecture compliance, anti-pattern scanning, and comment compliance.
      6 npm
      MIT