Skip to main content
Glama
rodlunt

engineering-audit

by rodlunt

begin_run

Create output directory and start or resume an engineering audit run, using crash recovery to preserve progress.

Instructions

Start a fresh audit run and create its output directory, or resume an interrupted one.

    assistant/model/repo_name/repo_commit/started are supplied by the
    calling agent; tool_version defaults to the installed package version
    if omitted. repo_dir is
    the path to the repository being audited, on disk; it is optional,
    but file_issues needs it to detect the GitHub repository to file
    against, unless a repo is given explicitly on that call instead.
    Calling this twice without finishing the first run (via
    render_report) is an error, since it would silently discard whatever
    domain results have already been recorded; pass replace=True to
    explicitly discard the in-progress run and start over.

    A run's progress is saved to a crash-recovery file in output_dir as it
    goes, so a server that stops mid-run (host restart, dropped
    connection, machine asleep) loses at most the domain in flight. When
    this call finds such a file for an unfinished run in output_dir it
    starts nothing and returns a description of it plus an instruction:
    run_started is False, "meta" is absent, and "resumable" says whether
    it can be continued at all. Call begin_run again with resume=True to
    continue that run (its recorded domains are kept, and the response
    lists which domains are still missing), or resume=False to discard it
    and start fresh. resume=False is the only way to overwrite saved
    results, and replace=True counts as the same explicit decision.
    Resuming a run for a DIFFERENT repository is refused outright, as is
    resuming one whose saved state cannot be read; either way, nothing is
    started and nothing is deleted until told.

    environment records the host facts the report header cannot carry, and
    its keys are a closed set: 'os' (e.g. "macOS 15.2", "Ubuntu 24.04"),
    'host_cli' (the CLI application driving this audit, e.g. "codex",
    "claude-code") and 'host_cli_version' (that CLI's version string).
    Collect them from the machine you are running on rather than guessing,
    and omit any key you cannot determine: an omitted fact and a guessed
    one are not the same thing. Any other key is refused outright, because
    this metadata is included in feedback issues filed publicly on the
    tool's own repository. Do not name the assistant, the model or the tool
    version here; all three are already fixed rows in the report header.

    The recorded metadata also stamps two provenance SHAs, best-effort:
    tool_commit (the git commit the installed tool build was made from,
    via its PEP 610 install record) and rules_pack_commit (the loaded
    rules pack directory's git HEAD, '-dirty' suffixed if it has
    uncommitted changes). Either is None when it could not be
    determined, which the report renders as "unknown" rather than
    guessing: a report must be traceable to the exact tool build and
    rules version that produced it, not just a package version number
    that can lag behind either.

    started is the caller's own claim about when the run began, taken on
    trust like everything else the calling agent asserts. This call also
    stamps meta.server_started from the server's own clock at the moment
    it runs, independent of that claim; render_report does the same for
    meta.server_finished. Neither figure is treated as more authoritative
    than the other in the rendered report: a resumed run genuinely spans a
    wall-clock gap that is not audit work, so the server's elapsed time is
    not automatically the truer duration, but an assistant-supplied
    duration that was never checked against anything is worse. The report
    states both and flags it when they diverge by more than expected,
    rather than presenting an unmeasured number as fact.

    The run also performs a best-effort tool update check, comparing
    tool_commit against the tool's latest tagged release on GitHub, and
    the rules pack against its own remote the same way. Each result
    lands in the returned meta (update_check and pack_update_check
    respectively), prefixed "current", "stale", "could-not-check" or
    "not-checked" (see engineering_audit.update_check for the exact
    strings). The calling agent MUST tell the user when either reports
    stale or could-not-check, rather than silently proceeding as if the
    installed build were confirmed current: this tool is installed via a
    pinned uvx reference, and a stale pin or cache would otherwise serve
    an old build forever with nothing to say so. This check runs
    automatically and discloses only the caller's IP address and the
    fact that this repository's tags were queried, no repository
    content, findings or paths; it can be turned off with
    --no-update-check or the ENGINEERING_AUDIT_NO_UPDATE_CHECK
    environment variable, in which case both fields read "not-checked",
    which is not something to warn the user about, since turning it off
    was their own choice.
    

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
modelYes
resumeNo
replaceNo
startedYes
repo_dirNo
assistantYes
repo_nameYes
output_dirYes
environmentNo
repo_commitYes
tool_versionNo

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault

No arguments

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden and does so thoroughly: it discloses that double-calling without finishing is an error, that replace=True discards results, that progress is crash-recoverable, that resuming a different repo is refused, and the update-check behavior including user warning obligations. This is exemplary behavioral disclosure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is long and dense, covering many edge cases and implementation details. While well-paragraphed and front-loaded with the core purpose, it could be trimmed; sentences like those detailing provenance SHA mechanics and elapsed-time philosophy, though valuable, add length that a more concise version might compress.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (11 params, resume/replace flow, crash recovery, provenance, update checks), the description covers all critical aspects: what the call does, what it returns (run_started, meta absence, resumable), side effects, and error conditions. The presence of an output schema is noted, but the description still explains return semantics thoroughly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, so the description must explain all parameters. It does: assistant/model/repo_name/repo_commit/started are caller-supplied, tool_version defaults, repo_dir is optional but needed for file_issues, environment has a closed key set, resume/replace semantics are defined, and started is a trust claim distinct from server-stamped times. Every parameter's meaning is addressed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a clear, specific verb+resource: 'Start a fresh audit run and create its output directory, or resume an interrupted one.' This distinguishes it from sibling tools like run_status or render_report, which query or finalize rather than initialize or resume.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit when-to-use guidance, including the requirement to call begin_run before file_issues when repo_dir is needed, the error on calling twice without finishing, and clear conditions for resume=True vs resume=False. It also states prohibitions like refusing to resume for a different repository, leaving no ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

Other Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/rodlunt/engineering-audit'

If you have feedback or need assistance with the MCP directory API, please join our Discord server