Skip to main content
Glama

Quick start

In the repository you want to check:

npx -y github:EmanueleCell4/elliot-mcp init

Restart your agent, then type:

/elliot-audit

A minute or three later the report is in elliot-report/<date>-<commit>/ at the root of the repo: report.html, report.md and claims.json. There is nothing to open, no dashboard and no account.

Requirements: Node 20 or newer, git, and Claude Code, Cursor or Codex.

As a Claude Code plugin

Inside Claude Code:

/plugin install elliot --marketplace EmanueleCell4/elliot-mcp

That adds the Elliot tools, a /elliot:audit command, and a skill that makes Claude verify its own "done" claims before reporting them.

On every pull request (GitHub Action)

Elliot reads the pull request description, checks each claim against the code ("fixes X", "adds tests for Y", "no behaviour change"), and comments with the proof. If an agent wrote the PR, this is where its "done" gets checked.

# .github/workflows/elliot.yml
name: Elliot
on:
  pull_request:
    types: [opened, edited, synchronize]
permissions:
  contents: read
  pull-requests: write
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: EmanueleCell4/elliot-mcp@v0.5.0
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}

The comment looks like this:

Verdict

Claim

Proof

❌ FALSE

Adds unit tests for the comment builder in test/comment.test.js

searched for comment.test.js: not found

✅ VERIFIED

The badge is only offered when the audit has no findings

src/audit-report.js:27

It runs Claude Code with your API key. One check usually takes 1–2 minutes. Nothing is sent anywhere except Anthropic's API and GitHub.

README badge

When every claim Elliot checks holds up, the report includes a badge you can paste into your README:

Docs verified by Elliot

Related MCP server: agentseed

Why

Coding agents say "Done ✅" when the work isn't done. They delete failing tests, skip checks, describe security that doesn't exist, and write READMEs that promise what the code never does. CI stays green because tests check the code the agent wrote, not the promises the product makes.

I found this out the hard way. My own legal-tech app had a security page promising role-based access enforced in the database. The data layer bypassed it by design, and every permission check was a function that ignored its input. The agent had told me it was done.

The usual fix is to ask a second AI to review the first one. That gives you another opinion from the same kind of system that made the mistake. In the first automatic audit I ran, the agent doing the checking cited line numbers that don't exist in 4 of 17 submissions.

Elliot works the other way around: the agent proposes, Elliot verifies. Every quote must really be in the file, at that line, at that commit: the audit fingerprints the tree when it starts (tracked changes and untracked files included) and refuses evidence once anything changes, and symlinks that lead outside the repository are not evidence. Every "this doesn't exist" is searched by Elliot itself. A fresh reviewer then tries to prove each finding wrong before it reaches the report.

What it found

Elliot 0.3 ran unattended inside Claude Code on 9 popular open-source tools from the AI coding-agent ecosystem (October 2026). A separate reviewer that had not seen the audits then checked every finding against the code.

Findings in the reports

Confirmed correct

Left unconfirmed by the reviewer

Wrong

Time per repo

13

12

1

0

1 to 3 minutes

"Wrong" means a factual error: the code does not show what the finding says.

Some of what it caught:

  • A tool whose README says "No Telemetry: no data collection or tracking" ships analytics that are on by default, with consent pre-set to "yes". (public issue)

  • An app where logging in turns telemetry back on after the user switched it off, then identifies the user by email. (public issue)

  • A security page that calls a script sandbox "read-only" while the code can give scripts file writes and MCP tools. Confirmed by the project's triage bot, with a docs fix sent upstream. (issue, PR)

  • Docs saying an API key is stored encrypted while the code writes it as plain JSON.

  • A popular agent tool that sends users' full AI prompts and responses to a third-party error tracker by default, under an option named "anonymousTelemetry". Reported privately to the maintainers; details will be published after a fix.

What it still misses: on the same 9 repos it found 6 of the 10 serious issues I had found by hand beforehand (a small, hand-picked set, so read it as a rough coverage figure, not a recall estimate). The most common miss is a missing authorization check (for example, a download endpoint that doesn't check who is asking). Elliot doesn't look for those yet; see the roadmap.

Benchmark: Claude with Elliot vs Claude alone

Eight widely used SDKs, 18 broken promises planted by flipping a value behind a documented default, two runs per condition, same model and instructions. A blind judge checked every finding that was not a planted bug.

Claude alone

Claude + Elliot

Planted bugs found (2 runs)

28 / 36 (78%)

30 / 36 (83%)

False alarms (judged not real)

21% of findings

6% of findings

Real unplanted problems found

8

5

Elliot raises about a third as many false alarms and finds slightly more planted bugs; Claude alone casts a wider net. When the two blind judges (Claude and Codex) disagreed, reading the code showed Codex had misstated the code 9 times: a second model is not a check unless something re-reads the source. Method, limits and all raw outputs: benchmark/planted-promises.

Benchmark: can anyone re-check what the agent says it verified?

56 real bug fixes (28 tasks, Opus and Haiku). Every fix was correct and no agent claimed "done" falsely. But in 19 of 56 runs (34%) the check the agent said it ran could not be re-run by anyone else: a deleted scratch script, a check described in prose, a path on the agent's own machine. The work was right; the evidence was gone. benchmark/done-claims

How it works

  1. Risk sweep (Elliot). Before the agent reads anything, Elliot scans the docs for risky promises (telemetry, privacy, encryption, access control, sandboxes, "never", "by default") and the code for signals that often contradict them: analytics SDKs, permission bypasses such as --dangerously-skip-permissions, privileged database clients, secrets written to disk. The audit cannot finish until every one is addressed or skipped with a reason.

  2. Read and propose (your agent). Your agent traces each claim into the code and proposes a verdict, quoting the docs and the code.

  3. Verify the proof (Elliot). Each quote is matched against the file at the pinned commit, within two lines of where the agent said it is. Each "absent" claim is searched by Elliot with git grep. Citations that don't check out are rejected, and the agent has to fix them.

  4. Materiality test. A FALSE or PARTIAL verdict must say what a user who trusts the docs would get wrong. Omissions, naming details and tiny delays become notes, not findings. Changelogs, roadmaps and internal developer docs are rejected as sources, because they are not promises to users.

  5. Independent review. A fresh subagent that never saw the audit tries to prove every finding wrong, and confirms, downgrades or retracts it. Elliot will not write the report until every finding has been reviewed.

  6. Report. HTML, Markdown and JSON in elliot-report/, plus a tamper-evident receipt for every check in .elliot/receipts.jsonl.

If you wrote code in the same session, the protocol tells the agent to run the audit in a fresh subagent, so the auditor is not the author.

Example report

The full report is in examples/lex-ai-report.html. It was produced by an earlier version, before the independent review step existed.

Commands

Set-up: init

npx -y github:EmanueleCell4/elliot-mcp init            # every agent it detects
npx -y github:EmanueleCell4/elliot-mcp init --claude   # only Claude Code
npx -y github:EmanueleCell4/elliot-mcp init --cursor   # only Cursor
npx -y github:EmanueleCell4/elliot-mcp init --codex    # only Codex
npx -y github:EmanueleCell4/elliot-mcp init --all      # all three

init is idempotent: running it twice changes nothing, and it never overwrites a command file you edited. It writes:

Agent

MCP server registered in

Command added

Claude Code

.mcp.json (project)

.claude/commands/elliot-audit.md → /elliot-audit

Cursor

.cursor/mcp.json (project)

.cursor/commands/elliot-audit.md → /elliot-audit

Codex

~/.codex/config.toml

~/.codex/prompts/elliot-audit.md → /prompts:elliot-audit

Audit a repo: /elliot-audit

Run it in your agent from the repository root. In Claude Code and Codex, anything you type after the command is passed to the agent as extra instructions, for example /elliot-audit focus on the security page.

Check a "done" claim

The same server also checks individual claims while the agent works. It ships instructions telling the agent to use them before it says "done".

The agent says

Tool

What Elliot looks at

"I fixed / cleaned / refactored the code"

elliot_snapshot, then verify_changes

the content hash of every file before and after the task

"The behavior works"

verify_outcome

runs the requested test command in a temporary OS sandbox; only exit code 0 is VERIFIED

"It's deployed / live"

verify_url

the HTTP status and the text really served at the URL

"PR merged", "CI is green", "pushed"

verify_github

the GitHub API: merge state, check runs, whether the commit exists

"Published to npm"

verify_npm

the npm registry: version exists, published after a given time

"What did my agent really do today?"

elliot_report

your receipts: claims checked, how many held up

These checks answer VERIFIED, NOT_VERIFIED or INCONCLUSIVE. Inconclusive means Elliot could not confirm, for example because of a GitHub rate limit. It never guesses.

All MCP tools

Tool

Used by

What it does

elliot_audit_start

the agent

Pins the commit, lists the files that make claims, runs the risk sweep, returns the protocol.

elliot_record_claim

the agent

Records one claim with a verdict. Checks the source quote, every evidence quote and every absence search.

elliot_record_behavior

the agent

Records risky code behaviour the docs never mention, with checked code quotes.

elliot_review_packet

the agent

Returns a self-contained prompt for a fresh reviewer subagent.

elliot_record_review

the reviewer

Confirms, downgrades or retracts a finding. A retraction needs a code quote.

elliot_audit_finish

the agent

Refuses to finish while risk-sweep items or reviews are missing, then writes the report.

elliot_snapshot

the agent

Fingerprints the working tree before a task.

verify_changes

the agent

Compares the tree with a snapshot, or with HEAD.

verify_outcome

the agent

Runs a command and arguments against a disposable copy, with writes confined to the copy and network blocked.

verify_attestation

anyone

Checks an Ed25519 signature and receipt digest, optionally against a separately trusted public key.

verify_url

the agent

Checks status and expected text at a URL.

verify_github

the agent

Checks pr_merged, checks_passing or commit_exists.

verify_npm

the agent

Checks a package version on the registry.

elliot_report

you

Summarises recent receipts.

Verdicts in an audit

Verdict

Meaning

VERIFIED

The code implements the claim. A small gap that misleads nobody is kept as a note.

PARTIAL

Implemented, with a gap that would make a user who relies on it do something wrong.

FALSE

The code contradicts the claim, or the feature, command, flag or option does not exist.

UNVERIFIABLE

Depends on a deployment, a contract or a service outside the repo.

UNDISCLOSED

Risky behaviour in the code that the docs never mention.

Manual installation

Use this if you'd rather not run init.

Claude Code

claude mcp add elliot -- npx -y github:EmanueleCell4/elliot-mcp

Codex (~/.codex/config.toml)

[mcp_servers.elliot]
command = "npx"
args = ["-y", "github:EmanueleCell4/elliot-mcp"]

Cursor (.cursor/mcp.json) and any other MCP client

{ "mcpServers": { "elliot": { "command": "npx", "args": ["-y", "github:EmanueleCell4/elliot-mcp"] } } }

Then add the command yourself, or ask your agent to "run an Elliot claim audit of this repository". For agents that support skills, also add skills/elliot/SKILL.md.

Configuration

All settings are optional environment variables.

Variable

Default

What it does

GITHUB_TOKEN

unset

Used by verify_github for private repos and higher rate limits.

ELLIOT_DIR

.elliot/ in the working directory

Where receipts and audit state are stored.

ELLIOT_GITHUB_API

https://api.github.com

GitHub API base URL, for GitHub Enterprise or tests.

ELLIOT_NPM_REGISTRY

https://registry.npmjs.org

Registry used by verify_npm.

ELLIOT_SIGNING_KEY

unset

Path to an Ed25519 PKCS#8 private key. When set, every saved receipt is signed. Keep it outside the repository.

ELLIOT_SIGNER_ID

public-key fingerprint

Human-readable signer label included in signed receipts. The label is not an identity proof by itself.

ELLIOT_ROLE

unset

Set to reviewer for a separate reviewer-only MCP server process.

ELLIOT_REVIEW_TOKEN

unset

Enables isolated review mode; configure the same non-empty secret on auditor and reviewer server processes.

Network access and data

Elliot has no telemetry and no server of its own. What it touches:

  • Audit (/elliot-audit, /elliot:audit): reads files in the repository and runs git locally. It makes no network calls.

  • verify_url: sends one GET request to the URL the agent passes in. Loopback, private-network and link-local addresses (including cloud metadata) are refused on every redirect hop unless you set ELLIOT_ALLOW_PRIVATE_URLS=1; one 15-second deadline covers the whole request and the body stops at 2 MiB.

  • verify_github: calls api.github.com (or ELLIOT_GITHUB_API). If a GitHub token is set (the GITHUB_TOKEN env var, or the plugin's optional github_token setting), it is sent only to that API.

  • verify_npm: calls registry.npmjs.org (or ELLIOT_NPM_REGISTRY).

  • GitHub Action: runs Claude Code in your CI with your ANTHROPIC_API_KEY, reads the PR description and code, and posts one comment through the GitHub API.

  • On disk: receipts in .elliot/ (or ELLIOT_DIR) and reports in elliot-report/, both inside your repository.

The Claude Code plugin runs the server bundled in this repository (node src/index.js). Claude Code installs its two npm dependencies from package-lock.json when you install the plugin.

Output

your-repo/
├── elliot-report/
│   └── 2026-10-22-dd84474/
│       ├── report.html     # open in a browser
│       ├── report.md       # paste into an issue or PR
│       └── claims.json     # everything, for scripts and CI
└── .elliot/
    ├── receipts.jsonl      # one tamper-evident receipt per check
    └── audits/             # audit state

Add .elliot/ to .gitignore. Commit elliot-report/ if you want the report to live with the code.

A receipt looks like this:

{
  "elliot": "0.3.0",
  "id": "rcpt_3f9a1c2b7d10",
  "check": "claim",
  "claim": "Role-based access is enforced at the database layer through row-level security.",
  "target": "lex-ai@dd84474",
  "verdict": "FALSE",
  "reason": "All data access uses the service-role client, which bypasses RLS.",
  "evidence": { "source": { "file": "app/security/page.tsx", "line_start": 14 }, "evidence": [ { "file": "lib/supabase/index.ts", "line_start": 10 } ] },
  "checked_at": "2026-10-21T19:42:11.000Z",
  "checker": "local",
  "digest": "sha256:…"
}

The digest detects changes to a receipt. For third-party verification, configure a private key outside the repository:

openssl genpkey -algorithm Ed25519 -out "$HOME/.elliot-signing-key.pem"
chmod 600 "$HOME/.elliot-signing-key.pem"
export ELLIOT_SIGNING_KEY="$HOME/.elliot-signing-key.pem"
export ELLIOT_SIGNER_ID="acme-ci"

Saved receipts then include the Ed25519 public key, its SHA-256 fingerprint, signer label and signature. A verifier checks the digest and signature with verify_attestation; valid is true only when a separately trusted public key is supplied and matches. Without that key, the verifier reports cryptographic validity while marking signer identity untrusted. The embedded key proves which key made the signature, not that the key belongs to a particular person or organization. Publish or distribute the trusted public key through a channel your verifiers already trust. Protect and back up the private key outside the repo. Receipts created without ELLIOT_SIGNING_KEY remain unsigned for compatibility.

Isolated review

For enforced reviewer separation, run Elliot twice with the same ELLIOT_DIR and a private ELLIOT_REVIEW_TOKEN (generate one with openssl rand -hex 32). The auditor process uses ELLIOT_ROLE=auditor; the independent reviewer process uses ELLIOT_ROLE=reviewer. The auditor server omits elliot_record_review; the reviewer server exposes only that tool. elliot_review_packet gives the reviewer the finding, checked quotes, impact and an audit-scoped capability, without repository access or the auditor's explanation. Elliot encrypts the capability in audit state, checks its hash and the reviewer server credential on submission, and rejects cross-audit reuse and duplicate decisions. Keep the shared ELLIOT_REVIEW_TOKEN in the two host configurations; only the single-audit capability travels in the review packet.

{
  "mcpServers": {
    "elliot-auditor": {
      "command": "node", "args": ["/path/to/elliot-mcp/src/index.js"],
      "env": { "ELLIOT_ROLE": "auditor", "ELLIOT_REVIEW_TOKEN": "<same-random-secret>", "ELLIOT_DIR": "/path/to/project/.elliot" }
    },
    "elliot-reviewer": {
      "command": "node", "args": ["/path/to/elliot-mcp/src/index.js"],
      "env": { "ELLIOT_ROLE": "reviewer", "ELLIOT_REVIEW_TOKEN": "<same-random-secret>", "ELLIOT_DIR": "/path/to/project/.elliot" }
    }
  }
}

The host must give the reviewer only the packet and the reviewer MCP connection. The default unconfigured mode retains the original combined tool surface for existing MCP clients. Separate processes and tool permissions enforce the boundary; deployments where the auditor can read reviewer host configuration or impersonate its MCP connection do not provide a separate trust domain.

Functional outcome checks

Call verify_outcome with an executable and argument array, for example command: "npm", args: ["test"]. Elliot copies the working tree to a temporary directory, then runs the command in the OS sandbox (sandbox-exec on macOS or bwrap on Linux), with network access blocked and writes limited to the temporary copy. A zero exit code is VERIFIED; a failing check is NOT_VERIFIED. If the OS sandbox is unavailable, the result is INCONCLUSIVE and the command is not run. Dependencies already present in node_modules are copied into the temporary tree. No shell expansion is performed; pass flags as separate arguments.

FAQ

Isn't this just an LLM judging an LLM? No. The agent proposes verdicts, but every piece of evidence is checked deterministically: quotes against files at a pinned commit, absences with Elliot's own git grep. A second agent that never saw the audit reviews every finding, and a retraction has to be backed by a code quote that Elliot also checks.

Does it send my code anywhere? Elliot itself makes no network calls during an audit. It reads files and runs git locally. Your agent reads the code as it normally does, with the model you already use. The verify_url, verify_github and verify_npm checks call those services only when the agent uses them.

Does it change my code? No. It writes only .elliot/ and elliot-report/, and init writes the config files listed above.

What does it cost? Nothing. Elliot is MIT-licensed and the audit runs on the agent and model you already pay for. An audit of a medium-sized repo took 1 to 3 minutes in our tests.

Can it audit private repos? Yes. It runs where your code is. Only verify_github needs GITHUB_TOKEN for private repos.

Can I trust a PASS? Treat a clean report as "nothing found among the claims checked", not as a certificate. The claim audit is a static reading of the code; it does not run your app (use verify_outcome for that). Authorization checks are covered only where sibling handlers check the caller and one does not; ownership checks are not covered yet.

I found a false positive. Please open an issue with the report's claims.json entry. False positives are the number Elliot is built to keep at zero.

Roadmap

  • File, URL, GitHub and npm checks with local receipts

  • /elliot-audit: claim audit with checked citations, report in elliot-report/

  • Risk sweep, undisclosed behaviours, materiality test and independent review

  • Authorization gaps: handlers that skip the auth check their sibling handlers perform

  • Ownership checks: handlers that check the caller is logged in but not that they own the record

  • Run functional outcome checks in an OS sandbox

  • Hooks that run Elliot automatically when the agent stops (Claude Code, Codex)

  • A GitHub Action that checks the claims in each pull request description

  • Ed25519 signed receipts with offline verification

Development

git clone https://github.com/EmanueleCell4/elliot-mcp && cd elliot-mcp
npm install
npm test     # 35 tests, including end-to-end runs that drive the server over stdio with the MCP client

The server is plain Node (ES modules) with two dependencies: the MCP SDK and zod. The main pieces:

File

What it does

src/index.js

MCP server, tool definitions, init entry point

src/audit.js

Audit state, citation checks, review gate, verdict resolution

src/risk.js

Risk sweep: risky doc promises and code signals

src/audit-protocol.js

The protocol the agent follows

src/audit-report.js

HTML and Markdown reports

src/init.js

Registers Elliot in Claude Code, Cursor and Codex

src/checks/

verify_changes, verify_outcome, verify_url, verify_github, verify_npm

Issues, false positives and stories of agents telling you they were done are very welcome.

License

MIT © 2026 Emanuele Cella

Available Tools

14 tools
elliot_audit_finishFinish the audit and write the reportA

Close the audit and write report.html, report.md and claims.json into elliot-report/-/ at the repository root.

NameRequiredDescriptionDefault
out_dirNoOverride the output folder (relative to the repository root).
skippedNoRisk-sweep items you looked at and decided need no claim, each with a reason. They are listed in the report.
audit_idYes

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and openWorldHint=false; the description usefully adds what is created and where (three named files under elliot-report/<date>-<commit>/). It does not disclose whether existing files are overwritten, whether the audit is locked afterwards, or what happens with unrecorded candidates, so it stops short of a full behavioral picture.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the action and its concrete outputs come first and nothing is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a state-mutating finalization tool with no output schema, the description covers the produced artifacts but omits the preconditions for a successful call, the effect on audit state, and what is returned on success or failure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 67%, and the description says nothing about any of the three parameters. In particular the significant 'skipped' array (risk-sweep items with reasons that appear in the report) and the 'out_dir' override are left entirely to the schema, so the description does not compensate for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Close the audit') and enumerates the exact artifacts produced (report.html, report.md, claims.json) plus their destination path. It is distinguishable from most siblings, but it does not clarify how it differs from the sibling elliot_report, which plausibly also emits a report.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: 'Close the audit' suggests this is the terminal step of an audit workflow, but there is no explicit statement of prerequisites (an open audit_id from elliot_audit_start, claims already recorded) or of when to prefer this over elliot_report.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

elliot_audit_startStart a claim auditA

Start auditing whether this repository does what its README, docs and security/pricing pages say. Pins the commit, lists the files that make claims, and returns the protocol to follow.

NameRequiredDescriptionDefault
pathNoAny path inside the repository (default: the server's working directory).
sourcesNoAudit only the claims made in these repo-relative files, e.g. a pull request description saved as .elliot-pr/PR.md. Turns off the repo-wide risk sweep.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=false and openWorldHint=false, and the description corroborates the mutating nature by disclosing that it 'pins the commit' and returns a protocol to follow. It does not say whether the pinned state is persistent, whether concurrent audits conflict, or what cleanup is required against elliot_audit_finish. Still, the key side effect is surfaced, which is real value beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense sentence whose first clause states the core purpose and whose remaining clauses front-load the concrete outputs. No filler, no repetition of the title, nothing that could be cut without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-required-param session-start tool with full schema coverage, no output schema and meaningful annotations, the description covers purpose, side effects, and the returned artifact ('the protocol to follow'). What is missing is lifecycle context: how long an audit stays open and its relationship to elliot_audit_finish. That is a modest gap, not a blocking one.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and both parameters are richly documented in the schema, including the default working directory and the fact that 'sources' disables the repo-wide risk sweep. The description adds nothing about either parameter, so it sits at the baseline. The phrase 'lists the files that make claims' loosely gestures at the sources scoping but no more than the schema already says.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb and resource (audit whether the repo matches its README/docs/security/pricing claims) and enumerates the concrete effects: pinning the commit, listing claim-bearing files, returning a protocol. That is far more than a restatement of the name. It does not name elliot_audit_finish as the counterpart, so sibling differentiation is only implied by the word 'Start'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The verb 'Start' plus 'returns the protocol to follow' implies this is the entry point of a multi-step audit before elliot_record_claim / elliot_audit_finish, but that sequencing is never stated. There are no explicit when-not conditions or named alternatives. The agent must infer the workflow position rather than being told it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

elliot_record_behaviorRecord undisclosed risky behaviorA

Record something risky the code does that the docs never mention (for example, it sends usage events, stores a key in plain text, or skips permission prompts) and no doc sentence promises otherwise. If a doc sentence does promise otherwise, use elliot_record_claim on that sentence instead. Never attach this to an unrelated sentence.

NameRequiredDescriptionDefault
signalNoThe code signal id from the risk sweep that led here, if any.
audit_idYes
behaviorYesWhat the code does, plainly, e.g. 'Sends the user email to PostHog on login'.
evidenceYesCode that does it.
severityYes
user_impactYesWho is affected and how.
docs_searchedYesWhere you looked in the docs and found nothing about it.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only tell the agent this writes to a closed-world target; the description adds the substantive preconditions (no doc sentence may promise otherwise, must belong to a specific sentence) that govern a valid record. It stops short of saying what recording does downstream — dedup, ID returned, effect on the audit — so it is not fully transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose and only three sentences long; the examples and the sibling routing each earn their space. The closing instruction 'Never attach this to an unrelated sentence' is slightly oblique since no sentence parameter exists in the schema, costing a point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter write tool with no output schema, the description covers the conceptual bar for behavior/docs_searched but omits what a successful call produces and how audit_id or signal should be sourced. Adequate to call correctly in principle, but an agent must infer the plumbing from the schema alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 71%, and the description adds real meaning for two of the trickiest fields: what counts as 'behavior' (with concrete examples) and the evidentiary standard behind docs_searched ('the docs never mention... no doc sentence promises otherwise'). It says nothing about audit_id, severity, or signal, leaving those entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource ('record something risky the code does that the docs never mention') with three concrete examples of qualifying behavior. It also names the sibling elliot_record_claim and the exact condition that separates the two, so an agent can route without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool (undocumented behavior with no doc promise), when to use the alternative instead ('If a doc sentence does promise otherwise, use elliot_record_claim on that sentence'), and an exclusion ('Never attach this to an unrelated sentence').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

elliot_record_claimRecord one audited claimA

Submit one claim with its verdict and evidence. Elliot checks that the claim's source quote is really in the docs at those lines, that every evidence quote is really in the code at those lines, and runs every 'absent' search itself. If anything does not check out the claim is rejected: fix the citation and submit again.

NameRequiredDescriptionDefault
noteNoFor VERIFIED: a minor gap worth mentioning that misleads nobody.
claimYesThe claim, quoted or tightly paraphrased, as a user would read it.
sourceYesWhere the project makes the claim.
verdictYesVERIFIED: the code implements it. PARTIAL: implemented with a material gap or narrower than claimed. FALSE: the code contradicts it or the feature does not exist. UNVERIFIABLE: depends on a deployment, contract or service outside the repo.
audit_idYes
evidenceYesCode that implements or contradicts the claim. Use type 'absent' when the claimed thing does not exist in code.
severityNoRequired for PARTIAL and FALSE: how much a user relying on the claim is hurt.
explanationYesOne or two plain sentences: what the code actually does.
user_impactNoRequired for PARTIAL and FALSE: what a user who relies on this sentence would get wrong, concretely. If there is no real consequence, use VERIFIED with a note.

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by disclosing the validation contract: source quotes are checked against the docs at those lines, evidence quotes against the code, and 'absent' searches are executed server-side. It does not say whether a rejected claim is persisted, whether resubmission replaces a prior claim, or what a successful submission returns, which are the remaining behavioral unknowns for a mutating call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, front-loaded with the action, then the validation mechanics, then the failure remedy. Nothing is repeated from the schema and no sentence is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Adequate for a 9-parameter nested-schema tool whose fields are 89% documented: the description covers the one thing the schema cannot, namely that quotes and absence checks are independently verified. It stops short of describing the success response, which matters slightly since there is no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 89%, so the baseline is 3; the description earns an extra point by clarifying that for 'absent' evidence the agent supplies only a pattern and Elliot performs the search, which is non-obvious from the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Submit one claim with its verdict and evidence") and names the payload triple, so an agent can distinguish it from elliot_record_behavior and elliot_record_review without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear retry guidance once validation fails ("fix the citation and submit again"), which is the main decision point for a per-claim submitter. It does not explicitly route against sibling recording tools, so the alternative-selection half of the guidance is left implicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

elliot_record_reviewRecord an independent review decisionA

For the reviewer process: confirm, downgrade or reject one finding. In isolated mode provide the audit-scoped reviewer_token from the evidence packet. Confirm needs evidence you checked yourself (a code quote or an absent search); reject needs a code quote showing the claim holds. All evidence is checked against the files.

NameRequiredDescriptionDefault
reasonYes
findingYesThe finding id, e.g. 3 or b1.
outcomeYesconfirm: real and material (pass severity to change it, e.g. low). downgrade: the mismatch is real but immaterial and becomes a note; do not pass a severity. reject: the claim actually holds.
audit_idYes
evidenceNo
severityNoCorrected severity, when the auditor's is wrong.
reviewer_tokenNoPer-audit capability included in the evidence packet when isolated review is enabled.

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by disclosing the enforcement rule that all evidence is verified against the actual files, which implies submissions can fail validation on unverifiable quotes. It also states the per-outcome evidence obligations. It stops short of saying what happens on a failed check or whether a decision can be revised later.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with the purpose before the mode prerequisite and evidence rules. The 'For the reviewer process:' opening is slightly clipped but no sentence is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must carry the result story, and it does not say what the tool returns or what an agent sees after a successful or rejected review. For a state-mutating review tool with 7 parameters, the mechanics of failure and the returned decision record are a material gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 57% and the description adds real meaning for outcome by spelling out what counts as valid evidence for confirm versus reject. However, it restates reviewer_token already documented in the schema, never mentions the severity parameter, and introduces an 'absent search' evidence kind that the schema's evidence.type const ('quote' only) cannot express.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (record/review) and resource (one finding) with the exact decision vocabulary (confirm, downgrade, reject), scoped to 'the reviewer process'. It is clearly distinct from sibling recorders like elliot_record_claim or elliot_record_behavior, though it never names those siblings to reinforce the split.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear situational frame ('for the reviewer process') plus a mode-specific prerequisite: supply the audit-scoped reviewer_token from the evidence packet in isolated mode. It does not route the agent away from alternatives (e.g. elliot_review_packet) or say what to do when a finding is not yet reviewable, so the exclusion half is missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

elliot_reportHonesty reportA
Read-only

Summarise recent receipts: how many claims were checked and how many held up. Show this to the user when they ask what the agent really did.

NameRequiredDescriptionDefault
limitNoHow many recent receipts (default 50).

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this as read-only and closed-world, so the safety profile is covered. The description adds that the report summarises recent receipts and includes counts of checked and upheld claims, which is useful context but not rich behavioral detail such as data range or return format.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with no wasted words. The purpose is front-loaded, and the usage instruction follows naturally.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only summary tool with one fully documented parameter and no output schema, the description provides the essential purpose and usage context. It could clarify the scope or meaning of 'recent' beyond the limit parameter, but otherwise it is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the sole limit parameter is fully documented in the schema with its default and range. The description does not add meaning beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: summarising recent receipts, including the metrics 'claims checked' and 'held up'. It is clear what the tool does, but it does not explicitly differentiate itself from sibling tools such as elliot_review_packet or the verify_* tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear usage trigger: show this when the user asks what the agent really did. It does not provide explicit exclusions or name alternative tools, so it is not a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

elliot_review_packetGet the independent review packetA

Returns only findings and checked evidence for review. Give it to a fresh reviewer process. With ELLIOT_REVIEW_TOKEN configured, the auditor endpoint cannot record review decisions; connect the reviewer only to a separate ELLIOT_ROLE=reviewer server.

NameRequiredDescriptionDefault
audit_idYes

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and openWorldHint=false, so the description carries real weight and delivers: the payload is limited to findings and checked evidence, an ELLIOT_REVIEW_TOKEN is required for the configuration described, and the auditor endpoint cannot record review decisions. That is genuinely useful operational context an agent would not get from the annotations alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the return contract before the security caveat. Every sentence carries information; no filler, though the security sentence is dense and could be split.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description says little about the shape of the packet beyond 'findings and checked evidence', which a fresh reviewer process must parse. The critical security/role guidance is present, but the caller lacks detail on packet structure and on audit_id semantics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is one required parameter (audit_id) and schema description coverage is 0%, and the description never mentions it. The schema's regex pattern at least signals the audit_<hex> format, but nothing in the prose explains where to obtain the id or what it identifies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource: it returns findings and checked evidence, and the word 'only' scopes the payload away from decisions/summaries that sibling tools like elliot_report or elliot_record_review presumably handle. It is clear what the tool produces, though it never explicitly names which sibling it replaces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Give it to a fresh reviewer process' states the intended consumer directly, and the closing sentence gives an actionable deployment constraint (use a separate ELLIOT_ROLE=reviewer server). There is no explicit 'do not use this when...' exclusions, but the intended use case is unambiguous.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

elliot_snapshotSnapshot the working treeA

Fingerprint every tracked and untracked file in the git repository before starting a task, so verify_changes can later prove what really changed.

NameRequiredDescriptionDefault
pathNoAny path inside the repository (default: the server's working directory).

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=false and openWorldHint=false, so the agent already knows this is a bounded, non-read-only operation. The description adds the scope (tracked + untracked files) but never explains what state the non-read-only behavior creates — where the snapshot persists, whether a prior snapshot is overwritten, or how expensive it is on a large repo.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence with the action and the downstream purpose front-loaded, and no filler. Every clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With zero required parameters, no output schema and a trivial schema, the definition covers purpose, timing and downstream linkage adequately. The remaining gap is what the fingerprint operation actually stores and returns, which matters somewhat given readOnlyHint=false.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The single `path` parameter is fully documented in the schema, including that any path inside the repo works and that the server's working directory is the default. The description adds no syntax or edge-case detail beyond that, which is acceptable given 100% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

It names a concrete action and resource — fingerprinting every tracked and untracked file in a git repository — and ties that output to a specific consumer, verify_changes. That is enough for an agent to distinguish it from the other verify_* and elliot_record_* siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'before starting a task' plus the reference to verify_changes later gives a clear trigger and pairing. It stops short of stating exclusions (e.g. don't run mid-task or on non-git directories), so it is clear context rather than a full when/when-not rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_attestationVerify a signed receiptA
Read-only

Verify an Ed25519-signed Elliot receipt and its digest. Returns valid=true only when the signature passes and an explicitly supplied public key matches; without one it reports cryptographic validity but marks signer identity untrusted.

NameRequiredDescriptionDefault
receiptYes
public_keyNoPublic key independently trusted by the verifier. Omit to check only mathematical signature validity; valid remains false.

TDQS

A4.2/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no output schema, the description carries the return-value burden and does so well: it discloses that valid=true requires both signature pass and an explicitly supplied matching key, and that omitting the key yields crypto validity flagged as untrusted signer. That is substantive semantics beyond the readOnly/openWorld annotations and does not contradict them.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences, front-loaded with the core action and then the conditional valid/public-key rule. No wasted text and the key caveat arrives early.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description adequately covers return semantics and the security caveat, making the tool safe to call correctly. The only shortfall is silence on the shape of the required nested receipt object, which an agent may need before invoking.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

At 50% schema coverage, the public_key behavior is reinforced by the description, but the required 'receipt' parameter — a nested free-form object — gets no explanation of its expected structure or fields. Marginal value added over the schema, no compensation for the gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — 'Verify an Ed25519-signed Elliot receipt and its digest' — plus the exact cryptographic scheme. This clearly distinguishes it from siblings like verify_changes, verify_url, or verify_outcome, which handle other artifact types.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains the decision condition for supplying public_key (trusted signer identity vs. math-only check), which is real usage guidance. However, it never states when to reach for this tool versus the many sibling verify_* tools, leaving tool-level routing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_changesVerify files actually changedA
Read-only

Check whether files really changed, compared with a snapshot taken before the task (or with HEAD if no snapshot is given). Use before claiming code was fixed, cleaned, refactored or written.

NameRequiredDescriptionDefault
pathNoAny path inside the repository.
claimNoThe claim being checked, in the agent's own words, e.g. 'Deployed the landing page'.
pathsNoOnly count changes under these repo-relative paths.
snapshot_idNoId returned by elliot_snapshot.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=false, so the description's added value is the comparison semantics: it compares against a prior snapshot or, failing that, HEAD. That is real behavioral context an agent cannot get from the annotations or schema alone, though it says nothing about what a negative or ambiguous result looks like.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler: the action and its baseline come first, the usage trigger second. Nothing could be removed without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-required-parameter, read-only verification tool this covers what to compare, against what, and when to call it. The one remaining gap is the result shape — whether it returns a boolean, a per-file diff, or a mismatch report — which is not covered by an output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so path, claim, paths and snapshot_id are all documented in the schema itself. The description only reinforces the snapshot_id-vs-HEAD fallback; it adds no meaning for the paths filter or the claim string beyond what the schema already says. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: checks whether files actually changed, with the comparison baseline named (snapshot taken before the task, or HEAD by default). This clearly separates it from verify_url, verify_github, verify_npm, which verify external artifacts rather than local file diffs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says to use it "before claiming code was fixed, cleaned, refactored or written", which anchors the tool to a concrete moment in the workflow. It also states the default baseline (HEAD when no snapshot is given), but does not name any sibling tool as an alternative or exclusion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_githubVerify a GitHub claimA
Read-only

Check what GitHub actually says: whether a PR is merged, whether CI checks passed on a PR or commit, or whether a commit/branch was really pushed. Set GITHUB_TOKEN for private repos.

NameRequiredDescriptionDefault
prNoPull request number.
refNoCommit sha or branch name.
repoYesowner/name
checkYes
claimNoThe claim being checked, in the agent's own words, e.g. 'Deployed the landing page'.

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=true, so the safety and external-call profile is covered structurally. The description adds a genuinely useful auth prerequisite ('Set GITHUB_TOKEN for private repos'), but says nothing about behavior on missing refs, rate limits, or whether a failed check is an error or a negative verdict.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core purpose, and every clause earns its place by naming a distinct check mode or a prerequisite. The colon-separated enumeration is dense but scannable, with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of explaining what comes back, and it never says whether the result is a boolean, a verdict with evidence, or an error on a failed check. The check-to-parameter mapping is also left implicit, so a caller can still mis-invoke it despite otherwise adequate coverage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 80%, so the schema already documents pr, ref, repo, and claim; the baseline of 3 applies. The description adds modest value by implying which check each input answers (PR-based vs. ref-based), but never states explicitly that `pr` is required for pr_merged/checks_passing while `ref` is required for commit_exists/checks_passing — a pairing the agent must infer.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ('Check what GitHub actually says') and then enumerates the three exact checks it performs — PR merge status, CI check status, and commit/branch existence — which map directly onto the check enum. This clearly differentiates it from siblings like verify_npm and verify_url, which cover other verification surfaces.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The enumerated checks give clear context for when each mode applies (verifying a merge claim vs. a CI claim vs. a pushed-commit claim), and the private-repo token note signals the prerequisite. There is no explicit when-not guidance or named alternative for overlapping checks, so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_npmVerify an npm package was publishedA
Read-only

Check the npm registry for a package and version. Use before claiming a package or release was published.

NameRequiredDescriptionDefault
nameYesPackage name, e.g. 'zod' or '@scope/pkg'.
claimNoThe claim being checked, in the agent's own words, e.g. 'Deployed the landing page'.
versionNoExact version; defaults to the latest dist-tag.
published_afterNoISO time the version must have been published after (proves it is a new publish).

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=true, so the agent knows this is a safe external read. The description adds the pre-claim verification intent, but says nothing about what a failed check looks like (error vs. negative result) or how the claim parameter influences the response — meaningful gaps for a verification gate with no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero waste, with the action stated first and the usage rule second. Nothing to trim and nothing buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a four-parameter verification tool with no output schema, the description never indicates the shape of the verdict the agent should expect or how to interpret a failure, which is exactly what an agent needs to act on the result. Adequate but with a clear gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all four parameters are already documented, including the published_after proof-of-new-publish semantics. The description only echoes name and version, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: check the npm registry for a package and version. The npm-registry scope implicitly separates it from verify_github and verify_url, but no sibling is named explicitly to confirm that distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Use before claiming a package or release was published" gives a concrete trigger condition tied to the other elliot_record_claim-style tools. It lacks an explicit when-not or a named alternative, which keeps it below a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_outcomeRun a functional check in isolationA
Read-only

Run a command and arguments in a disposable repository copy under the host OS sandbox. Writes stay in the copy, network is blocked, and VERIFIED requires exit code zero. For a shell pipeline use command bash with args ["-c", "..."]; Elliot adds set -o pipefail so a failing step is not hidden by the last one. Put every file the check needs in the repository: files elsewhere (such as /tmp scripts) do not exist inside the sandbox. If the OS sandbox cannot start, returns INCONCLUSIVE without running the command.

NameRequiredDescriptionDefault
argsNoExecutable arguments, e.g. ["test"] for npm test.
pathNoRepository path (default: server working directory).
claimNoThe claim being checked, in the agent's own words, e.g. 'Deployed the landing page'.
commandYesExecutable, for example npm, node or pytest. No shell is used.
timeout_msNoTimeout, default 120000ms.

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Substantially exceeds the annotations: it discloses disposable-copy isolation, network blocking, that writes are confined to the copy, the exit-code-zero rule for VERIFIED, pipefail injection for bash, and the INCONCLUSIVE fallback when the sandbox cannot start. This also explains why readOnlyHint=true holds despite commands that write files, which is genuinely additive context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and scope, then progressively narrower operational caveats in four tight sentences with no filler. Each sentence carries distinct information (isolation, shell recipe, file placement, failure mode).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter tool with no output schema, it covers the important unknowns: isolation, network policy, success criterion, and one failure terminal state. It does not enumerate the full result vocabulary beyond VERIFIED/INCONCLUSIVE, which is the only remaining gap an agent might hit when interpreting output.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds real meaning beyond the schema: the bash + ["-c", "..."] pattern for pipelines (the schema only says 'No shell is used'), and the constraint that every needed file must live in the repository, which implicitly scopes path. It says nothing about claim or timeout_ms, leaving those to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence states a specific verb and execution context: run a command and arguments in a disposable repository copy under the host OS sandbox. It is far more precise than the title alone and makes clear this is a functional-check executor rather than a metadata/URL/npm verifier. It never names the sibling verifiers (verify_changes, verify_npm, verify_url) it is distinguished from, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the constraints: it explains the shell-pipeline recipe (command bash, args ["-c", "..."]) and warns that files outside the repository do not exist in the sandbox. There is no explicit 'use this instead of verify_changes/verify_npm when...' routing, and no stated exclusions, so the agent must infer when to reach for this tool versus its siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verify_urlVerify a URL is liveA
Read-only

Check that a URL (a deploy, a preview, a page) actually responds with the expected status and, optionally, contains expected text. Use before claiming something is deployed or live.

NameRequiredDescriptionDefault
urlYesThe URL that should be live.
claimNoThe claim being checked, in the agent's own words, e.g. 'Deployed the landing page'.
expect_textNoText that must appear in the response body, e.g. a headline you just shipped.
expect_statusNoExpected HTTP status (default 200).

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and openWorldHint=true, so the safety and external-network profile is covered. The description adds that it checks status and optionally body text, but says nothing about timeouts, redirects, authentication, or what happens on failure. With annotations carrying the safety burden, this is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences, front-loaded with what the tool does and followed by the decision context. Every sentence earns its place with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple and all parameters are well documented in the schema. However, there is no output schema, and the description does not explain what a verification result looks like (e.g., pass/fail, observed vs expected status, text-match outcome). For a verification tool, that return-value gap keeps it at minimum viable completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter is already documented in the schema. The description restates the purpose of the URL, status, and text checks but adds no syntax or format detail beyond the schema. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('check/verify') and resource ('a URL') and clarifies the scope: response status and optional expected body text. It does not explicitly differentiate itself from narrower siblings like verify_github or verify_npm, leaving the agent to infer that this is the generic URL checker.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear usage context: 'Use before claiming something is deployed or live.' This tells the agent when the tool is appropriate. It does not name alternatives or state when not to use it, which keeps it below a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 14 tool updatesv0.6.0
    • First observedelliot_audit_finish
    • First observedelliot_audit_start
    • First observedelliot_record_behavior
    • First observedelliot_record_claim
    • First observedelliot_record_review
    • First observedelliot_report
    • First observedelliot_review_packet
    • First observedelliot_snapshot
    • First observedverify_attestation
    • First observedverify_changes
    • First observedverify_github
    • First observedverify_npm
    • First observedverify_outcome
    • First observedverify_url

TDQS

A3.8/5.0

Scored across 14 tools

Disambiguation4/5

Most tools target clearly different resources: verify_changes (file diffs), verify_url (HTTP), verify_outcome (sandboxed command), verify_github, verify_npm, verify_attestation are unambiguous. Weaker spots are the report-family (elliot_report summarises receipts, elliot_review_packet packages findings, elliot_audit_finish writes report files) and elliot_record_claim vs elliot_record_behavior, which are distinguished only by careful reading of the descriptions.

Naming Consistency4/5

There is a coherent, if dual, convention: elliot_* for audit-state tools and verify_* for external-evidence checks, each with readable verb/noun content. Minor deviations exist (elliot_audit_start / elliot_audit_finish use noun_verb ordering while elliot_record_claim, elliot_record_behavior, elliot_record_review use verb_noun).

Tool Count4/5

14 tools sits comfortably in the reasonable band, and each verify_* variant covers a genuinely different evidence source rather than padding. The only mild bloat is the three report/output tools, though each serves a different consumer (user summary, reviewer process, on-disk artifacts).

Completeness4/5

The audit lifecycle is well covered end to end: snapshot, audit_start, claim/behavior recording, review_packet, record_review, audit_finish, report. Gaps are minor: no tool to list or inspect individual previously recorded claims/receipts, and no ability to retract an already-accepted claim, plus verification is limited to GitHub and npm among external registries.

Maintenance

ActivityMaintained
ResponsivenessUnresponsive

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables AI coding agents to enforce spec-driven development and verify code before it is marked done, using six tools that catch invented APIs, scan for hallucinated content, check plugin conformance, sandbox-run tests, validate schemas, and record audit evidence.
    771 npm
    8
    PolyForm Noncommercial 1.0.0
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables coding agents to run deterministic completeness audits on code before declaring 'done'—catching unwired modules, placebo tests, swallowed errors, and stubs via AST analysis, entirely offline and free.
    MIT