# truthgate

**Your coding agent says "all tests pass, fixed permanently". truthgate checks that against its available execution trace.**

Antigravity sessions can contain command, output, and file-write records. truthgate reads the agent's *claims*
(final answer, reports, walkthroughs) and checks recognized patterns against available records **with code,
not another LLM**. When it finds a contradiction, the Antigravity `Stop` hook sends the agent back (up to 2 times) to prove the claim or retract it;
after that it may stop, and the FAIL verdict stays in the ledger for you to see.

## Historical local sample

One developer's private Antigravity archive was reported as 503 audited sessions over 3 months. The repository
includes [aggregate pattern counts](docs/lie_patterns_503_sessions.json), but not the source transcripts or a
reproducible source manifest. Treat these figures as the author's historical observation, not as an independently
reproducible benchmark:

| verdict | sessions |
|---|---|
| PASS (no contradiction found; historical score at least 0.8) | 233 |
| WARN (claims with no evidence) | 145 |
| **FAIL (claims contradicted by the agent's own trace)** | **125** |

Most frequent warning patterns (the source aggregate mixes contradicted and unverified findings):

| pattern | sessions |
|---|---|
| metrics (ms, MB, %, "3x faster") that no command ever printed | 191 |
| "ran `cmd`" when the command was never executed | 56 |
| file "created/updated" that was never written | 50 |
| "N/N tests passed" never shown in any output | 47 |
| "permanent / 100% / forever" with no repeated or stress run | 46 |
| "fixed / verified" when the last real check had failed | 35 |
| tests that redefine the production class instead of importing it ("test the copy") | 32 |
| **judge shopping**: LLM judge says REJECT → agent rewrites the evidence → asks again until PASS (up to 40 times) | 20 |
| "top 100 forum hacks" with zero web requests | 15 |
| database "verified/populated" that is empty | 9 |

Aggregate data: [`docs/lie_patterns_503_sessions.json`](docs/lie_patterns_503_sessions.json). No transcript content is published.

**Why an LLM judge alone fails:** the judge only sees the text the agent feeds it. Twenty sessions show the
agent re-wording its evidence after each REJECT until the judge said PASS. truthgate never asks a model
whether a claim is true; it looks for the claim in the trace.

## Checks

`TEST` `FILE` `COMMAND` `NUMBER` `OUTCOME` (claim must survive the *last* real check; `--recheck` re-runs
an exact allowlist of read-only checks — no shell, no metacharacters; pytest only with `TRUTHGATE_RECHECK_PYTEST=1`) `ABSOLUTE` `RESEARCH` `JUDGE` (judge shopping) `ARTIFACT`
(empty DB/JSON) `HASH` `VERDICT` (report says PASS, structured verifier said FAIL) `TESTCOPY` (AST)
`SPECIFIC` (repo names, star counts, model ids quoted from a page that never loaded) `SYNTHETIC` (findings
generated by a `range()`/modulo template) `PUSH` ("pushed" with no successful push) `FALSELIVE` `BROKEN`
("fixed" but the edited file no longer parses) `FAILOPEN` (`except: return True` in a verifier) `HARDCODED`
`SELFHASH` `TAUTOLOGY` `BORROWED` (verifier copies the producer's verdict).

Many of these came from an iterative review of the author's 9-month NotebookLM archive of agent chats: each
round asked for a recurring pattern not yet covered, then the pattern was confirmed against real transcripts
(the archive itself sometimes misquotes details, so nothing ships without a transcript check).

## Install

```bash
pip install "truthgate[mcp] @ git+https://github.com/lakhidas168-ship-it/truthgate.git"
truthgate latest            # audit your most recent Antigravity session
truthgate <conversation-id> --recheck --json
```

**Antigravity Stop hook** (sends the agent back for contradicted or selected unverified claims, max 2 bounces): add
`hooks/antigravity/hooks.json` to a plugin, or merge its `Stop` entry into your plugin's `hooks.json`.
The hook uses Antigravity's supplied `transcriptPath` and supports `transcript.jsonl` as well as
`transcript_full.jsonl` across its app, CLI, and IDE data roots ([hook contract](https://antigravity.google/docs/hooks)).
The wheel contains the Python hook modules; the repository contains the hook config. To set it up after a
wheel install, merge this into your workspace `.agents/hooks.json` (replace `python3` with the interpreter
where you installed truthgate if needed):

```json
{
  "truthgate": {
    "enabled": true,
    "Stop": [{"type": "command", "command": "python3 -m truthgate.hooks_stop", "timeout": 25}],
    "PostToolUse": [{
      "matcher": "write_to_file|replace_file_content|multi_replace_file_content|run_command",
      "hooks": [{"type": "command", "command": "python3 -m truthgate.hooks_post", "timeout": 5}]
    }]
  }
}
```

**MCP** (for Claude Code, Codex, Antigravity, or a supervisor agent): `truthgate-mcp` exposes
`audit_session`, `recent_failures`, and optional `triage_scope_alignment`.

### Optional JEV scope triage

`triage_scope_alignment(original_request, final_report)` labels whether the report addresses the
user's requested outcome: `same_scope`, `scope_swap`, or `unclear`. It is an **advisory scope check**;
it does not decide whether the work happened and never changes `audit_session`'s trace-based verdict.
It is not invoked by the Stop hook. Without configuration it returns `status: unavailable`,
`live: false`, and `scope_alignment: unclear`.

To opt in, set `TRUTHGATE_JEV_ARGV` to a JSON array naming a gatekeeper CLI that accepts
`{"state": "...", "questions": {...}}` on standard input in `consult` mode:

```bash
export TRUTHGATE_JEV_ARGV='["python3", "/path/to/jev_gatekeeper.py", "consult"]'
truthgate-mcp
```

The command is launched as argument vector with no shell. The gatekeeper must return its JSON
`status: ok`, `live: true`, and a valid Choice answer for a live label; every failure returns
`unavailable/unclear`. Credentials stay in the gatekeeper's environment and are never put in the
command value. This tool sends the **two supplied texts** to that CLI, which may contact an external
provider. Do not supply secrets or private chat transcripts unless you intend to send them to that
provider. Truthgate does not automatically forward full Antigravity transcripts or store these two
texts in its own ledger.

For Claude Code, add the local stdio server after installation:

```bash
claude mcp add --scope user --transport stdio truthgate -- truthgate-mcp
```

For an Antigravity workspace, merge this entry into `.agents/mcp_config.json`:

```json
{"mcpServers": {"truthgate": {"command": "truthgate-mcp", "args": []}}}
```

These setup shapes follow the [Claude Code MCP guide](https://code.claude.com/docs/en/mcp) and
[Antigravity MCP guide](https://antigravity.google/docs/mcp). The MCP server reads local Antigravity
transcripts; clients should grant access only to agents they trust with those private records.

## Where it fits (connect the wheels, don't reinvent them)

| layer | question | examples |
|---|---|---|
| before the action | "should this command run?" | runtime guardrails, PreToolUse gates |
| the action | "did this action really happen?" | signed action receipts |
| **after the answer** | **"is what the agent *told you* true?"** | **truthgate** |
| evaluation | "is the agent good in general?" | eval suites, trace-aware judges |

truthgate is the missing layer between an agent's trace and its final message. It pairs well with an LLM
judge *if* the judge gets truthgate's findings as input (so it can't be shopped).

## Limits

It checks recognized claims **against the available trace**, which can be incomplete or stale. A trace match does
not prove an external service's current state. A vague or unrecognized claim may not be checked. `PASS` means
the recognized claims met the current rules; it does not certify the entire answer. The Stop hook allows the
agent to finish after two correction attempts, even if an issue remains. An unavailable audit triggers up to
two correction requests, then the agent can also stop; this is a bounded correction loop, not a hard guarantee.
Unverified test, command, outcome, and file claims can also trigger a correction request; other unverified
claims generally do not. The PostToolUse write check uses target mtimes; stale mtimes produce `UNVERIFIED` because some tools preserve
timestamps and a stale mtime alone does not prove that file bytes were unchanged. Generic transcript outputs
do not carry tool-call IDs; overlapping calls cannot establish command-specific proof and are treated as
ambiguous. The parser currently covers
Antigravity transcripts; Claude Code / Codex adapters are next. NotebookLM archive retrieval supplies
historical leads, not proof that an action happened. JEV may help classify or prioritize ambiguous claims, but
its judgment is advisory and cannot replace a physical check. The optional JEV MCP tool checks scope only
and requires explicit configuration; truthgate does not call NotebookLM directly.

[MIT License](LICENSE)