buddie

Verify the claim. Know the context.

AI agent reliability toolkit

Why buddie?

An agent finishes a task and tells you that CI passed, that a source supports its summary, or that your laptop has CUDA and Python 3.12. It sounds certain, and often it is right. When it is wrong, the mistake hides inside a confident sentence.

buddie separates what can be checked from what is only claimed. It checks supported claims against the evidence itself, gives the agent real observations of your machine instead of guesses, and marks what it could not check as unchecked. In 929 checkable quotations from published deep-research reports, 212 were altered or absent from the page they cited; buddie flagged 207 of them (research). In Claude Code with the plugin, a seeded false quotation reached the reader in 24 % of paragraphs instead of 88 % (BDX-003).

Related MCP server: genpark-agent-hallucination-fact-verifier-skill

What buddie does

Verify. buddie checks supported claims against their evidence: a quotation against the page it links to, a file or commit against your repository, a merged PR or a green CI run against GitHub, a test result against the session's record of the command. No language model takes part in the check, so the agent cannot argue its way to a different verdict.

Catch mismatches. In Claude Code and Codex, buddie's hook checks supported claims at Stop and returns failures with the reason. In Claude Code the answer is already visible at that point; buddie asks for corrected lines instead of another copy of the whole answer. Supported message checks before delivery retain their existing behavior. Other clients call the checks through MCP or the command line and decide what to do with the result.

Observe. Before it suggests a command, an agent can ask buddie for the parts of your machine the task needs: the operating system, CPU, memory, GPU, Python, runtimes, tools. buddie reads only the categories you have granted, changes nothing, and records how each value was observed.

Discover. The compact Claude plugin includes a Discovery skill for evidence triage with the tools available in the current session. It uses real Core verification when connected, performs independent checks otherwise, and marks unavailable checks as not checked. Local hooks and MCP capabilities vary by surface; Cowork, Chat and mobile require separate live verification. Projects/Context remains a future design.

See it in action

Verify an agent's claim

An agent reports that it fixed a date parser and that the test suite is green. It marks the claim with a small anchor, ⚓ run:pytest -q => 41 passed, that buddie can check. The session transcript records what the test command actually printed:

Run it yourself from a clone; the report and transcript are in examples/work:

cd buddie/examples/work
buddie verify report.md --transcript session.jsonl --no-github

BLOCK anchor line 3 ⚓ run:pytest -q => 41 passed: output has no '41 passed'; it ends: ….......................................... 1 failed, 40 passed in 3.1s
note  pr:/ci:/run: anchors show what happened (PR state, checks on a commit, a command and its output), not that the code is correct
buddie FAIL: effects 0/1 hold, 1 broken

The files are a teaching fixture: buddie reads the recorded output rather than rerunning pytest. --json receipt.json also writes a receipt naming the evidence behind the verdict. Quotation, PR and CI examples, and which integration checks what: verification guide.

Know the environment

An agent is asked to set up a local development environment. Instead of guessing the operating system, GPU or Python version, it asks buddie for three categories, which the person has granted:

buddie snapshot --category system.os --category hardware.gpu --category software.python \
  --task "configure local development environment" --json diagnostics.json

buddie snapshot: diagnostics of 3 categories (grants active, policy absent; these govern the categories only)
  system.os: collected
    name: Windows 11  [os_interface]
  hardware.gpu: collected
    adapters: [model NVIDIA GeForce GTX 1650, source nvidia_smi_gpus, vendor NVIDIA, vram_bytes 4294967296]  [fixed_program]
  software.python: collected
    buddie_runtime: implementation cpython, pip True, venv True, version 3.14.6  [buddie]

Abridged from a run of buddie 2.0.0: the lines shown are unedited, and the machine's other observed values are left out. diagnostics.json holds the full document, with the method and time of each value.

Nothing is observed until you grant it (/buddie:init offers it). The template grants the low-sensitivity categories, Docker and Podman clients included:

buddie snapshot --grant-baseline          # writes the grants file, baseline only (never over yours)
buddie snapshot --grant-template          # prints it, to edit by hand
buddie snapshot --grants                  # what is granted now

Sensitive categories (the Docker daemon, disk usage, a project, a repository) are added to it one by one.

A category you have not granted is reported as denied, and its collector never runs. A value buddie cannot read is marked unavailable, with the reason. Diagnostics is read-only and installs nothing; it gives the agent better facts, not a guarantee that its next command is right. Categories, grants and the document.

Get started

buddie needs Python 3.11 or newer and has no dependencies.

Claude Code

The recommended start. The plugin adds the answer check, the MCP tools including environment diagnostics, and run: checks against the session transcript:

/plugin marketplace add tsyyan/buddie
/plugin install buddie@buddie

It installs the latest released plugin (the plugin-stable branch), not unreleased work on main. To allow diagnostics, save the categories you choose as environment.toml in ~/.config/buddie/ (%APPDATA%\buddie\ on Windows). For the demo above:

version = 1
allow = ["system.os", "hardware.gpu", "software.python"]
roots = []

Levels, the intro mode and what the hook reads: Claude Code guide.

Other integrations

• Codex: codex plugin marketplace add tsyyan/buddie, then codex plugin add buddie@buddie, and enable its hook in /hooks. The check runs at Stop; run: claims stay unchecked. Codex guide

• Any MCP client: uvx buddie serve starts the stdio server. There is no gate: the agent calls the tools. Integrations

• Command line and CI: pip install buddie, then buddie verify REPORT.md --json receipt.json. It exits 1 on FAIL; GAPS and EMPTY exit 0, so a CI step that requires PASS should read the receipt's outcome.

• Claude Desktop: the .mcpb bundle from Releases, without a gate. Desktop guide

Every surface in detail: getting started.

Release status

Upgrading from 2.0.5 changes nothing you have to do: 2.0.8 makes the hook's Python commands explicit for Anthropic directory inspection, preserving isolated mode, interpreter fallback and hook behavior. Claude Code's marketplace continues to install releases from plugin-stable.

Upgrading from 2.0.2: 2.0.4 includes canonical Discovery, the README-style dog icon and Toolkit positioning, Linux worker guard hardening, and Stop-hook guidance that asks for corrections without repeating the already displayed answer. No new permissions are required; on Linux a failed parent-death guard now prevents the regex worker from running. Discovery leaves unavailable checks unverified. The 2.0.3 candidate was superseded before publication.

Upgrading from 2.0.1 changes nothing you have to do: 2.0.2 adds the local feedback tools (feedback_start, feedback_resolve, feedback_accept), GitHub citation checks and versioned public release notes. Upgrading from 2.0.0 changes nothing; 2.0.1 fixed how the MCP server reads its input on Windows systems whose code page is not UTF-8. Upgrading from 1.1.1 changes nothing for a 1.x setup. What can differ, and why, is in the changelog.

Documentation

Limitations

buddie is not a truth detector. It checks claims in the forms it recognises, such as quotations with links and anchored claims about files, PRs, CI and commands, against the evidence it can read. Paraphrases and unanchored claims stay unchecked. PASS covers only what was checked, and GAPS and EMPTY are not passes.

The answer check exists only in Claude Code and Codex, and it returns an answer rather than locking it: the agent fixes the claim or discloses it as unverified. An agent running as your user can edit local evidence, settings and the diagnostics grants file; only an administrator's protected policy sets limits it cannot raise.

buddie runs locally, with no model calls and no telemetry. It fetches the pages a report cites and calls GitHub for PR and CI claims. All limitations · Security boundaries · Privacy

Apache-2.0 · Changelog · Citation · Issues