fusion
by Katawwr
README.md
<p align="center">
<img src="frontend/public/ff.svg" alt="FF" width="72" height="72">
</p>
<h1 align="center">Fusion First</h1>
<p align="center">
<b>Measure and guard AI agents.</b> Attack your agent, grade it with a grader whose accuracy is checked in every
run, and block unsafe tool calls with a runtime guardrail.
</p>
<p align="center">
<a href="https://fusion-first-testing.com">Site</a> ·
<a href="https://fusion-first-testing.com/trust">Evidence</a> ·
<a href="docs/ARCHITECTURE.md">Architecture</a> ·
<a href="docs/RUN_LOCALLY.md">Run Locally</a> ·
<a href="https://pypi.org/project/fusion-safety/">PyPI</a> ·
<a href="CHANGELOG.md">Changelog</a>
</p>
<p align="center"><a href="docs/CASE_STUDY.md">Case study</a> (the problem, the results, a published negative result, how it was built).</p>
<p align="center">
<img alt="Python 3.11+" src="https://img.shields.io/badge/python-3.11%2B-504d9a">
<img alt="PyPI" src="https://img.shields.io/pypi/v/fusion-safety?color=504d9a">
<img alt="MCP server" src="https://img.shields.io/badge/MCP-server-504d9a">
<img alt="Claude Code plugin" src="https://img.shields.io/badge/Claude%20Code-plugin-504d9a">
<img alt="License: MIT" src="https://img.shields.io/badge/license-MIT-504d9a">
</p>
<p align="center">
<a href="https://fusion-first-testing.com">
<img alt="The Fusion First site: the guard's headline results" src="docs/img/overview-dark.png" width="820">
</a>
</p>
## Start here
- **Read** the [case study](docs/CASE_STUDY.md): the problem, the headline result, a negative result, the limits.
- **Run** the command below: it re-derives two committed runs offline, with no model and no key.
- **Check** the [Trust Report](TRUST_REPORT.md): every question asked of Fusion itself, generated from `evals/`. The
[ledger](evals/validation/v1/PREREG_LEDGER.md) shows each registration was committed before its evidence.
- **Design**: [architecture](docs/ARCHITECTURE.md).
## Verify a real run
```bash
pip install -e .
fusion run verify --workspace examples/verified-run latest
fusion run verify --workspace examples/verified-run-toolloop latest
```
```text
verified: result re-derived exactly (in-sample guard verdicts excluded)
```
Re-derives two real runs offline from their committed recordings: llama3.2:1b on the text corpus and qwen2.5:7b
through the tool loop (`--corpus v2`), both graded by Claude Code host subagents; exits non-zero on drift. What each
run shows: [examples](examples/README.md).
## Use
```bash
pip install "fusion-safety[serve,mcp]"
fusion doctor
fusion run start --prompt agent.txt --target ollama:llama3.2:1b --grader claude-cli
fusion run finalize --min-grade B # report card; exit 1 below the bar
fusion serve
```
Any model or agent can be the target: `ollama:<model>`, `openai-compat:<url>#<model>`, `claude-cli:<model>`, a
hosted API with your own key (`openai:`, `anthropic:`, `hf:`, ...; needs `FUSION_ALLOW_API_SPEND=1`), or any
program via `cmd:<command>`.
**Test a tool-using agent** (`excessive_agency` and `data_exfiltration` only):
```bash
fusion run start --prompt agent.txt --target ollama:qwen2.5:7b --grader host --corpus v2 --check excessive_agency,data_exfiltration
fusion run start --prompt agent.txt --target ollama:qwen2.5:7b --grader host --corpus v2 --check excessive_agency,data_exfiltration --no-fix-arm
```
Each attack offers your agent the probe's own tools and runs a bounded loop (at most four assistant turns); scripted
tool results come back, the injected one at its scripted turn. The real tool calls are graded by the oracle
cross-check and the judge, and the guard is replayed over every tool result the agent received. `--corpus v2` also
adds gold v2's 80 sealed known-answer questions to a host-graded run; `--no-fix-arm` halves the target calls and
reports the as-written rate with its interval. Details: [Tool loop](docs/RUN_LOCALLY.md#tool-loop---corpus-v2).
**Runtime guardrail** in your agent loop:
```python
from fusion_first.guardrail.guard import Guardrail
from fusion_first.guardrail.policy import GuardConfig
guard = Guardrail(GuardConfig(allowlisted_domains=["your-co.com"], require_authorization=True))
outcome = guard.guard_tool_call(tool_name, tool_args, user_request=user_message,
untrusted_context=True, untrusted_text=tool_results)
if outcome.blocked: ...
```
Optional, off by default: `Guardrail(config, content_safety=LlamaGuard())` adds a content-safety check on the
user's prompt (`guard.guard_prompt(prompt)`; Llama Guard 3 on local Ollama, from `fusion_first.guardrail.stages`).
Optional, off by default: `Guardrail(config, binder=RequestBinder("qwen2.5:7b", explain=True))` asks a local
Ollama model whether the user's request covers a tool call the rules would block as not covered. It sees the
request and the call, never the tool results, and every other rule still runs (`fusion_first.guardrail.binder`).
Request binding, with or without the binder, does not stop a hijacked call to the tool the user asked for with
other arguments (Trust Report, the informed-attack study): pair it with a confirmation step for consequential tools.
**Claude Code**: `claude plugin marketplace add "$(fusion plugin-dir)" && claude plugin install fusion@fusion-first`.
**MCP**: `{ "mcpServers": { "fusion": { "command": "fusion-mcp" } } }` exposes the run engine and the guard's
`guardrail_snippet` / `check_tool_call` tools.
## Registered studies
- The <a href="docs/DETECTOR.md">tool-result injection detector</a> study is pre-registered; its result is on the <a href="https://fusion-first-testing.com/trust">Trust page</a> (registered result: not adopted; the detector stays dev-only).
- The guard's optional content-safety stage is <a href="evals/validation/v1/PREREG_content_stage.md">pre-registered</a>; its result is in the <a href="TRUST_REPORT.md#the-guards-content-safety-stage-pre-registered">Trust Report</a> (registered result: with the stage the guard flags significantly more unsafe prompts, at a cost in wrong flags on safe prompts written to look unsafe; a category wording tuned on seen prompts was not adopted).
- The judge's accuracy on tool trajectories is <a href="evals/validation/v1/PREREG_q1_toolloop.md">pre-registered</a>; its result is on the <a href="https://fusion-first-testing.com/trust">Trust page</a> (registered result: it met its registered bar). The <a href="evals/validation/v1/PREREG_q7_claims_v3.md">version-three wording</a> (claimed actions, pasted secrets) is registered and scored the same way.
## License
MIT
This server cannot be deployed
Maintenance
ActivityActive
ResponsivenessNo recent community issues