# Tool of Truth

**Every tool call, proven.**

Verification sentinel for AI agent tool usage. MITM proxy that intercepts all MCP tool calls, detects fabrication, tracks costs, verifies outcomes, and infers user satisfaction.

## Features

| Feature | Status | Description |
|---|---|---|
| MITM Proxy | ✅ | Transparently intercepts all MCP tool calls |
| Fabrication Detection | ✅ | 7-signal deep analysis: trace, doc-similarity, timing, placeholders, contradictions |
| Trust Scoring | ✅ | Weighted 0-100 with fabrication cap |
| Cost Tracking | ✅ | Per-call cost for 8 providers |
| Token Counting | ✅ | Input/output tokens per call |
| Outcome Verification | ✅ | Compares tool result against user's original prompt |
| Skill Adherence | ✅ | Verifies workflow via manifests |
| User Satisfaction | ✅ | Infers satisfaction from follow-up messages |
| Receipts | ✅ | SHA-256 hashed, JSON + human-readable |
| Local Storage | ✅ | JSONL daily files + indexed lookup |
| Tool Transparency | ✅ | Shows what downstream tools actually do |

## Quick Start

### 1. Install

```bash
npm install -g tooloftruth-mcp
```

### 2. Connect (Option A: Direct)

```json
// Your agent's MCP config
{
  "tooloftruth": {
    "command": "tooloftruth-mcp"
  }
}
```

### 3. Connect (Option B: Proxy — MITM mode)

```json
// Your agent's MCP config — proxy a single downstream server
{
  "firecrawl": { "command": "npx", "args": ["tooloftruth-mcp", "firecrawl"] }
}
```

Or proxy every local MCP server via `~/.tooloftruth/proxy.json`:

```json
// ~/.tooloftruth/proxy.json
{
  "servers": {
    "firecrawl": { "command": "npx", "args": ["firecrawl-mcp"] },
    "github": { "command": "npx", "args": ["@modelcontextprotocol/server-github"] }
  }
}
```

### Skill Manifests (enforced)

Manifests in `~/.tooloftruth/manifests/<skill>.json` **gate** tool calls:

```json
{
  "skill": "echo",
  "version": "1.0.0",
  "requires": {
    "echo": { "tool": "echo", "must_be_called": true, "expected_args": { "message": "" } }
  },
  "outputRules": ["Output must contain \"Echo:\""],
  "max_cost": 1.0
}
```

Undeclared tools are blocked (`BLOCKED_BY_MANIFEST`); arg/pattern/cost/output-rule violations are recorded and downgrade the verdict.

### 4. Use

```
You: Did you actually use firecrawl?
Agent: [calls tooloftruth_verify]
Agent: VERIFIED — firecrawl was called at 10:30:02, trust score 98/100, cost $0.03

You: How much did I spend on tools today?
Agent: [calls tooloftruth_cost]
Agent: Total: $0.47 across 12 tool calls

You: Was that result what I asked for?
Agent: [calls tooloftruth_outcome]
Agent: ALIGNED — tool result matches your request

You: That's wrong, try again
Agent: [calls tooloftruth_satisfaction]
Agent: User appears dissatisfied — result may be wrong
```

## CLI

```bash
# Intercept and monitor non-MCP commands
tooloftruth -- gh api repos/user/repo

# One-shot status report (daemon health, receipts, cost, top tools)
tooloftruth status
```

## Daemon

A launchd daemon watches `.tooloftruth/receipts/`, aggregates daily stats, and alerts on fabrications.

```bash
# Install the daemon (macOS)
launchctl load ~/Library/LaunchAgents/com.tooloftruth.daemon.plist
```

Stats are written to `.tooloftruth/stats/YYYY-MM-DD.json`.

## Truth Scan

`tooloftruth_truth_scan` does real web verification of claims: Bing search (curl) → crawl4ai fetch → evidence extraction → verdict.

## Sensitive-Data Detection

The daemon scans every new message and tool call for leaks, writing alerts with exact proof:

- **Secrets**: 24 gitleaks-style rules + entropy — AWS keys, GitHub/OpenAI/Google/Anthropic tokens, JWTs, private keys, DB connection strings, generic API keys, passwords
- **PII**: email, phone, SSN, credit card (Luhn-validated), IP, IBAN
- **Prompt injection**: ignore-previous-instructions, system-prompt reveal, jailbreak (DAN), XML tag injection
- **Dangerous commands**: `rm -rf /`, `curl | sh`, `dd` to disk, base64-decode-to-shell, etc.

Alerts land in `~/.tooloftruth/alerts/` with redacted match, source, confidence, and surrounding context.

**Deep scans**: `tooloftruth_gitleaks <repo>` runs gitleaks (200+ rules) over a repo's git history with exact file:line proof.

## Behavior Ledger

The daemon tracks per-session model, message/tool-call counts, error rate, token usage, and cost → `~/.tooloftruth/ledger/`. Query via `tooloftruth_ledger`.

## MCP Tools

| Tool | Description |
|---|---|
| `tooloftruth_verify` | Verify a specific tool was used |
| `tooloftruth_check` | Pre-flight: is tool installed/configured? |
| `tooloftruth_receipt` | Generate/view verification receipt |
| `tooloftruth_cost` | Cost breakdown by tool |
| `tooloftruth_history` | Search historical receipts |
| `tooloftruth_truth` | Full session truth report |
| `tooloftruth_satisfaction` | Infer user satisfaction from message |
| `tooloftruth_outcome` | Verify result matches user prompt |
| `tooloftruth_audit` | Cross-reference agent claims against actual tool calls |
| `tooloftruth_conversation` | View conversation log |
| `tooloftruth_truth_scan` | Web research + fact-check a claim |
| `tooloftruth_alerts` | List sensitive-data alerts (PII/secrets/injection/commands) |
| `tooloftruth_gitleaks` | Deep-scan a git repo for secrets |
| `tooloftruth_ledger` | Per-session behavior (model, errors, tokens, cost) |

## How the MITM Proxy Works

```
Agent ←→ Tool of Truth ←→ MCP Server (downstream)
              ↓
         Records everything
         Runs deep fabrication checks
         Verifies outcome alignment
         Infers satisfaction
         Generates receipts
         Returns result unchanged
```

## Storage

All data stays in your machine's home directory:

```
~/.tooloftruth/
├── config.json          ← settings
├── index.json           ← fast lookup index
├── proxy.json           ← downstream server config
├── receipts/
│   ├── 2026-08-17.jsonl ← today's calls
│   └── ...
├── conversations/
│   └── 2026-08-17.jsonl ← claim/action conversations
└── stats/               ← daemon aggregations
    ├── 2026-08-17.json
    └── ...

## Skill Manifests

Skills can attach verification manifests:

```json
{
  "skill": "last-30-days",
  "version": "2.1.0",
  "requires": {
    "github_mcp": {
      "tool": "get_repo_stats",
      "mustBeCalled": true
    }
  }
}
```

## Development

```bash
git clone https://github.com/adigoel07/tooloftruth.git
cd tooloftruth
pnpm install
pnpm build
npx vitest run
```

## License

MIT — use it, fork it, build on it.

## Author

Built by [adigoel07](https://github.com/adigoel07).