# Countersign

**The approval layer between AI agents and your PayPal account.**

**[Watch the demo (under 3 minutes)](https://youtu.be/FO_vsHLNWDA)**

AI agents can now refund customers, pay vendors and send invoices through PayPal's Agent Toolkit and MCP server. But an agent believes what it reads: one planted sentence in a customer email is enough to make it pay the wrong person. So most merchants keep agents away from the account that matters.

Countersign sits between the agent and PayPal. The agent calls the same PayPal tools it always did, but it holds a Countersign token instead of PayPal credentials, and every call that could move money gets a second signature before it runs:

1. **Ground truth from PayPal.** What was actually captured, what is still refundable, who paid. The agent's claims are not trusted.
2. **Deterministic policy.** Rules you write in plain English, compiled once by an AI model into code. Same call, same facts, same answer.
3. **An independent AI auditor.** A separate model call scores whether the action follows from your instructions or from text an outsider planted.
4. **Execute, hold, or block.** Routine work goes straight through. Anything over your limits waits for a one-click human approval and then runs byte-for-byte as approved. Everything is written to a hash-chained ledger.

![Agent desk: a compromised agent's payout is blocked before it reaches PayPal](docs/screens/blocked.png)

Built for the [PayPal AI Hackathon](https://paypalaihackathon.devpost.com/).

---

## Try it in two minutes (no keys needed)

Requires Node.js 22 or newer.

```bash
git clone https://github.com/MaxwellCalkin/countersign.git
cd countersign
npm install
npm run build
npm start
```

Open **http://localhost:8787**.

With no credentials, Countersign runs against a **built-in PayPal simulator**: the real PayPal Agent Toolkit and Countersign's REST client are pointed at a local stand-in for the PayPal API that implements the same endpoints and validation errors. Policy enforcement, approvals, the ledger, the red-team suite and the MCP endpoint all work. Things to try:

- **Agent desk → the "Attack" email → Replay a compromised agent.** An agent that obeys the hidden instruction tries to pay $2,400 to a look-alike vendor address. Watch it get refused.
- **Red team → Run the attack suite.** Eight attacks against the current policy; nothing is executed.
- **Approvals.** Approve or decline whatever is waiting.
- **Ledger.** Every decision, hash-chained. Export it as JSONL.
- **Insights.** A live AG Studio dashboard over every decision: click a bar to cross-filter, switch to Build to rearrange it.

### Run it against the real PayPal sandbox, with a live model

```bash
cp .env.example .env
```

Fill in the PayPal pair and one model key, then restart:

| Variable | Where to get it |
| --- | --- |
| `PAYPAL_CLIENT_ID`, `PAYPAL_CLIENT_SECRET` | [PayPal Developer Dashboard](https://developer.paypal.com/dashboard/applications/sandbox) → Apps & Credentials → Sandbox → your app |
| `GEMINI_API_KEY` (free, no card) | [Google AI Studio](https://aistudio.google.com/apikey) |

No Gemini key? Any one of `GROQ_API_KEY`, `OPENROUTER_API_KEY`, a local [Ollama](https://ollama.com) model or `ANTHROPIC_API_KEY` works too, and `LLM_PROVIDER=claude-code` uses a signed-in Claude Code CLI with no key at all; see [`.env.example`](.env.example). At startup Countersign asks the provider which models it offers, keeps the newest ones that actually answer, and moves to the next if a free daily quota runs out. On Gemini's free tier the Flash-Lite models have the roomiest quota: the demo video was recorded on `gemini-3.5-flash-lite` (set `LLM_MODEL` to pin a model).

Now every refund, payout and invoice is a real call to `api-m.sandbox.paypal.com`, the policy compiler and auditor run on the model you configured, and the built-in desk agent works the inbox for real. On first start Countersign seeds a few sandbox payments (using PayPal's sandbox test card) and one invoice so there is something to refund. **Reset demo data** in the sidebar re-seeds at any time.

Countersign refuses to start in live mode. It is a hackathon project; point it at sandbox only.

---

## Connect your own agent

Countersign is an MCP server that exposes the same tool names and schemas as PayPal's own, so switching an agent over is a one-line change. The **Connect an agent** page shows a ready-to-paste command with your agent's token.

```bash
claude mcp add --transport http countersign http://localhost:8787/mcp \
  --header "Authorization: Bearer <agent token from the Connect page>"
```

Then ask the agent to do something with PayPal. A call that policy holds returns:

```
HELD for human approval (apr_…). This action has NOT been executed yet. Reason: Refund is over $50
(policy "Refund limits"). A person will approve or decline it in Countersign; if approved it runs
exactly as you submitted it. Do not retry it or split it into smaller actions. …
```

Two extra tools help agents behave well: `countersign_policy_summary` (read the rules before acting) and `countersign_check_approval` (see whether a held action was approved).

---

## How it works

```
 any agent ──MCP──▶ ┌──────────────────────── Countersign ────────────────────────┐ ──▶ PayPal
 (Claude, Cursor,   │ classify   facts from   deterministic   independent   execute │     sandbox
  your own code)    │ the tool ─▶ PayPal    ─▶ policy       ─▶ AI auditor ─▶ hold   │
                    │                                                        block  │
                    │                     hash-chained ledger                       │
                    └───────────────────────────────────────────────────────────────┘
                                   ▲ approve / decline            ▲ plain-English policies
                                   human in the console           compiled by an AI model
```

**Every tool has a risk class.** All 47 tools in the PayPal Agent Toolkit plus 5 native ones are classified as read, create, charge, money-out or destructive ([`src/paypal/tools.ts`](src/paypal/tools.ts)). A tool Countersign has not classified is treated as destructive, so a toolkit upgrade can never silently add an unguarded way to move money.

**Facts come from PayPal, not from the agent.** Before deciding on a refund, Countersign fetches the capture and its order from PayPal: the captured amount, what has already been refunded, the currency, the payer ([`src/guard/intent.ts`](src/guard/intent.ts)). Two hard limits follow directly and cannot be overridden by any policy: a refund can never exceed what is still refundable, and it must be in the payment's currency.

**Policy is deterministic.** Rules are data: a scope (tools or risk classes), conditions over facts (`amount`, `recipient`, `out_24h_after`, `same_target_count_24h`, `risk`, …) and an effect. The strictest matching rule wins; if nothing matches, money-out and destructive actions need approval ([`src/guard/policy.ts`](src/guard/policy.ts)). No model is in this path.

**AI is used where judgment is needed, and nowhere else.**

| Where | What the model does | Why a model |
| --- | --- | --- |
| Policy compiler ([`compiler.ts`](src/guard/compiler.ts)) | Turns "refunds over $200 always need my approval" into rules, with notes on anything it could not express. The draft is replayed against past decisions before you activate it. | Merchants should not have to learn a rule language. |
| Auditor ([`auditor.ts`](src/guard/auditor.ts)) | Given the operator's task, the untrusted content the agent read, the agent's stated reason, the proposed call and the verified facts, returns a 0-100 risk score, a verdict and the quote that drove it. | Telling a customer's request apart from an instruction planted for the agent is a judgment call. |
| Desk agent ([`desk.ts`](src/agent/desk.ts)) | A tool-use loop that works the demo inbox through the gateway. | So the whole loop can be watched end to end. |

The auditor's score is just another fact (`risk`) that the deterministic engine evaluates, so the model can raise a flag but can never approve anything by itself. If the model is unavailable the auditor falls back to a pattern scanner and says so; it never fails open.

**What you approve is what executes.** A held call stores its exact arguments and their SHA-256. Approving re-checks the facts with PayPal (the payment may have been refunded elsewhere in the meantime), verifies the hash, and runs the stored arguments unchanged ([`gateway.ts`](src/guard/gateway.ts)).

**The ledger is tamper-evident.** Each entry's hash covers the previous entry's hash. `GET /api/ledger/export` gives you the chain as JSONL; edit one line and verification fails at that entry ([`store.ts`](src/store.ts)).

**Red-team your own policy.** The suite in [`redteam.ts`](src/guard/redteam.ts) sends vendor impersonation, look-alike recipients, inflated and mis-denominated refunds, structuring under the limit, velocity abuse and destructive actions through the real gateway in dry-run mode: facts are fetched from PayPal for real, the decision is real, nothing is executed.

---

**Insights you can reshape.** The Insights page embeds [AG Studio](https://www.ag-grid.com/studio/) over two live data sources (decisions and the hash chain) with a default two-page dashboard, Countersign's theme, a custom "Countersign tile" widget, and custom cell renderers for outcomes and auditor risk. In Build mode a "Countersign analyst" agent, built with the Studio Agent Framework, delegates to Studio's built-in data, planning, page and widget agents to add or edit widgets from a plain-English request. The model runs on Countersign's server ([`src/insights.ts`](src/insights.ts)); Studio executes the tool calls in the browser, so no API key ever reaches the page. The analyst is the newest and least proven part: a request takes many model turns, and on a small free model each turn can take tens of seconds or stall, so give it a stronger model.

![Insights: an AG Studio dashboard over every gateway decision](docs/screens/insights.png)

---

## PayPal platform used

| PayPal product | How Countersign uses it |
| --- | --- |
| **Agent Toolkit** (`@paypal/agent-toolkit` 1.11) | All 47 tools are wrapped in-process: their schemas are re-exposed to agents and their implementations do the actual PayPal calls. |
| **MCP** | Countersign is a drop-in MCP server (stateless Streamable HTTP) with PayPal's tool names. |
| **Orders v2** | Order lookups, sandbox seeding with authorize and capture intents, payer and refund history for fact checks. |
| **Payments v2** | Capture lookups, refunds, authorization capture and void. |
| **Payouts** | `create_payout` / `get_payout`, added on the REST API because the toolkit does not cover Payouts. |
| **Invoicing v2** | Create, send, remind, cancel, record payment. |
| **Disputes** | List, show, accept claim (always held for a human by the starter policy). |
| **Transaction Search, Subscriptions, Catalog, Shipment tracking** | Available to agents through the wrapped toolkit, with risk classes applied. |

While wrapping the toolkit we found that `accept_dispute_claim` in 1.11.0 validates its free-text `note` with the dispute-ID pattern, so any ordinary note is rejected. Countersign validates that tool with the intended schema instead ([`SCHEMA_FIXES`](src/paypal/tools.ts)).

## AI used

- **A provider-neutral model layer** ([`src/llm.ts`](src/llm.ts)). Countersign needs two things from a model: a schema-constrained answer (policy compiler, auditor) and a tool-use turn (desk agent, Insights analyst). Both work with Google Gemini, Groq, OpenRouter, a local Ollama model or any other OpenAI-compatible endpoint, with Claude through the official Anthropic SDK, and with a signed-in Claude Code CLI. Everything in this README, including the demo video, runs on Gemini's free tier.
- **Model Context Protocol** TypeScript SDK for the agent-facing endpoint.
- **AG Studio Agent Framework** for the Insights analyst, with the configured model behind a client-tool runner.

---

## Project layout

```
src/
  server.ts            entry point: API, MCP endpoint, console, simulator
  guard/
    gateway.ts         the decision pipeline and approvals
    intent.ts          tool call -> "what would this do to money?" + facts from PayPal
    policy.ts          deterministic rule engine and the starter policies
    compiler.ts        plain English -> rules (structured output)
    auditor.ts         independent risk audit (model) + pattern-scanner fallback
    redteam.ts         attack suite
  paypal/
    tools.ts           PayPal Agent Toolkit wrapper, risk classes, native REST tools
    http.ts            minimal PayPal REST client
    simulator.ts       local stand-in for the PayPal API
  agent/desk.ts        the built-in desk agent
  llm.ts               one interface over Anthropic and OpenAI-compatible providers
  mcp.ts               MCP server
  store.ts             state + hash-chained ledger
  insights.ts          model relay for the AG Studio analyst agent
web/                   React console (Vite, AG Grid, AG Studio)
test/                  end-to-end tests
video/                 the script that records the demo video from the running app
```

## Screens

| | |
| --- | --- |
| ![Approvals](docs/screens/approvals.png) | ![Red team](docs/screens/redteam.png) |
| ![Policies](docs/screens/policies.png) | ![Ledger](docs/screens/ledger.png) |

## Tests

```bash
npm test
```

The suite boots the whole server and drives the real PayPal Agent Toolkit against the simulator over HTTP: straight-through refunds, hard limits, hold-then-approve, tampered approvals, payout allowlists, velocity rules, ledger tampering, the red-team suite, a real MCP client handshake, the Insights agent relay, and a full desk-agent run on an OpenAI-compatible model stand-in.

## Deploy

[![Deploy to Render](https://render.com/images/deploy-to-render-button.svg)](https://render.com/deploy?repo=https://github.com/MaxwellCalkin/countersign)

AG Studio runs unlicensed in trial mode; to apply a licence key, set `VITE_AG_STUDIO_LICENSE` at build time (for local builds, in `web/.env.local`).

The included [`render.yaml`](render.yaml) asks for the PayPal pair and a model key and generates a `COUNTERSIGN_ADMIN_TOKEN`; when that is set, approving, editing policy and resetting require the token, so a public deployment is read-only to visitors.

## Limits and what is next

- Sandbox and simulator only, single merchant, file-backed storage. A production version needs a real database, per-user auth on the console, and PayPal webhooks to reconcile ledger entries with settled transactions.
- The auditor sees the full conversation context only for the built-in desk agent. External MCP agents supply a `countersign_reason`; richer provenance (which document triggered which call) would need agent-side support.
- Approvals are in the console. Push or chat approvals are an obvious next step.
- Hold-then-verify settlement: authorize a purchase, capture only once delivery is confirmed.

## License

[MIT](LICENSE)