Triadr - Self-Healing Multi-App Agent & Reliability Engine

Built for the Multi-App AI Agent Hackathon (multiappagenthackathon.com) Connected apps: GitHub (code audit) · Telegram (team approval) · Stripe (escrow payout) Stack: Model Context Protocol · Python 3.11+ · FastAPI · Next.js 14 · Tailwind CSS License: Apache 2.0

Triadr takes one sentence - "Audit PR #42, get team sign-off on Telegram, then release $2,500 from escrow to the contractor" - and carries it across three external apps as a six-step workflow. Every side effect passes through a reliability gate that validates, retries, reroutes, deduplicates and, when a step is genuinely unrecoverable, rolls the whole workflow back. Every decision lands on a SHA-256 hash chain you can verify yourself.

The guarantee: a run ends fully applied or fully reverted. Never half-executed.

Demo video

Watch the 2-minute demo - the whole workflow running against the real GitHub, Telegram and Stripe APIs, including a live approval pressed on a phone, a real payout, a fault storm the gate heals, and a rollback that undoes itself.

Related MCP server: 0nMCP

Live demo

The front end is on Vercel and proxies to a FastAPI control plane on a VPS; the second row is that control plane served directly, as a fallback if the proxy is unavailable.

The public deployment is deliberately credential-free: all three apps run in SIMULATED mode and the UI says so on every card. That is not a limitation of the integration - it is the only mode that works for a shared URL, because a live run posts an approval card to one specific person's Telegram and waits for them to press a button. The live runs against the real GitHub, Telegram and Stripe APIs are shown in the demo video and reproducible with your own credentials in five minutes - see §5.

Team

Solo entry. Repository: https://github.com/mrnetwork0001/Triadr

Submission checklist

Everything the hackathon asks for, and where it is in this repository.

Contents

1. Submission checklist - every requirement and where it lives

2. Demo video - 2 minutes, start here

3. What you will see

4. Prerequisites

5. Try it in five minutes - no accounts needed

6. Guided walkthrough of the dashboard

7. Try it live - real GitHub, Telegram and Stripe

8. Command reference

9. Configuration reference

10. Troubleshooting

11. How it works

12. Evidence

13. Triadr as an MCP server

14. Safety

15. Repository layout

1. What you will see

The fastest way to see it is the 2-minute demo video. Below is the same workflow in writing, and what can go wrong at each step:

Four scenarios exercise the happy path and every failure path - see §6.

Under a fault storm that fails ~75% of calls on first attempt, across 40 runs:

Reproduce it yourself: python3 scripts/campaign.py --runs 40

2. Prerequisites

3. Try it in five minutes - no accounts needed

Every app falls back to a simulator when its credential is absent, and says so in every result, so the whole demo runs offline.

3.1 Clone

git clone https://github.com/mrnetwork0001/Triadr.git
cd Triadr

3.2 Run the agent in your terminal (30 seconds)

python3 main.py --scenario all

You will see four runs and a comparison table:

  scenario   outcome      steps   faults  healed  undone  chain       ms
  clean      APPLIED      6/6     0       0       0       valid      602
  chaos      APPLIED      6/6     15      6       0       valid     1384
  rollback   ROLLED BACK  2/5     4       0       2       valid      612
  rejected   APPLIED      4/6     0       0       0       valid      460

How to read it: chaos absorbed 15 injected faults and still settled the payout once; rollback lost Stripe after the approval card was posted, so it undid the two side effects it had already made (undone 2) and moved no money; rejected skipped the payout by an explicit condition. Every run's audit chain verifies.

Each run also writes .triadr/<run_id>.jsonl and an attestation. Verify one:

python3 main.py --verify .triadr/<run_id>.jsonl

3.3 Run the web app

Two processes: the FastAPI control plane and the Next.js front end.

# one-time setup
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
npm install

# terminal 1 - control plane (API + live event stream)
.venv/bin/uvicorn server:app --port 8000

# terminal 2 - front end
npm run dev            # http://localhost:3000

If port 8000 or 3000 is taken on your machine, pick others and tell the front end where the API is:

.venv/bin/uvicorn server:app --port 8770
TRIADR_API_URL=http://127.0.0.1:8770 npx next dev --port 8771

4. Guided walkthrough of the dashboard

Open http://localhost:3000/dashboard. Left to right, top to bottom:

1. Left rail - the five sections of the console, the three connected apps with a green dot for live or grey for simulated, and whether the control plane is online.

2. Run console - the instruction (pre-filled), a scenario selector, and Run agent. Start with Clean run.

3. Press Run agent. Within a second:

   • KPI strip fills with measured numbers: reliability score, faults absorbed, steps self-healed, and the gate's own overhead in microseconds.

   • Execution tree shows the six steps turning from queued to running to applied. Click any step to open its gate attempts: each physical attempt, which endpoint it hit, the fault it got, and the backoff before the next try.

   • Gate event stream scrolls every decision as it happens: gate.fault, gate.backoff, gate.ok, saga.compensated and so on.

   • Cryptographic audit log seals when the run ends: entry count, Merkle root, and a Chain verified badge that was recomputed from the entry bodies, not stamped.

4. Now run Chaos storm. Watch steps go amber (self-healed) and expand one - you will see TIMEOUT on the primary gateway, RATE_LIMITED on the replica, then success. The payout still settles exactly once.

5. Run Stripe outage. Stripe is forced down after the approval card is posted. Watch the run stop, then the rollback: the Telegram card is retracted and the commit status reset, in reverse order. Result banner: rolled back cleanly, 2/2 side effects reversed, no partial state remains.

6. Run Reviewer rejects. The payout step is skipped by condition, and the receipt with it.

Everything you just watched came from the simulators. To make it real, keep going.

5. Try it live - real GitHub, Telegram and Stripe

Each app switches itself to LIVE the moment its credential is present. You can bring them up one at a time; the readiness check works per app. Full detail, including the minimum scopes, is in docs/LIVE_SETUP.md.

5.1 Credentials (about 15 minutes total)

cp .env.example .env     # then fill in the values below

GITHUB_TOKEN=github_pat_…      TRIADR_REPO=owner/repo       TRIADR_PR=1
TELEGRAM_BOT_TOKEN=123456:AA…  TRIADR_TELEGRAM_CHAT_ID=      # left blank - discovered below
STRIPE_SECRET_KEY=sk_test_…    TRIADR_CONTRACTOR_ACCOUNT=    # left blank - created below
TRIADR_APPROVAL_TIMEOUT=120

5.2 Prove each app, in stages

python3 scripts/live_check.py

Read-only. Prints PASS / FAIL / SKIP per tool with the exact fix for each failure. On the first run it also discovers your Telegram chat id and prints the line to add to .env.

python3 scripts/live_check.py --stripe-setup

Funds the test balance (a $100 test-mode charge) and creates a test payee through Stripe's Accounts v2 API, printing an acct_… id to add to .env and an onboarding link. Open the link and complete the form with test data - use the Test (Non-OAuth) bank and Stripe's test values (000 000 0000, DOB 01/01/1901, SSN 0000). Until this is done the payee's transfers capability is restricted and every transfer is refused.

python3 scripts/live_check.py --write --payout

A real commit status on your PR, a real Telegram message that deletes itself, and a real $1.00 transfer, replayed under the same idempotency key (same transfer id comes back - no double payment), then reversed.

TRIADR_APPROVAL_TIMEOUT=300 python3 scripts/live_check.py --run clean

The whole saga, live. A card lands in your Telegram - press ✓ Approve - and the $25 transfer settles, with the receipt posted under the card. Expected output:

  succeeded    audit          github.audit_pull_request     2641ms
  succeeded    status         github.set_commit_status      1763ms
  succeeded    approval_card  telegram.post_approval_card    933ms
  succeeded    approval       telegram.await_approval      48921ms   ← the human
  succeeded    payout         stripe.release_escrow         1356ms
  succeeded    receipt        telegram.post_message        10717ms

  6/6 steps applied across 3 apps. 0 fault(s) absorbed, 0 step(s) self-healed,
  0 steps left half-executed. Payout tr_… settled for USD 25.00.
  chain valid: True

--run rollback does the same with Stripe forced down: the card you just saw is retracted and the commit status reset, for real.

5.3 The live demo from the dashboard

Restart the control plane after editing .env (it reads the file at startup), reload /dashboard, and the app dots turn green. The run box now names your real repo, chat and payee. Press Run agent, then press Approve on your phone when the card arrives - the command bar tells you when the agent is waiting on you. Everything on screen is a real API call; injected faults (Chaos storm) are labelled [injected] in the event stream.

Each live clean run moves $25 of test money to the test payee. --stripe-setup tops the balance up again whenever it drops below $50.

6. Command reference

# The agent
python3 main.py                        # clean run
python3 main.py --scenario chaos       # clean | chaos | rollback | rejected | all
python3 main.py --instruction "Audit PR #7 in acme/api, sign-off on Telegram, pay acct_1X $50 USD"
python3 main.py --json                 # full result as JSON
python3 main.py --bench                # measured gate overhead, both phases
python3 main.py --verify <log.jsonl>   # re-verify a written audit chain

# Evidence
python3 scripts/campaign.py --runs 40  # regenerates every figure in this README
python3 -m pytest tests/ -q            # 176 tests

# Live readiness
python3 scripts/live_check.py                 # read-only checks per app
python3 scripts/live_check.py --write         # + a write and its undo per app
python3 scripts/live_check.py --approve       # + a real approval card; press a button
python3 scripts/live_check.py --stripe-setup  # fund test balance, create/inspect the payee
python3 scripts/live_check.py --payout        # + $1 transfer, replay, reversal
python3 scripts/live_check.py --run clean     # full saga, live
python3 scripts/live_check.py --run rollback  # full saga with Stripe forced down

# Web app
.venv/bin/uvicorn server:app --port 8000      # control plane (npm run api)
npm run dev                                   # front end (npm run build / npm start for production)
npm run typecheck

7. Configuration reference

All variables live in .env (see .env.example), loaded by every entry point. Existing environment variables always win over the file.

8. Troubleshooting

9. How it works

        "Audit PR #42, get sign-off on Telegram, then pay $2,500 from escrow"
                                      │
                                      ▼
                          ┌───────────────────────┐
                          │  Planner              │  deterministic plan shape,
                          │  agents/planner.py    │  parameters bound from text
                          └───────────┬───────────┘
                                      ▼
                          ┌───────────────────────┐
                          │  Saga orchestrator    │  6 steps, typed dependencies,
                          │  agents/orchestrator  │  every mutation has a rollback
                          └───────────┬───────────┘
                                      ▼
        ╔═════════════════════════════════════════════════════════════╗
        ║   RELIABILITY GATE - risk_gate.py                           ║
        ║   idempotency · schema · drift · rate shaping · breakers    ║
        ║   backoff+jitter · endpoint failover · compensation         ║
        ╚══════════┬═══════════════════┬══════════════════┬═══════════╝
                   ▼                   ▼                  ▼
            ┌────────────┐      ┌────────────┐    ┌────────────┐
            │  GitHub    │      │  Telegram  │    │  Stripe    │   MCP servers,
            │  6 tools   │      │  4 tools   │    │  4 tools   │   14 tools total
            └────────────┘      └────────────┘    └────────────┘
                   │                   │                  │
                   └───────────────────┼──────────────────┘
                                       ▼
                          ┌───────────────────────┐
                          │  SHA-256 hash chain   │  tamper-evident evaluation log
                          │  reliability_logger   │  + merkle root + HMAC signature
                          └───────────────────────┘

The gate, in execution order

Every call to any of the three apps goes through ReliabilityGate.guard():

1. Idempotency ledger - a replayed key returns the recorded result. A retried payout whose response was lost in flight cannot pay twice.

2. Input schema validation - zero-LLM, ~1.4 µs. A malformed payload is blocked before the executor runs, so a bad call has no side effect at all.

3. Token bucket - client-side rate shaping sized below each vendor's documented ceiling, so Triadr sheds load before the vendor does.

4. Circuit breakers - per endpoint, CLOSED → OPEN → HALF_OPEN. An open endpoint is never selected and costs no retry budget. If the whole fleet is open, one probe is still admitted rather than failing a workflow a single call would have saved.

5. Execution with typed faults - every failure is classified (RATE_LIMITED, TIMEOUT, AUTH_EXPIRED, PERMISSION_DENIED, …). Retryable faults back off exponentially with full jitter and honour Retry-After; terminal faults like a 403 are never retried.

6. Endpoint failover - attempts cycle across health-ranked gateways, not the same dead host.

7. Response contract checking - a declared outputSchema catches drift on the very first call; a learned per-tool fingerprint catches it thereafter. A vendor renaming decision to decision_v2 is caught and rerouted instead of silently corrupting the payout decision.

8. Compensation - if a critical step is unrecoverable, completed side effects are undone in reverse order, with a larger retry budget than the forward path.

The audit log

digest(n) = SHA256( digest(n-1) ‖ canonical_json(entry(n)) )

Editing, reordering or deleting any entry breaks verification at exactly the corrupted index. The attestation carries a Merkle root over all entries and, with TRIADR_LOG_KEY set, an HMAC signature. run.end is recorded before the seal, so the commitment covers the complete log.

10. Evidence

This is the section that answers "how did you test that it works".

Three independent kinds of evidence, all reproducible from this repository:

1. A test suite - 176 tests, run with python3 -m pytest tests/ -q.

2. A fault campaign - 40 full workflow runs under a storm that fails ~75% of calls on first attempt, run with python3 scripts/campaign.py --runs 40. Result: 356 faults absorbed, 0 runs left half-executed, 0 duplicate payouts, 40/40 audit chains verify.

3. A verifiable audit chain - every run writes a hash-chained log you can recompute with python3 main.py --verify .triadr/<run_id>.jsonl, so the numbers above are checkable rather than asserted.

Plus a live readiness check against the real APIs (python3 scripts/live_check.py), which is how the video's live run was proven end to end.

Measured on a laptop (python3 main.py --bench), never hardcoded:

The landing page re-measures this on every load when the control plane is up, and the dashboard shows the percentiles from your own run.

python3 -m pytest tests/ -q     # 176 tests

Covering schema validation edge cases, breaker transitions, backoff bounds, endpoint routing, idempotency, drift detection, chaos determinism, the MCP JSON-RPC surface, the LIVE request wiring for all 14 tools (method, URL, encoding, auth and idempotency headers, verified against a recorder), hash-chain tamper evidence, and an 8-seed property test asserting the core invariant: every run ends fully applied or fully reverted. The suite is hermetic: a developer's .env can never turn a unit test live.

The full write-up for judges is docs/RELIABILITY_BRIEF.md.

11. Triadr as an MCP server

The 14 tools are exposed over the standard MCP stdio transport, dependency-free - and every call an MCP host makes is gate-supervised, so the host inherits retry, rerouting, idempotency and rollback for free.

claude mcp add triadr -- python3 /path/to/Triadr/mcp_servers/stdio_server.py

Every tool that mutates remote state declares the tool that undoes it - enforced by a test.

12. Safety

• Stripe refuses to move money on an sk_live_ key unless TRIADR_ALLOW_LIVE_MONEY=1 is explicitly set.

• The planner's shape is fixed in code. An LLM may only extract parameters, never invent a step - a hallucinated extra release_escrow would be a financial incident.

• Condition evaluation is a deliberately tiny parser, not eval.

• Every payout carries an idempotency key derived from the workload, not the attempt.

• Every app reports LIVE or SIMULATED in results, dashboard and audit log; injected faults are labelled [injected].

• Secrets never reach a log: the Telegram bot token is redacted from every fault, and .env / .triadr/ are git-ignored.

13. Repository layout

License

Apache 2.0