MARL Cop & Thief - Dual AI Agents over MCP

A decentralized, partially-observable pursuit game between two autonomous AI agents - the Cop and the Thief - that converse in free natural language over MCP servers, decide moves with a game-theoretic minimax + self-play RL engine, render live in a web control panel, and email a mutually-agreed JSON match report via the Gmail API.

University of Haifa · Orchestration of AI Agents (ex06) · Dr. Yoram Segal. One command launches everything; one browser tab runs a whole match.

Highlights

• One-command node - python -m cop_thief.app boots both MCP servers, the public Cloudflare tunnels, and a browser control panel together (no orphan tunnels, no port juggling).

• Web control panel - live node status, copy-ready public URLs/tokens, an opponent challenge form, a one-click mirror self-test, and a live 5×5 game TV - all at http://127.0.0.1:8800.

• Real strategy - an Angel–Devil minimax engine (zero-sum Markov game, alpha-beta) with the Conway blocking game (Cop = Devil walls §4.3) and self-play RL weight learning, well beyond the assignment's baseline tabular Q. See docs/STRATEGY.md.

• Decentralized & cheat-resistant - no referee; both sides hash the result (SHA-256) and any disagreement scores 0/0. Inbound prose is treated as hostile: prompt-injection / coercion is screened, logged as evidence, and cannot change the outcome (no forfeit action exists).

• Single SDK boundary + API Gatekeeper - all logic behind CopThiefSDK; every external call (LLM, Gmail) funneled through a FIFO-backpressured gatekeeper with DeepSeek→Anthropic failover.

• Quality gates - pytest ≥ 85 % coverage, zero-violation ruff, ≤ 150 lines/file, uv-only.

Related MCP server: Police MCP Server

Quick start

uv sync                                   # install (uv is the ONLY package manager)
cp .env-example .env                      # fill in real values (see "Secrets")
uv run ruff check .                       # zero-violation lint gate
uv run pytest                             # full suite (>=85% coverage gate)
uv run python -m cop_thief.app            # launch the control panel + servers + tunnels

Then open http://127.0.0.1:8800.

Playing a match (control panel)

1. uv run python -m cop_thief.app → the panel opens; wait for Servers ● and Tunnels ● green.

2. The status card shows your two public …/mcp/ URLs + per-role tokens (copy buttons). Send these to your opponent along with docs/INTER_GROUP_TREATY_SPEC.md. Use the fill-in forms in match_setup/ (RULES.txt, OUR_DETAILS, OPPONENT_DETAILS).

3. Paste the opponent's two …/mcp/ URLs (and tokens, if any) into the challenge form, then START CHALLENGE - the 6 sub-games play cross-host on the TV and the report is emailed.

4. No partner yet? Click MIRROR SELF-TEST ⟳ - it fills your own localhost endpoints + tokens and plays you against yourself (ideal for strategy testing).

A game = 6 sub-games (per §4.1): we play Cop in 3 (home leg) and Thief in 3 (away leg), Thief-first, ≤ 25 moves each. Scoring is immutable: Cop capture → 20 / 5; Thief survives → 5 / 10.

Screenshots

The control panel - node status (Servers/Tunnels/Game), our shareable …/mcp/ URLs + tokens with copy buttons, the opponent challenge form, and the live game. Here a mirror self-test is running; note the [INTENT: BARRIER] line (the Cop walling an adjacent cell and staying put, §4.3), the green B barrier, and the ! capture.

Live board - the 5×5 grid with the Cop C (blue) and Thief T (red), beside the comms-intercept feed showing the natural-language [INTENT: MOVE] transmissions and the HOME LEG / Sub-game separators.

Leg transition - the runner crossing into the AWAY LEG (we play Thief) at sub-game 4/6, with a B barrier and ! capture still on the board.

Booting the node prints the shareable endpoints to the terminal (one process: servers + tunnels + panel):

Control panel  >  http://127.0.0.1:8800   (open in a browser)
╔══════════════════════════════════════════════════════════════════╗
║ LIVE PUBLIC MATRIX (Team Alpha)                                   ║
╠══════════════════════════════════════════════════════════════════╣
║ COP   (:8001)  https://acting-tomorrow-yard-raid.trycloudflare.com/mcp/   ║
║ THIEF (:8002)  https://dial-mean-courses-tramadol.trycloudflare.com/mcp/  ║
╚══════════════════════════════════════════════════════════════════╝
Tunnels live and written to config/setup.json. Share these /mcp/ URLs. Ctrl+C to stop.

Architecture

Entry points

Strategy in one paragraph

Each request_move is answered by depth-limited alpha-beta minimax over the zero-sum Markov game (Cop maximizes, Thief minimizes, optimal-adversary assumption). Progress-shaped terminal scores (±WIN ∓ turns) make the policy press for capture/survival, so draws are structurally avoided. The Cop's action set includes walling an adjacent cell (Conway "Devil" move, §4.3); the evaluation's containment feature is the Thief's flood-filled escape region, so the planner discovers legal herding-to-trap lines on its own. The linear evaluation weights are tunable by self-play TD (selfplay.train_weights). Three variant profiles (aggressive / balanced / defensive) field the required 3-agent roster. Full design: docs/STRATEGY.md.

Formal model - Dec-POMDP

The pursuit is modelled as a Decentralized, Partially-Observable Markov Decision Process, the tuple ⟨ n, S, {Aᵢ}, P, R, {Ωᵢ}, O, γ ⟩ (ex06 §11):

Partial observability is real: each agent decides from its belief of the board (its last observation + parsed opponent prose), never from global ground truth.

Orchestration challenges (the hard part)

Per ex06 §14 the value of the assignment is the orchestration, not the win. The hard problems and how we solve them:

• Free natural language, no predefined protocol. Agents converse in prose. We layer a thin deterministic contract - every message opens with exactly one signpost [INTENT: MOVE|BARRIER|HOLD] followed by a compass word - so the move is machine-resolvable while the body stays free NL. This keeps two independently-built engines in lock-step without a shared codebase.

• Linguistic ambiguity & untrusted input. Inbound prose is parsed deterministically (longest-match direction word, bracket-only intent so flavour text can't spoof it) with an optional LLM parse for unstructured opponent text; low confidence → a safe exploratory fallback (never crash, never forge a capture). Every field is treated as hostile - e.g. a non-numeric variant is coerced, never trusted.

• Ensuring mutual understanding. Both peers share a deterministic move language (encode/parse with no LLM needed), so a game is byte-reproducible. At the end both sides hash the canonical sub_games (SHA-256, K3); any mismatch ⇒ 0/0 for both - agreement is enforced, not assumed.

• Liveness over an unreliable network. Cross-host moves retry with reconnect; a sustained outage or a frozen peer (per-move 20 s timeout) forfeits that sub-game so the series always completes all 6 and the report still emails - a dead opponent can never stall the match.

Visualization & conclusive proofs (§11)

• GUI - the Screenshots above show the live 5×5 board, the [INTENT: …] comms-intercept feed, barriers (B), captures (!), and leg transitions.

• Cloud MCP communication - the boot block above prints the live public Cloudflare /mcp/ URLs, and the panel's comms feed streams the real [INTENT:] transmissions exchanged with the cloud servers; every turn is also appended to data/game_audit.jsonl and sealed into a tamper-evident per-game archive (data/archive/, bundle SHA-256 in the report).

• Learning - strategy weights are tuned by self-play TD (selfplay.train_weights); a tabular Q-learning baseline is retained for comparison. Design + curves: docs/STRATEGY.md.

Security & fair play

• Tokens - every MCP tool call requires a per-role revocable bearer token; exchanged out-of-band, rotatable to revoke. Servers are fail-closed.

• Anti-injection - inbound transmissions are untrusted; injection/coercion/impersonation/forgery are screened, flagged hostile:true in data/game_audit.jsonl, counted in the report, and have no effect on the engine-determined outcome. Codified for opponents in the treaty (§F).

• Mutual agreement - both sides hash the canonical sub_games; any mismatch ⇒ 0/0 (both_lose).

• Reporting - Gmail API over OAuth2 Desktop (scope gmail.modify, zero passwords); the group name rides in both the subject line and the JSON body; a safety guard defaults to the burner inbox.

Token budget & cost

Every external call is metered through the API Gatekeeper → TokenTracker, which streams live usage to data/token_usage.json (atomic write; excluded from the K3 agreement hash so cost never affects the result). All figures are config-driven (config/setup.json → token_budget / economics).

In practice the whole project has cost ≈ $0.01 on LLMs. Moves come from the local minimax engine and the move language is deterministic [INTENT: …] encode/parse - no LLM is needed to play or to send the report (Gmail API, not an LLM). The tiny budget + DeepSeek-first failover are guardrails for optional LLM-assisted natural-language parsing; the ceiling is set at $0.50 with plenty of headroom.

Secrets & configuration

• Copy .env-example → .env and fill: DEEPSEEK_API_KEY, ANTHROPIC_API_KEY, COP_MCP_TOKEN, THIEF_MCP_TOKEN, GMAIL_CREDENTIALS_PATH. A zero-dependency autoloader injects .env at startup - no export/source needed; existing shell exports always win.

• All tunables live in versioned config/*.json (no hardcoding). Secrets (.env, credentials.json, token.json) are git-ignored and never enter source control.

Documentation

PRD · PLAN · TODO · STRATEGY · RULES_AND_AGREEMENTS · INTER_GROUP_TREATY_SPEC

License

MIT.