# AgentRoom

**Two coding agents that share a workspace and coordinate finish jobs a single agent abandons.**

[![tests](https://github.com/seonglae/AgentRoom/actions/workflows/ci.yml/badge.svg)](https://github.com/seonglae/AgentRoom/actions/workflows/ci.yml)
[![Article](https://img.shields.io/badge/Article-seongland.com-3d5a80)](https://seongland.com/article/agentroom)
[![HuggingFace](https://img.shields.io/badge/🤗%20Demo-Space-c4553a)](https://huggingface.co/spaces/seonglae/AgentRoom)
[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)

A *room* is a state-management primitive: a runtime object exposing file-level
claim, status, and broadcast as MCP tools, sitting on top of a CRDT-merged
shared filesystem. Agents coordinate through room state rather than through
chat, and the substrate guarantees no edit is lost when they write at once.

```mermaid
flowchart TB
    A["Agent A"] --> R
    B["Agent B"] --> R
    C["Agent C"] --> R
    R["room MCP server<br/>claim · release · broadcast · state · read"] --> W
    W["CRDT-merged shared workspace<br/>character-level merge, no lost writes"]
```

The claim is not that concurrency helps. Concurrency without coordination
*hurts*: a parallel-merge baseline scores below a single agent, because the
second agent's `server.ts` silently overwrites the first's. Coordination is the
load-bearing part.

The uncoordinated concurrent baseline lands below the single agent
(parallel-merge 0.456 vs solo 0.544 on the hardest task); the full room tops
the matched-compute ablation at 0.669, with the substrate-only intermediates
between (0.575 shared, 0.588 with prompt).

The tighter result matters more than the higher one. Run-to-run spread roughly
halves on all three models, because the failure mode being removed is the tail
where a lone agent emits one stub file and exits.

## Try it in ten seconds

No API keys, no agents, no GPU.

```bash
pip install -e .
python -m agentroom.selftest
```

```
replica A after exchange:
  export function serve() {
    app.use(auth)
    app.get('/health', ok)
  }

kept replica A edit : True
kept replica B edit : True
replicas converged  : True

OK  both concurrent edits survived, replicas converged.
```

Two replicas edited the same file without seeing each other. Byte-level
last-writer-wins keeps one line. The CRDT keeps both and converges.

## Running an experiment

Requires the vendor CLIs you want to compare (Claude Code, Codex CLI, Gemini
CLI) on `PATH`, each already authenticated. Each run copies the bundled
`repos/express-app` starter (Express + TypeScript, one smoke test) into a
fresh sandbox.

```bash
agentroom --task tasks/30-fintech-ledger.json --agents claude-sonnet,claude-sonnet --mode concurrent
```

| Mode | What it is |
|---|---|
| `concurrent` | AgentRoom: CRDT workspace + room MCP tools + collaboration prompt |
| `shared-only` | same CRDT workspace, no MCP tools, no prompt |
| `shared-collab-noMCP` | CRDT workspace + prompt, no MCP tools |
| `parallel-merge` | independent agents, outputs unioned afterwards |
| `sequential` | one agent |
| `chatdev-style` | sequential three-phase role pipeline |

The ablation is a diff, so the control is checkable:

```bash
agentroom --task tasks/30-fintech-ledger.json --agents claude-sonnet,claude-sonnet --mode concurrent
agentroom --task tasks/30-fintech-ledger.json --agents claude-sonnet,claude-sonnet --mode shared-only
```

Same task, same agents, same budget. The only difference is the room.

## Reproducing the numbers

| Claim | Command |
|---|---|
| abandonment classification, all strata | `python scripts/classify_failures.py <runs-dir>` |
| every Tier I number in one invocation | `python scripts/tier1_recompute.py <runs-dir>` |
| matched-compute contrasts (Welch) | `python scripts/welch_contrasts.py <runs-dir>` |
| regex scorer | `python -m agentroom.quality <runs-dir>` |
| AST cross-check scorer | `node scripts/ast_scorer.mjs <run-dir>` |
| LLM-judge, and the cross-vendor panel | `python scripts/llm_rescore.py`, `python scripts/llm_rescore_panel.py` |

These read a run directory and derive the numbers from it. The run archive
itself is not in this repo; it is several gigabytes of sandbox snapshots.

## What did not work

Worth stating plainly, because a reader will find it anyway.

- **Collaboration is not universally good.** On the hardest task with Codex, two
  agents scored **0.80×** a single agent. Pairing can hurt.
- **More agents is not better.** Judged quality peaks at two and declines from
  three. Mean tests passing peaks at three and collapses at four. The two
  measures do not agree on the optimum.
- **The bundle attribution did not survive.** An earlier point estimate splitting
  the gain between the MCP layer and the CRDT substrate was retracted: the
  interval spans zero at the sample size available. The ordering holds, the
  percentage does not.
- **Heterogeneous pairs disagree between scorers.** Sonnet+Codex leads on the
  LLM-judge composite and trails on mean tests passing.
- **No execution oracle.** Tasks ship agent-authored test suites, so quality is
  judged, not verified. Cross-validated against regex and AST scorers, but this
  is not an execution-correctness claim.

## Design notes

- **`crdt_guard.py` is the merge primitive.** It watches the workspace and
  reconciles concurrent writes through `pycrdt` at character level. This is the
  substrate, and `selftest.py` demonstrates it in isolation.
- **`room_server.py` is the coordination interface.** `room_claim` is an
  advisory lock in the Chubby sense: enforcement is at the prompt layer, not the
  kernel. An agent that writes without claiming still has its bytes preserved by
  the substrate, and the room log surfaces the violation instead of blocking it.
  That is deliberate, and the observed cross-agent bug-fix behaviour depends on
  it.
- **`room_server_chat.py` exists to lose.** It is the chat-only baseline for the
  contrast that motivates state over messages.
- One scoring definition per metric, in `quality.py` and `stats.py`. No
  experimental result is a literal anywhere in this package.

## Paper

The workshop version is published at the ICML 2026 Workshop on Failure Modes
of Agentic AI ([OpenReview](https://openreview.net/forum?id=0aGLZqKJjt)); an
extended version is under review, and its arXiv link will be added on posting.
The interactive version is at
[seongland.com/article/agentroom](https://seongland.com/article/agentroom).

```bibtex
@inproceedings{
cho2026agentroom,
title={AgentRoom: Concurrent Multi-Agent Coding in a {CRDT}-Backed Shared Workspace},
author={Seonglae Cho and Donghyun Lee},
booktitle={Workshop on Failure Modes of Agentic AI at ICML 2026},
year={2026},
url={https://openreview.net/forum?id=0aGLZqKJjt}
}
```

To cite the software itself:

```bibtex
@software{agentroom,
  title   = {AgentRoom},
  author  = {Cho, Seonglae and Lee, Donghyun},
  year    = {2026},
  license = {MIT},
  url     = {https://github.com/seonglae/AgentRoom}
}
```

## License

MIT.