subhive
by Keradd
README.md
<div align="center">
# 🐝 subhive
### One live, shared brain for every AI agent your tools spawn.
**Semantic memory · dependency-aware task DAG · auto-scheduler · roles · real-time push · live dashboard — all over MCP.**
[](https://modelcontextprotocol.io)
[](https://nodejs.org)
[](https://www.sqlite.org/wal.html)
[](#-tools)
[](#-testing)
[](./LICENSE)
</div>
---
Point **Claude Code, Hermes, Cursor, Codex** — or any MCP client — at the same
hive and they share one brain instead of stepping on each other. Most "shared
context" MCP servers are a JSON file behind a lock. subhive is a **concurrent
SQLite (WAL) store** with a real event log, live WebSocket push, and the things
multi-agent work actually needs and JSON bridges lack.
```
Claude ┐
Codex ┤ ┌─────────────────────────────┐
Cursor ┼─ MCP ──▶ │ 🐝 subhive │ ──▶ live dashboard
Hermes ┤ │ memory · tasks · artifacts │ ──▶ WebSocket push
…any ┘ │ scheduler · roles · events │
└─────────────────────────────┘
```
## Table of contents
- [Why](#why)
- [Features](#features)
- [Quick start](#-quick-start)
- [Connect an MCP client](#-connect-an-mcp-client)
- [Live push & dashboard](#-live-push--dashboard)
- [Tools](#-tools)
- [The dependency handoff + scheduler](#the-dependency-handoff--scheduler-the-point)
- [Semantic memory](#semantic-memory)
- [Architecture](#-architecture)
- [Security](#-security)
- [Known limitations](#known-limitations)
- [Testing](#-testing)
- [License](#license)
---
## Why
Spin up three agents on one job and they immediately collide: no shared memory,
no idea who's doing what, no handoff. subhive gives them a single context to
coordinate through, with the **same 24 tools regardless of which tool launched
them**.
| Problem with JSON context-bridges | subhive |
|---|---|
| Rewrite-the-whole-file, lock contention | Concurrent SQLite **WAL**, atomic writes |
| No handoff — agents duplicate work | **Dependency DAG** — tasks unblock automatically |
| Hand-assign every task | **Auto-scheduler** load-balances across idle agents |
| Exact-key lookup only | **Semantic memory** search by meaning |
| Poll a file for changes | **Event log + live WebSocket push** |
| No access control | **Reader/writer/admin roles**, tokens, rate limit, audit |
| Corrupts on crash | **Crash recovery** requeues orphaned in-flight tasks |
---
## Features
- **🧠 Semantic memory** — `memory_search("how do we auth")` finds the `auth-strategy`
entry by meaning. Fully local embedder (trigram hashing + cosine). No API, no downloads.
- **🔗 Dependency-aware task DAG** — a task stays blocked and hidden from workers
until the tasks it `needs` are done, then unblocks automatically and fires an event.
Cycle detection prevents deadlock.
- **⚖️ Auto-scheduler** — `task_claim_next` hands each agent the highest-priority
*ready* task race-safely; `auto_assign` load-balances across the least-busy online agents.
- **📇 Late-join digest** — a new agent calls `digest` and gets a human-readable
briefing of the whole context: roster, memory, task board, recent activity.
- **♻️ Crash recovery** — on restart, ghost agents are dropped and orphaned
in-flight tasks are requeued so nothing gets stuck.
- **🔐 Roles + rate limiting + audit** — readers can't write, admins administer,
runaway agents get throttled, every denial is logged. Admin is **token-granted, not claimable**.
- **📡 Real-time** — every write emits an event; agents poll `get_events` or
subscribe over WebSocket for live push.
- **📊 Live dashboard** — open `http://host:port/` for a real-time roster, task
board, shared memory, and event stream. Single HTML file, no build step.
---
## 🚀 Quick start
```bash
npm install
npm start # → subhive serve on http://127.0.0.1:8787
npm run demo # self-contained 3-agent showcase (temp DB, no setup)
npm test # 5 suites: store · features · security · integration ×2
```
Or, once published:
```bash
npx subhive serve --port 8787
```
Create an isolated, token-protected context:
```bash
subhive context create myproject --token s3cret --desc "auth refactor"
subhive context list
```
---
## 🔌 Connect an MCP client
Point any MCP client at the HTTP endpoint and have it identify via headers:
```
URL: http://127.0.0.1:8787/mcp
Headers:
x-subhive-context: myproject # which hive (default: "default")
x-subhive-token: s3cret # required if the context has a token
x-subhive-agent: planner # this agent's display name (make it unique!)
x-subhive-tool: claude # claude / codex / cursor / hermes / …
x-subhive-role: reader # optional; reader/writer (admin needs the token)
```
> **The whole trick:** give each agent a **different** `x-subhive-agent` and the
> **same** `x-subhive-context`.
### Claude Code / Claude Desktop (`.mcp.json`)
```json
{
"mcpServers": {
"subhive": {
"type": "http",
"url": "http://127.0.0.1:8787/mcp",
"headers": {
"x-subhive-context": "myproject",
"x-subhive-token": "s3cret",
"x-subhive-agent": "claude-planner",
"x-subhive-tool": "claude"
}
}
}
}
```
### Zero-config: generate the client block for you
```bash
subhive connect claude --agent planner --context myproject --token s3cret
subhive connect cursor --agent coder
subhive connect codex --agent reviewer --role reader
```
Prints a ready-to-paste config for Claude, Cursor, VS Code, Codex, or generic
JSON — and tells you exactly which file it belongs in.
**Or let it write the file for you** — `--write` merges the entry straight into
the client's config file, **keeping any other MCP servers already there**:
```bash
subhive connect claude --agent planner --context myproject --token s3cret --write
# ✓ created .mcp.json (under "mcpServers") — agent "planner", context "myproject"
subhive connect vscode --agent coder --write # → .vscode/mcp.json
subhive connect claude --agent x --write --out ~/.claude/mcp.json # custom path
```
It refuses to clobber an existing `subhive` entry (re-run with `--force`) and
never overwrites a malformed file. `--write` works for `claude`, `cursor`, and
`vscode`; `codex`/`json` stay copy-paste.
---
## 📡 Live push & dashboard
**WebSocket** — first frame is a full `{type:'snapshot'}`, then `{type:'event', event}` per change:
```
ws://127.0.0.1:8787/ws?context=myproject&token=s3cret
```
**Dashboard** — open **`http://127.0.0.1:8787/`** in a browser for a real-time
roster (online status + task load), a four-column task board, shared memory, and
a live event stream — all driven by the same feed the agents use.
---
## 🧰 Tools
24 MCP tools, grouped by concern. Every call is role-checked, rate-limited, and audited.
| Tool | Purpose |
|------|---------|
| `whoami` · `roster` | your identity + role · who else is connected (with load) |
| `digest` | **human-readable briefing** of the whole context for late joiners |
| `memory_set` · `memory_get` · `memory_list` · `memory_delete` | shared tagged key/value memory |
| `memory_search` | **semantic search** over memory by meaning, not exact key |
| `task_create` | create work; `needs:[ids]` for deps, priority high/normal/low |
| `task_list` | filter by status / assignee; shows blocked state |
| `task_next` | **only unblocked** pending tasks, priority-ordered |
| `task_claim_next` | **atomically claim** the top ready task (race-safe scheduler) |
| `task_claim` · `task_update` | claim a specific task · set status/result; `done` auto-unblocks dependents |
| `auto_assign` | *(admin)* load-balance ready work across idle agents |
| `artifact_put` · `artifact_get` · `artifact_list` | shared blobs (code/plan/json/md/sql) |
| `send_message` · `inbox` | direct or broadcast messaging |
| `get_events` | event log since id N (poll to catch up) |
| `snapshot` | whole context in one call (agents, memory, tasks, ready, artifacts) |
| `set_role` · `reap` | *(admin)* manage roles · evict stale agents |
**Roles:** readers get read-only tools; writers get everything except admin ops;
admins get everything. Every denied call is written to the audit log.
---
## The dependency handoff + scheduler (the point)
```
planner: task_create "design schema" priority:high → t1
planner: task_create "build api" needs:[t1] → t2 (blocked)
worker: task_claim_next → t1 # highest-priority READY task, race-safe
worker: task_update t1 done # emits task.unblocked
worker: task_claim_next → t2 # now unblocked, claimed cleanly
```
`auto_assign` can instead fan all ready work out across the least-busy online
agents automatically. No polling races, no "is it my turn yet", no duplicated
work. Circular dependencies are rejected at creation time, so the scheduler can
never deadlock.
---
## Semantic memory
```
memory_set "auth-strategy" = "JWT with 15 minute expiry and refresh tokens"
memory_search "how do we authenticate users"
→ auth-strategy (score 0.33) # found by meaning, not the key
```
Embeddings are computed locally with a dependency-free hashing embedder
(trigrams + stopword removal, cosine similarity). **No external API, no model
downloads, no config.**
---
## 🏗 Architecture
```
src/
core/db.js SQLite (WAL) schema + additive migrations
core/store.js API over the DB; events, scheduler, search, digest, recovery
core/embed.js local hashing embedder for semantic memory search
core/guard.js role checks + per-session token-bucket rate limiter
tools/register.js 24 MCP tools, each guarded by role + rate limit + audit
tools/connect.js client-config generator (claude/cursor/vscode/codex/json)
transport/httpServer.js Streamable-HTTP MCP + WebSocket push + dashboard + REST
transport/dashboard.html live web dashboard (single file, no build)
server.js wires store + transport; runs crash recovery + reaper
cli.js serve / connect / context
test/
store.test.js unit: store + dependency logic
features.test.js unit: search, scheduler, digest, recovery, roles, rate limit
security.test.js auth-bypass, DM-leak, cycle, and privilege-escalation regressions
integration.test.js e2e: 2 MCP clients + WS listener over HTTP
integration_v2.test.js e2e: semantic search, scheduler, roles+audit, digest
```
**Data model:** `contexts` (auth boundary) → `agents` (with role), `memory`
(with embeddings), `tasks` + `task_deps` (dependency edges), `artifacts`,
`messages`, and an append-only `events` log that powers `get_events`, WebSocket
push, and the audit trail.
**Concurrency:** `better-sqlite3` is synchronous, so all writes serialize within
the process; the scheduler's atomic `UPDATE … WHERE status='pending'` guard makes
`task_claim_next` race-safe against simultaneous claimers.
**Durability:** SQLite WAL keeps committed writes safe across crashes. On startup
`recover()` drops ghost agent rows and requeues orphaned in-flight tasks; a
periodic reaper evicts agents whose heartbeat went stale.
---
## 🔒 Security
subhive shipped after a two-track audit — a manual sweep plus an independent
auditor agent running against a live server. **12 issues were found and fixed**,
each with a regression test in `security.test.js` / `features.test.js`.
| Severity | Issue | Fix |
|---|---|---|
| 🔴 Critical | Self-declared admin via `x-subhive-role: admin` header | Admin is **token-granted**; tokenless contexts cap at writer |
| 🔴 Critical | Direct messages fanned out to every WS subscriber | DM content redacted in broadcast; recipients read via `inbox` |
| 🟠 High | Circular dependency → permanent scheduler deadlock | BFS cycle detection rejects cycles + self-deps at creation |
| 🟠 High | Cross-context dependency edges broke isolation | same-context validation in `task_create` **and** `addDep` |
| 🟠 High | Crash-recovery events lost (event hook wired too late) | `onEvent` wired before `recover()` |
| 🟠 High | Context token leaked plaintext in snapshot / WS / REST | `publicContext()` redacts token, exposes `hasToken` flag |
| 🟠 High | Dangling `needs:[fake-id]` blocked a task forever | validates referenced task IDs exist |
| 🟡 Med | `task_update` accepted arbitrary status strings | status/priority enum validation |
| 🟡 Med | Rate-limiter clock skew → negative tokens → lockout | clamp elapsed ≥ 0, guard zero refill |
| 🟡 Med | Token compared with `===` (timing-observable) | `crypto.timingSafeEqual` |
| 🟢 Low | Stale tool count + dead `listMessages` paging | corrected count; `since` implemented via `created_at` |
**Security model:** named contexts are the auth boundary; optional per-context
tokens gate access; roles are server-enforced (never client-asserted); every
denied call is audited.
### Known limitations
Called out honestly rather than hidden:
- **Single-process deployment.** The scheduler is race-safe within one process
(synchronous SQLite). Running **multiple server processes against one DB file**
is not yet safe — `auto_assign` uses read-then-write without cross-process
locking. Run one `subhive serve` per DB.
- **WS DM filtering is coarse.** WebSocket connections aren't yet bound to an
agent identity, so DM *content* is redacted from the broadcast entirely;
recipients read private messages via `inbox`. Per-recipient WS delivery needs
identity binding (planned).
- Local embedder trades recall quality for zero-config/zero-dependency operation.
It's great for "find the decision," not a replacement for a real vector DB at scale.
---
## ✅ Testing
```bash
npm test # runs all 5 suites
# or individually:
node test/store.test.js # STORE OK
node test/features.test.js # ALL FEATURE TESTS OK
node test/security.test.js # SECURITY OK
node test/integration.test.js # INTEGRATION OK — tools=24
node test/integration_v2.test.js # INTEGRATION v2 OK
```
Integration tests start a real server, connect real MCP clients plus a WebSocket
listener, and verify auth, shared memory, semantic search, the full dependency
handoff, the scheduler, role enforcement + audit, artifacts, messaging, the event
log, and live push end-to-end. `security.test.js` reproduces every audited
vulnerability over the real MCP transport and asserts it's closed.
---
## License
[MIT](./LICENSE)
This server cannot be deployed
Maintenance
ActivityStale
ResponsivenessNo issues