OptiBot MCP Server
README.md
# OptiBot
Run Claude Code, Codex and Grok as background workers from Telegram: an MCP server, a routed inbox and a
session runner with reports.
<!-- screenshot: phone, Telegram demo chat with the bot. Nikola sends "Plan the CSV export for the demo-shop project, give it to Claude"; the bot answers "Processing…" and then "▶ #12 Claude · demo-shop — started". Demo project name and job number only, no real chats. -->
<!-- screenshot: phone, same demo chat about 20 minutes later: the completion message "[Claude job #12] ✅ CSV export plan — completed" with its Result / Why it matters / Unfinished / Action needed lines and the PDF "CSV-export-plan.pdf" attached below it. -->
<!-- screenshot: phone, the attached PDF opened in Telegram's viewer: first page of the demo plan, showing headings, a bullet list and a small table rendered by bin/render_report.py. -->
## The problem
I wanted to hand real work to coding agents from my phone and walk away: "plan this", "fix that", "run the
tests and tell me". A chat bot that blocks while the agent works breaks on anything longer than a few minutes,
and Telegram, agent CLIs and networks all fail in ways that can drop a message or run it twice. OptiBot stores
every message before acting on it, runs long work as tracked background jobs, and only reports "done" when
the worker proves it with a report.
## Architecture
```mermaid
flowchart TD
TG["Telegram Bot API"] --> TICK["bin/inbox_tick.py, every 10 s"]
TICK --> STORE[("Bot inbox, SQLite, store.py")]
STORE --> MEDIA["bin/media_worker.py downloads photos, files, voice notes"]
MEDIA --> STORE
STORE --> DISP["bin/inbox_dispatch.py routed dispatcher"]
DISP --> COORD[("dispatcher.sqlite, coordination.py")]
DISP --> GW["ACP gateways: Grok via acp_broker.py, Codex, Claude"]
GW --> AGENT["Chat agent turn"]
AGENT --> MCP["server.py MCP tools: list_unread, send_message, ack, fail, retry_reply"]
MCP --> STORE
STORE --> REPLY["bin/reply_sender.py"]
REPLY --> TG
AGENT --> SESS["bin/sessions.py start"]
SESS --> JOBS[("jobs.sqlite registry")]
SESS --> WORKER["systemd-run unit: claude, codex or grok CLI"]
WORKER --> MANIFEST["completion.json and Markdown reports"]
MANIFEST --> PDF["session_reports.py and render_report.py"]
PDF --> TG
SWEEP["sessions.py sweep, every 60 s"] --> JOBS
```
1. **Intake.** A systemd timer runs `bin/inbox_tick.py` every 10 seconds. It long-polls Telegram, checks the
chat allowlist and writes each update once into the bot's SQLite inbox (`store.py`), keyed by Telegram's
`update_id`. Control commands (`/new`, `/model`, `/sessions`, `/kill`, `/stop`, `/resolve`, …) are parsed
here by `routing.intake_command` and never reach a model.
2. **Dispatch.** `bin/inbox_dispatch.py` picks the next message per conversation (FIFO per chat,
round-robin across chats and bots), takes a backend slot and a workspace lease, and runs one turn on the
chat's selected agent over ACP (Agent Client Protocol, JSON-RPC over WebSocket, `acp_client.py`).
3. **Replies.** The agent answers through the MCP server (`server.py`): `send_message` for text, photos,
documents, voice and Markdown reports, then `ack`. Every reply is stored as a planned component first;
`bin/reply_sender.py` sends it and records Telegram's receipt.
4. **Long work.** Anything that will take more than a few minutes is started as a job with
`bin/sessions.py start`. The runner launches the Claude Code, Codex or Grok CLI headless in its own
`systemd-run --user` unit with memory and runtime caps, then the turn acks and ends.
5. **Completion.** The worker writes `completion.json` naming its Markdown reports. The runner validates it,
freezes the reports, renders PDFs and sends one completion message plus the PDFs to the chat that started
the job. A sweep timer reconciles lost units, retries refused sends and resumes jobs paused by provider
usage limits.
The full design (inbox state machine, routing identities, job lifecycle, failure handling) is in
[docs/architecture.md](docs/architecture.md). Running it as systemd user services is in
[docs/operations.md](docs/operations.md).
## Stack
- Python 3.12, standard-library `sqlite3` and `asyncio`; one SQLite file per bot, plus a coordination DB and a jobs registry
- [FastMCP](https://github.com/jlowin/fastmcp) + uvicorn for the streamable-HTTP MCP server, bearer-token auth
- `websockets` for ACP, `httpx` for the Telegram Bot API and Gemini / ElevenLabs text-to-speech
- `reportlab` with bundled DejaVu fonts for Unicode PDF reports
- systemd user units and timers (`systemd-run` for each job), Docker for the MCP server
- pytest (1,866 Python tests) and `node --test` for the Codex ACP patch
## Interesting engineering
- **Each message runs at most once, and none disappears silently.** Intake is idempotent on Telegram's
`update_id` (`update_receipts`, unique `inbox.update_id`). A dispatch moves through a checked state machine
(`routing._TRANSITIONS`: `claimed → submitted → delivered → acked`). If the dispatcher dies before the prompt
was submitted, the item goes back to `pending`; once it may have reached the model it becomes `uncertain`
and is never replayed. `ack` refuses unless the final reply has a Telegram receipt or a delegated job is
registered (`InboxStore.ack_dispatch`), and `fail` refuses while a reply is still being delivered. A stuck
item is cleared by the user with `/resolve N done` or `/resolve N abandon` in the chat; the controls worker
checks that the backend is free, then the decision, the router task and the reply commit in one transaction
(`InboxStore.finish_router_resolve`). Nothing is ever re-run from the chat.
- **Turn timeouts that don't block the chat for good.** Chat turns stop after 30 minutes
(`Settings.turn_timeout = 1800` in `bin/inbox_dispatch.py`). A timed-out turn stays `uncertain` and queues
exactly one notice per attempt (shared outbox key `uncertain:<item>:<attempt>`) telling the user how to
unblock it. `/stop` sends ACP `session/cancel` on the running turn and only reports "stopped" when the
prompt really ended. The wrapper prompt tells every agent to hand anything over ~5 minutes to a session.
- **Usage-limit pause and resume (`limit_wait`).** When the final event of a worker run is the provider's
usage-limit error (Claude, Codex or Grok wording), `finish` parks the job as `limit_wait` instead of failing
it, sends one "paused, continues at HH:MM" notice, and the sweep later moves it back to `queued` with a
single compare-and-set. The continuation resumes the **same agent session** (`claude -p --resume`,
`codex exec resume`, `grok --resume`) with a short "continue from where you stopped" prompt, at most 5
times (`meta.limit_resume_max`). Only the current run's slice of the log is inspected, so a tool output
that merely mentions a limit never counts.
- **A completion contract instead of exit codes.** Workers get `bin/sessions_preamble.md` prepended to their
prompt and must write `completion.json` (title, honest `completed`/`incomplete` outcome, four plain-language
summary fields, 1–4 Markdown reports). `session_reports.load_contract` validates it: a regular file, report paths that
resolve inside the project or uploads, sizes and real content. No valid manifest means the job ends `incomplete`, even with
exit code 0. Reports are frozen at completion, rendered by `bin/render_report.py` (a small Markdown subset,
never executes HTML), and named by content hash so a retry never renders twice.
- **Telegram sends are claimed, sent, then receipted, never inside a DB transaction.** Each completion
component (`text`, `document:N`, notices) is claimed in `delivery_claims`, sent with no transaction open,
and receipted in `deliveries`. A send that may have reached Telegram (lost answer) becomes `uncertain`, is
never resent automatically, and is settled with `sessions.py resolve-delivery`.
- **Capability-scoped MCP tools.** In routed mode each turn gets a fresh `dispatch_capability`; only its
SHA-256 digest is stored (`routing.py`). Tools only see and answer the item that capability owns, and an
explicit `chat_id` must equal the saved origin chat. Voice notes go to the agent as attachments with a
prompt to call an external `audio` MCP (`transcribe_audio`); the dispatcher adds such servers from its
config (`mcp_extra`).
## Repository layout
| Path | What it is |
|---|---|
| `server.py` | MCP server: `send_message`, `retry_reply`, `list_unread`, `ack`, `fail`; TTS voice notes; `/health` |
| `store.py` | The bot inbox: intake, conversations, dispatches, reply plans, outbox, `/resolve`, `/stop` |
| `routing.py` | Identities, capabilities, dispatch phase transitions, intake limits, retry policy |
| `coordination.py`, `resources.py` | Dispatcher owner lease and backend slots; workspace/job leases in `jobs.sqlite` |
| `acp_client.py`, `backends.py`, `acp_broker.py`, `isolated_acp.py` | ACP transport, per-agent adapters, Grok coordinator |
| `bin/inbox_tick.py`, `bin/media_worker.py`, `bin/reply_sender.py` | Intake, media download, reply and control workers |
| `bin/inbox_dispatch.py` | Routed dispatcher (`run`, `check`, `status`, `resolve`) |
| `bin/sessions.py`, `bin/session_reports.py`, `bin/render_report.py` | Job runner, completion contract, PDF renderer |
| `bin/sessions_preamble.md` | Rules and completion contract prepended to every worker prompt |
| `bin/migrate_bot_routes.py`, `route_transfer.py`, `legacy_*.py`, `startup.py` | Offline migration and poll-ownership tooling |
| `deploy/` | Compose example, systemd templates, example dispatcher config, Grok broker drop-in, Codex ACP patch |
| `tests/` | 58 pytest files and fixtures, one Node test |
## Run it locally
Needs Python 3.12. Configuration is environment variables; [`.env.example`](.env.example) lists every one the
code reads, grouped by component (`deploy/telegram-mcp.env.example` is the shorter file for the MCP container).
```bash
python3.12 -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt -r requirements-tick.txt pytest
```
**1. The session runner, no Telegram or agent CLI needed.** A `sh` job takes a shell script as its prompt.
`SESSIONS_NO_SYSTEMD=1` runs it as a detached process instead of a systemd unit, and
`SESSIONS_TELEGRAM_DRY=1` writes the completion message to `notify.json` instead of sending it. Projects are
directories under the workspace root (`OPTIBOT_WORKSPACE`, default `~/personal/workspace`), so use a throwaway
`HOME`:
```bash
export HOME=$PWD/.demo-home SESSIONS_NO_SYSTEMD=1 SESSIONS_TELEGRAM_DRY=1
unset OPTIBOT_SESSIONS_HOME # never point a demo at a real registry
mkdir -p $HOME/personal/workspace/demo $HOME/.config/optibot
printf 'TELEGRAM_BOT_TOKEN=change-me\nTELEGRAM_CHAT_ID=1000000001\n' > $HOME/.config/optibot/telegram-mcp.env
cat > /tmp/demo-job.sh <<'EOF'
printf '# Demo report\n\nA shell job wrote this file so the runner can turn it into a PDF.\n\n- one\n- two\n' > REPORT-demo.md
cat > "$OPTIBOT_REPORT_MANIFEST" <<'JSON'
{"schema_version": 1, "title": "Demo job", "outcome": "completed",
"summary": {"changed": "Wrote a demo report.", "why": "Shows the runner.", "unfinished": "None", "action": "None"},
"reports": [{"path": "REPORT-demo.md", "title": "Demo report"}]}
JSON
EOF
python bin/sessions.py start --kind sh --project demo --title "Demo" --prompt-file /tmp/demo-job.sh --json
python bin/sessions.py list # "#1 done · sh · demo — Demo" after a few seconds
cat $HOME/personal/.optibot-sessions/jobs/1/notify.json
find $HOME/personal/workspace/uploads/session-reports -name '*.pdf'
```
Real workers use `--kind claude|codex|grok` and need that CLI installed and logged in.
**2. The MCP server.**
```bash
MCP_TOKEN=change-me INBOX_DB=./data/inbox.sqlite BIND_HOST=127.0.0.1 python server.py
curl -s http://127.0.0.1:8765/health
```
Point an MCP client at `http://127.0.0.1:8765/mcp` with `Authorization: Bearer change-me`. With
`TELEGRAM_BOT_TOKEN` and `TELEGRAM_CHAT_ID` set (a bot from @BotFather), `send_message` delivers to your chat.
`docker build -t optibot-mcp .` builds the same server as an image; `deploy/telegram-mcp.yml` is an example
compose file behind a reverse proxy.
**3. The whole loop** (intake timer, dispatcher, reply/media/controls workers, sweep timer) runs as systemd
user services; see [docs/operations.md](docs/operations.md). The routed dispatcher needs ACP gateways for
the chat agents: the Grok one is `acp_broker.py` (it wraps the official Grok CLI's stdio ACP mode); the Codex
and Claude gateways are separate projects that are not part of this repository.
## Tests
```bash
pytest -q tests # about 7 minutes
node --test tests/test_codex_explicit_mcp.mjs
```
Result on a clean clone: **1,862 passed, 6 skipped, 0 failed**; the Node file passes 4/4. Telegram, the
agent CLIs and systemd are replaced by fakes, and `conftest.py` points both bots at synthetic stores. If
OptiBot already runs on the same machine, run the suite with a throwaway `HOME` and without
`OPTIBOT_SESSIONS_HOME` set.
- **6 skipped:** they check the Codex / Grok gateway projects, which live outside this repository.
- The PDF text checks need poppler-utils (`pdftotext`, `pdfinfo`) and skip cleanly without them.
## Status and limits
- A personal, single-user, self-hosted tool that runs my own Telegram bot day to day. There is no
multi-tenant story, no web UI and no hosted version.
- Linux only: the runner and all services assume systemd user units; jobs run under `systemd-run --user`.
- Workers run with their CLI's permission prompts off (`--permission-mode bypassPermissions`,
`--dangerously-bypass-approvals-and-sandbox`, `--always-approve`). The rules in
`bin/sessions_preamble.md` and the systemd caps are the guardrails, not a sandbox. Run it on a machine
you are fine handing to an agent.
- The workspace root is a setting: `OPTIBOT_WORKSPACE` (default `~/personal/workspace`) is read by the
session runner, the intake tick, the dispatcher and the Grok broker; uploads default to `<workspace>/uploads`.
A few other locations (the sessions home, the Grok home) are still per-variable defaults; see `.env.example`.
- Two intake modes live side by side: `legacy` (the tick wakes one agent directly and it drains the inbox
FIFO) and `routed` (the dispatcher). Several docstrings still describe the routed path as "staged"; they
predate its rollout.
- The worker preamble mentions an optional task-board hook (`SESSIONS_BOARD_HOOK`) that calls an external
board tool; that tool is not in this repository and the hook is off by default.
- Default model ids in `bin/sessions.py` (`DEFAULTS`) reflect what I used; override with `--model`/`--effort`.
## Licence
MIT, see [LICENSE](LICENSE). The DejaVu fonts in `fonts/dejavu/` keep their own free licence
(`fonts/dejavu/LICENSE`).
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo recent community issues