Streamfog MCP

📖 Installation Guide — quick start, manual setup, and troubleshooting 📖 Docs — Configuration · Tools · Troubleshooting · Development 🤖 llms-full.txt — full LLM-readable documentation

AI-driven AR lens orchestrator for live OBS streams. Control Streamfog face filters, AR effects, and Vtuber avatars through MCP tools via the local Streamer.bot WebSocket bridge. Your AI assistant becomes a stream producer.

What is Streamfog?

Streamfog is a Windows desktop app (https://streamfog.com) that streamers use to put AR effects on their face during live broadcasts — face filters ("beauty smooth", "cyber helmet"), animated lenses, background effects, and Vtuber-style avatars. It takes your webcam feed, applies the effect in real time, and renders the result into OBS Studio through a browser source.

Two things matter for automation:

1. Streamfog has no public API, CLI, or plugin SDK. You cannot script it directly, and it cannot tell you which lens is active. It is effectively a black box that only listens to Streamer.bot.

2. Streamer.bot is the remote control. Streamer.bot is a free automation tool for streamers. Streamfog registers itself with Streamer.bot as an integration, and Streamer.bot exposes a local WebSocket server that accepts DoAction commands by name (e.g. fire the action SetLens_BeautySmooth).

This project is the bridge between the two: it gives an AI assistant (or the dashboard, or any script) a clean tool interface — streamfog_set_lens, streamfog_clear_effects, streamfog_toggle_avatar, streamfog_status, streamfog_list_lenses — and translates each call into a Streamer.bot DoAction dispatch. Without this server you would have to open Streamer.bot and click actions manually; with it, your agent becomes a stream producer.

⚠️ Fire-and-forget: Streamer.bot does not report whether an action succeeded, and Streamfog cannot be queried. A tool returning "success" means dispatched, not applied — confirm visually on the stream.

How to use this server (3 steps)

1. Install the two external apps (one-time): Streamfog + Streamer.bot, enable Streamfog's integration inside Streamer.bot, and enable Streamer.bot's WebSocket server on port 8080. Full walkthrough: docs/ONBOARDING.md.

2. Map your lenses: create lenses.json linking human-readable lens names to the exact Streamer.bot action names that apply them (see Lens Map).

3. Start and drive it: start.ps1 boots the backend + dashboard; then either connect your MCP client (Claude Desktop / Cursor / opencode) over stdio or http://127.0.0.1:10994/mcp, click lenses on the dashboard, or call the REST API from scripts. All three surface the same bridge.

Architecture

Plain English: your AI talks to this server, this server talks to Streamer.bot, Streamer.bot tells Streamfog what to render, and OBS shows the result. Every hop is local.

┌─────────────┐     MCP            ┌──────────────────┐     WebSocket      ┌──────────────┐
│  LLM Agent  │ ───────────────→  │  streamfog-mcp   │ ────────────────→ │ Streamer.bot  │
│  (Claude,   │ ←───────────────  │  :10994 (FastMCP) │ ←──────────────── │ :8080         │
│   Gemini)   │   stdio / /mcp    │  :10995 (React)   │   DoAction JSON   │               │
└─────────────┘                  └──────────────────┘                    └──────┬────────┘
                                                                                │ Native Hook
                                                                         ┌──────▼────────┐
                                                                         │  Streamfog    │
                                                                         │  Desktop App  │
                                                                         └──────┬────────┘
                                                                                │ Browser Source
                                                                         ┌──────▼────────┐
                                                                         │  OBS Studio   │
                                                                         └───────────────┘

• MCP tools (for AI agents): served over stdio or the streamable HTTP endpoint at http://127.0.0.1:10994/mcp.

• REST API (for scripts + dashboard): /api/* on the same port, backed by the same bridge instance.

• Dashboard (for humans): React SPA at :10995 — lens grid, status KPIs, chat, logs.

The MCP endpoint is served at http://127.0.0.1:10994/mcp (streamable HTTP) in dual mode, alongside the REST API and dashboard.

Quick Start

uv sync
# Edit lenses.json with your Streamer.bot action names
# Set STREAMFOG_MCP_STREAMERBOT_TOKEN in .env if using auth
.\start.ps1

MCP-only via stdio (for Cursor, Claude Desktop):

uv run -m streamfog_mcp --stdio

Prerequisites

1. Streamfog installed and running

2. Streamer.bot installed and running

3. Streamfog → Streamer.bot integration enabled in Streamfog's Integrations panel

4. Streamer.bot WebSocket server enabled (Settings → WebSocket Server)

5. Actions created in Streamer.bot (e.g. SetLens_BeautySmooth, ClearEffects, ToggleAvatar)

6. lenses.json populated with your action→lens mappings

See docs/ONBOARDING.md for the step-by-step walkthrough.

Configuration

Lens Map (lenses.json)

{
  "beauty_smooth": "SetLens_BeautySmooth",
  "cyber_helmet": "SetLens_CyberHelmet",
  "vtuber_avatar": "SetLens_VTuberAvatar"
}

Keys are human-readable lens identifiers used in MCP tool calls. Values are the corresponding Streamer.bot action names. Keys starting with _ are ignored; unknown identifiers fall back to the action SetLens_{identifier}.

MCP Tools (5)

Lens Control

Discovery — READ_ONLY

REST API

Web Dashboard

Dark SOTA dashboard at :10995 (React + Vite + Tailwind + Zustand):

• Dashboard — hero + KPI cards (server, tools, bridge, uptime), lens grid, quick actions, exponential-backoff health

• Tools — dynamic tool list with schemas from /api/tools

• Chat — skill-first local LLM chat (Ollama/LM Studio/vLLM), 4 personalities, localStorage history, export

• Settings — backend health + LLM provider detection

• Help — architecture, env reference, troubleshooting

• Logs — live ring-buffer modal (Ctrl+L)

Keyboard: Ctrl+Scroll zoom, Ctrl+0 reset, Ctrl+L logs.

Project Structure

streamfog-mcp/
├── src/streamfog_mcp/
│   ├── _mcp.py              FastMCP singleton + resources
│   ├── server.py            FastAPI gateway (REST + /mcp mount)
│   ├── __main__.py          CLI entry (--stdio / --serve)
│   ├── config.py            Pydantic settings (STREAMFOG_MCP_ prefix)
│   ├── errors.py            _error_response() with logger.exception
│   ├── logging_ring.py      Ring buffer for /api/logs
│   ├── skills/streamfog/    SKILL.md (chat preprompt)
│   ├── tools/core_tools.py  5 @mcp.tool() decorators
│   └── services/streamerbot.py  Streamer.bot WebSocket client
├── webapp/                  Vite + React 19 + Tailwind + Zustand
├── native/                  Tauri 2.0 desktop wrapper
├── lenses.json              Lens → action mapping
├── pyproject.toml / uv.lock
├── start.ps1 / start.bat
├── justfile
└── tests/                   13 tests (units + endpoint + e2e)

Verification

just lint          # ruff
just test          # pytest (coverage-gated)
uv run pyright src/  # types
cd webapp && bun run check && bun run biome:ci
cd webapp && bunx playwright test   # e2e

Known Limitations

• Streamfog does not expose a native CLI or local API — all control goes through Streamer.bot

• Lens activation is fire-and-forget (Streamer.bot does not report success/failure for actions)

• No lens preview or thumbnail retrieval (Streamfog desktop is a black box)

• Lumia/Crowd Control bridge path is documented but not yet implemented as an alternative transport