Scribe's Hoard

A private, local recorder for meetings, interviews and voice notes that transcribes on your own PC and makes every session searchable — exposed to an assistant ("Faustus") through MCP. Nothing leaves the machine: no cloud, no telemetry, no network call besides 127.0.0.1 (plus one download of the speech model the first time).

The one trick that makes it useful: on Windows the microphone and the system audio are captured as two separate tracks. Whatever came from your microphone is labelled yo, whatever came out of the speakers (the other people in the call) is labelled otros. No speaker diarization, no voice recognition, no guessing — the channel tells you who spoke.

What it does

• Record (Grabar): pick a title, kind (reunión / entrevista / nota de voz / otro), the sources (micrófono, sonido del sistema) and the language. While recording, a voice-activity detector cuts the audio into utterances and the model transcribes them on the fly, so the transcript grows live. When you stop, the whole recording is transcribed again in one pass and the final segments replace the provisional ones.

• Sessions (Sesiones): list with search, kind, tag and date filters. A session page has a player with a light waveform synced to the transcript (click a line to jump, the current line follows playback), editable title, notes and tags (saved immediately), export TXT / SRT / Markdown, delete with confirmation.

• Import: drop an existing file (wav, mp3, m4a, ogg, flac, mp4, webm…); it is converted with ffmpeg if it is on PATH, otherwise with the decoder that ships with faster-whisper (PyAV), and transcribed in the background.

• Search (Buscar): full-text search (SQLite FTS5, diacritics-insensitive, title weighted) across every transcript, grouped by session, each hit with its time offset and a snippet; click to open the session at that moment.

• Settings (Ajustes): model size, device (auto/cuda/cpu), compute type, live transcription on/off, VAD sensitivity, default sources/kind/language, model download state, ffmpeg detection, data folder and storage used.

• A "Grabando…" banner with elapsed time and a stop button on every page while a recording is running.

Related MCP server: migas-mcp

Requirements

• Windows 10/11 for two-track capture (soundcard, WASAPI loopback of the default speaker + default microphone). On Linux/macOS the sounddevice backend records the microphone only; the whole pipeline after capture runs anywhere and is tested with a fake backend that streams WAV fixtures.

• Python 3.11+ (3.13 works). Node 22 only to build the client.

• CPU works out of the box (int8). With an NVIDIA GPU, device: auto picks CUDA when ctranslate2 can see it. The CUDA 12 + cuDNN 9 runtime libraries come from the nvidia-cublas-cu12 / nvidia-cudnn-cu12 wheels in requirements-windows.txt (the app puts their bin folders on the DLL search path itself). If CUDA fails at load or at the first inference, the app falls back to CPU, finishes the session, and says why in Ajustes.

• ffmpeg on PATH is optional (imports of compressed formats are faster and more robust with it).

Install and run (Windows)

git clone <this repo> scribe-hoard
cd scribe-hoard
python -m venv venv
venv\Scripts\pip install -r requirements-windows.txt
npm install
npm run build
venv\Scripts\python scripts\launch.py

scripts\launch.py starts the server on the first free port from 5185 and opens the browser. python -m scribe starts it without opening anything. Data (SQLite DB, session audio, models, MCP token) lives in <repo>\data or in SCRIBE_DATA_DIR.

The first transcription downloads the model (small by default, ~460 MB; tiny is ~75 MB) into data\models. You can trigger it from Ajustes → Descargar ahora.

Self-test

venv\Scripts\python scripts\selftest.py            # lists devices, records 5 s from mic + loopback, transcribes
venv\Scripts\python scripts\selftest.py --fake     # same pipeline with a WAV fixture, works anywhere

Options: --seconds, --model tiny, --device cpu|cuda, --transcriber fake, --fixture path.wav (or mic.wav,system.wav), --no-system.

Environment variables

Access from your phone (behind a tunnel)

The server binds 127.0.0.1 and only answers requests whose Host is localhost, 127.0.0.1 or [::1]. To reach it from your phone through a tunnel that fronts the app (a private mesh network, a reverse proxy), list the extra host names in SCRIBE_ALLOWED_HOSTS, comma-separated, exact names or *.suffix: SCRIBE_ALLOWED_HOSTS=my-pc.example,*.ts.net. Port and letter case are ignored, and the Origin of API calls must resolve to one of those hosts too (any scheme or port). Cross-site fetches are still refused; opening the app from another page (a link, a bookmarklet, the share sheet) is a normal navigation and works.

Once opened through the tunnel, the browser offers to install it (PWA).

How recording works

1. The backend opens one stream per source. Blocks of 30 ms (16 kHz mono int16) land in a queue with the track name.

2. Each track has its own WAV writer (data/sessions/<id>/mic.wav, system.wav) and its own VAD chunker (webrtcvad, with an energy-based fallback). Tracks are kept time-aligned by padding with silence when a device stops delivering (a WASAPI loopback goes quiet when nothing plays).

3. Every closed utterance (≥ 240 ms of speech, closed after 700 ms of silence or live_chunk_max_s) is queued for the transcriber; the resulting segments are stored with live = 1, labelled by track, and pushed to the UI through GET /api/sessions/{id}/live (SSE).

4. On stop the session goes to processing; a single worker thread transcribes each full track with the model's own VAD filter, merges the per-track segments by time (yo / otros, or S1 for a single track), joins consecutive lines of the same speaker, replaces the live segments, mixes the tracks into audio.wav for the player and marks the session done (or failed with the error; Reintentar re-queues it).

5. Sessions left in recording or processing by a crash are finalised from whatever audio was written when the app starts again.

6. Silence and hallucinations (scribe/transcribe/filter.py, shared by the live chunks and the final pass): audio without speech (RMS/peak energy and VAD voiced ratio) is never sent to the model; segments with no_speech_prob > 0.6, avg_logprob < -1.0 or compression_ratio > 2.4 are dropped, as are well-known Whisper hallucinations ("Subtítulos realizados por la comunidad de Amara.org", "Gracias por ver el vídeo", "Thank you for watching"…). Counters land in the session's stats; a session whose final pass yields no text is done with the note "(sin voz detectada)" in the UI and in scribe_sessions.

API

All JSON, bound to 127.0.0.1 only, errors as { "error": "…" }.

from/to accept ISO dates or phrases (hoy, ayer, esta semana, hace 3 días, today, yesterday).

MCP

mcp_server.py is a stdio MCP server that fetches the tool list from the running app and proxies every call to POST /api/agent/call; it never opens the database. faustus-plugin.json at the root lets Faustus discover the app by its health endpoint and page title and connect it with the form prefilled.

Tools: scribe_status, scribe_sessions, scribe_transcript (paginated by time), scribe_search, scribe_start (only when the user asks), scribe_stop, scribe_note, scribe_tag, scribe_export, scribe_delete (destructive). The instructions tell the assistant to quote transcripts as transcripts, to cite timestamps, never to summarise a session it has not read, that yo/otros come from the channel and not from voice recognition, and never to start a recording unless explicitly asked in the current message.

Manual MCP client config:

{ "command": "C:\\path\\scribe-hoard\\venv\\Scripts\\python.exe",
  "args": ["C:\\path\\scribe-hoard\\mcp_server.py"],
  "env": { "SCRIBE_URL": "http://127.0.0.1:5185" } }

Development

python -m venv venv && venv/bin/pip install -r requirements.txt
npm install
npm run dev        # uvicorn --reload on 5185 + vite on 5173 (proxying /api)
npm test           # pytest -q (fake backend + fake transcriber, plus one real faster-whisper tiny run)
npm run build      # client → scribe/static (served by the app)

Tests never touch the real data folder; each one gets a temporary SCRIBE_DATA_DIR. tests/test_whisper_real.py uses data/models when it exists (or SCRIBE_MODELS_DIR), otherwise downloads tiny into a temp dir.

Layout: scribe/ (FastAPI app; audio/ capture backends, transcribe/ transcriber backends, vad.py, recorder.py, pipeline.py, merge.py, store.py, export.py, agent_tools.py, api/), mcp_server.py, client/ (React 19 + Vite + Tailwind 4), scripts/, tests/.

Privacy notes

• Audio is stored as 16 kHz mono WAV per track under data/sessions/<id>/; delete a session and the folder goes with it.

• The MCP token is regenerated at every start and only readable locally.

• Transcripts are automatic speech recognition output and may contain errors.