# ask-pb

A local podcast knowledge pipeline and read-only MCP server, starting with the Pinkbike Podcast. Independent project; not affiliated with Pinkbike or Outside.

**Working foundation:** RSS → episode catalog → audio cache → timestamped transcription → SQLite full-text search → MCP evidence tools.

The initial live sync found 363 episodes (375.4 declared hours). Metadata alone cannot answer bike questions: import transcripts or run the local transcription worker to populate search.

## Setup

Python 3.11+ and SQLite with FTS5; the ingestion CLI supports macOS/Linux (uses POSIX file locking).

```sh
python3 -m venv .venv
.venv/bin/pip install '.[dev]'
.venv/bin/ask-pb sync
.venv/bin/ask-pb coverage
.venv/bin/ask-pb episodes --limit 10
```

The default data directory is `./data`. Set `ASK_PB_DATA_DIR` or pass **`--data-dir` before the subcommand** for a stable absolute location. All audio, database files, and model outputs belong outside version control. `probe/` contains the original access investigation and catalog snapshot.

For development, use `pip install -e '.[dev]'` where editable installs work, or reinstall with `pip install .` after edits. Tests load `src/` directly.

## Populate evidence

### Import an existing transcript

Copy an episode ID from `episodes`. Import JSON matching this schema:

```json
{
  "provider": "your-transcriber/model-version",
  "language": "en",
  "segments": [
    {"start": 12.5, "end": 20.0, "text": "Original transcript wording.", "speaker": null}
  ]
}
```

```sh
.venv/bin/ask-pb import-transcript EPISODE_ID /absolute/path/transcript.json
.venv/bin/ask-pb search 'suspension'
```

Timestamps are seconds from the start of the audio, finite, ordered, and positive in duration. Speaker is optional; do not assign a real person's name without verification. The example wording is synthetic, not a Pinkbike quote. Entire imports validate before replacement and commit atomically. Reimporting replaces the episode's search index; old evidence IDs change when the transcript changes.

### Transcribe locally

```sh
.venv/bin/pip install '.[transcribe]'
.venv/bin/ask-pb process --limit 1 --model small.en
# Or select a specific pending episode:
.venv/bin/ask-pb process --episode-id EPISODE_ID --model small.en
```

The optional adapter uses faster-whisper on CPU with int8 computation by default. The first run downloads model weights. No cloud transcription API is called. Native dependency availability varies by Python/platform; Python 3.11 or 3.12 is a reasonable fallback if installation fails. Speech recognition itself has not been benchmarked against the Pinkbike archive yet.

`process` downloads and transcribes newest pending episodes, bounded by `--limit` (default 1). Completed episodes are skipped. Failed episodes retain an error and are retried on a subsequent invocation. A failed batch exits nonzero. Successful transcripts include a SHA-256 of the downloaded audio. Downloaded audio is cached and hash-checked; partial files are discarded on failure. Maximum file size is 1 GB.

For a different provider, implement a callable that takes an audio path and returns the same JSON document; pass it to `core.process`, or export JSON and use `import-transcript`.

## Stand up the MCP

### Local stdio (recommended)

Configure your MCP client to launch the installed executable with **absolute paths**:

```json
{
  "mcpServers": {
    "pb": {
      "command": "/absolute/path/ask-pb/.venv/bin/ask-pb",
      "args": ["--data-dir", "/absolute/path/ask-pb/data", "serve"]
    }
  }
}
```

The client owns the server process; there is no separate daemon to start. All diagnostics go to stderr. The server exposes:

| Tool | Purpose |
| --- | --- |
| `get_coverage` | Check corpus size, feed check times, and transcription coverage |
| `list_episodes` | Browse metadata and transcription status |
| `search_bike_evidence` | Search short topic/product keywords; all query words must match |
| `get_evidence` | Retrieve a passage's original wording and provenance |
| `get_episode_passages` | Read surrounding transcript passages by episode and time |

Tools cannot download audio, import data, or run models. Evidence includes timestamps, source URL, audio URL with a media time fragment, provider, and transcript hash. Time-fragment playback depends on the audio player; start/end seconds are also returned explicitly.

### Local HTTP

```sh
.venv/bin/ask-pb --data-dir /absolute/path/ask-pb/data serve --transport streamable-http --port 8000
```

Connect a Streamable HTTP MCP client to `http://127.0.0.1:8000/mcp`. The server binds to loopback. For hosted deployment, see [Deploy on Render](docs/render.md): Docker, persistent storage, bearer authentication, a health endpoint, and scheduled ingestion. Render terminates TLS. OAuth and multi-tenant access are not implemented.

## Keep the archive updated

Rerun these commands from an external scheduler or manually:

```sh
.venv/bin/ask-pb --data-dir /absolute/path/ask-pb/data sync
.venv/bin/ask-pb --data-dir /absolute/path/ask-pb/data process --limit 3
```

These local commands do not install a schedule. The Render deployment starts its own bounded ingestion worker. `sync` honors ETag/Last-Modified when provided, upserts by feed URL + GUID, and preserves transcripts through metadata-only changes. Changed audio URLs invalidate cached audio and mark transcripts stale; stale passages disappear from search until reprocessed. One ingestion CLI writer per data directory is enforced; MCP readers can remain running.

Feed omissions do **not** delete old records: feeds may be truncated. Takedown/deletion reconciliation is not implemented. A publisher changing audio bytes at an unchanged URL is also not automatically detected. Dynamic advertisements may shift playback timestamps across later requests; recorded audio hashes identify which file was transcribed. Back up the data directory with workers stopped (or use SQLite's backup API).

## Scope and next steps

This is evidence retrieval, not yet archive-wide analysis. Search uses SQLite FTS5 with AND matching and ranks individual transcript segments. It does not perform semantic search, infer product generations, identify speakers, deduplicate video/podcast recordings, or prove consensus. Use several focused searches and check coverage before synthesizing. Source text is untrusted data, never instructions to an AI client.

Next: evaluate one real transcription; add overlapping retrieval windows and semantic search; normalize product generations; verify speakers; then add cross-episode research with explicit coverage and counterevidence. YouTube ingestion remains outside this foundation because the initial caption probe did not retrieve text.

Public feed access does not establish content reuse rights. The software license does not license Pinkbike audio, transcripts, or metadata. Keep corpus distribution separate from code distribution.

## Development

```sh
.venv/bin/python -m pytest -q
```

Tests use synthetic transcripts and mock network responses. They cover repeat syncs, revisions, invalid input, timestamp citations, atomic downloads, retry behavior, CLI operations, and a real MCP stdio session. They do not download podcast audio or model weights.