tubeprobe
by avalidurl
README.md
# tubeprobe
**Keyless, open-source YouTube research for humans and AI agents.**
One small CLI + a local [MCP](https://modelcontextprotocol.io) server: search, video & channel metadata, transcripts,
comments, playlists, related videos, trending, live — plus optional speech-to-text with *your* engine and model,
and optional export to *your* notes app.
- **No API key, no account, no hosted service.** Runs on your machine; nothing is proxied through anyone's server.
- **Tiny core.** Pure Python standard library + one bundled 0.7 MB JavaScript file + the official ~3 MB yt-dlp release.
No ML libraries, no models, no `node_modules`, zero required pip dependencies.
- **Bring your own transcription.** For videos without captions, plug in MacWhisper, whisper.cpp, mlx-whisper,
faster-whisper, OpenAI Whisper, or any CLI — any model, any language. Off until you configure it.
- **Bring your own notes app.** Save a clean Markdown note (metadata, chapters, timestamped transcript linking back to
the video) to a folder, Obsidian, Notion, Apple Notes, a webhook, or any command. Off until you configure it.
- **Works with every MCP client**: Cursor, Claude Desktop, Claude Code, Codex, Gemini CLI, and others.
> Status: beta (0.1). Built on [yt-dlp](https://github.com/yt-dlp/yt-dlp) and [YouTube.js](https://github.com/LuanRT/YouTube.js),
> which do the hard work of speaking YouTube's internal web API. See [Limitations](#limitations).
---
## Contents
[How it compares](#how-it-compares) · [Install](#install) · [Quickstart (CLI)](#quickstart-cli) · [Use it from an AI agent (MCP)](#use-it-from-an-ai-agent-mcp) ·
[Configuration](#configuration) · [Transcription](#transcription-optional) · [Export to notes](#export-to-notes-optional) ·
[Tools reference](#tools-reference) · [Security](#security) · [Limitations](#limitations) · [Legal](#legal--responsible-use) · [Contributing](#contributing)
## How it compares
| | **tubeprobe** | Hosted YouTube-research APIs / MCPs (paid) | YouTube Data API v3 | yt-dlp alone |
|---|---|---|---|---|
| Cost | Free, MIT | Subscription / credits | Free quota (10k units/day), then quota requests | Free |
| API key / account | None | Required | Required (Google Cloud project) | None |
| Where it runs | Your machine | Their servers (your queries go to them) | Google | Your machine |
| Search, video, channel, playlist metadata | ✅ | ✅ | ✅ | Partial (no search filters, no suggest) |
| Transcripts (manual + auto captions, translation) | ✅ | ✅ | ❌ (only your own videos) | ✅ (download only) |
| Videos **without** captions | ✅ your own STT engine, any language | Varies / extra cost | ❌ | ❌ |
| Comments + replies, community posts, hashtags, live, suggest | ✅ | Varies | Partial | Comments only |
| Export to notes (Obsidian, Notion, Markdown, webhook…) | ✅ built in, opt-in | ❌ | ❌ | ❌ |
| MCP server for AI agents | ✅ local (stdio) | ✅ (remote) | ❌ | ❌ |
| Rate limits | YouTube's own per-IP throttling | Provider plan | Quota | YouTube's per-IP throttling |
**Trade-off, honestly:** a hosted service runs a fleet of IPs and absorbs YouTube's anti-bot measures for you.
tubeprobe talks to YouTube from your own IP, so very heavy use (hundreds of transcript fetches in a few minutes)
gets temporarily throttled. For personal research it's plenty; for bulk scraping it's the wrong tool (and see [Legal](#legal--responsible-use)).
## Install
Requirements: **Python ≥ 3.11** and **Node.js ≥ 20** on `PATH`. Optional: `ffmpeg` (only for speech-to-text).
```bash
# 1. install the CLI (pick one)
uv tool install git+https://github.com/avalidurl/tubeprobe # or: pipx install git+https://github.com/avalidurl/tubeprobe
# 2. fetch the official yt-dlp release (~3 MB), pinned and SHA-256 verified
tubeprobe setup
# 3. check
tubeprobe status
```
(Not on PyPI yet — install from GitHub as above.) Prefer pip-managed yt-dlp? Install
`"tubeprobe[yt-dlp] @ git+https://github.com/avalidurl/tubeprobe"` instead and skip `tubeprobe setup`.
Keep yt-dlp current — YouTube changes often: run `tubeprobe setup` again after upgrading tubeprobe.
Upgrade: re-run the install command with `--reinstall` (uv caches builds by version number, so plain `--force`
can reinstall the old code), e.g. `uv tool install --reinstall git+https://github.com/avalidurl/tubeprobe`.
From source:
```bash
git clone https://github.com/avalidurl/tubeprobe && cd tubeprobe
uv tool install --reinstall . # or: pip install .
tubeprobe setup
```
**Install size** (measured): a 0.2 MB wheel (0.8 MB installed, including the bundled bridge) + the 3 MB yt-dlp release
= **about 4 MB** on disk. Speech-to-text engines and their models are separate and entirely your choice.
## Quickstart (CLI)
Every command prints JSON, so it pipes nicely into `jq`.
```bash
tubeprobe search "espresso machine review" --type video --upload_date this_month --sort_by view_count
tubeprobe video dQw4w9WgXcQ --fields title,views,likes,published
tubeprobe transcript "https://youtu.be/dQw4w9WgXcQ" --language en --text_only
tubeprobe transcript dQw4w9WgXcQ --translate_to de # YouTube's machine translation
tubeprobe comments dQw4w9WgXcQ --sort new
tubeprobe channel @veritasium
tubeprobe tab @veritasium --tab shorts --limit 20
tubeprobe playlist PLFgquLnL59alCl_2TQvOiD5Vgm1hCaGSI
tubeprobe trending --region GB
tubeprobe save dQw4w9WgXcQ # export to your configured notes app(s)
tubeprobe --help # all commands
```
## Use it from an AI agent (MCP)
`tubeprobe mcp` speaks MCP over **stdio** (the default and recommended transport). Register it once per client.
If a client says it can't start the server, replace `"tubeprobe"` with the absolute path from `which tubeprobe`
(desktop apps don't inherit your shell's `PATH`).
<details open><summary><b>Cursor</b> — <code>~/.cursor/mcp.json</code></summary>
```json
{ "mcpServers": { "tubeprobe": { "command": "tubeprobe", "args": ["mcp"] } } }
```
</details>
<details><summary><b>Claude Desktop</b> — <code>claude_desktop_config.json</code></summary>
macOS: `~/Library/Application Support/Claude/claude_desktop_config.json` · Windows: `%APPDATA%\Claude\claude_desktop_config.json`
```json
{ "mcpServers": { "tubeprobe": { "command": "tubeprobe", "args": ["mcp"] } } }
```
GUI apps don't see your shell's `PATH`, so use the absolute path from `which tubeprobe`
(e.g. `"/Users/you/.local/bin/tubeprobe"`). tubeprobe itself also looks for `node`/`ffmpeg` in `~/.local/bin`,
Homebrew, `/usr/local/bin`, Volta and nvm; set `TUBEPROBE_NODE` if yours lives elsewhere.
</details>
<details><summary><b>Claude Code</b></summary>
```bash
claude mcp add --scope user tubeprobe -- tubeprobe mcp
```
</details>
<details><summary><b>Codex CLI</b> — <code>~/.codex/config.toml</code></summary>
```toml
[mcp_servers.tubeprobe]
command = "tubeprobe"
args = ["mcp"]
```
</details>
<details><summary><b>Gemini CLI</b> — <code>~/.gemini/settings.json</code></summary>
```json
{ "mcpServers": { "tubeprobe": { "command": "tubeprobe", "args": ["mcp"] } } }
```
</details>
Then just ask: *"Find the three most-viewed videos about sourdough from this month, read their transcripts and summarize
the techniques — save each to my notes."*
**HTTP transport (opt-in, local only).** `tubeprobe mcp --http --port 8765` serves `POST /mcp` on **127.0.0.1 only**
(never exposed to your network). Set `TUBEPROBE_HTTP_TOKEN` to require `Authorization: Bearer <token>`; requests with a
non-local `Origin` header are rejected (DNS-rebinding protection). Prefer stdio unless your client needs HTTP.
## Configuration
Optional file: `~/.config/tubeprobe/config.toml` (override with `$TUBEPROBE_CONFIG`). Start from
[`config.example.toml`](config.example.toml). Everything is off by default.
| Environment variable | Meaning |
|---|---|
| `TUBEPROBE_CONFIG` | Path to config.toml |
| `TUBEPROBE_ENGINE` | Comma list overriding `[transcription] engines` |
| `TUBEPROBE_LANGUAGE` | Overrides `[transcription] language` |
| `TUBEPROBE_AUTO_SAVE` | `1`/`0` overrides `[export] auto` |
| `TUBEPROBE_CACHE_DIR` | Response cache (default `~/.cache/tubeprobe`) |
| `TUBEPROBE_CACHE_TTL` | Cache lifetime in seconds (default 3600) |
| `TUBEPROBE_PROXY` | Proxy URL passed to yt-dlp (e.g. `socks5://127.0.0.1:1080`) |
| `TUBEPROBE_COOKIES` | Optional Netscape `cookies.txt` for yt-dlp (see Limitations — use a throwaway account) |
| `TUBEPROBE_DATA_DIR` | Where `setup` puts yt-dlp (default `~/.local/share/tubeprobe`) |
| `TUBEPROBE_YTDLP` | Use a specific yt-dlp zipapp |
| `TUBEPROBE_NODE` | Path to the `node` binary (if it isn't on PATH or in a standard location) |
| `TUBEPROBE_HTTP_TOKEN` | Bearer token for `mcp --http` |
| `YOUTUBE_API_KEY` | Optional: lets `trending` use YouTube's official most-popular chart |
## Transcription (optional)
Most videos have captions and tubeprobe uses them (manual first, then auto-generated, any language, with optional
translation). Only for videos **without** captions does it fall back to speech-to-text — and only if you've configured
an engine. tubeprobe never ships, picks, or downloads a model. No language is assumed: set `language` to whatever your
model supports, or leave it empty for auto-detect, and override per call with `--language`.
```toml
[transcription]
engines = ["whisper-cpp"] # tried in order
language = "" # auto-detect
[transcription.whisper-cpp]
model = "~/models/ggml-large-v3-turbo.bin"
```
| Engine | Install (your choice of model) | Notes |
|---|---|---|
| `macwhisper` | [MacWhisper](https://goodsnooze.gumroad.com/l/macwhisper) ≥ 13 (`mw` CLI) | Uses the app's models (Whisper / Parakeet). Check the app's "translate to English" setting is off if you want the original language. |
| `whisper-cpp` | `brew install whisper-cpp` + any ggml model | Fast on CPU / Metal / CUDA |
| `mlx-whisper` | `uv tool install mlx-whisper` | Apple Silicon. Downloads the model you name on first use. |
| `whisper-ctranslate2` | `pipx install whisper-ctranslate2` | faster-whisper (CTranslate2) |
| `openai-whisper` | `pipx install openai-whisper` | Reference implementation (PyTorch) |
| `command` | anything | `cmd = ["my-stt", "{audio}", "{language}", "{out_dir}"]`, run without a shell |
Requires `ffmpeg`. Audio is downloaded to a temp dir, converted to 16 kHz mono WAV, transcribed, and deleted.
`tubeprobe status` shows what's configured; `tubeprobe stt <video> --engine X` forces a specific engine.
## Export to notes (optional)
Add one or more `[[export.sinks]]` to your config. Then `tubeprobe save <video>`, `tubeprobe transcript <video> --save`,
the MCP tool `save_to_notes`, or `[export] auto = true` to save on every transcript fetch.
```toml
[export]
auto = true
[[export.sinks]]
type = "obsidian"
vault = "~/Obsidian/MyVault"
folder = "sources/youtube/{channel_slug}"
filename = "{date}-{slug}"
frontmatter = { type = "source", created = "{today}" }
tags = ["youtube"]
```
Sink types: **`markdown`** (any folder — Logseq, Foam, Zettlr, a git repo…), **`obsidian`** (+ optional link line in an
existing daily note), **`notion`** (database row; token from `$NOTION_TOKEN`), **`apple-notes`** (macOS), **`webhook`**
(POST JSON — n8n, Zapier, Make, your server; header values can reference `${ENV_VARS}`), **`command`** (any program,
markdown on stdin, no shell). Full options are in [`config.example.toml`](config.example.toml).
Each note contains: YAML frontmatter (title, video_id, url, channel, published, duration, views, likes, comments,
transcript source/language, fetched-at, your extra keys and tags), chapters with timestamp links, the description, and
the transcript in ~1-minute paragraphs, each starting with a `[mm:ss](https://youtu.be/ID?t=S)` link.
Notes are **idempotent per video id**: saving again updates the same file (or `on_existing = "skip"`), keeps the
original `created` value, and preserves anything you write below the `<!-- tubeprobe:end -->` marker.
Folder/filename templates are sanitized — placeholder values are slugified and paths can't escape the sink directory.
Writing a new sink is ~20 lines: add `def sink_<name>(cfg, md, note, fields) -> dict` to `tubeprobe/sinks.py` and register it.
## Tools reference
| MCP tool | CLI | What it does |
|---|---|---|
| `youtube_search` | `search` | Search with filters: upload date, type, duration, sort, features (HD, 4K, subtitles, live, …), channel, region/lang, paging |
| `youtube_search_suggest` | `suggest` | Autocomplete suggestions |
| `youtube_search_hashtag` | `hashtag` | Videos for a #hashtag |
| `youtube_trending` | `trending` | Popular right now (official chart with `YOUTUBE_API_KEY`, else a keyless proxy) |
| `youtube_live_now` | `live` | Live streams right now |
| `youtube_video_get` | `video` | Full metadata, chapters, tags, formats summary, `fields` filter |
| `youtube_videos_batch` | `videos` | Up to 50 videos in parallel |
| `youtube_video_transcript` | `transcript` | Timestamped transcript; language/kind/translation; STT fallback; optional save |
| `youtube_related_videos` | `related` | Up-next / related |
| `youtube_comments` | `comments` | Top or newest comments, paged |
| `youtube_comment_replies` | `replies` | Replies to a comment |
| `youtube_channel_get` | `channel` | Channel details (subscribers, description, links, …) |
| `youtube_channels_batch` | `channels` | Up to 50 channels |
| `youtube_channel_tab` | `tab` | Channel videos / shorts / streams / playlists |
| `youtube_channel_community` | `community` | Community posts |
| `youtube_playlist` | `playlist` | Playlist metadata + items, paged |
| `youtube_url_resolve` | `resolve` | Any YouTube URL / handle → canonical ids (+ start time) |
| `save_to_notes` | `save` | Export a video (+ transcript) to configured sinks |
| `transcription_status` | `status` | Show configured engines and sinks |
## Security
- **Local by default.** stdio MCP transport; the optional HTTP transport binds to 127.0.0.1 only, supports a bearer token,
rejects foreign `Origin`s and bodies over 1 MB.
- **No shell, ever.** External programs (STT engines, `command` sinks, ffmpeg, node) are run with argument lists, never
through a shell, so titles/URLs can't inject commands.
- **Validated inputs.** Video ids, channel ids/handles, playlist ids and URLs are checked against strict patterns and
YouTube host names before use; MCP arguments are type/enum/range-checked and unknown arguments are rejected.
- **Safe file writes.** Note paths are built from slugified values and confined to the configured directory
(path-traversal guard); writes are atomic.
- **Secrets stay out of config.** Notion tokens come from env vars; webhook headers can reference `${ENV}`.
- **Supply chain.** Zero required Python dependencies. The JS bridge is built from a locked `package-lock.json`.
yt-dlp is downloaded from its official GitHub release and verified against a pinned SHA-256.
CI runs ruff (incl. bandit rules), tests, `pip-audit`, and `npm audit`.
- **No telemetry.** tubeprobe only talks to YouTube (and to sinks/engines you configure).
Report vulnerabilities privately — see [SECURITY.md](SECURITY.md).
## Limitations
- **Per-IP throttling.** YouTube rate-limits heavy use from one IP (HTTP 429 / "confirm you're not a bot"), especially
for transcripts. tubeprobe caches responses and falls back to alternate endpoints for metadata, but bursts of many
dozens of transcript requests can still pause for 5–10 minutes. Cloud/datacenter IPs are throttled far sooner than home connections.
- **Unofficial interfaces.** yt-dlp and YouTube.js use YouTube's internal web API. When YouTube changes, update yt-dlp
(`tubeprobe setup`) and tubeprobe.
- **Trending:** YouTube retired its Trending page in 2025. Without `YOUTUBE_API_KEY`, `trending` is a proxy (fresh uploads
ranked by views across configurable seed queries), not YouTube's chart.
- **No age-restricted / members-only / private content** by default. `TUBEPROBE_COOKIES` can pass a cookies file to
yt-dlp, but logged-in automation can get an account flagged — if you use it, use a throwaway account and keep the file private.
- `notion` and `apple-notes` sinks are newer and less battle-tested than `markdown`/`obsidian`/`webhook`/`command`.
## Legal & responsible use
tubeprobe is an independent project, not affiliated with or endorsed by YouTube or Google. It reads publicly available
pages the same way a browser does, but YouTube's [Terms of Service](https://www.youtube.com/t/terms) restrict automated
access and downloading. **You are responsible for how you use it.** It's intended for personal research, accessibility,
note-taking and education. Don't use it to mass-scrape, to redistribute other people's content, or to circumvent
access controls; respect creators' copyright. Speech-to-text downloads audio temporarily to your machine and deletes it.
## Contributing
Issues and PRs welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). New sinks and engines are deliberately small, self-contained functions.
## License
[MIT](LICENSE). Third-party components: [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo recent community issues