Gobbler

CLI-first content conversion to Markdown for humans and AI agents

Gobbler converts YouTube transcripts, audio/video files, documents, web pages, and intentionally selected browser tabs into structured Markdown. The public automation contract is the gobbler CLI; the Skills in this repository teach agents to use that same CLI.

Project status

Gobbler is an active beta. Current supported surfaces are:

• gobbler for conversion, diagnostics, batches, jobs, providers, and browser automation.

• Local Markdown Skills for CLI-capable AI agents.

• An optional Chromium extension and local relay for tabs deliberately placed in a Gobbler tab group.

Websites and AI chat products can change their DOM without notice, so browser integrations are more fragile than direct file and URL conversion.

Related MCP server: Python MCP Markdownify Server

Install from source

Requirements: Python 3.11+ and uv. Docker is optional for document/webpage conversion; ffmpeg is required for audio/video transcription.

git clone https://github.com/Enablement-Engineering/gobbler.git
cd gobbler

# Reproducible source-checkout environment
uv sync
uv run gobbler --version
uv run gobbler doctor --json

Run commands from the checkout with uv run gobbler ..., or install an isolated global CLI:

uv tool install .
gobbler --version

For local development, use uv sync --extra dev. See the installation guide for Docker and browser setup.

Convert content

# YouTube transcript; URL can also come from stdin
uv run gobbler youtube "https://youtube.com/watch?v=VIDEO_ID" -o transcript.md

# Local faster-whisper transcription
uv run gobbler audio meeting.mp3 --model small -o meeting.md

# Docling service required
uv run gobbler document report.pdf --no-ocr -o report.md

# Crawl4AI service required
uv run gobbler webpage "https://example.com" --no-proxy -o page.md

Without -o, normal Markdown output is written to stdout. --format json produces a single JSON object for individual conversions. Batch commands use --json for newline-delimited JSON events.

Service readiness

YouTube and local audio conversion do not need Docker. Document and webpage conversion use the services in docker-compose.yml:

make start-docker
uv run gobbler doctor --json

• Crawl4AI: http://localhost:11235

• Docling: http://localhost:5001

gobbler doctor --json is the broad agent-friendly diagnostic. gobbler status --json focuses on conversion-provider readiness and exits nonzero when overall status is degraded while still emitting JSON.

Batch and queued work

# Playlist output directory is required
uv run gobbler batch youtube-playlist \
  "https://youtube.com/playlist?list=PLAYLIST_ID" \
  -o ./transcripts --dry-run

# Local files
uv run gobbler batch directory ./documents -o ./markdown --pattern "*.pdf"

# URL list; one URL per line
uv run gobbler batch webpages urls.txt -o ./pages --dry-run

# Explicitly queue a webpage batch in the SQLite job store
uv run gobbler batch webpages urls.txt -o ./pages --queue --json
uv run gobbler jobs worker start
uv run gobbler jobs list

Queued work is opt-in through --queue. The current queue is SQLite-backed; Redis/RQ is not required.

Providers and configuration

Inspect the runtime instead of relying on a static provider list:

uv run gobbler providers list --format json
uv run gobbler config path
uv run gobbler config show
uv run gobbler config init

The default config file is ~/.config/gobbler/config.yml. Configuration is deep-merged over built-in defaults. CLI flags take precedence for the commands that expose them. Environment variables are used by specific integrations, not as a general override system:

• TRANSCRIPTAPI_KEY: paid YouTube transcript fallback.

• WEBSHARE_USER / WEBSHARE_PASS or YOUTUBE_PROXY: YouTube proxy configuration.

• CRAWL4AI_PROXY: webpage proxy fallback.

• OPENAI_API_KEY: openai-whisper transcription provider.

• CRAWL4AI_API_TOKEN: Docker Compose API token.

• GOBBLER_MODELS_PATH: Docling model-cache mount.

See the configuration guide for the current schema.

Browser extension

The extension communicates with a local relay on port 4625. Most browser operations auto-start it; browser status is intentionally read-only and does not.

uv run gobbler relay start
uv run gobbler browser status
uv run gobbler browser list
uv run gobbler browser extract -o page.md
uv run gobbler notebooklm query "Summarize the current notebook"

Browser command guards target tabs whose group ID matches the extension's stored Gobbler group. Use Allow & Add/Add Tab to create or populate it and grant origin access for extraction and page APIs. A different group with the same title does not match, while manually moving a tab into the existing managed group makes it eligible for commands; debugger-based browser exec does not require the separate origin permission. The extension does not bypass authentication, access controls, rate limits, or bot detection. See the browser extension guide.

AI-agent Skills

# Inspect available Skills
npx skills@latest add Enablement-Engineering/gobbler --list

# Interactive install
npx skills@latest add Enablement-Engineering/gobbler

# Install the main conversion Skill globally
npx skills@latest add Enablement-Engineering/gobbler \
  --skill gobbler --global --yes

This installs Skill files only; it does not install the Gobbler CLI. The repository currently provides:

• gobbler: conversion and batch workflows.

• gobbler-browser: authenticated-tab and supported AI-chat workflows.

• gobbler-setup: installation, diagnostics, and service troubleshooting.

Output contract

Successful Markdown conversions include YAML frontmatter when the selected converter provides metadata. For automation:

• Individual --format json commands emit one JSON object.

• Batch --json commands emit JSON Lines.

• Versioned conversion, diagnostic, batch, and job JSON contracts include schema_version: 1.

• Configuration, provider-registry, and browser JSON outputs are command-specific and currently unversioned.

• Error payloads use stable error_code values where documented.

• Credential-bearing URL components are sanitized in diagnostic payloads.

Development

uv sync --extra dev
uv run pytest tests/unit/ -q --tb=short
uv run ruff format --check src/ tests/
uv run ruff check src/ tests/
uv run --extra docs mkdocs build --strict

Project layout:

src/gobbler_cli/       Typer CLI
src/gobbler_core/      converters, providers, configuration, utilities
src/gobbler_queue/     SQLite job queue and worker
src/gobbler_relay/     browser-extension HTTP/WebSocket relay
browser-extension/     Chromium extension
skills/                CLI instruction Skills

See CONTRIBUTING.md, SECURITY.md, and the full documentation.

License

MIT License. See LICENSE.