Augent — The Audio Layer for Agents

If the answer is trapped in audio or video, this is the way through.

Augent turns any audio or video source into structured, searchable intelligence for agents. Give it URLs or files. It downloads, transcribes, indexes, and stores everything in persistent memory. Search by keyword or meaning, find where concepts intersect, identify speakers, generate chapters and notes, batch process entire libraries, and more. One install, full pipeline, entirely on your machine.

If you want the quality info from content without sitting through it, the fastest way, this is it.

Preferred setup: run the one-line installer in your terminal. One command installs Augent, all dependencies, and the MCP server config. Works on macOS and Linux. Windows: install via pip. Works with Claude Code, Codex, and any MCP client. New install? Start here: Getting started.

Install

curl -fsSL https://augent.app/install.sh | bash

Works on macOS and Linux. Installs everything automatically.

Windows: pip install "augent[all] @ git+https://github.com/AugentDevs/Augent.git"

The installer is a single bash script (source). Every dependency is open source:

No background services. No telemetry. No sudo on macOS.

Related MCP server: video-analyzer

How it works (short)

graph TB
    A["URL / File"] --> B["Download + Separate"]
    B --> C["Transcribe"]
    C --> D["Memory + Tag"]

    D --> E["Search"]
    D --> F["Analyze"]
    D --> G["Export"]

    style A fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style B fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style C fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style D fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style E fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style F fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style G fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px

    linkStyle default stroke:#00f060,stroke-width:1.5px

Full architecture →

Project Structure

augent/
├── mcp.py          # MCP server — 22 tools for agents
├── config.py       # User configuration (~/.augent/config.yaml)
├── core.py         # Transcription engine (faster-whisper)
├── search.py       # Keyword search
├── embeddings.py   # Semantic search, chapters, visual scoring
├── speakers.py     # Speaker diarization (pyannote-audio)
├── separator.py    # Audio source separation (Demucs v4)
├── tts.py          # Text-to-speech (Kokoro)
├── memory.py       # Three-layer memory (SQLite)
├── graph.py        # Obsidian graph view (wikilinks, MOCs, frontmatter)
├── clips.py        # CLI clip extraction (audio segments around matches)
├── export.py       # Export formats (JSON, CSV, SRT, VTT, MD)
├── cli.py          # CLI interface
└── web.py          # Web UI (FastAPI)

MCP Tools

The primary way to use Augent. Any MCP client gets direct access to all tools.

Add to ~/.claude.json (global) or .mcp.json (project):

{
  "mcpServers": {
    "augent": {
      "command": "augent-mcp"
    }
  }
}

Restart Claude Code. Run /mcp to verify connection.

Full tool reference →

"Download these 10 podcasts and find every moment a host covers a product in a positive or unique way. Not just brand mentions, only real endorsements or life-changing recommendations. Give me the timestamps and exactly what they said: url1, url2, url3, url4, url5, url6, url7, url8, url9, url10"

CLI

Full CLI for terminal-based workflows. Works standalone or with any agent.

Eyes & Ears

Someone explains their entire workflow in a video. Augent transcribes it, builds the workflow files, maps the sequencing, the decision points, the tool stack. Every piece structured into something an agent can act on.

But some steps are inherently visual. Augent detects where visual context is needed and exports multiple screenshots at those moments, giving the agent frame-by-frame context of the flow being described. Audio intelligence plus visual context equals a complete, replicable system.

graph TB
    A["Expert explains workflow or automation"] --> B["Augent transcribes + structures"]
    B --> C["Builds workflow files + sequencing"]

    C --> D["Maps decision logic"]
    C --> E["Identifies tools + platforms"]
    C --> F["Flags visual gaps"]

    F --> G["Exports screenshots for context"]
    D --> H["Ready to run"]
    E --> H
    G --> H

    style A fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style B fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style C fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px
    style D fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style E fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style F fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style G fill:#0a0a0a,stroke:#00f060,color:#00f060,stroke-width:2px
    style H fill:#0d2618,stroke:#00f060,color:#00f060,stroke-width:2px

Read more →

X/Twitter Spaces

Download or live-record Twitter/X Spaces audio. Auto-detects whether a Space is live or ended and handles both. Live Spaces record from the current moment using ffmpeg, ended Spaces download the full recording via yt-dlp. All downloads run in the background so your agent keeps working.

One-time setup

X/Twitter requires authentication to access Space audio. Any account works, including a burner.

1. Log into x.com in any browser

2. Open DevTools (F12 or Cmd+Option+I) > Application > Cookies > https://x.com

3. Copy the auth_token and ct0 values

4. Create ~/.augent/auth.json:

{"auth_token": "PASTE_HERE", "ct0": "PASTE_HERE"}

Tokens are stored locally and only sent to Twitter's servers to fetch audio. Augent never posts, DMs, follows, or modifies anything on your account. To revoke access, log out of Twitter or delete ~/.augent/auth.json.

Claude Code Skill

Claude and Codex already know how to use Augent's tools from their descriptions. The skill adds advanced workflows on top: multi-step note-taking pipelines, auto-tagging rules, translation flows, quiz formatting, and optimal search strategies.

mkdir -p ~/.claude/skills/augent
curl -o ~/.claude/skills/augent/SKILL.md \
  https://raw.githubusercontent.com/AugentDevs/Augent/main/skills/augent/SKILL.md

Works globally across all projects. One install, every conversation benefits.

OpenClaw

Augent is available as an OpenClaw skill on ClawHub.

Install via ClawHub:

npx clawhub@latest install augent

Or set up manually:

augent setup openclaw

If you installed Augent with curl -fsSL https://augent.app/install.sh | bash, OpenClaw is detected and configured automatically. These commands are only needed for manual setup or pip installs.

Obsidian Graph View

Every transcription builds a node. Every shared tag builds a connection. Your audio memory becomes a navigable knowledge graph, entirely automatic.

Every take_notes call, every transcription, every tag creates structure: YAML frontmatter, [[wikilinks]] between related content, and MOC hub files that cluster topics. Run rebuild_graph once to upgrade existing memory. The graph grows on its own from there.

Use Augent inside your main Obsidian vault, alongside your personal notes, journals, and projects. Everything compounds together. Full guide.

Using Claude Code or Codex with Obsidian? Set up augent-obsidian to make every .txt and .md file on your Mac open directly in Obsidian, with automatic sync for external edits.

Multilingual

Augent transcribes audio in its original language with full accuracy, powered by OpenAI's Whisper, supporting 99 languages including Chinese, French, Spanish, Japanese, Arabic, Hindi, Korean, German, Russian, Portuguese, and many more. Language is auto-detected, no configuration needed. Translation to English is handled by Claude (or your LLM), producing far better translations than any local model.

• When a transcription returns a non-English language, the MCP response includes a translation offer

• Accepting stores a clean English (eng) sibling file in memory alongside the original

• Both the original and translated versions appear in the Memory Explorer

Model Sizes

tiny is the default. Handles everything from clean studio recordings to noisy field audio. Use small or above for heavy accents, poor audio, or lyrics.

Configuration

Customize defaults and disable tools you don't need via ~/.augent/config.yaml:

# ~/.augent/config.yaml
model_size: tiny           # Default Whisper model
output_dir: ~/Downloads    # Default download directory
notes_output_dir: ~/Desktop # Notes, clips, TTS output
clip_padding: 15           # Seconds of padding around clips
context_words: 25          # Words of context in search results
tts_voice: af_heart        # Default TTS voice
tts_speed: 1.0             # TTS speed multiplier
disabled_tools: []         # Hide tools from MCP clients

Per-call arguments always override config. No config file needed, all values have sensible defaults.

Configuration docs →

Web UI

Local web interface. Runs 100% locally. No internet, no API keys, no data leaves your machine.

augent-web

Open: http://127.0.0.1:8282

Search view:

1. Upload an audio file or paste a YouTube/video URL to download audio directly

2. Enter keywords separated by commas

3. Click SEARCH and results stream live with timestamps and context

4. YouTube timestamps are automatically hyperlinked when the source is YouTube

Clip export:

• Click the film icon on any search result to create a visual region on the waveform, or drag on the waveform to select any range manually

• Nudge buttons (±1s / ±5s) on each edge for precise boundary adjustment

• Preview plays only the selected range so you hear exactly what will be exported

• Export MP4 downloads only the selected segment, not the full video

• Keyboard shortcuts: Space preview, Enter export, Esc close

Memory Explorer:

• Browse all stored transcriptions, including files transcribed via MCP or CLI. Every tool writes to the same memory.

• View full transcripts with clickable YouTube timestamps

• Delete individual transcriptions from memory

• Show Audio to reveal the source audio file in Finder

• Show Transcript to reveal the .md transcript file in Finder. Drag it into a Claude Code session to run the full MCP pipeline on a previously transcribed file.

• Share as HTML to download a self-contained, shareable transcript page

• Search across all memories by keyword to find matches across every transcription in your library

Source URL persistence: When audio is downloaded from any URL (YouTube, Twitter/X, TikTok, Instagram, SoundCloud, and 1000+ sites) the source URL is permanently stored by file hash. Any future search or transcription of that file, even weeks later or from a different path, automatically links back to the original source. No need to re-enter the URL.

Star History

Contributing

PRs welcome. Open an issue for bugs or feature requests.

License

MIT