Agentic Coach Builder

Agentic Coach Builder turns the YouTube talks, websites and PDF books you choose into your own AI coach for Claude Code and Cowork. Every answer cites the exact second of a talk, the paragraph of an article or the page of a book. The coach remembers your context on your machine, and an evaluation framework measures retrieval and answers before a coach ships. No fine-tuning and no training: change the Sources and rebuild.

• Any subject, your Sources: YouTube playlists (captions only), any website (a polite crawler), PDF books you own (kept private).

• Verified knowledge only: an LLM extracts advice and quotes; code locates every quote in the source before it can be cited.

• One engine, many coaches: a coach is a Pack, a configuration file plus skills. Three ship as examples: a startup founder coach, a software coding coach and a long-term investor coach.

• Local-first: search and memory run in a small MCP server on your computer; your conversations and memory stay there.

• Measured: a retrieval benchmark, coverage-honesty checks and coach gates on the real host decide when a build is good enough.

How it works

flowchart LR
    subgraph S["Your Sources"]
        YT["YouTube playlists"]
        WEB["Websites"]
        PDF["PDF books (private)"]
    end
    S --> ING["Ingest<br/>sync · clean · extract · verify"]
    ING --> LIB[("Library<br/>Verified items + index")]
    LIB --> TAG["Tag by Domain"]
    TAG --> PK["Packs<br/>one per coach"]
    PK --> PL["Coach plugins<br/>MCP server + skills"]
    PL --> H["Claude Code · Cowork"]
    LIB -.-> EV{{"Evaluation<br/>retrieval · coverage · coach gates"}}
    EV -.->|gates| PL

Zoom in: pipeline data flow · how a coach answers · Domain tagging · evaluation framework

Related MCP server: cc-history

Quickstart

git clone https://github.com/kaushalag29/Agentic-Coach-Builder.git && cd Agentic-Coach-Builder
uv venv --python 3.12 && source .venv/bin/activate
uv pip install -e ".[extract,index,serve,eval,pack,dev,pot,web,pdf]"
cp sources.example.yaml sources.yaml && cp .env.example .env   # your Sources and your LLM endpoint
ytbrain ops                                                    # ingest -> tag -> evaluate -> build the coach plugins
claude --plugin-dir dist/plugin                                # then, in Claude Code: /founder-coach:setup

Step by step, with every LLM backend and the Cowork install: docs/getting-started.md. To make a coach of your own: docs/build-your-own-coach.md.

Requirements

Pipeline phases and commands

Every Step is checkpointed per Document: stop anything with Ctrl+C and run it again. Reference: docs/commands.md · docs/pipeline.md · docs/ops.md.

Evaluation

• Retrieval: Tuning questions generated from your Documents, labelled by two judges and a tie-breaker from other model families; nDCG@10, recall and MRR with 95% intervals, and a regression gate. The released benchmark in eval/ ships ids, titles, links, timestamps and grades, never transcript text (docs/eval-spec.md).

• Coverage honesty: ytbrain eval gap measures how often a coach answers a question its Library doesn't cover, or refuses one it does.

• Coach gates on the real host: Citations supported (G2), pushback on weak plans (G4), memory across weeks (G5), every part of a multi-part question (G6), Projects kept apart (G7), no security picks (G8).

• Experiments: can a small local classifier replace the tagging LLM? A blind three-judge panel, no hand labels, decision rules fixed in advance. It couldn't (−13.9 points), and the study found where tagging really loses accuracy.

FAQ

Is this fine-tuning? No. Nothing is trained. The coach is a Claude session grounded in your Sources through search over Verified quotes, so it updates the day your Sources change and every claim can be checked.

How is it different from plain RAG? Only quotes found verbatim in the source are indexed; every hit carries a deep link; each search reports coverage so the coach names what it doesn't know; memory of your goals and decisions persists per Project; and quality is gated by evals, not eyeballed.

Can I run it with local models only? Yes for ingestion, tagging and the coaches: Ollama is the default, and LM Studio works through the OpenAI-compatible backend. The eval judges need an OpenAI-compatible endpoint that serves models from different families (two judges and a tie-breaker).

Does my data leave my machine? Coach memory and the usage log are stored only on your machine; Claude sees what the coach returns in a conversation, as with any tool. Ingestion sends Source text to the LLM endpoint you configure.

Are my books shared? No. PDF books are a private Source: indexed for your own coach, excluded from release builds, never seeded into the released benchmark (ADR-0014).

What does it cost? Extracting about 700 YC talks with deepseek/deepseek-v4-flash on OpenRouter cost $0.40–0.80; a local model costs nothing but time (tested setup).

Does it work on my phone? No: Claude's mobile apps can't run local plugins.

Documentation

Contributing

Issues and pull requests are welcome: see CONTRIBUTING.md. The offline test suites need no keys, models or network.

Security and privacy

Secrets live only in .env (git-ignored), and CI scans every tracked file for keys. Coach memory, its backups and the usage log stay on your machine. See SECURITY.md.

License

Source-available under the Business Source License 1.1, converting to Apache 2.0 on 2030-01-01. Free for personal use, education, research, open-source projects, and a person or company coaching itself and its own team. Offering it, or a coach or knowledge base built with it, to others commercially needs a license: ask through this repository.

Disclaimer

Coaches summarise the Sources you give them. Their output is educational and may be wrong, outdated or out of context; it is not legal, financial, investment or tax advice. You are responsible for your right to download and process what you ingest, and for following the terms of YouTube, websites and your LLM provider. Provided "as is".