# hookline

hookline is a self-hosted research agent for short-form video. You describe a niche in one sentence. hookline
collects public metadata about Shorts, TikToks and Reels in that niche, keeps only the videos that are really about
it, finds the outliers (videos that did far better than their creator usually does), explains what they have in
common, and turns one of them into a script brief in your own voice.

It all runs on your own computer: one Node.js process and one SQLite file, with a web UI, a CLI, a REST API and an MCP
server for AI assistants. It works end to end with **no API keys** on a synthetic demo dataset, using a rule-based
judge and template writers. Add a language model key, or point it at a local model, when you want model-written
verdicts and prose.

Status: 0.1.0, the first release. Expect rough edges, and read [DATA-SOURCES.md](DATA-SOURCES.md) before you connect a
real platform.

## What it does

- **Niche research agents.** An agent is an intent ("Find short videos about home espresso for beginners, not machine
  reviews"), ordered keywords, excluded terms, platforms and optional seed creators. It runs once or on a schedule
  (daily, weekly, monthly or a cron at most once a day). Keyword suggestions come with a quality score.
- **Outlier detection.** A kept video with a known follower count gets a size-weighted virality score,
  `ln(views / followers) × ln(followers)`, shown in the UI as the outlier score with five tiers (breakout, surge,
  standout, warm, baseline). It also gets a **creator baseline**: how the video did against the same creator's usual
  views on comparable videos, with a confidence level, plus velocity and engagement. Counts that a platform only shows
  rounded are kept as ranges and marked approximate, so a tier that depends on the rounding says so.
- **Clearance, a calibrated relevance judge.** Every video that reaches you was judged on topic. The judge asks yes/no
  questions and returns probabilities. Off-topic videos are dropped, videos that match only through stuffed hashtags
  are dropped or held, and unsure ones are held for you to review instead of being guessed. You calibrate the judge
  with your own labels in a keyboard-first Review screen.
- **Hooks library.** The opening line of each video, kept verbatim, typed (question, POV setup, comparison and so on)
  and searchable across agents, with lift tables that compare hook types and formats.
- **Reports and trends.** A report per run with themes, baseline beaters, posting times, lift tables and ideas to
  test; hashtag and sound aggregates; and a trend list with a lifecycle (new, rising, steady, fading). Every number
  is computed in code, and a model-written sentence that states a number the data does not hold is removed.
- **Script briefs.** One outlier becomes a beat-by-beat brief ("Original beat" next to "Your beat") with a timing
  plan, a word budget, one call to action and a check that your draft does not copy the original.
- **Webhooks.** Signed deliveries with retries, Discord and Slack formats, and outlier alerts that fire once per video.
- **REST and MCP.** A native API for the UI and CLI, a compatible `/v1` REST subset (see
  [Compatibility](#compatibility)), and an MCP server over stdio and HTTP with 24 tools.

## 5-minute start

You need Node.js 22.18 or newer (hookline uses the built-in `node:sqlite`) and git.

```sh
git clone <repository URL> hookline
cd hookline
npm ci --omit=optional   # about 25 MB; plain `npm ci` also fetches the optional Agent SDK (see below)
npm run doctor           # checks Node, SQLite, the home folder and every provider
npm run demo             # loads the synthetic dataset and runs three demo agents
npm start                # serves the UI and the API at http://127.0.0.1:4477/
```

`npm install` works as well; `npm ci` installs exactly the versions in `package-lock.json`. On a fresh clone the
install ends with `added 96 packages, and audited 99 packages`.

`npm run doctor` checks each part and ends with a verdict. Its warnings about a missing language model and the
off-by-default plugins are expected on a fresh install. An excerpt:

```text
ok    judge           First backend: auto; usable: heuristic.
warn  llm             Running without a language model. Verdicts and prose are rule-based.
warn  llm:claude-cli  claude-cli is off (optional; personal use only).

The zero-key demo can run. 9 ok, 4 warning(s), 0 failure(s).
```

`npm run demo` runs three demo agents in fictional niches over three simulated days, in well under a minute (about
7 seconds on a desktop). An excerpt of its output, with `…` where lines are cut:

```text
Clear aligners for adults (demo) (clear-aligners)
  runs: day 1 completed · day 2 completed · day 3 completed
  funnel (last day): 91 hits → 70 unique · 2 other language · 2 excluded · 10 off-topic · 15 held → 41 cleared (filter rate 20%)
  top outliers:
    tiktok @demo_clear_aligners_06 · 22100000 views · score 43.2 breakout · on topic (p 0.99)
    tiktok @demo_clear_aligners_06 · 15400000 views · score 38.0 breakout · on topic (p 0.97)
    youtube @demo_clear_aligners_05 · 263916 views · score 30.7 surge · on topic (p 0.99)
    …
  top hooks:
    "Is it a myth that attachments stain easily?" [question]
    …
    "POV: you just clicked in a brand new set of trays." [pov_setup]
  themes: Videos about “attachments” · Question hooks in myth-busting videos · …

Home espresso for beginners (demo) (home-espresso)
  …
Open http://127.0.0.1:4477/ to explore (run `hookline serve` or `npm start` first if it is not running).
```

Then `npm start` and open <http://127.0.0.1:4477/>. You will see the three demo agents with their outliers, reports,
hooks and trends under a "Synthetic demo data" ribbon. Nothing in the demo contacts a real platform.

Two things happen during the install that you should know about:

- **The optional Agent SDK.** Plain `npm ci` (or `npm install`) also fetches `@anthropic-ai/claude-agent-sdk` and its
  platform package, about 245 MB, which only the opt-in `claude-cli` provider uses. It is distributed under its own
  terms (see [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)). Install it only if you will use that provider.
- **Git hooks.** Inside a git clone, the install also sets `core.hooksPath` to `.githooks`, so your commits run the
  secret and legal scans. `git config --unset core.hooksPath` turns that off.

### Running the CLI

The `hookline` command is not on your PATH after `npm ci`. In the install folder, run it as `npx hookline <command>`;
from anywhere, as `node /path/to/hookline/bin/hookline.mjs <command>`. `npm link` in the install folder puts a
`hookline` command on your PATH. The rest of this page and the other docs write it as `hookline` for short, and the
npm scripts (`npm run doctor`, `npm run demo`, `npm start`, `npm run judge:eval -- …`) call the same program.
`hookline --help` lists every command, and `hookline <command> --help` its options. Commands log JSON lines to stderr
at the `info` level; `--quiet` or `--log-level warn` hides them.

### Where things live

- **Your data** lives in the hookline home, `~/.hookline` by default: one SQLite database plus logs, eval runs and
  temporary files. Choose another home with `--home <folder>`, the `HOOKLINE_HOME` environment variable, or `"home"`
  in the config file.
- **The config file** is optional. `hookline config init` writes `hookline.config.json` (a copy of
  `hookline.config.example.json`) into the current folder, so run it in the install folder. hookline looks for it with
  `--config`, then `HOOKLINE_CONFIG`, then the current folder, then the install folder, then `<home>/config.json`.
  Restart hookline after editing the file. Changes made in Settings or with `hookline config set` are stored in the
  database and apply to a running server on their own.
- `hookline config path` prints the config file, the home, the database and the `.env` files that were read.

### Keys and `.env`

API keys come only from the environment or a `.env` file, never from the config file. A `.env` is read from the folder
of the config file in use and from the home, and **never from the current folder**. With no config file, only
`<home>/.env` is read: if you copied `.env.example` to `.env` in the install folder, run `hookline config init` there
first so the two sit side by side. `hookline config path` and `hookline doctor` warn about a `.env` there that is not
read.

[.env.example](.env.example) lists every variable a `.env` can set: the Anthropic key, an OpenAI-compatible base URL
with its key and model names, judge servers, the paid collector's key, the yt-dlp path, and the server's host, port,
log level, plugins and API token. `HOOKLINE_HOME` and `HOOKLINE_CONFIG` decide where `.env` files are read, so they are
never taken from one: set them in your shell or in your MCP client's `env`.

## Use real data

### What works with no keys

- The synthetic demo (`npm run demo`, or **See it work** on the first screen), including demo agents you create
  yourself (**Use the synthetic demo data** in the New agent form, or `search_keywords` with `demo: true` over MCP).
- Importing videos you choose: URLs, CSV or JSON (`hookline import`, or **Import** in the UI).
- Public YouTube channel feeds: add channels as seed creators by channel id (`UC…`) and a real agent reads their newest
  Shorts.
- The rule-based judge, tagger and report writer for everything above.

### Manual import

Import is the safest source: hookline stores only what you give it. A CSV needs a `url` column (or `platform` plus an
id); views, likes, comments, the publish date, the creator and the follower count are optional, and counts may be
written as `1.2K` or `9.36M`. A JSON file is an array of rows, and a text file lists one video URL per line.

```sh
hookline import videos.csv                # or videos.json, urls.txt, or - for a URL list on stdin
```

Rows that cannot be read are listed with the reason. The sample file `fixtures/import/sample.csv` has two broken rows
on purpose:

```text
Imported 3 new and 0 updated videos (csv); 2 failed.
  - row 4: url is not a supported video URL
  - row 6: views must be a whole number
```

Imported videos go into the shared library. An agent links the ones that match its keywords on its next run.

### YouTube channel feeds

The built-in `youtube-rss` collector reads a channel's public feed of its 15 newest Shorts: one small, cached request
per channel. Add channels as seed creators by channel id when you create an agent:

```sh
hookline agent create --intent "<one sentence>" --keywords "<a>,<b>,<c>" --platforms youtube --seed youtube:UC<channel id>
```

A feed has publish times and view counts but no follower counts. Its videos get a creator baseline from the channel's
other videos, and a size-weighted score only once a follower count comes from another source (an import with a
`followers` column, or the yt-dlp plugin).

### The yt-dlp plugin (opt-in)

YouTube keyword search and TikTok seed creators come from `@hookline/collector-ytdlp`, a separate package in this
repository ([packages/collector-ytdlp/README.md](packages/collector-ytdlp/README.md)) that drives a yt-dlp binary you
install yourself. It stays off until you do all three of these: list it under `plugins` in the config file, enable
its collectors, and type an acknowledgment for each platform (`hookline acks set ytdlp-youtube`,
`hookline acks set ytdlp-tiktok`, or the dialog in Settings → Collectors).

What it does: it reads public metadata only. That means search results pages, channel and creator pages (follower
counts and recent videos), per-video details and, for YouTube, auto-captions. It spaces its requests, and it cools a
platform down when it sees a rate limit or a sign-in wall.

What it does not do: download videos, audio or thumbnails (every yt-dlp process runs with `--skip-download`, and the
tests check it); log in or use browser cookies (unless you explicitly turn on its `cookies` option); sign requests,
impersonate clients, rotate proxies or solve CAPTCHAs; or leave page dumps and caption files behind (they are written
to a private temporary folder and deleted right away).

Automated access may break a platform's terms, and the platform may block or rate-limit you. You are responsible for
that choice.

### The paid ScrapeCreators adapter (unverified)

The built-in `scrapecreators` collector calls the ScrapeCreators data API with **your** key and **your** credits
(TikTok search, profiles and transcripts; Instagram Reels search and view counts). It needs `SCRAPECREATORS_API_KEY`,
its acknowledgment (`hookline acks set scrapecreators`) and a credit budget above zero
(`budgets.perRun.providerCredits`). **It was built and tested against a local fake server only, and has not been
verified against the live API.** Treat it as experimental. [docs/plugins.md](docs/plugins.md) lists its options.

### Your first real agent

1. **Config.** In the install folder, `hookline config init` writes `hookline.config.json` from
   `hookline.config.example.json`. Every key is optional.
2. **A language model (optional).** Put `ANTHROPIC_API_KEY=...` in a `.env` file next to your
   `hookline.config.json` or in your hookline home, or see
   [Language models and the judge](#language-models-and-the-judge) for the other options.
3. **Sources.** Seed YouTube channels by channel id, import videos, or turn on the yt-dlp plugin as described above.
   Without any real source, a new real agent's run stops at its first step and says which collector is missing.
4. **Create and run.** In the UI choose **New agent**, write one sentence about what you want to find, click
   **Suggest keywords**, then **Create & run**. From the terminal:
   `hookline agent create --intent "<one sentence>" --keywords "<a>,<b>,<c>"`.
5. **Calibrate** (see [Calibrate the judge](#calibrate-the-judge)). Until then, verdicts use wider thresholds and hold
   more items for review.

A `quick` run over six YouTube keyword searches and five TikTok seed creators, with the yt-dlp plugin and a language
model, took about 9 minutes on a Windows desktop. Most of that was the plugin's polite pacing between channel pages.

## Language models and the judge

Every language model is optional. Without one, the judge, tagger, report writer, keyword suggestions and briefs use
rule-based code, and the doctor says so. The options:

- **Anthropic API (the documented default).** Set `ANTHROPIC_API_KEY`. It serves the `anthropic-api` judge, tagging,
  reports, keyword suggestions and briefs.
- **Any OpenAI-compatible server**, local or hosted (Ollama, llama-server, vLLM, OpenAI, OpenRouter): set
  `LLM_BASE_URL`, plus `LLM_API_KEY` when it needs one and `LLM_MODEL_FAST` / `LLM_MODEL_MID` for the model names.
  That covers tagging, reports, keywords and briefs. To use such a server as the **judge** too, set `JUDGE_BASE_URL`
  and `JUDGE_MODEL` (and `JUDGE_API_KEY` if it needs one); a local Ollama is `http://127.0.0.1:11434/v1`. hookline
  probes the judge server once. When it returns token log-probabilities, the judge reads the yes/no probability
  straight from them (logprob mode, one tiny call per question); otherwise it asks the model to state its
  probabilities, in batches.
- **A System One-compatible judge on your own key** (`systemone-http`). There are presets for Typesafe
  (`TYPESAFE_API_KEY`), the Vercel AI Gateway, OpenRouter and Cloudflare Workers AI, plus a custom URL for any server
  that speaks the format, including another hookline (`judge.systemoneHttp.baseUrl` with
  `HOOKLINE_SYSTEMONE_API_KEY`). The first preset whose key is set is used. Only judging runs there, and hookline's
  own extension fields are stripped before a request leaves.
- **Your own Claude Code session (`claude-cli`)**, for personal use only. It needs the full `npm ci` (the Agent SDK),
  `hookline config set llm.providers.claude-cli.enabled true` (or its switch in Settings → Judge & LLM) and its
  acknowledgment (`hookline acks set claude-cli`, or the dialog in Settings). It works only while the server listens on
  this computer alone. Log in by running `claude` in your own terminal: hookline has no login and never reads Claude
  credentials.
- **The heuristic judge, always on.** A small logistic model over topic words in the transcript, on-screen text,
  caption and title, plus hashtag statistics. It answers when nothing else is set up, it ends every fallback chain (a
  failing backend never lets a video through unjudged), and when another judge is in charge it judges every video
  alongside it, so large disagreements are queued for review.

The writers (tagger, reports, keywords, briefs) take the first available of `anthropic`, `openai-compatible` and
`claude-cli` (`llm.order`). The judge takes the first healthy of `systemone-http`, `openai-compatible`,
`anthropic-api`, `claude-cli` and `heuristic` (`judge.order`). Settings → Judge & LLM changes the judge order, and
`hookline config set` changes either order or pins a provider per purpose (`llm.purposes`). `hookline doctor` names
what each provider is missing (no key, no model name, server not reachable).

Model calls are capped by budgets, by default $1 and 300 calls per run and $5 per day (`budgets` in the config file).
A spent budget falls back to the rule-based path, and the run records why. When a model judges or tags, the video's
text (title, caption, hashtags, the start of the transcript, on-screen text) goes to the provider you chose. Creator
handles, follower counts and URLs never do.

The judge also answers the System One request format at `POST /v1/systemone`, so you can call it directly. This is a
real answer from the heuristic on a fresh install:

```sh
curl -s http://127.0.0.1:4477/v1/systemone -H 'content-type: application/json' -d '{
  "model": "heuristic",
  "state": {"platform": "youtube", "title": "Day one with clear aligners as an adult",
            "caption": "what nobody tells you about the first week of trays",
            "hashtags": ["clearaligners", "invisalign", "fyp"]},
  "questions": {"on_topic": {"type": "noul", "instructions": "Is this video about clear aligner treatment for adults?",
    "x_template": "clearance.on_topic@1",
    "x_lexicon": {"keywords": ["clear aligners", "aligner trays"], "excludes": ["teeth whitening"]}}}}'
```

```json
{"model":"heuristic:heuristic-v1","answers":{"on_topic":{"type":"noul","noul":0.88}},"usage":{"input_tokens":108,"output_tokens":13},"x_backend":"heuristic","x_calibration":{"on_topic":{"status":"uncalibrated","key":null}}}
```

The heuristic answers only the Clearance question templates (the `x_template` above); a model backend answers any
question. [docs/judge.md](docs/judge.md) covers the backends, the routing rules and the eval commands.

### Calibrate the judge

A raw model probability is not a calibrated one. hookline fits a calibration map per backend, model, prompt and
question from **your labels only** (model outputs are never used as labels):

1. Open **Review** (`g` then `r`) and press **Seed sample (60)**. It picks a uniform sample of the agent's judged,
   unlabeled videos.
2. Label with the keyboard: `y` / `n` on topic, `s` / `d` hashtag stuffing, `0`–`3` fit, `Enter` to save and go on.
   Review is blind by default: the judge's verdict appears only after you save (`r` reveals it).
3. When the readiness panel says there are enough labels (30 yes/no labels with at least 5 of each answer), press
   **Fit now** in the agent's Clearance tab, or run
   `npm run judge:calibrate -- --backend <backend> --question on_topic`.

The fit cross-validates each candidate method on your labels, keeps the plain probabilities unless a method clearly
improves them, and derives accept and reject thresholds with a certified error rate. The per-run audit sample keeps
supplying unbiased labels, and a map whose accuracy drifts is flagged for a refit. `npm run judge:eval` and
`npm run judge:compare` measure and compare judges on the same labels. The demo ships synthetic labels, and a map
fitted on them applies only to demo agents.

## Use it from an AI assistant (MCP)

hookline's MCP server lets an assistant create agents, read results, look up creators and write script briefs through
24 tools. Assistants start stdio servers in their own working folder, so pass your config file explicitly with
`HOOKLINE_CONFIG`. A fresh install has no config file yet: run `npx hookline config init` in the install folder first
(the MCP server refuses to start when `HOOKLINE_CONFIG` names a missing file). Replace `/path/to/hookline` with your
install folder.

Claude Code:

```sh
claude mcp add hookline -e HOOKLINE_CONFIG=/path/to/hookline/hookline.config.json -- node /path/to/hookline/bin/hookline.mjs mcp
```

Claude Desktop and other clients configured with JSON:

```json
{
  "mcpServers": {
    "hookline": {
      "command": "node",
      "args": ["/path/to/hookline/bin/hookline.mjs", "mcp"],
      "env": { "HOOKLINE_CONFIG": "/path/to/hookline/hookline.config.json" }
    }
  }
}
```

On Windows, write the paths as `C:\\path\\to\\hookline\\bin\\hookline.mjs` inside JSON. While `npm start` is running,
the same tools are also served over streamable HTTP at `http://127.0.0.1:4477/api/mcp/mcp`.

If you keep your data in another home (`--home` or `HOOKLINE_HOME` when you start the server), add the same
`HOOKLINE_HOME` to the client's `env`, or set `"home"` in the config file, so the assistant opens the same database.
Settings → API & MCP shows the command filled in with the running server's own home and config file.

At startup `hookline mcp` writes the config path, home and database it opened to stderr, and whether it reused a
running server's worker. If results look empty, check that line first, because a wrong config opens a different,
empty database. To try it without keys, run `npm run demo`, then ask your assistant to call `search_keywords` with
`demo: true` and a demo topic (for example clear aligners for adults). [docs/mcp.md](docs/mcp.md) lists the tools,
resources and prompts, and [examples/mcp-clients.json](examples/mcp-clients.json) has more snippets, including HTTP
with a token.

## Script briefs

Pick an outlier in an agent's **Hooks** tab and choose **Make brief**, or use the CLI:

```sh
hookline brief <video id or URL> --seconds 30 --voice-file <your voice file> --out <your project folder>/
```

With a folder as `--out`, hookline writes `brief.json` and `brief.md` into it. A path ending in `.md` writes only the
Markdown. The voice file is any plain-text or Markdown guide to how you talk, up to 16 KB; a longer file is cut to its
first 16 KB of whole lines. The writer is told never to state facts about you that your notes or voice guide do not
give. Without a model, the brief is a template with the original beats and a slot for each of yours. This is the top
of one made from the first demo outlier:

```text
# Script brief (30 s)

Source: https://tiktok.demo.invalid/9932146133123398556 · tiktok · @demo_clear_aligners_06
Views 22,100,000 · followers 961,027 · outlier score 43.19 · baseline × 163.28 · published 2026-09-19
Writer: rule-based template · confidence 80%

## Timing plan

[0:00–0:03] HOOK · [0:04–0:25] BODY · [0:26–0:30] CTA

Word budget: 81 spoken words (2.7 words per second).
```

## Webhooks

Add an endpoint in the UI (**Webhooks**) or through the `/v1/webhooks` routes. Each delivery carries
`X-Hookline-Signature: sha256=<HMAC-SHA256 of the raw body>`, keyed with the endpoint's secret, which is shown once.
Failed deliveries are retried after 30 s, 2 min, 10 min, 1 h, 6 h and 24 h, and an endpoint that fails 20 times in a
row is turned off. Events: run completed, outlier detected (once per agent and video, above a score or ratio you can
set), review needed and lookup completed. Endpoints must use https unless they are on this computer.
[examples/webhook-receiver.mjs](examples/webhook-receiver.mjs) is a small receiver that checks the signature.

## Compatibility

hookline answers a subset of Virlo's public v1 REST API under `/v1` on your own server, so clients, scripts and
assistant setups written for that API can point at a local hookline server instead. Its MCP server keeps the names of
17 tools from that service's MCP surface and adds 7 of its own. The judge accepts the System One request and response
format published by Typesafe at `POST /v1/systemone`, so System One clients can use hookline's judge. Webhook
management follows the same v1 API, while signatures and headers are hookline's own.

Compatibility covers names and shapes only: paths, parameter and field names, enum values, error codes, envelopes, and
a published scoring formula. Every description, prompt, document, UI text and line of code in this repository was
written for hookline. [PROVENANCE.md](PROVENANCE.md) lists each compatibility artifact and where it came from.
[docs/api-compat.md](docs/api-compat.md) lists what is served, what answers 404 and where hookline behaves differently.

## Not affiliated

hookline is an independent open-source project. It is not affiliated with, endorsed by or sponsored by Virlo,
Typesafe, Anthropic, OpenAI, ScrapeCreators, the yt-dlp project, or any video platform, including YouTube, TikTok and
Instagram. Their names appear here only to say what hookline is compatible with or what it can read (nominative
use), and they belong to their owners. When you connect hookline to a platform or a paid service, that platform's or
service's terms apply, and you are responsible for following them.

## Data sources and responsible use

[DATA-SOURCES.md](DATA-SOURCES.md) lists every collector, whether it is on by default, what it reads and its risk.
The short version:

- Only the demo, manual import and YouTube channel feeds are on by default. Everything else is off until you enable it
  and type an acknowledgment. Acknowledgments live in your database, never in a config file, so a copied config
  cannot switch a risky source on.
- hookline reads public metadata. It never downloads or re-hosts media, never asks for platform passwords, and does
  not evade rate limits or blocks. Requests to each host are spaced, and one agent run happens at a time per hookline
  home.
- Platform terms of service apply to you. Automated collection from YouTube, TikTok or Instagram may break their
  terms even when it only reads public pages. Use the opt-in collectors for your own research, at a gentle pace, and
  stop when a platform pushes back.
- It stores no email addresses or phone numbers. `hookline forget <platform> <handle>` removes a creator's personal
  data, and a retention job prunes raw payloads and old snapshots.

## Architecture

hookline is plain Node.js (ES modules, no build step) on the built-in `node:sqlite`. `hookline serve` runs the HTTP
server, a job worker and a scheduler in one process; the CLI and the MCP server open the same database file and reuse
the server's worker when one is running. An agent run is a checkpointed job that walks twelve steps (plan, collect,
dedupe, language, excludes, enrich, clearance, profile, score, tag, aggregate, report), so a run killed at any point
resumes where it stopped. Modules sit in tiers from contracts up to entry points, and a test fails on any upward
import. [docs/architecture.md](docs/architecture.md) explains the tiers, the data model, the pipeline, the judge and
calibration.

## Good to know

- One agent run happens at a time per hookline home, across every process that shares it.
- `hookline report rebuild <agentId>` rebuilds the analysis of the agent's newest finished run and replaces that run's
  earlier analysis and trends. The old analysis id stops resolving.
- Model-written report sentences that state a number, or name an id, that is not in the data are removed. The count is
  in the analysis's `x_provenance.numguard_removed`.
- YouTube rate-limits caption downloads hard. Expect runs with few or no transcripts; the judge then works from
  titles, captions, hashtags and on-screen text, and the run notes it.
- Verified against real services before this release: the yt-dlp plugin (YouTube search and TikTok seed creators),
  the `claude-cli` judge and writers, MCP over stdio and HTTP, briefs, and webhook delivery to a local receiver. The
  `anthropic`, `openai-compatible` and `systemone-http` backends and the `scrapecreators` adapter were tested against
  local fakes only.

## Security

The server binds to `127.0.0.1` and needs no login while only this computer can reach it. It requires API tokens
when it can tell that it is exposed: a non-loopback bind, an outside host in `server.publicUrl` or
`server.allowedHosts`, or a request that carries proxy headers. Behind any proxy or tunnel, set
`server.auth: "token"` yourself. Read [SECURITY.md](SECURITY.md) before exposing it, and report vulnerabilities
privately as described there.

## Development

```sh
npm test                                   # every test (node:test), offline and without keys; about 70 s on a desktop
npm run test:unit                          # one layer: also test:contract, test:e2e, test:arch, test:legal, test:perf
npm run lint:legal                         # names, banned phrases, colors and forbidden files
npm run scan:secrets                       # keys and tokens
node scripts/ownership-check.mjs --verify  # every tracked file has exactly one owner in OWNERSHIP.json
```

The pre-commit hook runs both scans on staged files, and CI runs the tests and all three checks on Windows and Linux
with Node 22.18 and 24. Live tests against real services run only with `HOOKLINE_LIVE_TESTS=1`, never in CI.
`node scripts/smoke-live.mjs --yes --seed tiktok:@<creator>` runs one real agent end to end on a machine where the
yt-dlp plugin is set up.

Before you open a pull request, read [CONTRIBUTING.md](CONTRIBUTING.md). It has the clean-room rules (names only,
everything else in your own words, synthetic fixtures), the contributions we do not accept (request signing, client
impersonation, cookie logins, proxy rotation, CAPTCHA solving, media downloads) and the code conventions.

## Documentation

| Document | What it covers |
|---|---|
| [docs/architecture.md](docs/architecture.md) | tiers, the context, the data model, a run, the judge and calibration, jobs, surfaces |
| [docs/api-compat.md](docs/api-compat.md) | the compatible `/v1` REST subset |
| [docs/mcp.md](docs/mcp.md) | MCP tools, transports and client setup |
| [docs/judge.md](docs/judge.md) | judge backends, calibration, thresholds and evals |
| [docs/plugins.md](docs/plugins.md) | the built-in collectors, and writing a collector plugin |
| [docs/ui.md](docs/ui.md) | the web UI screens and keyboard shortcuts |
| [DATA-SOURCES.md](DATA-SOURCES.md) | every collector, its default and its risks, and what hookline never does |
| [packages/collector-ytdlp/README.md](packages/collector-ytdlp/README.md) | the opt-in yt-dlp plugin: what it reads, how to turn it on, its options |
| [.env.example](.env.example), [hookline.config.example.json](hookline.config.example.json) | every environment variable, and every config key at its default |
| [examples/mcp-clients.json](examples/mcp-clients.json) | MCP client snippets (stdio and HTTP) |
| [SECURITY.md](SECURITY.md) | threat model and how to report a vulnerability |
| [CONTRIBUTING.md](CONTRIBUTING.md) | clean-room rules, contributions we do not accept, how to run the checks |
| [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md), [PROVENANCE.md](PROVENANCE.md) | licenses, method credits, compatibility sources |
| [CHANGELOG.md](CHANGELOG.md) | release notes |

## License

MIT, see [LICENSE](LICENSE). Dependencies keep their own licenses ([THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)).