# Netscribe

A Chrome (Manifest V3) extension that captures network traffic across your tabs
and lets you search it with a query language — without opening DevTools.

> **Netscribe was formerly Request Finder — they're the same project.** The old
> name is still the internal identifier throughout, and that's deliberate: it's
> what the MCP server is registered as and where the daemon keeps its state, so
> renaming it would break existing setups. Wherever you see `request-finder`,
> read *Netscribe*:
>
> | Where | Value |
> |-------|-------|
> | MCP server id | `request-finder` (in `claude mcp add`) |
> | Daemon state | `~/.request-finder/` (token, `history.db`, logs) |
> | npm package / scripts | `request-finder`, `npm run bridge:uninstall` |
> | Extension name in Chrome | Request Finder |

**What it captures**, across every tab, all the time:

- **HTTP API calls** — XHR/fetch and ordinary page loads: URL, method, status,
  headers, and timing.
- **GraphQL** — captured like any other HTTP call (typically `POST /graphql`).
  Under Deep Capture its query and variables are searchable body text, so
  `body:` will match an operation name.
- **RPC** — JSON-shaped RPC (JSON-RPC, tRPC, Connect in JSON mode) is captured
  in full, bodies included. Protobuf-shaped RPC (gRPC-Web, Connect in proto
  mode) is captured too — URL, status, headers, timing — but its body is binary
  protobuf, so `body:` won't match it.
- **WebSockets** — the handshake plus the individual frames sent and received
  (ADR 0006), including binary ones. RPC tunneled over a socket is searchable
  frame by frame.

**Ask Claude about a request.** With the bridge daemon running, Netscribe
exposes your captured traffic to Claude Code over MCP — so you can ask "why did
this call 401?" or "what changed between these two requests?" and Claude can
read the real headers and bodies instead of guessing. Each request has a
**Request Ref** you can paste to point Claude at one exact call. See
[Claude Code integration](#claude-code-integration-bridge-daemon) below.

The search UI lives in a dedicated full-tab page — **click the Request Finder
toolbar icon** to open it.

> **Privacy note:** Netscribe records request/response headers (including
> `Authorization`, `Cookie`, API keys) and — under Deep Capture — request and
> response bodies, in plaintext IndexedDB on your machine. It is a personal
> debugging tool for your own traffic. Use the **Discard list** (domains never
> captured — the button right of Purge all), **Pause**, and **Purge all**
> guardrails, and the secret-masking toggle. Purge all clears the extension's
> history and, if you tick the box, the Claude/MCP daemon's history too
> (otherwise the daemon keeps its own copy — ADR 0008). See `docs/adr/0003`.

## Install & build

A `Makefile` wraps the npm scripts, so building the extension is one command:

```bash
make               # fresh build → dist/, ready to load unpacked
```

`make` wipes `dist/`, installs dependencies if a lockfile has moved, then runs
the typecheck and production build. Other targets (`make help` lists them all):

```bash
make dev           # Vite + CRXJS watch build
make test          # Vitest unit suite (parser, evaluator, index, IDB)
make test-e2e      # Playwright smoke (builds first — it needs dist/)
make check         # unit suite + a fresh build
make clean         # drop build output; `make distclean` also drops node_modules
```

The underlying `npm run build` / `npm test` scripts still work if you prefer them.

Load the unpacked extension: `chrome://extensions` → Developer mode → **Load
unpacked** → select `dist/`.

## Capture model

| Path | API | Scope | Captures | Cost |
|------|-----|-------|----------|------|
| **Metadata Capture** | `chrome.webRequest` | all tabs, always on | URL, method, headers, status, timing | silent |
| **Deep Capture** | `chrome.debugger` | opt-in, per tab | the above **+ request/response bodies** | shows a debugging banner; can't coexist with DevTools |

`body:` and `response:` only match Deep-Captured requests and degrade
gracefully on the rest (ADR 0001). Storage is a ring buffer (default 50k,
configurable) — the oldest request is evicted past the cap (ADR 0002).

<img width="2880" height="1534" alt="Screenshot_2026-07-20_blurred" src="https://github.com/user-attachments/assets/7a000901-479d-4369-8ae5-5119521c767f" />


## Query language

```
method:POST has:authorization          # space = AND
status:500 OR status:401               # explicit OR (binds looser than AND)
domain:slack.com -status:200           # '-' negates
(status:500 OR status:401) method:POST # parentheses group
```

| Operator | Meaning |
|----------|---------|
| `has:<name>` | header present (request or response) |
| `header:<name>` | header present (alias of `has:`) |
| `header:<name>=<value>` | header value contains `<value>` |
| `method:<m>` | exact method |
| `status:<spec>` | `500`, class `5xx`, or comparison `>=400` / `<500` |
| `url:<substr>` | substring of the full request URL |
| `page:<substr>` | substring of the page URL the request fired from (live SPA route) |
| `domain:<host>` | host or any subdomain (`slack.com` matches `api.slack.com`) |
| `cookie:<name>` | cookie present in the request `Cookie` header |
| `tab:<substr>` | substring of the originating tab's title (as captured) |
| `is:ws` / `is:sent` / `is:received` / `is:http` | WebSocket frames (any / sent / received) or HTTP records |
| `body:"…"` | substring of the **request** body (HTTP, Deep Capture only) or a **sent** WS frame's payload (always captured) |
| `response:"…"` | substring of the **response** body (HTTP, Deep Capture only) or a **received** WS frame's payload (always captured) |
| bare word | substring across URL, headers, bodies, and frame payloads |

WebSocket **messages are captured by default** — a content script wraps
`window.WebSocket` (ADR 0006/0007), so frames are recorded with no debugger, no
banner, and **even while DevTools is open**. Deep Capture is *not* involved in
WS frames at all (it stays HTTP-only, for response bodies).

The Finder Page has a top-level **Requests | WebSockets** switch. The
**WebSockets** view shows frames only — `▲ WS` sent / `▼ WS` received — and the
search box takes the same operators: `response:"chat_message"` (received
payload), `is:sent`, `domain:hiver.space`, `tab:outlook`. The handshake itself
appears in the **Requests** view as a `websocket` request (recorded at the
`101`, since a live socket never "completes").

> The content script must be in place before the page opens its socket, so
> **reload the page once** after installing/reloading the extension. Frames have
> their own ring-buffer budget (`maxFrames`, default **100k**) so a chatty
> socket can't evict HTTP history (`maxRequests`, default 50k).
>
> Heartbeat frames (JSON `{"type":"ping"}` / `"pong"`) are **dropped at capture
> by default** to cut noise and save the frame budget. The WebSockets toolbar
> has a **Record pings** checkbox to keep them.
>
> Binary frames are **size-only by default**. The **Log binary** checkbox
> captures their payloads — decoded as text when the bytes are valid UTF-8 (so
> text-as-Blob stays searchable), else base64 — capped at `maxBodyBytes`. When
> off, the bytes aren't even read/encoded.

Matching is case-insensitive. Use `"quotes"` for values with spaces.

<img width="2880" height="1534" alt="Screenshot_2026-07-20_1525_blurred" src="https://github.com/user-attachments/assets/705d09f6-97af-4a86-b497-97b31aec654e" />
<img width="1440" height="767" alt="Screenshot 2026-07-20 at 15 21 39" src="https://github.com/user-attachments/assets/05702f6c-85a7-4270-9e71-ae9f7d6240b0" />


## Architecture

```
service worker ── webRequest / debugger ──▶ IndexedDB (source of truth, ring buffer)
      │                                            ▲
      └── live feed (port) ──▶ Finder Page ── load ┘
                                   │
                                   ├─ Web Worker: in-memory index (URL + headers
                                   │   + structured fields), evaluates queries.
                                   │   Body-dependent matches come back as
                                   │   "needs body scan" …
                                   └─ … which the page resolves (Pass 2) by
                                       reading full records from IndexedDB.
```

The query evaluator is one three-valued (Kleene) function run twice: in the
Worker against in-RAM data (body text → `MAYBE`), then against full records from
IndexedDB to resolve the `MAYBE`s.

### Layout

```
src/
  background/   service worker: webRequest + debugger capture, batched IDB writes
  shared/       types, IndexedDB layer, query engine (tokenizer→parser→evaluate)
  worker/       in-memory RequestIndex + Worker entry
  finder/       React full-tab UI (search, virtualized list, detail panel)
docs/adr/       architecture decision records
CONTEXT.md      glossary (ubiquitous language)
```

## Claude Code integration (bridge daemon)

A small always-on local daemon (`bridge/`) lets Claude Code **watch traffic
live, query history, and act** (toggle Deep Capture, pause, purge) over MCP.
The extension streams captures to it over a localhost WebSocket; the daemon
keeps its own durable SQLite history and exposes an MCP server. See ADR 0004.

```
extension SW ═══WS═══▶ bridge daemon ◀═══MCP/HTTP═══ Claude Code
             ◀══WS═══  (actions: deep-capture, pause, purge)
```

### Setup

```bash
cd bridge && npm install            # native better-sqlite3 build
cd .. && npm run bridge             # starts the daemon on 127.0.0.1:8787
```

The daemon prints a **token** on first run (and on every start; it's stable,
stored at `~/.request-finder/token`). Then:

1. In the Finder Page → **⚙ Bridge** → paste the token, set the URL
   (`ws://127.0.0.1:8787/ws`), and check **Enable bridge**.
2. Register with Claude Code (the daemon prints this exact line):
   ```bash
   claude mcp add --transport http request-finder http://127.0.0.1:8787/mcp \
     --header "Authorization: Bearer <token>"
   ```
   Or use the committed `.mcp.json` with `export RF_BRIDGE_TOKEN=<token>`.

### MCP tools

| Tool | What it does |
|------|--------------|
| `requests_search` | search history with the query language (newest-first) |
| `request_get` | full record by Request Ref `rid` (the `rf_…` the UI's "Copy for Claude" yields) or `seq` |
| `requests_tail` | cursor long-poll for live watching (call in a loop) |
| `tabs_list` | tabs the extension sees + Deep Capture state |
| `stats` | counts, time range, connection + Deep Capture status |
| `deep_capture_set` | turn Deep Capture on/off for a tab (to see bodies) |
| `capture_set_paused` | pause/resume Metadata Capture |
| `history_purge` | wipe daemon + extension history (`confirm: true`) |

**Security (ADR 0004):** loopback bind only, a shared token on both the
WebSocket and every MCP call, and an Origin check that rejects web-page
WebSocket connections. The daemon serves captured secrets — keep the token
private; `history_purge` clears everything.

### Always-on (macOS LaunchAgent)

Instead of `npm run bridge` each time, run the daemon at login and keep it
alive (`deploy/com.requestfinder.bridge.plist`, launched via
`bridge/run-daemon.sh` which resolves your nvm Node):

```bash
cp deploy/com.requestfinder.bridge.plist ~/Library/LaunchAgents/
launchctl load ~/Library/LaunchAgents/com.requestfinder.bridge.plist   # starts now + every login
# stop/uninstall:
launchctl unload ~/Library/LaunchAgents/com.requestfinder.bridge.plist
```

Logs: `~/.request-finder/daemon.log` (and `daemon.err.log`). On another machine,
edit the two absolute paths in the plist.

### The `watch-requests` skill

`.claude/skills/watch-requests/SKILL.md` teaches Claude Code the common
workflows over these MCP tools — "tail any 5xx on the gmail tab", "find the
auth-bearing calls to api.slack.com and show me one", "deep-capture tab 42 and
inspect the failing response". It activates automatically when you ask for that
kind of thing (the `request-finder` MCP server must be registered).

## Running it day-to-day

| Part | When | How |
|------|------|-----|
| **Extension** | always (whenever Chrome runs) | Load unpacked from `dist/` **once**; Chrome remembers it across reboots. After `make`, click **↻** on the extension card. The Finder Page (toolbar icon) works on its own. |
| **Bridge daemon** | only for Claude | LaunchAgent (above) runs it at login; or `npm run bridge` on demand. |
| **Claude registration** | once | `claude mcp add …` + paste the token into the extension's ⚙ Bridge panel. Persists. |

## Uninstall / cleanup

Two separate stores:

- **Extension data** — cleared **automatically** by Chrome when you remove the
  extension (its IndexedDB + settings). Nothing to do.
- **Bridge daemon** — a separate local process + SQLite at `~/.request-finder`,
  independent of the extension; uninstalling the extension does **not** touch
  it. The extension can't purge it on uninstall (no code runs then). Clean it
  one of two ways:
  - **Before uninstalling:** Purge all → tick **Claude/MCP (daemon) history**.
  - **Any time:** `npm run bridge:uninstall` — removes **only** Netscribe's
    own artifacts: its LaunchAgent, its running daemon (only if the process on
    :8787 is Netscribe's), and `~/.request-finder` (history.db, token,
    logs). It touches nothing else.