ietf-mcp-worker

An IETF research MCP for Cloudflare Workers, written in TypeScript and Effect. Public readers share corpora, origin fetches, refresh jobs, and embedding work. Code mode is the default: discover an operation, compose calls in JavaScript, and return only the evidence the model needs.

Based on mnot/ietf-llm at 63b61f4f289db7b53201d9153760924a04890114 and its code-mode proposal. This is a new storage and execution design, not a Python compatibility layer. See coverage for differences and remaining limitations.

Run locally

Requires Node 24 or newer.

npm ci
printf 'ADMIN_TOKEN=replace-with-a-local-secret\n' > .dev.vars
npm run dev

Connect an MCP client to http://localhost:8787/mcp. Reads are public, and anyone can gather a missing corpus with start_gather (for example {"name":"httpbis"}). Public gathers draw on a shared daily budget (DAILY_GATHERS), and a corpus finished in the last 24 hours is returned as is. The operator token is required to cancel jobs, set refresh schedules, change an operator-registered corpus, and gather the rfcs and groups mirrors.

A group covered by the ietf-llm seed store starts from its verified prebuilt snapshot, readable within minutes. The full gather starts automatically afterwards and replaces it when it publishes.

Each full gather then derives two record kinds. people merges author aliases using Datatracker person ids (from email addresses and GitHub usernames), then identical multi-word display names. topics clusters threads and issues with k-means over Workers AI embeddings. summarize writes an on-demand, cached summary of a thread or issue. grep_corpus also accepts RE2 regular expressions. get_rfc_info returns the full RFC reference graph from rfc.fyi.

export IETF_MCP_URL=http://localhost:8787/mcp
export IETF_MCP_TOKEN=replace-with-a-local-secret
node scripts/client.ts gather '{"name":"httpbis","group":"httpbis","since":"2026-01-01","refreshHours":24}'
node scripts/client.ts status httpbis
node scripts/client.ts search httpbis 'cache invalidation'

For a small first smoke test, gather {"name":"http-caching","rfcs":[9111]}. Wait for job.state to be done. A failed refresh keeps the previous snapshot. Full groups can involve thousands of fetches; the alarm processes one persisted task at a time. Check status for the exact failing source when a gather fails.

Custom corpora accept drafts, lists, repos, and rfcs, in addition to group. A name without sources is interpreted as a group. Standalone lists should explicitly set lists. groups gathers the chartered-group mirror; rfcs gathers the published RFC series. Both are substantial initial jobs.

// execute tool: model-written JavaScript, isolated from credentials and networking
async () => {
  const result = await ietf.call("chunks", {
    corpus: "httpbis",
    kind: "threads",
    since: "2026-01-01",
    chars: 0,
    limit: 100,
  });
  return {
    sendersOnThisPage: [...new Set(result.items.map((item) => item.author))],
    nextOffset: result.nextOffset,
  };
};

search discovers operation names and schemas; it is not a corpus search. The generated function calls ietf.call(operationName, arguments). Operations include orientation, search, exact matching, structured records, drafts, RFCs, issues/PRs, meetings, live Datatracker records, and norms. Use /mcp/classic to expose these operations directly without sandbox charges.

Related MCP server: agentic-rag

Skills, CLI and export

The four upstream IETF skills and their reference documents are preserved with attribution. They are exposed as MCP resources, skill prompts, and read_skill.

node scripts/client.ts install-skills .agents/skills
node scripts/client.ts stdio
node scripts/client.ts call overview '{"corpus":"httpbis"}'
node scripts/client.ts export httpbis ./httpbis-export

The stdio command bridges clients to the same hosted data. Export produces Markdown files and a citation manifest suitable for manual NotebookLM upload. Installation and export refuse to overwrite existing files.

Deploy

npm run check
npx wrangler login
npx wrangler secret put ADMIN_TOKEN
npm run deploy

For a free temporary preview, use npm run deploy:temporary. This uses wrangler.preview.jsonc, which omits Worker Loader and scheduled refresh. The preview exposes direct tools at /mcp; production keeps code mode as the default. Cloudflare rejected native code mode on the temporary account with error 10195: Dynamic Workers require a paid plan. Set the preview's operator secret with npx wrangler secret put ADMIN_TOKEN --temporary --config wrangler.preview.jsonc. Temporary credentials and claim URLs belong outside this repository; unclaimed previews expire after 60 minutes.

The default configuration has no model calls or Vectorize dependency. Search explicitly reports lexical ranking until semantic search is configured:

npx wrangler vectorize create ietf-passages --dimensions=384 --metric=cosine

Add these entries to wrangler.jsonc, and add "EMBEDDINGS": "on" to vars:

{
  "ai": { "binding": "AI" },
  "vectorize": [{ "binding": "VECTORS", "index_name": "ietf-passages" }]
}

Redeploy and gather the desired corpora. Workers AI uses @cf/baai/bge-small-en-v1.5 (384 dimensions). Text embeddings are cached by model and content hash across corpora; unchanged documents avoid Vectorize upserts. Vector results are checked against the published text hash so staged or obsolete records cannot leak into answers. Vectorize is eventually consistent. An optional GITHUB_TOKEN secret increases GitHub's upstream rate allowance.

Bounds and costs

• Public requests: 30/minute per client IP, enforced by Cloudflare's regional rate limiter. This is not a globally exact per-IP quota.

• Code mode: 100 executions/day globally by default; set DAILY_EXECUTIONS to change it. The counter is atomic across users. Classic reads still work when the budget is exhausted.

• Each execution: 100 ms CPU, 10 seconds async deadline, 40 host operations, no outbound network, 16,000 code characters, 24,000 response characters.

• Fetches: two concurrent requests per origin, conditional GET, shared cache, bounded retries, and shared Retry-After cooldown.

• Sources: 8 MiB fetched body, 300 PDF pages, 1.9 MB document row; exceeding a bound fails the refresh instead of silently publishing an incomplete result.

At 100 executions/day for 30 days, plan for about $9/month, plus storage, AI, and other usage beyond included allowances. The dominant variable is the Dynamic Worker creation fee, not embedding inference. See the assumptions, larger scenarios, and optimization options in costs.

Verify

npm run check
node scripts/cost.ts 3000
# Against a running local or deployed service, with operator credentials:
node --env-file=.dev.vars scripts/smoke.ts

Tests run in Cloudflare's local runtime and cover writer-to-reader ingestion, shared-fetch deduplication, failed/cancelled refreshes, atomic execution quotas, PDF text extraction, pagination, trust boundaries, and actual sandbox/MCP calls. Live Workers AI and Vectorize require a configured Cloudflare account. See the validation record for what passed locally and on the temporary Cloudflare deployment.

Design

Corpus Durable Objects own each corpus's SQLite records, FTS index, task queue, and publication pointer. SharedFetch objects coordinate origin traffic and cache HTTP/mail/embedding results. Effect models upstream effects, retries, timeouts, and concurrency. MCP uses the official SDK and Cloudflare Code Mode. There is no per-user corpus, container, polling server, or always-on process. See architecture.

The software is MIT licensed with upstream attribution. Vendored skill text is CC-BY-4.0; reproduced RFCs retain their separate notices. See skills/VENDORED.md.