Skip to main content
Glama
zoebnomi
by zoeb-nomi

zoebnomi

A free check catches every citation that points at the wrong document. It cannot catch one that misstates the right one. On the held-out half of CrossSource's labelled legal citations, word overlap caught 10 of 10 wrong-document citations and was right on all 16 calls it made, with no bad citation ever passed as supported. The 30 citations that were faithful or quietly misstated (a reversed holding, a swapped standard) looked identical to it. So check_citations decides the first kind for free and hands the second kind to a judge, with CrossSource's own judge prompt.

An MCP server, a CLI and a Claude Skill from Zoeb Nomi. Every tool is read-only and deterministic, and costs nothing to call. No API keys and no model calls on the server.

Try it in 60 seconds

Add https://mcp.zoebnomi.com/mcp as a connector (Claude: Customize > Connectors > Add custom connector, "No sign in"; Claude Code: claude mcp add --transport http zoebnomi https://mcp.zoebnomi.com/mcp), then ask:

  1. "Use check_citations on the answer and sources in https://github.com/zoeb-nomi/zoebnomi-mcp/blob/main/examples/answer.json" (or paste your own answer with [id] citations and its sources). On the example:

    • [smith:1] is a wrong_document: the cited passage shares 0% of the claim's words; [jones:0], a different case, shares 83%.

    • [smith:9] is an unsupported_claim: that id is not among the sources (fabricated or mis-copied).

    • [smith:0] needs_judgment, with the judge prompt attached: overlap alone cannot tell support from a misstatement.

  2. "verify_claim: CrossSource strict prompting lifted citation precision to 0.999" returns number_mismatch: the site says 0.981 to 0.994 at n = 25, with the source link.

  3. "fit_to_role" with a job description returns which requirement lines have public evidence (quoted, with links) and which have none, stated as counts.

The portfolio is the source. The tools read zoebnomi.com live (llms.txt and the case-study pages), cached for 15 minutes. A new build, essay or note on the site shows up in verify_claim, fit_to_role and the resources with no release. If the site doesn't answer, each page falls back to a snapshot bundled at build time. On top of the live text sits a small set of curated claims with n and date. Each one is kept only while its exact quote is still on the site, and /health lists any that have dropped out.

Tool

What it does

check_citations(answer, sources)

For each inline citation ([doc_id:chunk_id] or [id]), decides what word overlap can decide and hands the rest back. It uses CrossSource's claim-span pairing and failure taxonomy (supported, right_doc_wrong_passage, wrong_document, unsupported_claim). A cite to an id that isn't in sources, or one where another passage clearly covers the claim, gets a deterministic verdict with the reason. Everything else returns needs_judgment with CrossSource's judge prompt, for your model to apply. With use_client_model: true your client's model judges them through MCP sampling (stdio clients that support it).

verify_claim(claim)

Checks a claim or number about Zoeb Nomi against what zoebnomi.com says right now. Returns found, number_mismatch or not_found, with the site's quote, source URL, evidence link, and n and date where the site gives them.

fit_to_role(job_description)

Maps a JD to his public work with quoted evidence picked from the live site, and lists requirements with no public evidence as gaps. It never answers compensation, availability, location or work authorization.

The lead tool packages the citation check from CrossSource, his open eval harness for citation accuracy in legal RAG, so any agent can call it.

Resources, all generated from the live site: zoebnomi://llms.txt, zoebnomi://case-studies (every case study with its key facts), zoebnomi://writing (every essay and note with its finding), and zoebnomi://harness, a live, read-only summary from the public agent ledger behind the harness panel on zoebnomi.com.

Related MCP server: voltcrawl

Eval of the tools themselves

npm run eval is deterministic and offline. Full tables are in eval/RESULTS.md.

check_citations is scored on CrossSource v0.3 citation_check: 75 items with final adjudicated labels, each a claim, a cited chunk and 5 context chunks. The split was fixed before tuning: 25 dev items, 50 test items. Thresholds were tuned on dev only.

Test split (n = 50)

Decided deterministically

16 of 50 (32%)

Correct when decided

16 of 16

Called "supported" wrongly

0 (it never says supported on its own)

wrong_document caught

10 of 10

right_doc_wrong_passage caught

6 of 10 (4 deferred)

supported and unsupported_claim

all 30 deferred to the judge, by design

Word overlap reliably catches a citation that points at the wrong place. It can't tell a faithful paraphrase from a reversed holding, so those go to a judge with CrossSource's prompt. For reference, the CrossSource harness reports that prompt with claude-sonnet-5-5 at citation_check accuracy 1.00 on a 30-item sanity sample, not a full run.

verify_claim is scored on 28 queries written before the matcher ran: 15 true claims, 6 with a wrong number, and 7 the site does not support. It runs against the same index the server builds from the site, made here from the snapshot so the run is reproducible. 26 of 28 pass. The first miss is vc-08, which matched the site's own note "Claude cited my site on 0 of 63 probes. Twice.". That is the same fact, but the fixed pass rule expects the curated quote, so it is scored as a miss. The second: "the eyecite pull request was merged" comes back found, quoting the site's line about the open, not merged pull requests. A word lookup can't see negation, so the tool tells callers to repeat the site's quote, not their own wording.

Install

The remote endpoint is live at https://mcp.zoebnomi.com/mcp (health: https://mcp.zoebnomi.com/health). The npm package is not published yet, so the npx zoebnomi lines work once it is; until then use the remote endpoint or the local build (see Develop).

Claude Code

claude mcp add --transport stdio zoebnomi -- npx -y zoebnomi mcp
claude mcp add --transport http zoebnomi https://mcp.zoebnomi.com/mcp

Claude (claude.ai, desktop): Customize > Connectors > Add custom connector, paste https://mcp.zoebnomi.com/mcp, and choose "No sign in".

ChatGPT: add a custom MCP server with the endpoint https://mcp.zoebnomi.com/mcp and no authentication. OpenAI's docs describe this flow under Plugins ("Add custom MCP server"). Older guides call it developer mode. Check the current path in OpenAI's docs.

Cursor: add this to .cursor/mcp.json or ~/.cursor/mcp.json:

{ "mcpServers": { "zoebnomi": { "command": "npx", "args": ["-y", "zoebnomi", "mcp"] } } }

or use the install link cursor://anysphere.cursor-deeplink/mcp/install?name=zoebnomi&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsInpvZWJub21pIiwibWNwIl19.

MCP Registry: listed as io.github.zoeb-nomi/zoebnomi once published (server.json).

Claude Skill: skill/evals-the-zoeb-way covers golden datasets, judge calibration and the citation check. In Claude Code, copy the folder to ~/.claude/skills/. In claude.ai, zip the folder and upload it under Customize > Skills.

CLI

npx zoebnomi
npx zoebnomi check-citations answer.json --strict
npx zoebnomi verify "CrossSource lifted citation precision to 0.994"
npx zoebnomi fit jd.txt
  • With no command, it prints the card.

  • check-citations takes a file holding {"answer": "...", "sources": [{"id": "...", "text": "..."}]}, or - to read stdin. --strict exits 1 when any citation is decided as a failure, so it can gate CI. --json prints the full result, judge prompts included.

  • npx zoebnomi mcp runs the MCP server on stdio.

Develop

Node 20+. TypeScript 7, the MCP TypeScript SDK, and no other runtime dependencies besides zod.

npm install
npm test
npm run eval
npm run build
npm run worker:dry-run && node scripts/smoke.ts bundle .wrangler-dry/worker.js
  • npm test: 39 tests. They cover the tools and the MCP client end to end, over the in-memory/stdio path and over Streamable HTTP through the Worker handler, plus MCP sampling. Live behaviour is tested against a fake site: a new note is found and listed, a changed number drops the curated claim, a dead site falls back to the snapshot, and contact lines are redacted. House rules are tests too: every quote must be on the site, and there must be no dashes, banned overstatements or personal details.

  • npm run worker:dry-run bundles the Worker without contacting Cloudflare. The smoke script then lists and calls every tool and reads every resource against that bundle.

  • Deploys: Cloudflare Workers Builds deploys main to mcp.zoebnomi.com after npm ci && npm test pass; a failing test stops the deploy. Rollback is the previous version in the Worker's Deployments.

  • node scripts/smoke.ts https://mcp.zoebnomi.com is the post-deploy check. GET https://mcp.zoebnomi.com/health is the heartbeat.

  • npx @modelcontextprotocol/inspector --cli node dist/stdio.js --method tools/list runs the official Inspector against the stdio build.

  • npm run snapshot refreshes the fallback snapshot. If a curated quote has left the site, npm test then fails; live, /health shows it under curated_dropped.

Layout: src/server.ts (tools and resources), src/worker.ts (Cloudflare Worker: /mcp stateless Streamable HTTP, /health), src/stdio.ts, src/cli.ts, src/site.ts (live read, cache, fallback, redaction), src/siteindex.ts (parses the site into quotable entries, layers the curated claims), src/citations.ts, src/verify.ts, src/fit.ts, src/data/claims.ts (curated claims), src/data/snapshot.ts (generated fallback), eval/, skill/.

Provenance and limits

  • Facts come only from public zoebnomi.com text (llms.txt and the case-study pages), read live with a snapshot fallback. Every response says which one it used (index.source: live, snapshot or mixed). Contact, booking and location lines are removed before any tool sees the text.

  • Cost: no model calls. Each process or Worker isolate fetches four small pages at most once every 15 minutes, with a Cloudflare edge-cache hint, so traffic does not multiply requests to the site.

  • check_citations keeps CrossSource's pairing, taxonomy, missing-id rule and judge prompt. It differs in two places: it accepts any [id] as well as [doc_id:chunk_id], and it trims punctuation left over from the previous sentence.

  • fit_to_role maps keywords. A hit means related public work exists. It does not mean the requirement is fully met, so read the quotes.

  • Remote over HTTP is stateless, so MCP sampling (use_client_model) works over stdio only.

MIT. The CrossSource-derived eval data carries CrossSource's MIT notice (eval/data/README.md).

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      B
      maintenance
      Enables agents to verify whether citations exist and match canonical records, and whether URLs resolve and contain expected content, with evidence-backed confirmed, contradicted, or unknown verdicts.
      2
      45 PyPI
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables coding agents to retrieve verified web evidence via headless Chromium rendering, sitemap navigation, and search candidate discovery, returning clean Markdown, structured JSON, and verbatim quoted proof.
      12 npm
      MIT
    • F
      license
      Not graded
      quality
      A
      maintenance
      Enables AI agents and applications to find authoritative primary data sources from natural-language queries, receive step-by-step website navigation guidance, and integrate evidence-backed external facts into their workflows.
      -