Recommend Agentic Trust Layer

A claim goes in. A truth score 0–100 and a separate, honest confidence come out — with the sources and the reasoning, not just a verdict.

The agentic trust layer for your AI stack: a deterministic verification pipeline your agents call before they act on something. Not another agent — the thing that tells your agents what's true.

Ask any LLM "how sure are you?" and it says ~95–100% on almost everything, including its mistakes. The product is not the answer — anyone can produce an answer. The product is what the answer is worth. Nothing here is answered from model memory: every judgement is made against text fetched live for that specific claim, and the confidence falls when the evidence is thin, stale, or argues with itself.

Two numbers, deliberately separate:

• Truth score 0–100 — above 50 leans true. This is about the claim.

• Confidence — how much to trust our verdict. Falls when lanes disagree, when a lane comes back empty, when the adversarial pass breaks a lane, or when the claim is the kind evidence cannot settle.

No framework, no build step: one Python stdlib server (server.py), one HTML file (index.html), SSE streaming so you watch the evidence lanes land live.

Quickstart

git clone https://github.com/recommend-dev/recommend-agentic-trust-layer.git
cd recommend-agentic-trust-layer
pip install -r requirements.txt        # just `requests` (+ google-auth for the Vertex path)
cp .env.example .env                   # add GEMINI_API_KEY + EXA_API_KEY — that's the minimum
python3 server.py                      # → http://localhost:8899

Works with two keys: a free Gemini key (aistudio.google.com/apikey) and an Exa key. Better with more — a lane whose key is missing simply sits the check out and the verdict says so (a silent lane costs confidence; it never fakes coverage).

Related MCP server: stillos-notary-mcp

Pipeline

claim
 └─ 1. STRUCTURE (Gemini)  → normalized falsifiable sentence, claim_type, entities,
                             sub-claims, 3 query angles (one hunting contradiction)
 └─ 2. LANES (parallel, routed by claim_type)
        always      : Grounded Web · Semantic Web · Live Index
        + deep      : Deep Research            (checkbox)
        + predictive: Prediction Market        (auto)
        + causal /
          statistical: Research Literature     (auto)
 └─ 3. JUDGE per lane   — only against what THAT lane fetched, background knowledge forbidden
 └─ 3b CHALLENGE per lane — opposing counsel attacks it; if it breaks, strength × 0.55
 └─ 4. SUB-CLAIMS       — each atomic assertion rated against pooled evidence
 └─ 5. AGGREGATE        — strength-weighted; disagreement and silence both cost confidence
 └─ 6. CALIBRATION      — one-way ratchet: may only LOWER confidence, never raise it
 └─ 7. READOUT          — 2-3 plain sentences on what the evidence showed and why

Lane routing (deliberate)

Firing every source at every claim is slower and noisier. No prediction market exists for settled history; a news write-up doesn't settle cause-and-effect. Knowing which authority settles which kind of question is part of the design.

Behaviour that took work to get right

• Unfalsifiable claims never get a factual verdict. "X is going to attack Y" used to return REFUTED at 96%. Absence of reporting about the future is not proof. It now returns a capped-confidence lean with a note — "nobody credible is reporting this" is real information, but we don't pretend to close an open question.

• The intake step never corrects the claim. A false claim must be checked as asserted — silently flipping a myth into its debunking makes the system report the opposite of what was asked.

• A prediction market is belief, not fact. Its strength is capped so it can inform a lean but never carry a verdict on its own.

• Every prompt knows today's date. Without it, models guess — and penalise good evidence for being "in the future".

• (p_true or 0.5) when every lane said 0. 0.0 is falsy in Python. The classics.

MCP — give your agent a verify_claim tool

The whole pipeline is exposed as one MCP tool over streamable HTTP (JSON-RPC over POST /mcp), gated on a bearer key. A demo key is auto-generated into keys.json on first run and printed at startup (and shown in the UI on localhost).

claude mcp add --transport http recommend-trust \
  http://localhost:8899/mcp \
  --header "Authorization: Bearer <key from startup output>"

Returns verdict, score, confidence, per-lane breakdown with challenge flags, verified sub-claims with verdicts, and sources. Point your agent at it and stop letting it answer factual questions from memory.

Tracking

Any verdict can be pinned to your session and is re-checked on a schedule (TRACK_EVERY_H, default 24h), so you can watch a claim's score move as the evidence does.

Spend guards

Every check costs real API credit, so the server ships with per-IP and global daily caps (PER_IP_DAY=25, GLOBAL_DAY=400) — tune them in .env before putting an instance on a public host.

Tests

Real-Chrome E2E tests (macOS Chrome path is hardcoded in the scripts — adjust executablePath for your OS):

cd tests && npm install
node uitest.js      # UI invariants (modal, gauge, MCP card, vendor-name leak check)
node tracktest.js   # tracking flow
node cmptest.js     # compare endpoint
node scaletest.js   # layout at widths

Deploying behind a reverse proxy

The front-end derives every path from location.pathname, so it works at / locally and under any path prefix in production — don't reintroduce absolute /api/... paths. If your proxy buffers responses (Caddy, nginx), disable buffering for this route (Caddy: flush_interval -1) or SSE will never stream. Set HOST=0.0.0.0 to listen beyond localhost.

Verify a knowledge base (OKF) — two directions

Google's Open Knowledge Format v0.2 records who generated agent-written knowledge and who verified it — but "verified" is a name typed into a field. A signature, not a check, and any agent can type it. This repo ships the machine behind the field, in two complementary directions:

1. External — world truth (okf_verify.py)

"Is this claim actually true out there?" Extracts each concept's externally checkable claims and runs them through the full pipeline — live evidence lanes, grounded judge, adversarial challenge, calibrated confidence.

python3 okf_verify.py examples/okf-demo        # --dry to judge without writing

Three honest outcomes, all shown in examples/okf-demo:

2. Internal — citation integrity (okf_adapter.py)

"Does the source this document CITES actually state this?" No web search at all: the evidence is the bundle's own declared sources — bundle-relative files read directly, external URLs fetched full-text. This is the mode for internal knowledge bases, where the web has no opinion: policies, metric definitions, schema docs.

python3 okf_adapter.py path/to/bundle          # --dry · reports in <bundle>-reports/

Per-claim verdicts: supports · contradicts · related-only · no-evidence · unverifiable (unreachable source). A claim can be perfectly true and still fail here — true-but-miscited is an attribution failure, caught by an explicit "does the cited source state this?" check. Reasons outrank labels: a supports whose own reasoning denies support is downgraded mechanically. Output: a per-concept report plus a diff table (claims checked · survived · caught · reason), and only concepts where every cited claim survives get the stamp. Needs only a Gemini key — in our scale test a 26-concept bundle cost ~$0.02 end to end.

Both directions

• Stamps are spec-conformant and additive: a verified: {by: recommend-trust-layer/0.1, …} event (the machine-confirmed trust tier per SPEC §5.3) plus an x_verification extension block with per-claim results. Existing frontmatter is never deleted.

• The gate is the product: a verifier that stamps everything is a rubber stamp. Refuted, inconclusive and unverifiable concepts stay unstamped, with the reasons written down.

• Deprecated concepts and Attested Computations are noted and skipped — deterministic SQL checking is different machinery, and we don't pretend otherwise.

Upload UI

python3 okf_server.py                          # → http://localhost:8898

Drop .md concept files, watch claims extracted and evidence lanes land live, download the stamped file.

Roadmap

• Pluggable lanes. Add your own evidence lane (Brave Search, internal corpus, BigQuery schema checks, …) without touching the pipeline.

• Continuous verification. Re-run on a schedule and on every bundle change; pair with stale_after so verification ages honestly.

License

MIT · built by Recommend