Skip to main content
Glama
searchstefano

agenticschema

AgenticSchema · GitHub license npm version PRs Welcome

Turn any webpage into typed JSON, callable MCP / WebMCP tools, or AI-native Epistemic Context — in 3 ms, using 2.7× to 28× fewer tokens than a text scraper and up to 1,449× fewer than raw HTML.

Human scrapers treat a webpage as a flat text document. An LLM doing Live Search experiences the web completely differently: it already has language in its pre-trained weights, suffers from Transformer self-attention dilution on rhetorical filler, is vulnerable to temporal drift on old articles, and needs active navigation toward hidden APIs and primary sources. AgenticSchema closes that gap for both sides of the web:

  • For AI Live Search, scrapers & agent pipelines (@agenticschema/core, @agenticschema/server): Extract normalized entities, Dual-Plane Canonical Evidence Graphs (evidence) with zero-bloat Inline Epistemic Sigils ([✓S1], [~S2], [#S3], [§S4]), Multi-Facet Epistemic Sufficiency Verdicts (sufficiency: ANSWERABLE | PARTIAL_ESCALATE | ABSTAIN), Surprisal-Distilled Epistemic Atoms (atoms), Relativistic Temporal Cones (temporal), Shadow JSON APIs (machineTwins), Information-Foraging Frontier Hops (frontier), Adaptive L0 -> L1 -> L2 Wavefronts (forageUrls), and Copy-Discounted Bayesian Truth Fusion (fuseUrls).

  • For website owners (@agenticschema/browser): Drop one <script> tag onto your site to expose your existing markup as W3C WebMCP tools on document.modelContext. You write no new API and run no backend.

                         Website / HTML (442 KB mean)
                                      │
          JSON-LD · Microdata · RDFa · OpenGraph · <dl>/<table> · __NEXT_DATA__
                                      ▼
                            AgenticSchema (3 ms)
                                      │
      ┌───────────────────┬───────────┴───────────┬───────────────────┐
      ▼                   ▼                       ▼                   ▼
  L0 Pulse           Submodular b=120        L1 Epistemic        MCP / WebMCP
 (99 tokens)          (110 tokens)           (413 tokens)       (1,013 tokens)
 1,449× vs HTML      1,305× vs HTML          348× vs HTML        142× vs HTML
      └───────────────────┴───────────┬───────────┴───────────────────┘
                                      ▼
                     AI Agent (or npx @agenticschema/server --live)

Why AgenticSchema beats normal scraping: the numbers

Everything below is measured and reproducible from this repository (npm run test:corpus and npm run bench:report) across 177 real pages pulled from Common Crawl (shops, news, recipes, reference, books) and 120 agent trials.

1. Token & payload compression vs. HTML and text scrapers

Tokens per page, averaged across all 177 real pages (442.0 KB mean raw HTML), counted with o200k_base:

What the model reads

Mean tokens / page

Mean size / page

vs. Raw HTML

vs. Competent text scraper

Raw HTML, as served

143,770

442.0 KB

—

—

Extracted text (Readability-style scraper: <script>, <style>, markup stripped)

2,752

~11.2 KB

52×

—

AgenticSchema MCP tools (toTools — full deduplicated entities + tools)

1,013

3.0 KB

142×

2.7×

AgenticSchema toContext L1 (Epistemic Core: Schema + Specs + Surprisal Atoms + Inline Sigils)

413

1.2 KB

348×

6.7×

AgenticSchema toContext Submodular (Multi-Facet Query Bottleneck, hard budget=120)

110

0.4 KB

1,305×

25×

AgenticSchema toContext L0 Pulse (1-line Semantic DNA strand with [✓S1]/[~S2] sigils)

99

276 B

1,449×

28×

Nobody serious feeds raw HTML to a model: a competent scraper strips the markup first, accounting for 52× of the reduction. Against a competent text scraper, AgenticSchema cuts token consumption by another 2.7× (full MCP tools), 6.7× (L1 Epistemic Core), 25× (Multi-Facet Submodular Query Bottleneck under a strict 120-token ceiling), and 28× (L0 Holographic Pulse) — while returning machine-typed fields and verified epistemic claims instead of unstructured paragraphs:

Vertical

Pages

Raw HTML

Text scraper

MCP tools

L1 Epistemic

Submodular (b=120)

L0 Pulse

book (Goodreads)

2

287,454

11,534

191 (61×)

460 (25×)

100 (115×)

95 (121×)

reference (Wikipedia, MDN)

50

62,730

3,699

208 (18×)

550 (6.7×)

108 (34×)

121 (31×)

ecommerce (IKEA, Patagonia, Backcountry)

75

240,151

2,912

1,567 (1.9×)

412 (7.1×)

112 (26×)

99 (29×)

news (Al Jazeera)

25

90,530

882

528 (1.7×)

259 (3.4×)

110 (8.0×)

67 (13×)

recipe (Marmiton)

25

58,454

1,549

1,514 (1.0×)

292 (5.3×)

109 (14×)

89 (17×)

All corpus pages (mean)

177

143,770

2,752

1,013 (2.7×)

413 (6.7×)

110 (25×)

99 (28×)

(Multipliers in parentheses are vs. a competent text scraper on the same vertical.)

2. AI-Native Live Search Signals extracted across the 177 real pages

Beyond schema normalization, toContext(), forageUrls(), and fuseUrls() extract seven AI-native layers that traditional text scrapers discard or miss completely:

AI-Native Layer

Measured on 177-page Corpus

What it gives the LLM during Live Search

Dual-Plane Evidence Graph & Inline Sigils (evidence)

100% of structured & tabular facts

Typed EvidenceNode[] with 4D confidence $C=(C_{\text{extraction}}, C_{\text{agreement}}, C_{\text{source}}, C_{\text{freshness}})$ + zero-bloat sigils ([✓✓S0] cross-origin consensus, [✓S1] intra-page consistent $C_{\text{agreement}}=0.92$, [~S2] unverified markup, [#S3] <dl>/<table>/SSR, [§S4] prose atom, [!CONFLICT]).

Multi-Facet Sufficiency & Absence Taxonomy (sufficiency)

110 tokens mean (at budget=120)

Enforces $\text{ANSWERABLE} \iff \forall f_i \in F_{\text{required}}, \text{Evidence}(f_i) \ge \tau_i$, and diagnoses every facet as SATISFIED, NOT_PUBLISHED, NOT_FOUND, CONFLICTED, or UNVERIFIABLE.

Active Inference EVI Controller (controller)

$< 0.1\text{ ms}$ decision

Computes residual query entropy $H(Q \mid S)$ and Expected Value of Information ($\text{EVI}$) to choose STOP_AND_ANSWER, ESCALATE_L2, QUERY_MACHINE_TWIN, FOLLOW_FRONTIER, or ABSTAIN.

Surprisal-Distilled Epistemic Atoms (atoms)

679 verified atoms (3.8 / page)

Strips zero-surprise rhetorical filler and requires a non-parametric anchor (quantity/unit, constraint/negation, or temporal date) subordinated to query relevance.

Relativistic Temporal Light-Cone (temporal)

125 / 177 pages (71%)

Anchors publishedAt / modifiedAt / priceValidUntil, flags expired offers (⚠️EXPIRED_OFFER), and warns when older pages use indexical words ("currently", "this year", "latest").

Shadow APIs & Machine Twins (machineTwins)

102 endpoints discovered

Declaratively discovers same-origin JSON/API twins (wp-json, /llms.txt, /_next/data/...json, Shopify .json, feeds) with SSRF/private-IP and credential filtering.

Information-Foraging Frontier & Adaptive Wavefront (forageUrls)

361 high-scent hops

Scans all $N$ pages at L0 (99 tok), promotes top resonant pages to L1 (413 tok), and dynamically escalates to L2 only when sufficiency.missingFacets is non-empty.

3. Answer accuracy & hallucination rate (sonnet, 120 trials)

Fewer tokens only matter if the agent answers at least as well. Over 120 isolated trials (5 questions per page: single-fact, multi-part, and unanswerable questions designed to catch hallucinations), comparing an agent given the scraped page text against the same agent given only AgenticSchema MCP tools:

Benchmark metric

Competent text scraper (text)

AgenticSchema (tools)

Delta

Overall agreement (mappable pages, neutral referee)

85%

89%

+4 pt

E-commerce questions (including ProductGroup variants & prices)

100%

100%

11/11 prices

Recipe questions (recipeYield, timings, ingredients)

60%

80%

+20 pt

Two-part questions (e.g. "What is the rating, and what does it cost?")

83%

92%

+9 pt

Hallucinations on unanswerable questions (NOT_ON_PAGE)

10% (1/10)

0% (0/10)

Zero inventions

Mean cost per trial (API-equivalent USD)

$0.076

$0.050

−34% cost

Why the tools arm wins on complex and recipe questions: prose scatters facts across the page (or omits recipeYield from body text while declaring it in JSON-LD), causing a reader model to drop one half of a two-part question or lose a product name in boilerplate. Two targeted tool calls return exact fields with zero prose noise.

4. Runtime speed & web-scale coverage

Metric

Value

Why it matters

Core pipeline & Zero-DOM toContext() fast path (extract → guard)

3 ms / page (0 ms DOM parse)

Server/Edge toContext() and JSON-LD toTools() run in 3 ms directly on raw HTML strings without instantiating a DOM.

Full DOM construction (linkedom / happy-dom when Microdata/RDFa tree walk is needed)

~38–41 ms / page

Only paid on pages that require a full DOM tree walk (excluding network fetch time).

Streaming Early-Abort (earlyAbort: true)

Up to 90% less bandwidth

Aborts the HTTP stream right after </head> once structured data is captured.

Domains publishing SearchAction + urlTemplate

10,000,000+

Over 10M domains already declare machine-readable search endpoints (Google Crawl stats, 2026-07).

Browser CDN bundle (@agenticschema/browser)

30.6 KB gzip

Single self-contained classic IIFE script, W3C WebMCP polyfill included.


Related MCP server: neuron-inspector

Contents


Quick start: 3 ways to use it

1. Scrape any URL to structured JSON or MCP tools from the terminal

No browser and no configuration. Given a real product page:

world.openfoodfacts.org/product/3017620422003

  read    get_web_site
  read    get_organization
  action  search_web_site(search_term_string)

search_web_site is an executable tool derived automatically from the page's SearchAction. Run it as an MCP server for Claude Desktop / Cursor / Claude Code / Antigravity, or dump the structured JSON / Epistemic Context directly for Live Search:

# Start a persistent Live Search MCP server (exposes fetch_evidence, forage_urls, fuse_sources for ANY URL):
npx @agenticschema/server --live

# Or start a page-bound MCP server over stdio for one or more URLs (fetched concurrently):
npx @agenticschema/server https://world.openfoodfacts.org/product/3017620422003

# Or print a 1-turn Hybrid LLM Context (Structured Facts + Specs + Inline Sigils + Deduplicated Prose):
npx @agenticschema/server --context --early-abort https://en.wikipedia.org/wiki/Backpack

# Multi-Facet Submodular Bottleneck: project a page down to a strict 120-token ceiling (~110 tok mean):
npx @agenticschema/server --context --query "waterproof warranty weight" --budget 120 https://en.wikipedia.org/wiki/Backpack

# Holographic Multi-Resolution: get a ~99-token L0 Semantic DNA Pulse or ~413-token L1 Epistemic Core:
npx @agenticschema/server --context --resolution L0 https://en.wikipedia.org/wiki/Backpack

# Adaptive Foveated Wavefront: explore N search results (L0 -> L1 -> automatic L2 escalation on missing facets):
npx @agenticschema/server --forage --query "price weight battery" https://example.com/p1 https://example.com/p2 https://example.com/p3

# Copy-Discounted Knowledge Graph Fusion: fuse N search results via SimHash independence + Bayesian Consensus:
npx @agenticschema/server --fuse --query "price weight battery" https://example.com/p1 https://example.com/p2

# Or scrape any page directly to structured JSON (with OpenGraph/meta/form fallback):
npx @agenticschema/server --json --fallback https://en.wikipedia.org/wiki/Backpack

Drop-in Agent Skill included: Point Claude Code, Antigravity, Cursor, or Codex at skills/agentic-live-search/SKILL.md (also mirrored in .agent/skills/agentic-live-search/SKILL.md) so your AI coding/research agent automatically uses agenticschema (--live MCP or CLI --context/--forage/--fuse) instead of raw WebFetch.

Human scrapers treat a web page as a static text document. An LLM doing Live Search (WebFetch, OAI-SearchBot, Perplexity, RAG) experiences the web completely differently: it already has language in its pre-trained weights, suffers from Transformer attention dilution on rhetorical filler, is vulnerable to temporal drift on old articles, and needs active navigation toward hidden APIs and primary sources.

toContext(), forageUrls(), and fuseUrls() implement 7 AI-native paradigms:

  1. Dual-Plane Canonical Evidence Graph (evidence) & Zero-Bloat Inline Epistemic Sigils:

    • Exposes a full typed EvidenceNode[] graph in code with a 4D Epistemic Confidence Vector $C = (C_{\text{extraction}}, C_{\text{agreement}}, C_{\text{source}}, C_{\text{freshness}})$ while embedding 2-token Inline Epistemic Sigils in the LLM prompt:

      • [✓✓S0]: Cross-origin consensus corroborated across $\ge 2$ independent SimHash clusters in fuseUrls() ($P(v \mid \mathcal{D}) \ge 0.80$).

      • [✓S1]: Intra-page cross-representation consistency — structured Schema.org/Microdata/RDFa datum verified in visible DOM text ($C_{\text{agreement}} = 0.92$, ruling out SEO cloaking or template drift, without claiming 1.0 ontological truth from a single origin).

      • [~S2]: Unverified structured markup not echoed in visible prose ($C_{\text{agreement}} = 0.78$).

      • [#S3]: Tabular <dl>/<table> specification or SSR hydration state (__NEXT_DATA__).

      • [§S4]: High-surprisal quantitative/constraint/temporal prose atom.

      • [!CONFLICT]: Explicit contradiction between markup and visible text, or across sources.

  2. Multi-Facet Sufficiency Calculus (sufficiency) & Counterfactual Absence Taxonomy:

    • Decomposes multi-part queries into orthogonal Query Facets (facets), maximizes marginal epistemic utility $U(f, q) = R(f, q) \cdot C(f) \cdot T(f) \cdot N(f \mid S)$, and enforces the strict rule: $$\text{ANSWERABLE} \iff \forall f_i \in F_{\text{required}},\quad \text{Evidence}(f_i) \ge \tau_i \land \neg\text{Conflicted}(f_i)$$

    • Diagnoses every facet in sufficiency.facetDiagnostics as SATISFIED, NOT_PUBLISHED (structured entity exists but omits the property), NOT_FOUND (absent everywhere), CONFLICTED, or UNVERIFIABLE (expired/weak $< \tau_i$), preventing the LLM from confusing absence with non-existence.

  3. Active Inference & Expected Value of Information Controller (controller):

    • Computes residual query uncertainty $H(Q \mid S)$ and Expected Value of Information $\text{EVI}(a) = \frac{\mathbb{E}[\Delta I(a)] \times \text{Impact}(a)}{\text{Cost}{\text{tokens}}(a) + \text{Cost}{\text{latency}}(a)}$ to output the optimal next step: STOP_AND_ANSWER, ESCALATE_L2, QUERY_MACHINE_TWIN, FOLLOW_FRONTIER, REPORT_CONFLICT, or ABSTAIN.

  4. Adaptive Holographic Resolution (L0 -> L1 -> L2) & Foveated Wavefront (foragePages / forageUrls):

    • L0 Pulse (~99 tokens/page — 1,449× smaller than HTML, 28× smaller than text): A 1-line "Semantic DNA strand" ([L0 Product] Name | [✓S1]price=... | t=2025-11-10 | atoms: [§S4] ... | twin=...).

    • L1 Epistemic Core (~413 tokens/page — 6.7× smaller than text): Schema + Specs + Surprisal-Distilled Epistemic Atoms + Temporal Cone + Machine Twins + Frontier (zero narrative prose).

    • forageUrls() Adaptive Escalation: Scans all $N$ URLs at L0, promotes the highest-resonance pages to L1, checks sufficiency.missingFacets, and automatically escalates from L1 to L2 Deep Inspection only when a query facet remains unresolved in L1.

  5. Surprisal-Driven Epistemic Distillation (atoms) & Relativistic Temporal Light-Cone (temporal): Strips zero-surprise rhetorical scaffolding, requires a non-parametric anchor (quantity, constraint, or date) subordinated to query relevance, flags expired offers (⚠️EXPIRED_OFFER), and anchors indexical time words ("currently", "this year").

  6. Declarative Shadow API & Machine-Twin Autodiscovery (machineTwins): Detects same-origin JSON/API twins (wp-json, /llms.txt, /_next/data/...json, Shopify .json, feeds) while blocking private/loopback SSRF addresses, credentials, and cross-origin hosts.

  7. Claim-Level Copy-Discounted Log-Odds Bayesian Fusion (fusePages / fuseUrls): Evaluates both page-level and claim-neighborhood SimHash similarity (so a comparator copying a manufacturer's price is discounted on price while retaining full independence on its own lab-measured weight), and computes calibrated Log-Odds Bayesian Posteriors $P(v \mid \mathcal{D}) \in (0, 0.99]$.

import { contextPages, forageUrls, fuseUrls, createLiveSearchServer, createAgentMiddleware } from '@agenticschema/server';

// A. Adaptive Foveated Multi-Page Exploration (L0 -> L1 -> automatic L2 escalation on missing facets):
const wavefront = await forageUrls(
  ['https://lab.example/solid-state', 'https://blog.example/tips', 'https://news.example/trends'],
  { query: 'fast charge cycles at -30°C', foveaCount: 1 }
);
console.log(wavefront.context);      // Unified Foveated Wavefront with [✓S1]/[§S4] sigils
console.log(wavefront.sufficiency);  // { verdict, facetDiagnostics: [{ facet, status, maxConfidence }] }
console.log(wavefront.controller);   // { action: 'STOP_AND_ANSWER' | 'FOLLOW_FRONTIER', evi, target }

// B. Single-page Live Search with Multi-Facet Submodular Bottleneck or Holographic Resolution:
const [page] = await contextPages(['https://en.wikipedia.org/wiki/Backpack'], {
  query: 'internal frame load capacity',
  resolution: 'L1', // 'L0' (~99 tok pulse) | 'L1' (~413 tok epistemic) | 'L2' (full hybrid)
});
console.log(page.evidence);      // Typed EvidenceNode[] with 4D confidence (extraction, agreement, source, freshness)
console.log(page.controller);    // Active Inference EVI decision plan

// C. Claim-Level Copy-Discounted Log-Odds Bayesian Fusion:
const fused = await fuseUrls(
  ['https://store-a.example/camera', 'https://review-b.example/camera'],
  { query: 'price sensor weight', tokenBudget: 400 }
);

// D. Edge Middleware (Cloudflare Workers / Next.js / Hono):
const agentMiddleware = createAgentMiddleware({ mcpEndpoint: '/mcp' });

3. Expose your own website to in-browser AI agents (W3C WebMCP)

Two tags in development, one in production. The first reads your page's markup and registers WebMCP tools on document.modelContext; the second bridges them to local desktop MCP clients during development:

<!-- ① registration: read this page's Schema.org markup, publish it as WebMCP tools -->
<script src="https://cdn.jsdelivr.net/npm/@agenticschema/browser@latest"></script>

<!-- ② transport (development only): bridge those tools to a local MCP relay -->
<script src="https://cdn.jsdelivr.net/npm/@mcp-b/webmcp-local-relay@4/dist/browser/embed.js"></script>

Point your MCP client (Claude Desktop, Cursor, Claude Code) at the relay:

{
  "mcpServers": {
    "webmcp-local-relay": {
      "command": "npx",
      "args": ["-y", "@mcp-b/webmcp-local-relay@latest"]
    }
  }
}

Open the page, and the tools appear in your client. Verify with webmcp_list_sources (or inspect Chrome DevTools → Application → WebMCP in Chrome 146+).

Four things worth knowing before you paste that in:

  • Order matters. Put the relay embed tag after the registration tag.

  • Leave off type="module". The script-tag build is a classic IIFE script so it works out of the box in Google Tag Manager and Cloudflare Zaraz and preserves document.currentScript. See How the adapter finds its own tag.

  • Tag ② is for development only. Shipping it to real visitors makes their browsers probe 127.0.0.1. See Keep the relay out of production.

  • Pin your versions in production.

Try it in the browser with nothing installed:


Read this first: registration is not transport

This is the single thing that trips people up on the browser path, so it comes before the browser reference.

Getting a page's data to an external desktop agent takes two steps, and @agenticschema/browser only does the first one:

   ①  REGISTRATION                        ②  TRANSPORT
   @agenticschema/browser                 a WebMCP-capable browser,
   reads the page's Schema.org            an extension, or a local relay
   markup and registers tools             carries those tools to the agent
   on document.modelContext
        │                                        │
        └──────────► document.modelContext ◄─────┘
                     (the meeting point)

@agenticschema/browser writes tools into document.modelContext. It does not open a connection to anything, because a browser tab cannot listen on a port. See Three constraints.

If you only want the browser's own built-in AI agent (Chrome WebMCP) to call your tools, step ① is all you need. If you want an external desktop program (Claude Desktop, Cursor) to call them, you also need a transport (step ②). See Choosing a transport.

The symptom of forgetting step ② is very specific: the tools show up in Chrome DevTools (Application → WebMCP) but your external MCP client reports zero sources. DevTools reads document.modelContext in-process; your external MCP client is a separate program that cannot.


The script tag, in full

Where the tag goes

The build is a single classic script (dist/cdn/auto.js, ~30.6 KB gzipped) with no bundler and no package install behind it:

Platform

Where

WordPress

The theme's header template, or any plugin that inserts scripts into <head>.

Shopify

theme.liquid, before the closing </head>.

React, Next.js, Astro

The shared layout or document component, alongside your other third-party tags.

Plain HTML

In <head>, or before </body>.

Google Tag Manager, Cloudflare Zaraz

A Custom HTML tag. See Through a tag manager.

Put it in the shared layout rather than on one page. In a single-page app the adapter follows history.pushState, replaceState, popstate, and DOM mutations automatically, so a single tag covers every route.

How the adapter finds its own tag

To read its data-* options the adapter tries three things, in order:

  1. document.currentScript, set while a classic script runs (including one injected by a tag manager), and null in a module script,

  2. script[data-agenticschema], an explicit marker,

  3. script[src*="agenticschema"], the src of the standard snippet.

Since the build is an IIFE, the plain snippet takes rule 1 and everything works with no marker:

<script data-max-tools="8"
        src="https://cdn.jsdelivr.net/npm/@agenticschema/browser@latest"></script>

Adding type="module" gives up rule 1. If you self-host under a custom filename and use type="module", add data-agenticschema:

<script type="module" data-agenticschema data-max-tools="8"
        src="/assets/webmcp-bundle.js"></script>

Through a tag manager

Google Tag Manager and Cloudflare Zaraz inject a plain <script src> without type="module". Because the CDN bundle is an IIFE, it runs cleanly in any tag manager and reads data-* attributes via document.currentScript.

  • Fire it on every page, as early as possible.

  • Keep the local relay embed (embed.js) out of tag managers.

Every attribute

All options are optional:

Attribute

Values

Default

What it does

data-agenticschema

present / absent

absent

Marks the tag so the adapter can find it when document.currentScript is unavailable and src does not contain agenticschema.

data-actions

off

actions generated

Turns off executable tools entirely. Read tools are unaffected.

data-max-tools

integer > 0

24

Ceiling on generated tools.

data-watch

off

watching on

Stops the adapter following DOM changes and History API navigations.

data-allow-hosts

comma-separated hosts

page origin only

Extra hosts an action's destination may point at, beyond the page's own origin. Read Security first.

data-fallback

present / true / on

off

Synthesizes Schema.org entities from OpenGraph, Twitter Cards, <meta>, and HTML <form method="get"> search forms when the page has no structured data.

data-exposed-to

comma-separated origins

none

Cross-origin parent origins allowed to discover and invoke tools when embedded in an iframe (forwarded to registerTool options).

A page that uses all of them:

<script data-agenticschema
        data-actions="off"
        data-max-tools="8"
        data-watch="off"
        data-fallback
        data-allow-hosts="api.example.com, search.example.com"
        data-exposed-to="https://partner.example.com"
        src="https://cdn.jsdelivr.net/npm/@agenticschema/browser@latest"></script>

Content-Security-Policy

A page with a CSP has to allow cdn.jsdelivr.net in script-src, or self-host dist/cdn/auto.js:

Content-Security-Policy: script-src 'self' https://cdn.jsdelivr.net;

The development relay embed (embed.js) also creates a blob: iframe and opens a local WebSocket:

Content-Security-Policy: script-src 'self' https://cdn.jsdelivr.net;
                         frame-src blob:;
                         connect-src ws://127.0.0.1:9333;

Choosing a transport

Transport

Who calls the tools

Setup

Good for

Native browser WebMCP

the browser's own agent

none: the browser provides document.modelContext

the end state (Chrome 146+ flag / Chrome 149+ origin trial)

Chrome origin trial

the browser's own agent, on visitors' Chrome

register a token for your origin and serve it

running the native path on real traffic today

Local relay (embed.js)

desktop MCP clients: Claude Desktop, Cursor, Claude Code

one script tag + npx @mcp-b/webmcp-local-relay

development, testing, personal automation

Browser extension

whatever the extension is wired to

install the extension

using tools across sites you do not control

Node MCP server (@agenticschema/server)

any MCP client over stdio or HTTP

npx @agenticschema/server <url>

server-side agents & scraping without a browser

Keep the relay out of production

Gate embed.js on your build's development flag so real visitors never probe 127.0.0.1:9333:

// Next.js / React
{process.env.NODE_ENV === 'development' && (
  <script src="https://cdn.jsdelivr.net/npm/@mcp-b/webmcp-local-relay@4/dist/browser/embed.js" />
)}
---
// Astro
---
{import.meta.env.DEV && (
  <script src="https://cdn.jsdelivr.net/npm/@mcp-b/webmcp-local-relay@4/dist/browser/embed.js"></script>
)}

The JavaScript API & Universal Scraper

Universal scraping with scrape() and extractMeta() (@agenticschema/core)

@agenticschema/core has zero runtime dependencies and runs in Node, Bun, Deno, Cloudflare Workers, or the browser:

import { scrape, extractMeta } from '@agenticschema/core';
import { schemaOrgProfiles } from '@agenticschema/profiles';

const result = scrape(htmlOrDocument, {
  ...schemaOrgProfiles,
  baseUrl: 'https://shop.example.com/p/backpack',
  fallback: true, // enabled by default in scrape()
});

result.primary;      // Primary entity as clean JSON (e.g. { type: 'Product', name: '...', ... })
result.entities;     // All non-action entities materialized on the page
result.meta;         // { title, description, canonical, language, openGraph, twitter }
result.actions;      // [{ name: 'search_web_site', type: 'SearchAction', parameters: ['q'], required: ['q'] }]
result.tools;        // Ready-to-call ToolDescriptor[] (with execute())
result.diagnostics;  // Diagnostic[]

Filtering and paginating list_* group tools

When a page publishes multiple secondary entities of the same type (e.g. 30 related products or reviews), AgenticSchema collapses them into a single list_<type> tool instead of flooding the agent with 30 tools. Every list_* tool supports built-in filtering and pagination:

const listProducts = result.tools.find((t) => t.name === 'list_product');

// Filter case-insensitively by keyword and paginate:
const page1 = await listProducts.execute({
  query: 'waterproof',
  limit: 5,
  offset: 0,
});

Browser programmatic API (@agenticschema/browser)

import { start } from '@agenticschema/browser';

const handle = await start({
  maxTools: 12,
  actions: 'auto',
  fallback: true,
  allowedHosts: ['api.example.com'],
  exposedTo: ['https://partner.example.com'],
});

handle.tools();         // ToolDescriptor[]: what is currently registered
handle.diagnostics();   // Diagnostic[]: what the pipeline skipped, and why
await handle.refresh(); // remap now; a no-op if the markup has not changed
handle.stop();          // unregister everything and stop watching

start() accepts every pipeline option plus five browser-specific options:

Option

Type

Default

What it does

document

Document

the page's own

The document to read.

watch

boolean

true

Follow route changes and markup edits (including container insertions and [itemprop]/[property] mutations) in SPAs.

debounceMs

number

250

How long to wait after a DOM change before remapping.

exposedTo

readonly string[]

undefined

Cross-origin parent origins allowed to call tools when embedded in an iframe.

modelContext

ModelContext

document.modelContext

The WebMCP surface to register on. Injectable for tests.

In compliance with the W3C WebMCP specification, @agenticschema/browser passes strictly opt-in annotations (readOnlyHint: true, untrustedContentHint: true, consequentialHint: true when applicable, never : false) to document.modelContext.registerTool().


The core pipeline

import { toTools } from '@agenticschema/core';

const { tools, diagnostics, graph } = toTools(documentOrHtmlString, options);
                    ┌──────────────── @agenticschema/core ─────────────────┐
 Document │ HTML    │                                                      │
 │ JSON-LD ───────► │  extract ──► normalize ──► select ──► map ──► guard  │ ──► ToolDescriptor[]
                    └──────────────────────────────────────────────────────┘
                                              │
                          ┌───────────────────┴───────────────────┐
                          ▼                                       ▼
             @agenticschema/browser                   @agenticschema/server
             document.modelContext                    stdio / fetch handler / CLI JSON
             (script tag, WebMCP)                     (works with any MCP client today)

Stage

Does

extract

Pulls out JSON-LD, Microdata ($O(N)$), and RDFa ($O(N)$), plus optional OpenGraph / Twitter Card / <meta> / <form> fallback nodes.

normalize

Flattens @graph, resolves @id, strips vocabulary prefixes, merges nodes sharing an @id, guards against prototype pollution (__proto__, toString).

select

Identifies the primary subject of the page (mainEntity, mainEntityOfPage, about, or non-boilerplate root) and suppresses page chrome and protocol machinery (EntryPoint, *Action).

map

Applies type profiles (case-insensitively, following dotted paths like hasVariant.offers), transitively deduplicates materialized child entities, and memoizes static read payloads.

guard

Validates tool names, strips HTML/control characters, caps payloads with a UTF-8 fast path, and wraps execute() in an error boundary returning { isError: true }.

Every pipeline option

Category

Option

Type

Default

What it does

Extraction

formats

Array<'jsonld' | 'microdata' | 'rdfa'>

all three

Which structured data syntaxes to read.

Extraction

fallback

boolean

false (true in scrape())

Synthesize entities from OpenGraph, Twitter Cards, <meta>, and <form method="get"> when Schema.org markup is absent.

Normalisation

baseUrl

string

page URL in browser

Base for resolving relative @id values and deriving pageOrigin.

Normalisation

maxDepth

number

12

Maximum nesting depth before emitting depth-limit.

Mapping

profiles

Profile[]

generic profile only

Type profile registry (@agenticschema/profiles supplies 20+ curated profiles).

Mapping

ancestorsOf

(type: string) => string[]

none

Schema.org 924-class hierarchy resolver from @agenticschema/profiles.

Mapping

maxTools

number

24

Ceiling on generated tools (tool-limit diagnostic).

Actions

actions

'auto' | 'off'

'auto'

'off' disables executable tool generation.

Actions

pageOrigin

string

derived from baseUrl

Origin that action URLs are vetted against.

Actions

allowedHosts

readonly string[]

[]

Extra hosts allowed beyond pageOrigin.

Actions

timeoutMs

number

10000

Per-request timeout on action fetch calls.

Actions

fetchImpl

typeof fetch

global fetch

Custom fetch implementation.

Guard

maxDescriptionLength

number

320

Maximum length of a tool description.

Guard

maxPayloadBytes

number

32000

Maximum UTF-8 byte length of a tool result (field-truncated).

Custom

custom

readonly CustomTool[]

[]

Hand-declared tools (defineTool). Wins on name clashes.


The Node server & CLI scraper

@agenticschema/server fetches pages concurrently (with scheme, credential, and SSRF validation), parses HTML with linkedom when needed (skipping DOM construction entirely on JSON-LD-only pages), and speaks MCP 2026-07-28 over stdio or HTTP — or outputs Epistemic Context / scraped JSON directly.

npx @agenticschema/server [<url>...] [options]

Flag

Default

What it does

--live

off

Start a persistent Live Search MCP server over stdio exposing fetch_evidence, forage_urls, and fuse_sources for any public URL on the fly (with SSRF protection).

--context

off

Generate a 1-turn Hybrid LLM Context (toContext: structured facts + <dl>/<table> specs + inline [✓S1]/[~S2] sigils + deduplicated prose) and print it to stdout.

--forage

off

Adaptive Foveated Multi-Page Wavefront (forageUrls): scan all URLs at L0, promote top resonant pages to L1, and auto-escalate to L2 if query facets remain missing.

--fuse

off

Multi-Page Knowledge Graph Fusion (fuseUrls): SimHash copy-cluster discounting + Bayesian Truth Consensus or Markdown Comparison Matrix.

--resolution <L0|L1|L2>

L2

Holographic resolution level (L0 ~99 tok pulse, L1 ~413 tok epistemic core, L2 full hybrid).

--query <text>

none

Focus extraction via the Multi-Facet Submodular Query Bottleneck and compute sufficiency.

--budget <tokens>

none

Hard BPE-calibrated token ceiling (tokenBudget) for --context, --forage, or --fuse.

--json

off

Scrape all given URLs concurrently and print structured JSON (primary, entities, meta, actions, tools) to stdout instead of starting an MCP server.

--early-abort

off

Abort the HTTP response stream right after </head> once JSON-LD is found, saving up to 90% of bandwidth on large pages.

--fallback

off

Synthesize Schema.org entities and search actions from OpenGraph, Twitter Cards, <meta>, and <form method="get"> when Schema.org is missing.

--max-tools <n>

24

Cap on generated tools per page.

--no-actions

actions on

Do not build executable tools from potentialAction.

--allow-host <host>

none

Extra host allowed for actions. Repeatable.

--http

off

Serve MCP over HTTP instead of stdio. Binds 127.0.0.1 only, with body-size cap (413 Payload Too Large above 1 MB) and Host/Origin validation.

--port <n>

3111

Port for --http.

--quiet

off

Keep diagnostics off stderr.

-h, --help

n/a

Print usage and exit.

Over HTTP / Cloudflare Workers

import { createHttpHandler } from '@agenticschema/server';

const handler = await createHttpHandler(['https://example.test/product'], {
  fallback: true,
});

export default { fetch: (request) => handler.fetch(request) };

Results of tools/list, resources/list, and resources/read include the MCP 2026-07-28 ttlMs (300000 ms default) and cacheScope (public for tool listings, private for page resources) cache hints.


Deep dive: the 177-page corpus & 120-trial benchmark

How the corpus is built and measured

The 177 pages in the corpus are pulled from an immutable Common Crawl snapshot (CC-MAIN-2026-30) via byte-range WARC requests so reruns are 100% reproducible and never touch live origin servers:

npm run corpus:fetch                  # fetch from Common Crawl into fixtures/local
npm run corpus:report                 # inspect raw page types independent of AgenticSchema
npm install --no-save gpt-tokenizer   # optional: enable o200k_base token columns
npm run test:corpus                   # run all 177 pages through the pipeline

What improves token efficiency so sharply across the corpus:

  • Transitive entity deduplication (markMaterialized): When get_recipe or get_product materializes child entities inline (Offer, AggregateRating, NutritionInformation, Person, HowToSection, ImageObject) up to depth 2, those child entities are marked consumed and never re-emitted as separate duplicate tools. This eliminated 149 redundant tools across the corpus (from 900 down to 751 tools) and brought recipes from 3,787 tokens (0.4× vs text) down to 1,514 tokens (1.0× vs text).

  • Boilerplate & machinery suppression: Navigation ItemLists, WPHeader/WPFooter, BreadcrumbList, and protocol nodes (EntryPoint, SearchAction) are suppressed when a primary subject exists, so SearchAction becomes a callable search_* action tool and never a noisy get_search_action read tool.

  • Dotted-path resolution (ReadSpec.from): Real-world e-commerce pages store variant prices in hasVariant.offers (46 of 75 shop pages in the corpus are ProductGroups) and group ratings in isVariantOf.aggregateRating. Following multi-hop paths took e-commerce price accuracy from 8/11 to 11/11 (100%).

Full details are in docs/corpus.md and docs/bench.md.


Security

The library takes untrusted page content and puts it into an AI model's context. Six defences are enforced in core and inherited by every adapter:

  1. Prompt injection isolation: Page text never enters a tool's name or description — only its data payload. @type is used for naming only when it matches /^[A-Za-z][A-Za-z0-9]{0,39}$/; hostile @type strings containing instructions fall back to Thing. All descriptions and materialized data strings are stripped of HTML tags and control characters in guaranteed $O(N)$ linear time.

  2. Untrusted content annotations: Browser tools carry untrustedContentHint: true on document.modelContext so the host agent knows tool outputs originate from third-party page markup.

  3. SSRF & exfiltration prevention on urlTemplate:

    • Only idempotent action types (SearchAction, FindAction, ReadAction, ViewAction) with GET are auto-exposed. Side-effecting actions (OrderAction, POST, etc.) are skipped with an action-skipped diagnostic.

    • Destinations must be http:/https:, carry no embedded credentials (user:pass@), and match pageOrigin (or allowedHosts) both before and after RFC 6570 level-1 template expansion.

    • Placeholders in the scheme, host, port, or protocol-relative authority (//{host}/...) are rejected at map time.

    • Action requests pass redirect: 'error' so a 3xx redirect cannot escape the origin check.

  4. HTML action response cleaning & structured HTTP errors: When an action endpoint returns an HTML page instead of JSON, <script>, <style>, <svg>, <noscript>, and markup tags are stripped before payload capping. HTTP 4xx/5xx responses return isError: true with structured recovery hints (code, retryable, suggestion).

  5. Prototype pollution immunity: JSON-LD, Microdata, and RDFa extractors and normalizers block __proto__, constructor, and prototype keys and check Object.hasOwn() everywhere, preventing crashes or pollution from properties named toString or valueOf.

  6. Tool & payload budgets: Tool count is capped at maxTools (default 24) and output size at maxPayloadBytes (default 32,000 bytes, cut cleanly on UTF-8 code-point boundaries).

Read SECURITY.md for the complete threat model.


Custom tools

For actions with side effects (POST, checkout, booking) or private API endpoints, declare custom tools via custom / defineTool():

import { start } from '@agenticschema/browser';

start({
  custom: [{
    name: 'check_stock',
    description: 'Check in-store availability for a postal code.',
    annotations: { readOnlyHint: true },
    inputSchema: {
      type: 'object',
      properties: { postalCode: { type: 'string', description: '5-digit postal code' } },
      required: ['postalCode'],
      additionalProperties: false,
    },
    execute: async ({ postalCode }) => ({
      content: [{ type: 'text', text: await (await fetch(`/api/stock?cap=${postalCode}`)).text() }],
    }),
  }],
});

Custom tools pass through the same guard (name validation, description sanitization, payload cap, and async error boundary) and take precedence over generated tools on name clashes.


Troubleshooting

Symptom

Cause

Fix

Tools visible in DevTools → Application → WebMCP, but external MCP client shows 0 sources

Registration done, transport missing

Add the relay embed tag in dev, or use @agenticschema/server. See Choosing a transport

Page has no JSON-LD / Microdata / RDFa and produces 0 tools

Site only publishes OpenGraph or <meta> tags

Pass fallback: true, --fallback, or data-fallback on the <script> tag

A data-* option has no effect

type="module" on a self-hosted src without agenticschema in the filename

Drop type="module", or add data-agenticschema

The script tag never runs

CSP blocks cdn.jsdelivr.net

Allow it in script-src, or self-host dist/cdn/auto.js

No action tool from a SearchAction

Cross-origin target, POST, placeholder in host, or no baseUrl/pageOrigin

Check diagnostics for action-skipped; pass baseUrl when running headless

list_* instead of several get_*

Same-type secondary entities collapse into one filterable tool

Pass { query, limit, offset } when calling list_*


Packages

Package

Purpose

@agenticschema/core

Extraction, normalization, selection, mapping, guarding, and scrape(). Zero runtime dependencies.

@agenticschema/profiles

20+ curated Schema.org type profiles + compact 924-class Schema.org inheritance hierarchy.

@agenticschema/browser

W3C WebMCP adapter (document.modelContext). Self-contained IIFE script-tag build (~30.6 KB gzip, polyfill included).

@agenticschema/server

MCP 2026-07-28 server over stdio and HTTP (createServer, createHttpHandler) + concurrent multi-page scraper (scrapePages, --json).


Development

npm install
npm test          # 286 tests across 16 test files in < 1s
npm run typecheck # strict TypeScript check across all workspaces
npm run build     # build all 4 packages + CDN IIFE bundle
npm run size      # enforces single-file IIFE < 30 KB (30,720 B) gzipped

Contributions welcome. See CONTRIBUTING.md and CODE_OF_CONDUCT.md.


Status and disclaimer

Early, pre-1.0, API not stable. WebMCP is a W3C specification in active browser rollout (available in Chrome via chrome://flags/#enable-web-model-context / #enable-webmcp-testing and Chrome 149+ origin trial; polyfilled automatically everywhere else).

Provided as is, with no warranty of any kind, express or implied. See LICENSE and SECURITY.md.

MIT.

Maintenance

ActivityMaintained
ResponsivenessUnresponsive

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    B
    maintenance
    Turns Chrome into a local MCP server, giving AI agents 51 browser tools to inspect DOM, automate clicks, run JavaScript, audit security and accessibility, mock APIs, extract data, record demos, and more from any MCP client.
    92
    615 npm
    MIT
  • A
    license
    Not graded
    quality
    A
    maintenance
    Connects MCP-compatible AI agents to an anti-detection browser server, exposing dozens of tools for navigation, page interaction, search, content extraction, downloads, multi-tab handling, and persistent sessions. It lets agents automate and scrape the web at scale using accessibility snapshots, CSS-selector fallbacks, saved profiles, and batch workflows over stdio or authenticated HTTP transport.
    971 npm
    MIT