Groundhog

Web search, read and research for AI agents, through a real, stealth-patched Chrome. Groundhog is an MCP server that returns clean Markdown a model can trust: text no human could see is stripped before the model reads it and reported, every source comes with a provenance receipt, and a real browser reads pages that block plain fetchers.

agent / crawler  ──MCP──▶  Groundhog (search, read_url, research)  ──CDP──▶  stealth Chrome  ──▶  the web

Measured against other fetch layers

Twenty-eight pages carry text a human can't see, and three controls carry none. A case passes when the payload doesn't reach the model and the article text still does.

Other extractors often drop hidden text by accident, but none of them tells the caller anything was removed. The three cases Groundhog doesn't report (a hidden input value, a meta description, an iframe srcdoc) never reach any extractor. The corpus is public at dmytrome.github.io/groundhog; the harness and its limits are in benchmark/.

Related MCP server: firecrawl-mcp-server

Quick start

Claude Code:

claude mcp add groundhog -- uvx groundhog-mcp

Claude Desktop, Cursor, Windsurf:

{
  "mcpServers": {
    "groundhog": {
      "command": "uvx",
      "args": ["groundhog-mcp"]
    }
  }
}

On the first fetch Groundhog starts the Chrome you already have, headless, in its own profile at ~/.groundhog/chrome. Your own browsing, tabs and logins are never touched. If Chrome is missing, status and the error say what to install.

GROUNDHOG_BROWSER=stealth runs the hardened browser container instead (Docker or Podman; the first pull takes a few minutes). To run it yourself:

docker run -d --rm --name groundhog-browser --shm-size 512m \
  -p 127.0.0.1:9222:9222 -- ghcr.io/dmytrome/groundhog:latest

CDP_URL points Groundhog at any browser and is used as given: http:// is probed for /json/version, ws:// or wss:// (what hosted browsers hand out) is connected to directly. Auto-start only runs for a local address that isn't answering, and GROUNDHOG_AUTO_START_BROWSER=false turns it off. From source: cd mcp && uv sync && uv run groundhog-mcp.

All four tools are annotated readOnlyHint: they write nothing and change no remote state. The one side effect is auto-start, which may launch Chrome or, with stealth, run the container after removing an unreachable one named groundhog-browser.

What makes it different

• Hidden text is stripped and reported. A rendered DOM shows what a human would see; everything else goes, and each removal is listed in threats. It's a strong heuristic, not a proof: see the limits.

• Every source carries a receipt: a SHA-256 of the extracted content, canonical URL, language, word count, and author and date when the page declares them.

• Safe by default. An SSRF guard checks every document request, redirects and frames included, with per-domain rate limiting. See Security.

• No automation tell. Groundhog drives Chrome over raw CDP and never enables the Runtime domain that anti-bots detect in Puppeteer, Playwright and Selenium.

• A real fingerprint. Real Chrome means real TLS, HTTP/2, WebGL and canvas. In headless mode the HeadlessChrome token is replaced and client hints are copied from the browser; shared and service workers aren't covered. The stealth image runs headful under Xvfb.

• No model, no API key. research returns extracts and your agent does the synthesis.

Tools

read_url(url, format="markdown", max_tokens=None, query=None, include_hidden=False)

Stripped by default: display:none, visibility:hidden, content-visibility:hidden, opacity ≤ 0.05, font-size < 4px, zero-size and display:contents boxes that render nothing, .sr-only-style sub-pixel boxes, clip: rect(...), transparent or background-coloured text, off-page positioning, collapsed <details>, <template> content, long alt / aria-label / aria-description / title text, and zero-width, bidi and Unicode Tag characters. HTML comments are reported too. include_hidden=True keeps the text but still reports it.

Treat threats as untrusted: its excerpts are page-authored text, cleaned and length-capped.

At most 50 findings per page are returned (10 per source in research), with the two classes capped separately so decoys can't bury a real payload. Notices are appended after the cap.

query swaps head-truncation for BM25 passage ranking on the sanitized content, keeping passages within half the best score up to the budget.

search(query, limit=10)

Returns hits (title, url, snippet, engine, score, published) and the backend that answered. Nothing is fetched. With SEARXNG_URL set it uses your SearXNG instance, which needs formats: [html, json]; otherwise it renders a search page through the browser. Hit text is cleaned and capped, and a hit whose URL would change under cleaning, isn't http(s), carries credentials or exceeds 2048 characters is dropped. A backend that can't answer raises an error instead of returning an empty web.

research(query, max_sources=5, max_tokens=None, include_domains=None, exclude_domains=None, published_after=None, aim=None)

Searches, reads the top sources and ranks passages from all of them in one pass.

• passages: id, text, source_url, heading, score, start, end, clamped, fence. text is exactly markdown[start:end] unless clamped. A passage cut from a long code block carries the block's opening line in fence: reopen with it when text doesn't start with it, and close the block when text doesn't end with a closing fence. id stays stable while the page doesn't change.

• sources: url, title, status (ok / blocked / timeout / error), page_status (as in read_url), threats, provenance, filtered. A failed source doesn't fail the call. A challenge, an error page or a page with no text (whose page_status stays ok) counts as not_content and contributes no passages.

• One page per registrable domain unless include_domains is set, and then no more pages per site than GROUNDHOG_MAX_CONCURRENT_PAGES; max_sources goes up to 10.

include_domains reads only those domains and their subdomains, exclude_domains never reads them, and published_after (YYYY-MM-DD) drops pages dated earlier. Up to 8 domains can be included and 50 excluded. A page that redirects into an excluded domain or out of the included ones, is dated too early, or repeats the URL or text of a page already used is listed with filtered and contributes no passages. The query sent to the engine names at most eight domains; further excluded ones are filtered after the search. aim (up to 200 characters) says what the evidence must show and is echoed back without changing the search or the ranking.

receipt holds the executed_query, the distinct hits and hits_by_domain, the hits not fetched (hits_excluded, hits_outside_domains, hits_before_date, hits_same_domain, hits_unread past the source limit), each source counted once as failed, not_content, filtered or usable, outside_domains, unknown_dates, and the passages omitted, set_aside or clamped. Read it before concluding the web has no answer.

status()

Returns browser_reachable, cdp_url (scheme, host and port only) and a hint when the browser can't be reached.

Configuration

MCP server:

Browser container:

The stealth Chrome container

Headful Chrome under Xvfb with a CDP endpoint, usable by any CDP client. It reports Chrome rather than HeadlessChrome, sets navigator.webdriver to false, and sets the UA at launch. With PROXY, timezone and locale match the exit IP, credentials go through a local tinyproxy, and WebRTC is pinned to the proxy. WebGL uses a detected GPU, else Mesa llvmpipe.

Against a fresh container (Chrome 151, no proxy), over raw CDP:

iphey flags any datacenter IP, so it's tracked but not scored. The live table is in RESULTS.md. Puppeteer, Playwright and Selenium are still flagged because they enable Runtime; examples/ has clients for each, and OTHER_TOOLS.md covers more.

Security

The CDP endpoint is unauthenticated: anyone who reaches the port controls the browser. Keep it on localhost or a private network. The stealth image runs --no-sandbox, so keep the container isolated. Report vulnerabilities via SECURITY.md.

The SSRF guard resolves each host and refuses loopback, private, link-local (including 169.254.169.254), reserved, multicast, unspecified, CGNAT and IPv4-mapped IPv6 addresses, non-http(s) schemes, and URLs with credentials. It checks every document request Chrome is about to send, redirects and cross-site frames included, and final_url once more. Sub-resources are left to Chrome's Local Network Access rule, and the Chrome that Groundhog launches has preloading turned off.

Limits of the guard:

• Only documents are checked. An older Chrome, or a site granted local-network access in the profile, isn't covered for sub-resources.

• Preloading stays on in the stealth container and in a browser you supply.

• Groundhog and Chrome resolve DNS separately, so a short-TTL rebinding window remains.

• Fetches share one profile, so cookies set by one page persist into later fetches, and page JavaScript can make its own requests from it.

GROUNDHOG_BLOCK_PRIVATE_IPS=false turns the guard off; use it only where reaching internal addresses is intended.

Limits of hidden-text detection

An empty threats list isn't a clean bill of health. Detection runs in an isolated world the page can't tamper with, and the live page is never modified. But:

• Thresholds can be sat just inside (opacity: 0.06), and unmodelled techniques such as clip-path, text-indent or transform: scale(0) pass.

• The invisible-character set is a denylist.

• Closed shadow roots can't be read, so their content is left out entirely.

• When the page wins the cascade against the hiding stylesheet or hides its own <body>, text is rebuilt from markup, line breaks are guessed, and strip_incomplete says so.

Limits of block detection

• A challenge is called with certainty from vendor headers (cf-mitigated, x-vercel-mitigated, x-amzn-waf-action, x-dd-b, x-datadome-cid) or challenge assets (Cloudflare, DataDome, PerimeterX, Imperva). Other vendors fall back to English wording on a near-empty page.

• A soft block, a 200 with the data quietly withheld, looks like content.

• unknown isn't ok, but it isn't treated as a block either.

• Vendor presence (server: cloudflare, cf-ray) is deliberately ignored.

A note on "stealth"

Best effort. It beats common open-source detectors, not commercial anti-bot systems that gate on IP reputation and behaviour. Use it for legitimate, authorized automation.

Privacy Policy

Groundhog is self-hosted software and talks to no servers the project runs. It collects nothing: no telemetry, analytics, crash reports or accounts.

Pages are fetched by a browser on your machine and returned to your MCP client. The server keeps no database and writes no logs. The browser profile does persist cookies and storage: ~/.groundhog/chrome for local Chrome (delete the folder to discard it), or the container for stealth (docker rm -f groundhog-browser).

Your traffic reaches the sites you fetch, your search backend (your SearXNG, or a public search engine without one), an IP-geolocation service for the proxy's exit IP only when PROXY is set, and PyPI and GHCR at install time.

Questions: SECURITY.md or https://github.com/dmytrome/groundhog/issues.

License

MIT