Skip to main content
Glama
foomworks

agent-web

by foomworks
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Remote

agent-web

URL → LLM-ready markdown. A charter-clean, polite, robots-respecting web reader for AI agents — honors robots.txt, identifies honestly, and never bypasses anti-bot/CAPTCHA or paywalls. No account, no API key — free per-call access.

🌐 Live: https://agent-web.foomworks.workers.dev · 🔌 MCP: https://agent-web.foomworks.workers.dev/mcp

What it gives an agent

Give it one publicly reachable URL; get back the page as clean, LLM-ready markdown — title, word count, and the text — without parsing raw HTML yourself. Server-rendered (static) HTML pages. Ideal for summarizing an article, extracting docs, or feeding a page into an LLM.

Related MCP server: Webustler

Connect as an MCP server (recommended)

Streamable HTTP, JSON-RPC 2.0:

https://agent-web.foomworks.workers.dev/mcp

Tool

Cost

Returns

read_url

free

fetch a URL → full LLM-ready markdown (title, word count, markdown)

read_url_preview

free

the first ~600 characters of the markdown — a cheap look before the full pull

Or call it over HTTP

BASE=https://agent-web.foomworks.workers.dev
curl -s "$BASE/read?url=https://example.com/&ref=readme"          # full markdown
curl -s "$BASE/read/preview?url=https://example.com/&ref=readme"  # first ~600 chars

Discovery: /.well-known/mcp.json (MCP) · /.well-known/agent-card.json (A2A) · /openapi.json · /policy (Acceptable-Use Policy)

Charter-clean by design (the bright line)

This is a polite reader, not a scraper:

  • Honors the origin's robots.txt for our user-agent — a disallowed path is refused without fetching; an unconfirmable robots.txt (5xx/error) is treated conservatively as disallowed.

  • Identifies honestly with a descriptive user-agent — never spoofs a browser.

  • Read-only GET of the single URL you supply — no crawl, no proxy rotation.

  • Never bypasses anti-bot/CAPTCHA, paywalls, or login walls.

  • SSRF-guarded: private/loopback/link-local/internal hosts are blocked and every redirect hop is re-validated.

You are responsible for your right to fetch the URL you submit.

Limits

  • Static HTML only for now — JavaScript-rendered pages, screenshots, and PDF are a later increment.

  • Body cap ~2 MB, fetch timeout ~12 s.

  • The service holds no keys and never initiates payments.

Install the AgentSkill

SKILL.md is a portable AgentSkill (ClawHub / skills.sh / Claude Skills).

About

agent-web is operated by FOOM — an AI-operated, human-supervised autonomous service. License: MIT.

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • fetch

    RMRanjit
    A
    license
    Not graded
    quality
    D
    maintenance
    Fetches web pages and converts them to markdown for LLM consumption, supporting chunked reading and raw content extraction.
    MIT

  • skrape

    AmokOrg
    A
    license
    Not graded
    quality
    D
    maintenance
    Converts any webpage into clean, LLM-ready Markdown, removing noise and supporting JavaScript rendering.
    MIT