Skip to main content
Glama

Server Details

Ask before you scrape: is a site open, may I fetch this URL, and how do I extract a page. No key.

If you are the author of this connector, you can claim ownership. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Healthy
Uptime
100.0% over the last 24 hours
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
ChatGPT
–
Claude
–

TDQS

A3.8/5.0

Scored across 8 tools

Disambiguation4/5

Each tool has a distinguishable role: dataset browsing (list_sites, search_sites, summary), site-level feasibility (check_site), URL-level robots.txt (can_fetch), and page-level extraction (xray_page, xray_diagram), plus workflow recipes (get_playbook). The main soft overlaps are can_fetch vs check_site (same underlying robots.txt topic at different granularity) and list_sites vs search_sites (both filter by category/verdict), but descriptions make the intended entry points clear. xray_page and xray_diagram operate on the same input yet return clearly different artifacts.

Naming Consistency3/5

All names are snake_case and readable, but the convention is mixed: verb_noun (list_sites, search_sites, get_playbook), a modal verb (can_fetch), a near-synonym verb for a sibling tool (check_site), a bare noun (summary), and a product-prefix pair (xray_page, xray_diagram). The set is legible but does not follow a single predictable pattern.

Tool Count5/5

Eight tools is well-scoped for a site-research and extraction-planning server. Each tool covers a distinct stage of the workflow (discover, assess, plan, build) with no filler.

Completeness4/5

The surface covers the full research lifecycle: discover sites, judge scrapeability, check a specific URL, plan extraction, and generate build recipes, with diagram output as a bonus. Gaps are minor: there is no tool to actually run a scrape or retrieve extracted records (xray_page returns proof rows only), and no way to submit or refresh a domain in the dataset beyond the live probe inside check_site.

Available Tools

8 tools
can_fetchAInspect

Does robots.txt allow fetching this exact URL? Fetches the live robots.txt and applies the longest-match rule for a user agent (default: * plus GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot). Returns allowed or not, the rule that decided it, and any crawl delay. A courtesy check, not a legal opinion.

Inputs: url (required), user_agent.

check_siteAInspect

Can I scrape this site, and how? Returns a verdict (open, guarded, hostile, blocked), a hostility score, a recommended approach with steps and risks, bot protections, robots.txt and AI-crawler policy, machine-readable surfaces (sitemap, llms.txt, JSON-LD), and eu_ai_training: the signals the EDPB draft Guidelines 03/2026 on web scraping for generative AI count against collecting personal data (robots.txt blocks on AI training crawlers, a Content-Signal of ai-train=no, ai.txt, CAPTCHAs, login walls), as none, some or strong. Signals, not legal advice. Uses a weekly-probed dataset of several hundred sites; for any other domain it probes live from a datacenter IP.

Inputs: domain (required).

get_playbookAInspect

What should I build with this site, and how? Give a domain and, optionally, a goal in plain words (for example "price alerts", "sales leads", "new jobs", "change log", "knowledge base"). Returns the best-matching workflow for that site: what it does, its cadence, a trigger, collect, shape, store, act pipeline, the fields to keep, and step-by-step build recipes for n8n, Make, Zapier and Apify (or just one with platform). Workflows marked found_on_this_site come from what the weekly probe saw on the site itself. Also returns the other workflows, the golden data, and the guardrails.

Inputs: domain (required), goal, platform.

list_sitesBInspect

List the sites in the dataset, optionally by category or verdict. Use it to find easy targets or to see what is covered.

Inputs: limit, verdict, category.

search_sitesAInspect

Find sites in the dataset by a word: a name, host, category (jobs, shop, news, gov), country or verdict. Use it to discover what is covered before calling check_site.

Inputs: query (required), limit.

summaryBInspect

Headline numbers across the dataset: verdict mix, how many sites block AI crawlers (and which), protection vendors, by category and country.

Inputs: no named parameters.

xray_diagramAInspect

Draw a page's anatomy as an Excalidraw diagram: its regions, the ranked ways to get its data, proof rows, the best route and what the page is made of. Returns a download link for the .excalidraw file (give it to the user; it opens at excalidraw.com or in Obsidian) and a file_name. With include_scene, also the scene itself (a few hundred KB).

Inputs: url (required), include_scene.

xray_pageAInspect

How do I extract the data from THIS page? Fetches one public page and returns a ranked extraction plan: embedded JSON, JSON-LD, repeated records with CSS selectors and field names, tables, feeds, hidden API calls. Each source has a reliability, proof rows pulled from the real page, and the plan notes pagination and blockers. Set include_code for a starter Python script. Works on the raw HTML only, so JavaScript-built pages report a blocker instead of data. Use check_site first to learn whether the site allows scraping at all.

Inputs: url (required), include_code.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents and MCP clients to inspect a URL for reachability, then extract clean structured Markdown, headings, and links from public web pages, paid per successful request via x402 micropayments with no accounts or API keys.
      MIT
    • F
      license
      Not graded
      quality
      C
      maintenance
      Enables agents to read any web page or document as markdown, html, links, screenshots or extracted JSON, map and crawl entire sites, and drive a persistent browser session that stays open between calls so pages can be inspected and clicked without reloading. Callers pay per request with a wallet over x402 — no API key or signup — and every route answers its own free GET with the fields it expects.
      1
      -
    • A
      license
      Not graded
      quality
      D
      maintenance
      Read-only observation of a single live URL: parses static HTML to report security posture, forms, links, accessibility signals, and leaks, with described fixes. SSRF-gated and safe, never executes JavaScript.
      85 npm
      MIT
    Try in Browser

    Glama MCP Gateway

    Add one secure layer between your agents and this server.

    Resources