PyreCrawl

Overview Schema Source Related Servers Discussions Admin

scrape

Convert any URL into clean, LLM-ready markdown. Bypasses anti-bot blocks with automatic fallback methods to extract readable content for analysis, summarization, or extraction.

Instructions

Scrape a single URL → LLM-ready markdown.

Use this when the user shares a URL and wants its content (read, analyze, summarize, extract). Auto-escalates through fast→stealth→llm when blocked.

Args: url: Target URL (http/https). prefer: "auto" | "fast" | "stealth" | "llm". auto = fast first, escalate to stealth on block/short page. fast = cheap HTTP only (no JS). stealth = real Chromium + Cloudflare solver. llm = full Crawl4AI browser + BM25 fit-markdown. timeout: per-attempt timeout in seconds. include_html: include raw HTML in the response (large; off by default). js: (stealth only) JS expression evaluated against the live page after it settles. The value comes back in meta.js_result. Use for data that lives in DOM properties (e.g. an input's .value) rather than in serialized HTML. wait_for: (stealth only) JS predicate expression polled until truthy (bounded by timeout). Use to wait for content that arrives asynchronously after network_idle.

Returns: {url, final_url, status, markdown, title, method, elapsed_ms, meta} or {error, url, method} on failure.

Input Schema

TableJSON Schema

Name	Required	Default
`js`	No
`url`	Yes
`prefer`	No	auto
`timeout`	No
`wait_for`	No
`include_html`	No

Output Schema

TableJSON Schema

Name	Required	Description	Default
No arguments

Schema Changelog

Changes observed during successful MCP inspections.

First observedv0.7.1Sep 8, 2026

TDQS

A4.6/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the full behavioral burden and succeeds. It discloses the auto-escalation path ('fast→stealth→llm when blocked'), explains what each mode does (cheap HTTP, real Chromium + Cloudflare solver, full Crawl4AI browser), and scopes js and wait_for to stealth only. It also documents the success and failure return shapes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is tightly structured: a one-line purpose, a single usage sentence, a compact Args block, and a Returns block. Every line is information-dense with no filler, and the most important scoping details are front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a complex tool with 6 parameters, 4 execution modes, JS evaluation, and async waiting. The description accounts for every parameter, the escalation behavior, return values, and failure output. There are no obvious omissions that would prevent an agent from invoking it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has no descriptions and only bare titles/defaults, so the description must compensate. It does: url is constrained to http/https, prefer enumerates all four modes with escalation semantics, timeout is defined as per-attempt seconds, include_html warns about response size, and js/wait_for get exact behavioral definitions with an example for js.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb-resource statement: 'Scrape a single URL' and 'LLM-ready markdown'. It names a clear trigger ('when the user shares a URL and wants its content'), and 'single URL' separates it from multi-URL siblings like crawl and batch_scrape. However, it does not explain how it differs from sibling extract, so differentiation is incomplete.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a concrete usage context: 'Use this when the user shares a URL and wants its content (read, analyze, summarize, extract)'. That is clear context, but it does not list exclusions or point to alternative tools such as extract or crawl, leaving the when-not-to-use judgment to the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server