PyScrappy

Overview Schema Related Servers Score Discussions

scrape_url

Fetch any HTTP(S) page and extract text, links, images, tables, and metadata into structured JSON. Renders JavaScript when needed and supports pagination for multi-page scraping.

Instructions

Scrape any HTTP(S) URL and return a ScrapeToolResult whose data holds one object per page containing extracted text (with word_count), links, images, tables, and page metadata.

Fetches the page over the network and parses the HTML; no data is stored or mutated. By default it makes a plain static HTTP request, so pages built client-side with JavaScript come back nearly empty. When that is detected, the returned errors list gets a hint to retry with render_js=true; set render_js=true to render with a headless browser instead (requires the pyscrappy[browser] extra). On empty or failed results, data is [], count is 0, and errors describes the problem rather than raising.

Input Schema

TableJSON Schema

Name	Required	Description
`url`	Yes	String, the page URL to scrape including scheme, e.g. "https://example.com/products". Required, no default.
`max_pages`	No	Integer, follow "next"-style pagination up to this many pages, e.g. 3. Default 1 (scrape only the given URL).
`render_js`	No	Boolean, render JavaScript with a headless browser backend, e.g. True. Default False; allowed values True or False, and True needs the pyscrappy[browser] extra installed.
`selectors`	No	Optional dict mapping output field name to CSS selector to extract specific values into each data item, e.g. {"title": "h1", "price": ".amount"}. Default None (returns only the standard text/links/images/tables/metadata).

Output Schema

TableJSON Schema

Name	Required	Description	Default
`data`	No
`count`	No
`errors`	No
`scraper`	No
`source_urls`	No

Schema Changelog

Changes observed during successful MCP inspections. Dates show when Glama detected each change.

Changed4 schema fields changedJul 31, 2026

changedInput schema / properties / max_pages / description

Previous value: -"Follow pagination up to this many pages (default 1)."New value: +"Integer, follow \"next\"-style pagination up to this many pages, e.g. 3. Default 1 (scrape only the given URL)."

changedInput schema / properties / render_js / description

Previous value: -"Render JavaScript with a browser backend (needs pyscrappy[browser])."New value: +"Boolean, render JavaScript with a headless browser backend, e.g. True. Default False; allowed values True or False, and True needs the pyscrappy[browser] extra installed."

changedInput schema / properties / selectors / description

Previous value: -"Optional CSS selectors, e.g. {\"title\": \"h1\", \"price\": \".amount\"}."New value: +"Optional dict mapping output field name to CSS selector to extract specific values into each data item, e.g. {\"title\": \"h1\", \"price\": \".amount\"}. Default None (returns only the standard text/links/images/tables/metadata)."

changedInput schema / properties / url / description

Previous value: -"The page to scrape."New value: +"String, the page URL to scrape including scheme, e.g. \"https://example.com/products\". Required, no default."

Changed22 schema fields changedv1.3.3Jul 31, 2026

addedInput schema / additionalProperties
```
Added value: +false
```

addedInput schema / properties / max_pages / description

Added value: +"Follow pagination up to this many pages (default 1)."

removedInput schema / properties / max_pages / title
```
Removed value: -"Max Pages"
```

addedInput schema / properties / render_js / description

Added value: +"Render JavaScript with a browser backend (needs pyscrappy[browser])."

removedInput schema / properties / render_js / title
```
Removed value: -"Render Js"
```

addedInput schema / properties / selectors / description

Added value: +"Optional CSS selectors, e.g. {\"title\": \"h1\", \"price\": \".amount\"}."

removedInput schema / properties / selectors / title
```
Removed value: -"Selectors"
```
addedInput schema / properties / url / description
```
Added value: +"The page to scrape."
```
removedInput schema / properties / url / title
```
Removed value: -"Url"
```
removedInput schema / title
```
Removed value: -"scrape_urlArguments"
```

removedOutput schema / $defs

Removed value: -{
-  "ToolError": {
-    "description": "A non-fatal problem encountered while scraping.",
-    "properties": {
-      "message": {
-        "title": "Message",
-        "type": "string"
-      },
-      "url": {
-        "title": "Url",
-        "type": "string"
-      }
-    },
-    "required": [
-      "url",
-      "message"
-    ],
-    "title": "ToolError",
-    "type": "object"
-  }
-}

removedOutput schema / properties / count / title
```
Removed value: -"Count"
```
removedOutput schema / properties / data / title
```
Removed value: -"Data"
```
removedOutput schema / properties / errors / items / $ref
```
Removed value: -"#/$defs/ToolError"
```

addedOutput schema / properties / errors / items / description

Added value: +"A non-fatal problem encountered while scraping."

addedOutput schema / properties / errors / items / properties

Added value: +{
+  "message": {
+    "type": "string"
+  },
+  "url": {
+    "type": "string"
+  }
+}

addedOutput schema / properties / errors / items / required
```
Added value: +[
+  "url",
+  "message"
+]
```
addedOutput schema / properties / errors / items / type
```
Added value: +"object"
```
removedOutput schema / properties / errors / title
```
Removed value: -"Errors"
```
removedOutput schema / properties / scraper / title
```
Removed value: -"Scraper"
```
removedOutput schema / properties / source_urls / title
```
Removed value: -"Source Urls"
```
removedOutput schema / title
```
Removed value: -"ScrapeToolResult"
```

Addedv1.2.0Jul 26, 2026
RemovedJul 26, 2026
First observedv1.1.6Jul 26, 2026

TDQS

A4.5/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and delivers: it discloses network fetching, that no data is stored or mutated, the plain-HTTP limitation with JS pages, the render_js=true retry path, and that failures return empty data/errors instead of raising. This is exemplary.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured: the first sentence front-loads the purpose and result shape, followed by useful behavioral details. Every sentence earns its place, and there is no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given an output schema exists and annotations are absent, the description covers result structure, failure handling, JS-rendering behavior, and installation note. It is complete enough for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters. The description adds some context about render_js and error behavior, but it does not substantially enrich parameter meaning beyond what the schema provides; baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it scrapes any HTTP(S) URL and returns a ScrapeToolResult with extracted text, links, images, tables, and metadata. The 'any URL' framing distinguishes it from specialized sibling scrapers like scrape_wikipedia or scrape_stock.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context: it works for any HTTP(S) URL and provides a specific retry instruction when JavaScript-rendered pages return empty. It does not explicitly name alternatives or exclusions, but the general-purpose wording makes the use case obvious.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/mldsveda/PyScrappy'

If you have feedback or need assistance with the MCP directory API, please join our Discord server