olostep-mcp

Scrape Website

scrape_website

Read-onlyIdempotent

Retrieve webpage content from a URL and return clean markdown, HTML, or text after JavaScript rendering; extract structured JSON via schema or parser.

Instructions

Use this when you have a URL and need what is on the page. Returns the page as clean markdown (default), html or text, with its url and title, after rendering JavaScript. For structured fields, set output_format to "json" with an llm_extract schema or a parser. Long pages are cut at max_characters (default 20,000) with a link to the full text. For many known URLs use batch_scrape_urls; for many pages of one site use create_crawl.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "scrape_website"
  }
}

Input Schema

Name	Required	Description	Default
`url`	No	The URL of the website you want to scrape.
`parser`	No	Optional parser ID for specialized extraction (e.g., "@olostep/amazon-it-product").
`country`	No	Optional country code (e.g., US, GB, CA) for location-specific scraping.
`llm_extract`	No	Defines what to pull when output_format is "json". Provide a `schema` (a JSON-schema object of the fields you want, e.g. { "type": "object", "properties": { "title": { "type": "string" } } }) and/or a `prompt`. Required for JSON output unless a `parser` is given.
`output_format`	No	Output format. "markdown" (default), "html", and "text" need no extra config. "json" returns STRUCTURED data and requires either a `parser` OR an `llm_extract` schema describing the fields to pull. Do not request "json" on its own.	markdown
`url_to_scrape`	No	Alias for `url`.
`max_characters`	No	Max characters of page content to return (default 20000). Longer pages are cut and link to their full text; raise this to read more.
`wait_before_scraping`	No	Wait time in milliseconds before scraping (0-10000). Useful for dynamic content.

Schema Changelog

Changes observed during successful MCP inspections.

Changed8 schema fields changedv1.0.21Nov 13, 2026

removedInput schema / $schema

Removed value: -"http://json-schema.org/draft-07/schema#"

addedInput schema / properties / llm_extract

Added value: +{
+  "additionalProperties": false,
+  "description": "Defines what to pull when output_format is \"json\". Provide a `schema` (a JSON-schema object of the fields you want, e.g. { \"type\": \"object\", \"properties\": { \"title\": { \"type\": \"string\" } } }) and/or a `prompt`. Required for JSON output unless a `parser` is given.",
+  "properties": {
+    "prompt": {
+      "type": "string"
+    },
+    "schema": {
+      "additionalProperties": {},
+      "type": "object"
+    }
+  },
+  "type": "object"
+}

addedInput schema / properties / max_characters

Added value: +{
+  "default": 20000,
+  "description": "Max characters of page content to return (default 20000). Longer pages are cut and link to their full text; raise this to read more.",
+  "minimum": 500,
+  "type": "integer"
+}

changedInput schema / properties / output_format / description

Previous value: -"Choose format (\"html\", \"markdown\", \"json\", or \"text\"). Default: \"markdown\""New value: +"Output format. \"markdown\" (default), \"html\", and \"text\" need no extra config. \"json\" returns STRUCTURED data and requires either a `parser` OR an `llm_extract` schema describing the fields to pull. Do not request \"json\" on its own."

changedInput schema / properties / parser / description

Previous value: -"Optional parser ID for specialized extraction (e.g., \"@olostep/amazon-product\")."New value: +"Optional parser ID for specialized extraction (e.g., \"@olostep/amazon-it-product\")."

addedInput schema / properties / url

Added value: +{
+  "description": "The URL of the website you want to scrape.",
+  "format": "uri",
+  "type": "string"
+}

changedInput schema / properties / url_to_scrape / description

Previous value: -"The URL of the website you want to scrape."New value: +"Alias for `url`."

removedInput schema / required
```
Removed value: -[
-  "url_to_scrape"
-]
```

Addedv1.0.0Jan 3, 2026

TDQS

A4.7/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive, openWorld), so the description's job is to add operational behavior — and it does: JavaScript is rendered, long pages are truncated at max_characters with a link to the full text, and json output requires a parser or llm_extract schema. It omits auth requirements, rate limits, and failure modes (e.g., blocked pages), which keeps it short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four tight sentences, front-loaded with the primary use case, then output formats, then truncation, then alternatives. No filler or repetition of the tool name, and every sentence carries operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 8-parameter, nested-schema tool with no output schema, the description still tells the agent what comes back (markdown by default, url and title included), how truncation behaves, and how to get structured data. An agent can select and invoke this correctly without further context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is already 100%, so the baseline is 3; the description still adds value by stating the cross-parameter dependency in prose: output_format="json" is only meaningful with an llm_extract schema or a parser, and max_characters governs where truncation happens. It does not explain parser IDs or country codes beyond what the schema says, so it stops short of a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('scrape a URL, get what is on the page') and immediately clarifies the deliverable (clean markdown/html/text, plus url and title, after JavaScript rendering). It also distinguishes itself from the two most confusable siblings, batch_scrape_urls and create_crawl, by name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Opens with an explicit triggering condition ('when you have a URL and need what is on the page') and routes multi-URL cases to batch_scrape_urls and single-site multi-page cases to create_crawl. Use-vs-alternative guidance is complete enough that the agent need not open the sibling schemas.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server