olostep-mcp

Batch Scrape URLs

batch_scrape_urls

Read-only

Scrapes a known list of URLs in one batch (up to 100k), returning page content immediately for small batches or a batch_id for larger runs to retrieve later.

Instructions

Use this when you have a known list of URLs to scrape, such as URLs from a CSV or from search results (up to 100k in one batch). Waits up to wait_for_completion_seconds (default 45) and returns the content if the batch finishes in that time. Larger batches usually take 5-8 min and return a batch_id: get their content with get_batch_results, a set at a time, passing next_cursor for the rest. For one or a few URLs, scrape_website is faster; to discover a site's pages, use create_crawl.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "batch_scrape_urls"
  }
}

Input Schema

Name	Required	Description	Default
`urls`	No	Array of URLs to scrape — plain URL strings, or objects with "url" and optional "custom_id".
`parser`	No	Optional parser ID for specialized extraction (e.g. @olostep/google-search).
`country`	No	Optional country code for location-specific scraping.
`output_format`	No	Choose format for all URLs. Default: "markdown".	markdown
`urls_to_scrape`	No	Alias for `urls`.
`wait_before_scraping`	No	Wait time in milliseconds before scraping each URL.
`wait_for_completion_seconds`	No	Seconds to wait for the batch to finish and return its content in this call. 0 returns the batch_id right away.

Schema Changelog

Changes observed during successful MCP inspections.

Changed7 schema fields changedv1.0.21Nov 13, 2026

removedInput schema / $schema

Removed value: -"http://json-schema.org/draft-07/schema#"

changedInput schema / properties / parser / description

Previous value: -"Optional parser ID for specialized extraction."New value: +"Optional parser ID for specialized extraction (e.g. @olostep/google-search)."

addedInput schema / properties / urls

Added value: +{
+  "description": "Array of URLs to scrape — plain URL strings, or objects with \"url\" and optional \"custom_id\".",
+  "items": {
+    "anyOf": [
+      {
+        "format": "uri",
+        "type": "string"
+      },
+      {
+        "additionalProperties": false,
+        "properties": {
+          "custom_id": {
+            "type": "string"
+          },
+          "url": {
+            "format": "uri",
+            "type": "string"
+          }
+        },
+        "required": [
+          "url"
+        ],
+        "type": "object"
+      }
+    ]
+  },
+  "maxItems": 100000,
+  "minItems": 1,
+  "type": "array"
+}

changedInput schema / properties / urls_to_scrape / description

Previous value: -"JSON array of objects with \"url\" and optional \"custom_id\"."New value: +"Alias for `urls`."

changedInput schema / properties / urls_to_scrape / maxItems
```
Previous value: -10000New value: +100000
```

addedInput schema / properties / wait_for_completion_seconds

Added value: +{
+  "default": 45,
+  "description": "Seconds to wait for the batch to finish and return its content in this call. 0 returns the batch_id right away.",
+  "maximum": 900,
+  "minimum": 0,
+  "type": "integer"
+}

removedInput schema / required

Removed value: -[
-  "urls_to_scrape"
-]

Addedv1.0.0Jan 3, 2026

TDQS

A4.4/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/openWorld/non-destructive, and the description adds genuinely non-obvious behavior: the call blocks up to wait_for_completion_seconds (default 45), larger batches take 5-8 minutes and instead return a batch_id, which must then be drained via get_batch_results using next_cursor. It stops short of describing result payload shape or failure/timeout behavior for partial batches.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the selecting condition, then scale, then async behavior, then alternatives - every sentence carries distinct information with no restatement of the name or title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter batch/async tool with no output schema, the description covers the blocking-vs-async split and the follow-up call, which is the critical missing piece an agent needs. The unexplained alias parameter and lack of any note on partial failures or error handling keep it from fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so parameters are already documented; the description only reinforces wait_for_completion_seconds and 0-means-immediate-return, which the schema also states. It does not explain the urls_to_scrape alias, the parser/country/output_format effects, or why urls is not marked required despite being conceptually needed, and it references next_cursor which is not a parameter of this tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('scrape' a 'known list of URLs'), quantifies scale ('up to 100k in one batch'), and explicitly separates itself from scrape_website and create_crawl. An agent can select it without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit when-to-use condition ('a known list of URLs... from a CSV or from search results') plus named alternatives with their selecting conditions: scrape_website for one or a few URLs, create_crawl for site discovery, and get_batch_results for large batches. Exclusions and routing are both present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server