Spicrawl

Submit a batch scrape job

spicrawl_batch_submit

Submit many URLs to be scraped as one asynchronous job, with shared settings applied to every URL. Give EITHER urls or items, not both (at least one URL, unless open is true). Returns the job object (id, status, estimated_credits, progress). Poll with spicrawl_batch_status, read with spicrawl_batch_results or spicrawl_batch_task_content. Use this instead of many spicrawl_scrape calls when you have tens or thousands of URLs. Set open: true to keep the job accepting more URLs via spicrawl_batch_add_items; an open job never finishes until you call spicrawl_batch_close.

Input Schema

TableJSON Schema

Name	Required	Description
`name`	No	A human label for the job.
`open`	No	Keep the job accepting items after submission (append with spicrawl_batch_add_items, finish with spicrawl_batch_close).
`urls`	No	Shorthand: URLs scraped with the shared settings and no per-item overrides. Mutually exclusive with `items`.
`wait`	No	Milliseconds to wait after load (rendered scrapes).
`items`	No	Long form: one /v1/scrape request object per item. Besides `url` and `external_id`, any /v1/scrape field (e.g. `js_render`, `wait_for`, `response_format`) overrides the shared setting for that item only. Mutually exclusive with `urls`.
`format`	No	Output format for every item (`response_format`). spicrawl_batch_submit defaults to 'markdown', the same as spicrawl_scrape; an item's own `response_format` overrides it.
`render`	No	Render every URL with a browser (`js_render`).
`max_cost`	No	Ceiling in credits for ONE item (not the whole job — see credit_budget).
`priority`	No	Scheduling priority of the job relative to your other jobs.
`wait_for`	No	CSS selector to wait for before capturing (rendered scrapes).
`concurrency`	No	Max items in flight at once for this job (may be capped by the server; see warnings).
`max_attempts`	No	Per-item retry attempts on failure.
`credit_budget`	No	Ceiling on the WHOLE job, in credits; the run aborts when it would exceed this.
`premium_proxy`	No	Coming soon: not available yet, do not send. Use residential managed-pool exits for every URL.
`proxy_country`	No	Coming soon: not available yet, do not send. ISO-3166 alpha-2 exit country; requires premium_proxy.
`custom_headers`	No	Extra request headers sent with every URL.
`block_resources`	No	Resource types to block while rendering, e.g. ["image","font","media"].
`failure_threshold`	No	Abort the job once this many items have failed.
`main_content_only`	No	Strip each page to its main article.
`webhook_endpoint_id`	No	Id of a registered webhook endpoint notified (with the job object) when the job completes.

Output Schema

TableJSON Schema

Name	Required	Description
`id`	Yes	Job id. Pass it as `job_id` to the other spicrawl_batch_* tools.
`name`	No	The label given at submission, if any.
`open`	No	True while the job accepts appended items (it will not complete until closed).
`params`	No	The job-level scrape settings as stored.
`status`	No	Lifecycle: queued, running, paused, cancelling, completed, failed or cancelled. Keep polling while queued/running/paused/cancelling; stop at completed, failed or cancelled.
`priority`	No	Effective scheduling priority.
`progress`	No	Live item counters and the credits charged so far.
`warnings`	No	Decisions made on the caller's behalf at submission (a clamped concurrency, items not yet queued).
`error_code`	No	Machine code of a job-level failure.
`project_id`	No	Project the API key belongs to.
`started_at`	No	ISO-8601 time the first item started.
`status_url`	No	API path to poll for this job: /v1/batch/{id}.
`concurrency`	No	Effective per-job concurrency after the server's clamping.
`finished_at`	No	ISO-8601 time the job reached a terminal status.
`results_url`	No	API path of the job's results: /v1/batch/{id}/results.
`total_items`	No	Number of items in the job.
`max_attempts`	No	Effective attempts per item.
`submitted_at`	No	ISO-8601 time the job was accepted.
`error_message`	No	Human-readable job-level failure reason.
`estimated_credits`	No	Credits reserved at submission (a hold, not a charge; compare with progress.credits_charged).
`failure_threshold`	No	Failed-item count that aborts the job, when set.
`results_expire_at`	No	ISO-8601 deadline after which results answer 410 Gone.
`cancel_requested_at`	No	ISO-8601 time a cancel was accepted.
`estimated_credits_micro`	No	Same as estimated_credits in micro-credits.

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedOct 13, 2026

changedOutput schema / (root)

Previous value: -nullNew value: +{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "additionalProperties": true,
+  "properties": {
+    "cancel_requested_at": {
+      "description": "ISO-8601 time a cancel was accepted.",
+      "type": [
+        "string",
+        "null"
+      ]
+    },
+    "concurrency": {
+      "description": "Effective per-job concurrency after the server's clamping.",
+      "type": "number"
+    },
+    "error_code": {
+      "description": "Machine code of a job-level failure.",
+      "type": [
+        "string",
+        "null"
+      ]
+    },
+    "error_message": {
+      "description": "Human-readable job-level failure reason.",
+      "type": [
+        "string",
+        "null"
+      ]
+    },
+    "estimated_credits": {
+      "description": "Credits reserved at submission (a hold, not a charge; compare with progress.credits_charged).",
+      "type": "number"
+    },
+    "estimated_credits_micro": {
+      "description": "Same as estimated_credits in micro-credits.",
+      "type": "number"
+    },
+    "failure_threshold": {
+      "description": "Failed-item count that aborts the job, when set.",
+      "type": [
+        "number",
+        "null"
+      ]
+    },
+    "finished_at": {
+      "description": "ISO-8601 time the job reached a terminal status.",
+      "type": [
+        "string",
+        "null"
+      ]
+    },
+    "id": {
+      "description": "Job id. Pass it as `job_id` to the other spicrawl_batch_* tools.",
+      "type": "string"
+    },
+    "max_attempts": {
+      "description": "Effective attempts per item.",
+      "type": "number"
+    },
+    "name": {
+      "description": "The label given at submission, if any.",
+      "type": [
+        "string",
+        "null"
+      ]
+    },
+    "open": {
+      "description": "True while the job accepts appended items (it will not complete until closed).",
+      "type": [
+        "boolean",
+        "null"
+      ]
+    },
+    "params": {
+      "anyOf": [
+        {
+          "additionalProperties": {},
+          "type": "object"
+        },
+        {
+          "type": "null"
+        }
+      ],
+      "description": "The job-level scrape settings as stored."
+    },
+    "priority": {
+      "description": "Effective scheduling priority.",
+      "type": "number"
+    },
+    "progress": {
+      "additionalProperties": true,
+      "description": "Live item counters and the credits charged so far.",
+      "properties": {
+        "bytes": {
+          "description": "Total response bytes of finished items.",
+          "type": "number"
+        },
+        "cancelled": {
+          "description": "Items not run because the job was cancelled.",
+          "type": "number"
+        },
+        "completed": {
+          "description": "Items in any terminal status (succeeded + failed + cancelled + skipped).",
+          "type": "number"
+        },
+        "credits_charged": {
+          "description": "Credits actually charged so far (whole credits, rounded down).",
+          "type": "number"
+        },
+        "credits_charged_micro": {
+          "description": "Same as credits_charged in micro-credits (1 credit = 1,000,000).",
+          "type": "number"
+        },
+        "failed": {
+          "description": "Items that failed after all attempts; re-run them with spicrawl_batch_retry.",
+          "type": "number"
+        },
+        "percent_complete": {
+          "description": "completed / total x 100.",
+          "type": "number"
+        },
+        "remaining": {
+          "description": "Items not yet finished (total - completed).",
+          "type": "number"
+        },
+        "skipped": {
+          "description": "Items skipped (for example after failure_threshold or credit_budget stopped the run).",
+          "type": "number"
+        },
+        "succeeded": {
+          "description": "Items that succeeded. Only these are charged.",
+          "type": "number"
+        },
+        "total": {
+          "description": "Items in the job (grows when items are appended to an open job).",
+          "type": "number"
+        }
+      },
+      "type": "object"
+    },
+    "project_id": {
+      "description": "Project the API key belongs to.",
+      "type": "string"
+    },
+    "results_expire_at": {
+      "description": "ISO-8601 deadline after which results answer 410 Gone.",
+      "type": [
+        "string",
+        "null"
+      ]
+    },
+    "results_url": {
+      "description": "API path of the job's results: /v1/batch/{id}/results.",
+      "type": "string"
+    },
+    "started_at": {
+      "description": "ISO-8601 time the first item started.",
+      "type": [
+        "string",
+        "null"
+      ]
+    },
+    "status": {
+      "description": "Lifecycle: queued, running, paused, cancelling, completed, failed or cancelled. Keep polling while queued/running/paused/cancelling; stop at completed, failed or cancelled.",
+      "type": "string"
+    },
+    "status_url": {
+      "description": "API path to poll for this job: /v1/batch/{id}.",
+      "type": "string"
+    },
+    "submitted_at": {
+      "description": "ISO-8601 time the job was accepted.",
+      "type": "string"
+    },
+    "total_items": {
+      "description": "Number of items in the job.",
+      "type": "number"
+    },
+    "warnings": {
+      "anyOf": [
+        {
+          "items": {
+            "type": "string"
+          },
+          "type": "array"
+        },
+        {
+          "type": "null"
+        }
+      ],
+      "description": "Decisions made on the caller's behalf at submission (a clamped concurrency, items not yet queued)."
+    }
+  },
+  "required": [
+    "id"
+  ],
+  "type": "object"
+}

Changed1 schema field changedOct 13, 2026

removedInput schema / properties / engine

Removed value: -{
-  "description": "Pin the scrape engine for every item: `fetch` or `obscura` (must be allowed by the plan and served by the deployment); `camoufox` is coming soon, do not pin it yet. `chromium` cannot be pinned in a batch — use `render: true`, or spicrawl_scrape.",
-  "enum": [
-    "fetch",
-    "obscura",
-    "camoufox"
-  ],
-  "type": "string"
-}

First observedOct 1, 2026

TDQS

A4.7/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare non-read-only, non-idempotent, non-destructive behavior. The description adds genuine context beyond that: the job is asynchronous, it returns a job object, and an open job 'never finishes until you call spicrawl_batch_close' — a lifecycle trap the annotations do not convey. It stops short of stating credit/auth side effects or that concurrent submission of the same URLs will duplicate work.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first clause, followed by constraints, return shape, routing, and the alternative-tool guidance. Six sentences, each carrying distinct information, with no restatement of the name or title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 20-parameter, nested-object, async submission tool with a full output schema and 100% schema coverage, the description supplies exactly the missing layer: mutual exclusivity, open-job lifecycle, and the sibling tools to poll/read/extend with. No essential calling information is absent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds real meaning beyond the schema by stating the mutual exclusion of `urls` and `items` ('not both') and the minimum-content rule ('at least one URL, unless `open` is true'), which the schema does not express as a constraint.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource (submit many URLs as one asynchronous job) and immediately scopes it with 'shared settings applied to every URL'. It explicitly distinguishes itself from the scrape sibling and from batch_add_items/batch_close, so an agent can identify it without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

States the selection condition outright: 'Use this instead of many spicrawl_scrape calls when you have tens or thousands of URLs.' It also names the downstream lifecycle tools (batch_status to poll, batch_results/batch_task_content to read, batch_add_items/batch_close for open jobs) and the exclusive condition for urls vs items.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Spicrawl tools Inspect

Try in Browser

Give agents eyes on the webHosted screenshot MCP. Send a URL, get a pixel-perfect image. $0.002 per grab. Works with Claude, Cursor, and Codex.grabbit.liveAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources

Need Help?