Skip to main content
Glama
Sign In
Crawlora-org

Crawlora MCP

Official
by Crawlora-org
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Remote

web_crawl

Submit a bounded resumable web crawl. Accepts an owner-scoped durable crawl job. max_credits is required; the accepted request and active reader/source revisions are frozen for idempotent replay.

Instructions

Submit a bounded resumable web crawl. Accepts an owner-scoped durable crawl job. max_credits is required; the accepted request and active reader/source revisions are frozen for idempotent replay. A repeated matching idempotency key returns the original job without preparing another reservation.

Input Schema

NameRequiredDescriptionDefault
requestYesBounded crawl request

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv1.17.12

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does reasonably well: it discloses that max_credits is required, that the job is owner-scoped and durable, and that a matching repeated idempotency key returns the original job without a new reservation. It omits the async/non-blocking nature, auth/permission requirements, rate limits, and credit cost behavior, keeping it short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and kept to four sentences. Some jargon redundancy ("bounded resumable" vs "durable") and a dense final sentence cost a point, but nothing is wasted or buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description should arguably say what a submission returns (a job id) and how to follow up via web_crawl_status/results. For a complex nested request object with a job lifecycle, the idempotency coverage is good but the lifecycle pointers are missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds real meaning for idempotency_key (matching repeats replay the original job) and flags max_credits as required, but it does not clarify the nested limits, scope, fetch, or sitemap options beyond what the schema already carries.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ("Submit a bounded resumable web crawl") and adds characteristics (bounded, resumable, owner-scoped, durable) that distinguish it from a one-shot scrape. However, it never names the sibling lifecycle tools (web_crawl_status, web_crawl_results, web_crawl_cancel) or contrasts with web_search/web_scrape, so sibling differentiation is left to inference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description says what the tool is but not when to choose it over alternatives. There is no guidance such as "use this for multi-page crawls; use web_scrape for a single page" and no mention of the polling follow-up flow. Usage must be inferred entirely from the name and the shadow of sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools