Skip to main content
Glama

edf_crawl_site

Start a bounded crawl of a public website or sitemap to discover external hostnames, optionally check them, and return a job ID for polling.

Instructions

Start a bounded EDF crawl of one public website or sitemap, discover external hostnames, and optionally check them. Same-origin page queue, at most 500 pages/1000 domains. Returns job ID; poll edf_get_job. Native regex crawler does not render JavaScript.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYes
modeNosite
delay_msNo
max_pagesNo
max_domainsNo
check_domainsNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.0.0

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly=false, openWorld=true, idempotent=false, destructive=false, but the description adds meaningful context beyond them: hard bounds (500 pages / 1000 domains), a same-origin queue, async job-ID return pattern, and the critical limitation that the native regex crawler does not render JavaScript. It does not discuss rate limiting or failure behavior, keeping it just short of 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense and front-loaded: purpose first, then scope limits, then the return/poll pattern, then the JS caveat. Four tight sentences with little waste, though 'EDF' is used without expansion and the ordering could group the caps with the parameter meaning more tightly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter tool with 0% schema coverage and no output schema, the description covers purpose, bounds, async return, and a key limitation, but leaves delay_ms and the meaning of the returned job payload unaddressed. Adequate but with clear gaps for an agent configuring a crawl.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the burden. It maps some params to behavior — 'website or sitemap' (mode), 'at most 500 pages/1000 domains' (max_pages/max_domains), and 'optionally check them' (check_domains) — but delay_ms and the url constraint are never explained, and no value-level semantics beyond the caps are given.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Start a bounded EDF crawl of one public website or sitemap'), plus the side-effect of discovering external hostnames. It is clearly distinguishable from siblings like edf_check_domains or edf_dns_lookup, and it names the follow-up tool edf_get_job.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives implicit context for use (start a crawl, then poll edf_get_job) but offers no explicit when-to-use vs when-not, and does not contrast with alternatives such as edf_check_domains for just checking hostnames. The async handoff to edf_get_job is the only routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.