Crawl Website Source and Wait
ethora-sources-site-crawl-v2-waitSubmit a website URL for crawling and wait until the job reports completed or failed. Returns job status, ID, poll count, and duration, so you can ingest the result directly.
Instructions
Crawl a website URL and wait for the crawl to finish: enqueues the job, then polls it until it reports completed or failed. Returns { done, status, jobId, polls, durationMs, result }; done: false with a note means the budget ran out while the job was still running (it usually finishes server-side anyway).
Requires: a selected app (ethora-app-select) or an explicit appId.
Auth: app-token mode OR B2B mode with an explicit appId. Errors: 401/403 wrong auth; 400 malformed url; 504/timeout if it takes longer than timeoutMs (the job may still complete server-side — check with ethora-sources-site-list-v2).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Absolute URL to crawl, e.g. `https://example.com/docs`. | |
| appId | No | 24-char hex appId to ingest into. Required in B2B mode unless already set via `ethora-app-select`; ignored in app-token mode. | |
| timeoutMs | No | How long to poll for the crawl to finish, in milliseconds. Default 45000, chosen to stay under the ~60s request timeout most MCP clients enforce. Caps at 600000 (10 min) for clients that allow longer calls. | |
| followLink | No | If true, also crawl in-domain links reachable from `url`. Can ingest many pages — use with care. |