ingest
Point an app at a website and it becomes data: crawl a site, land every page as markdown in the project's files, and index it into the project's search — one call instead of wiring up a crawler, a file store, and a search index yourself. Always runs as a background job (crawls take minutes) — this call returns immediately with a job_id; the jobs API reports its status, and search_query against the "ingest" index returns results once it completes.
The crawl never follows a link off the seed URL's exact host — it can't wander onto another site. Re-ingesting the same URL overwrites the same file and search item rather than duplicating it, so re-running ingest after a docs site updates is safe.
Example:
{ "project_id": "my-saas", "url": "https://docs.example.com", "max_pages": 40 }
// Returns: { "job_id": "job_...", "status": "queued" }Then, once the job completes:
{ "project_id": "my-saas", "index": "ingest", "query": "how do I authenticate" }
// via search_query — returns hits pulled straight from the crawled pages.Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"project_id": "<project_id>",
"url": "<url>"
},
"name": "ingest"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Seed http(s) URL to start crawling from, max 2048 chars. Only pages on this exact host are ever crawled. | |
| max_pages | No | Maximum pages to crawl (1-200, default 50). | |
| project_id | Yes | Project ID (UUID), subdomain, or slug — resolved server-side, no UUID lookup needed. 'default' also works when the account has exactly one project; multi-project accounts must name one (the error lists them). | |
| path_prefix | No | Project files folder pages land under, default "ingest". Pages land at /{path_prefix}/{hostname}/{page-path}.md. | |
| idempotency_key | No | Stable key for one logical creation. The same key with identical arguments is safe after a lost response; independent work takes a new key. |