Skip to main content
Glama

ingest

Idempotent

Point an app at a website and it becomes data: crawl a site, land every page as markdown in the project's files, and index it into the project's search — one call instead of wiring up a crawler, a file store, and a search index yourself. Always runs as a background job (crawls take minutes) — this call returns immediately with a job_id; the jobs API reports its status, and search_query against the "ingest" index returns results once it completes.

The crawl never follows a link off the seed URL's exact host — it can't wander onto another site. Re-ingesting the same URL overwrites the same file and search item rather than duplicating it, so re-running ingest after a docs site updates is safe.

Example:

{ "project_id": "my-saas", "url": "https://docs.example.com", "max_pages": 40 }
// Returns: { "job_id": "job_...", "status": "queued" }

Then, once the job completes:

{ "project_id": "my-saas", "index": "ingest", "query": "how do I authenticate" }
// via search_query — returns hits pulled straight from the crawled pages.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "project_id": "<project_id>",
      "url": "<url>"
    },
    "name": "ingest"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlYesSeed http(s) URL to start crawling from, max 2048 chars. Only pages on this exact host are ever crawled.
max_pagesNoMaximum pages to crawl (1-200, default 50).
project_idYesProject ID (UUID), subdomain, or slug — resolved server-side, no UUID lookup needed. 'default' also works when the account has exactly one project; multi-project accounts must name one (the error lists them).
path_prefixNoProject files folder pages land under, default "ingest". Pages land at /{path_prefix}/{hostname}/{page-path}.md.
idempotency_keyNoStable key for one logical creation. The same key with identical arguments is safe after a lost response; independent work takes a new key.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnlyHint=false, openWorldHint=true, idempotentHint=true, destructiveHint=false), the description discloses the traits that actually matter here: execution is asynchronous and returns immediately with a job_id, the crawl is host-bounded and cannot wander off-site, and re-ingest overwrites the same file and search item instead of duplicating. That last point independently corroborates the idempotentHint annotation rather than contradicting it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core behavior is front-loaded in the first sentence and the async caveat follows immediately, which is the right order. The two JSON examples cost space and the second one illustrates search_query rather than ingest, but it does document the natural follow-up call, so the length is mostly earned.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return contract and does so explicitly: an immediate { job_id, status: "queued" } plus how to poll and how to query results later. Combined with the host restriction, overwrite semantics, and page limits, an agent has everything needed to call this correctly and interpret what comes back.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description still adds meaning the schema doesn't: it frames url as a seed whose exact host bounds the crawl, describes the overwrite semantics that motivate idempotency_key, and shows a concrete call with max_pages in the example. It doesn't add syntax detail for path_prefix beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence gives a concrete verb chain and resource: crawl a site, land every page as markdown in project files, and index it into project search. It also explicitly scopes the crawl to the seed URL's exact host and points to search_query for reading results, so an agent can tell this apart from generic browsing or file tools without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear usage context ('one call instead of wiring up a crawler, a file store, and a search index yourself') and explains the follow-up workflow: this call returns a job_id, the jobs API reports status, and search_query against the "ingest" index returns hits once complete. It also states re-running after a docs update is safe. It stops short of naming an alternative tool for one-off page fetches or explaining when not to ingest, which keeps it at 4.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources