Skip to main content
Glama
Sign In
Ethora MCP Server

Ethora MCP Server

by dappros
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

Crawl Website Source and Wait

ethora-source-site-crawl-wait
Destructive

Crawl a website URL and wait for the job to finish, then add the crawled pages to an agent's knowledge base without deleting existing documents.

Instructions

Adds the crawled pages to an agent's knowledge base (the one named by agentId, else the app's default agent); existing documents are kept and nothing is deleted. A URL that is already indexed is refused with 409 unless force: true re-crawls it. Crawl a website URL and wait for the crawl to finish: enqueues the job, then polls it until it reports completed or failed. Returns { done, status, jobId, polls, durationMs, result }; done: false with a note means the budget ran out while the job was still running (it usually finishes server-side anyway). Requires: a selected app (ethora-app-select) or an explicit appId. Auth: app-token mode OR B2B mode with an explicit appId. Errors: 401/403 wrong auth (403 AGENT_NOT_WRITABLE when agentId is someone else's agent); 400 malformed url; 409 already indexed; 504/timeout if it takes longer than timeoutMs (the job may still complete server-side - check with ethora-source-site-list, or stop it with ethora-source-site-crawl-cancel). Related: ethora-agent-knowledge-get shows what the agent can search afterwards.

Input Schema

NameRequiredDescriptionDefault
urlYesAbsolute URL to crawl, e.g. `https://example.com/docs`.
appIdNo24-char hex appId to ingest into. Required in B2B mode unless already set via `ethora-app-select`; ignored in app-token mode.
forceNoRe-crawl a URL that is already indexed and overwrite the stored copy. Without it an already-indexed URL is refused with 409.
agentIdNoAgent whose knowledge base gets the content (24-char hex, from `ethora-agent-list` / `ethora-agent-create`). Must be your own agent or one of this app's. Omit to file it under the app's default agent (the one answering its website widget); an app still on the shared Support Agent first gets its own copy of it.
timeoutMsNoHow long to poll for the crawl to finish, in milliseconds. Default 45000, chosen to stay under the ~60s request timeout most MCP clients enforce. Caps at 600000 (10 min) for clients that allow longer calls.
followLinkNoIf true, also crawl in-domain links reachable from `url`. Can ingest many pages — use with care.
agentIdOrAddressNoAlias for `agentId` - either name is accepted, pass whichever you have.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv26.11.0

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations: it discloses that existing documents are kept and nothing is deleted, the 409 refusal on already-indexed URLs unless force is set, the exact return shape, the done:false budget-exhausted caveat, auth modes, and specific error codes (401/403, 403 AGENT_NOT_WRITABLE, 400, 409, 504). This is exactly the behavioral context annotations cannot carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the purpose, then behaviors, requirements, errors, and related tools in scannable sentences. Dense but each clause carries information; only minor tightening is possible.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutating, long-running, auth-sensitive tool with no output schema, the description compensates fully: it documents the return contract, timeout semantics, and every relevant error. Nothing needed to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents force, timeoutMs, followLink, and the agentId alias in detail; the description mostly corroborates rather than extends these. It adds the 409/force interplay, but that is already present in the schema, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Crawl a website URL') plus the blocking behavior ('wait...enqueues the job, then polls until completed or failed'), which distinguishes it from the sibling crawl-cancel and site-list tools. An agent knows exactly what this does without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit preconditions are given ('requires a selected app via ethora-app-select or an explicit appId'), and failure paths route to named alternatives ('check with ethora-source-site-list, or stop it with ethora-source-site-crawl-cancel'). The force/re-crawl condition is also spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools