Skip to main content
Glama

tealkit_crawl_and_index

Recursively crawl website documentation and index pages into a local offline RAG knowledge base for retrieval. Set a root URL and page limit to build searchable offline docs.

Instructions

Recursively crawls a website documentation tree (up to max_pages) and indexes it into the local offline RAG knowledge base.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesRoot URL to start recursive crawling (e.g. https://docs.flutter.dev)
max_pagesNoMaximum number of pages to crawl (default: 30, max: 200)

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.0.0

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It does not say whether the crawl is synchronous or returns a job id (a live question given tealkit_get_job_status / tealkit_run_background siblings), how long it may take, whether it overwrites or appends to the existing index, or whether it respects robots.txt/rate limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that names the action, the scope constraint, and the destination. No filler or redundant restatement of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a long-running crawl-and-index mutation with no annotations and no output schema, the description omits the two things an agent most needs: whether the call blocks or returns a job handle, and what happens to previously indexed content. The surrounding job-management siblings make that omission material.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both url and max_pages are already documented in the schema with examples, defaults and bounds. The description only restates max_pages as a scoping limit and adds no format or constraint detail beyond what the schema provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('recursively crawls a website documentation tree') and the outcome ('indexes it into the local offline RAG knowledge base'), with scope limited by max_pages. It implicitly distinguishes itself from the sibling tealkit_index_local_docs by specifying a website rather than local files, though it never names that sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'offline RAG knowledge base' hints at the purpose, but there is no explicit when-to-use guidance, no mention of the alternative tealkit_index_local_docs for local files, and no note on when to use this versus the job-management siblings (schedule_job, run_background).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.