Skip to main content
Glama

AIsa Web Search & Research

Graph-based website traversal tool using Tavily Crawl.

post_tavily_crawl
Read-onlyIdempotent

Walk a site from a root url and return the content of the pages it finds. Steer it with natural-language instructions plus regex path and domain filters, and bound it with max_depth, max_breadth and limit. Returns base_url and results[] with url and raw_content. Measured at about 4.5 seconds for a 3-page limit; cost and time grow with the bounds you set, so set them. Use it for broad coverage of one site — documentation, a catalogue, a competitor's blog. It answers synchronously, which post_firecrawl_crawl does not: that one runs as a background job and suits crawls too large to wait on. For a handful of known pages post_tavily_extract is far cheaper; to size a site before paying to crawl it, run post_tavily_map first.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesThe root URL to begin the crawl.
limitNoTotal number of links the crawler will process before stopping.
formatNoFormat of the extracted web page content.markdown
timeoutNoMaximum time in seconds to wait for the crawl operation.
max_depthNoMax depth of the crawl.
max_breadthNoMax number of links to follow per level of the tree.
instructionsNoNatural language instructions for the crawler.
select_pathsNoRegex patterns to select only URLs with specific path patterns.
exclude_pathsNoRegex patterns to exclude URLs with specific path patterns.
extract_depthNoDepth of the extraction process.basic
include_usageNoInclude credit usage information in the response.
allow_externalNoInclude external domain links in the final results list.
include_imagesNoInclude images in the crawl results.
select_domainsNoRegex patterns to select crawling to specific domains or subdomains.
exclude_domainsNoRegex patterns to exclude specific domains or subdomains from crawling.
include_faviconNoInclude the favicon URL for each result.
chunks_per_sourceNoMaximum number of relevant chunks returned per source.

Output Schema

TableJSON Schema
NameRequiredDescriptionDefault

No arguments

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly, openWorld, idempotent, and non-destructive hints. The description adds valuable behavioral context beyond those: it notes the synchronous nature ('It answers synchronously, which post_firecrawl_crawl does not') and provides a measured performance estimate ('about 4.5 seconds for a 3-page limit') plus a warning that cost/time grow with bounds. This is meaningful additional disclosure, though not exhaustive (no error behavior or rate limits mentioned).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a compact paragraph of about five sentences that front-loads the main action and then layers in constraints, performance, and alternatives. It is appropriately sized for a tool with 17 parameters and conveys high signal per sentence. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (17 params, multiple sibling tools), the description is exceptionally complete. It covers the core operation, use cases, performance expectations, cost implications, and clear routing to alternatives. The output schema exists, so return values are already documented, and the description doesn't need to repeat them. An agent has everything needed to decide whether and how to call it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers 100% of parameters, so the baseline is 3. The description adds conceptual value by grouping key parameters ('Steer it with natural-language instructions plus regex path and domain filters, and bound it with max_depth, max_breadth and limit') and by describing the return structure ('Returns base_url and results[] with url and raw_content'), which goes beyond the schema's isolated field descriptions. It doesn't explain each parameter in depth but provides a useful high-level model.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource: 'Walk a site from a root url and return the content of the pages it finds.' It clearly states the core function and immediately distinguishes itself from sibling tools by naming post_firecrawl_crawl, post_tavily_extract, and post_tavily_map, so an agent can tell them apart without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit guidance on when to use it: 'Use it for broad coverage of one site — documentation, a catalogue, a competitor's blog.' It also contrasts with alternatives: post_firecrawl_crawl for background/large crawls, post_tavily_extract for a handful of known pages, and post_tavily_map for sizing a site. This gives clear selection criteria and exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources