tavily-map
Map website URLs to analyze structure, navigation, and content. Ideal for site audits, content discovery, and understanding website architecture. Start with a root URL to visualize site paths and organization.
Instructions
A powerful web mapping tool that creates a structured map of website URLs, allowing you to discover and analyze site structure, content organization, and navigation paths. Perfect for site audits, content discovery, and understanding website architecture.
Input Schema
Name | Required | Description | Default |
---|---|---|---|
allow_external | No | Whether to allow following links that go to external domains | |
categories | No | Filter URLs using predefined categories like documentation, blog, api, etc | |
instructions | No | Natural language instructions for the crawler | |
limit | No | Total number of links the crawler will process before stopping | |
max_breadth | No | Max number of links to follow per level of the tree (i.e., per page) | |
max_depth | No | Max depth of the mapping. Defines how far from the base URL the crawler can explore | |
select_domains | No | Regex patterns to select crawling to specific domains or subdomains (e.g., ^docs\.example\.com$) | |
select_paths | No | Regex patterns to select only URLs with specific path patterns (e.g., /docs/.*, /api/v1.*) | |
url | Yes | The root URL to begin the mapping |