tavily-map
Map website URLs to analyze site structure, content organization, and navigation paths. Ideal for site audits, content discovery, and understanding website architecture.
Instructions
A powerful web mapping tool that creates a structured map of website URLs, allowing you to discover and analyze site structure, content organization, and navigation paths. Perfect for site audits, content discovery, and understanding website architecture.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| allow_external | No | Whether to allow following links that go to external domains | |
| categories | No | Filter URLs using predefined categories like documentation, blog, api, etc | |
| instructions | No | Natural language instructions for the crawler | |
| limit | No | Total number of links the crawler will process before stopping | |
| max_breadth | No | Max number of links to follow per level of the tree (i.e., per page) | |
| max_depth | No | Max depth of the mapping. Defines how far from the base URL the crawler can explore | |
| select_domains | No | Regex patterns to select crawling to specific domains or subdomains (e.g., ^docs\.example\.com$) | |
| select_paths | No | Regex patterns to select only URLs with specific path patterns (e.g., /docs/.*, /api/v1.*) | |
| url | Yes | The root URL to begin the mapping |
Input Schema (JSON Schema)
{
"properties": {
"allow_external": {
"default": false,
"description": "Whether to allow following links that go to external domains",
"type": "boolean"
},
"categories": {
"default": [],
"description": "Filter URLs using predefined categories like documentation, blog, api, etc",
"items": {
"enum": [
"Careers",
"Blog",
"Documentation",
"About",
"Pricing",
"Community",
"Developers",
"Contact",
"Media"
],
"type": "string"
},
"type": "array"
},
"instructions": {
"description": "Natural language instructions for the crawler",
"type": "string"
},
"limit": {
"default": 50,
"description": "Total number of links the crawler will process before stopping",
"minimum": 1,
"type": "integer"
},
"max_breadth": {
"default": 20,
"description": "Max number of links to follow per level of the tree (i.e., per page)",
"minimum": 1,
"type": "integer"
},
"max_depth": {
"default": 1,
"description": "Max depth of the mapping. Defines how far from the base URL the crawler can explore",
"minimum": 1,
"type": "integer"
},
"select_domains": {
"default": [],
"description": "Regex patterns to select crawling to specific domains or subdomains (e.g., ^docs\\.example\\.com$)",
"items": {
"type": "string"
},
"type": "array"
},
"select_paths": {
"default": [],
"description": "Regex patterns to select only URLs with specific path patterns (e.g., /docs/.*, /api/v1.*)",
"items": {
"type": "string"
},
"type": "array"
},
"url": {
"description": "The root URL to begin the mapping",
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
}