crawl4ai-mcp

crawl_website

Crawl a website from a specified URL, extracting content up to a defined depth and page limit. Returns structured data for crawled pages.

Instructions

Crawl a website starting from the given URL up to a specified depth and page limit.

Args: url: The starting URL to crawl. crawl_depth: The maximum depth to crawl relative to the starting URL (default: 1). max_pages: The maximum number of pages to scrape during the crawl (default: 5).

Returns: List containing TextContent with a JSON array of results for crawled pages.

Input Schema

Name	Required	Description	Default
`crawl_depth`	No
`max_pages`	No
`url`	Yes

Input Schema (JSON Schema)

{
  "properties": {
    "crawl_depth": {
      "default": 1,
      "title": "Crawl Depth",
      "type": "integer"
    },
    "max_pages": {
      "default": 5,
      "title": "Max Pages",
      "type": "integer"
    },
    "url": {
      "title": "Url",
      "type": "string"
    }
  },
  "required": [
    "url"
  ],
  "title": "crawl_websiteArguments",
  "type": "object"
}

This server cannot be installed

Other Tools from crawl4ai-mcp

Related Tools

crawl
@Dumpling-AI/mcp-server-dumplingai
ingestWebsite
@pbteja1998/sourcesyncai-mcp
tavily-crawl
@MertAtesmen/tavily-mcp-python
generate-site-map
@bsmi021/mcp-server-webscan
fetch_website_nested
@flutterninja9/better-fetch
firecrawl_crawl_process
@spences10/mcp-omnisearch

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/ritvij14/crawl4ai-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server