Open Search MCP

MIT License

Overview InspectNew Schema Related Servers Reviews Score

batch_crawl_urls

Extract main text content and links from multiple web pages simultaneously. Specify URLs, manage concurrent requests, and set delays for efficient batch crawling with Open Search MCP.

Instructions

Crawl and extract content from multiple web pages

Input Schema

Name	Required	Description
`delay`	No	Delay between batches in milliseconds
`extractLinks`	No	Extract all links from pages
`extractText`	No	Extract main text content
`maxConcurrent`	No	Maximum concurrent requests
`urls`	Yes	Array of URLs to crawl

Input Schema (JSON Schema)

{
  "properties": {
    "delay": {
      "default": 1000,
      "description": "Delay between batches in milliseconds",
      "maximum": 5000,
      "minimum": 500,
      "type": "number"
    },
    "extractLinks": {
      "default": false,
      "description": "Extract all links from pages",
      "type": "boolean"
    },
    "extractText": {
      "default": true,
      "description": "Extract main text content",
      "type": "boolean"
    },
    "maxConcurrent": {
      "default": 3,
      "description": "Maximum concurrent requests",
      "maximum": 5,
      "minimum": 1,
      "type": "number"
    },
    "urls": {
      "description": "Array of URLs to crawl",
      "items": {
        "type": "string"
      },
      "maxItems": 10,
      "type": "array"
    }
  },
  "required": [
    "urls"
  ],
  "type": "object"
}

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/flyanima/open-search-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server