Which integrations are available for this server?

Provides web search and content extraction capabilities for GitHub Copilot, allowing it to search the web and crawl URLs to retrieve up-to-date information and documentation. Integrates real-time web search and crawling tools directly into OpenAI's API, enabling GPT models to search the web and extract content from URLs through the MCP protocol.

How do I use crawleo-mcp?

1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@crawleo-mcp search for latest AI news and summarize top 3 articles" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

crawleo-mcp

by Crawleo

Overview Schema Related Servers Score Discussions

JavaScript

Remote

Crawleo MCP Server

Real-time web search and crawling capabilities for AI assistants through Model Context Protocol (MCP).

Overview

Crawleo MCP enables AI assistants to access live web data through two powerful tools:

web.search - Real-time web search with multiple output formats
web.crawl - Deep content extraction from any URL

Related MCP server: firecrawl-mcp-server

Features

✅ Real-time web search from any country/language
✅ Multiple output formats - Enhanced HTML, Raw HTML, Markdown, Plain Text
✅ Device-specific results - Desktop, mobile, or tablet view
✅ Deep content extraction with JavaScript rendering
✅ Zero data retention - Complete privacy
✅ Auto-crawling option for search results

Installation

Option 1: NPM (Recommended for local usage)

Install globally via npm:

npm install -g crawleo-mcp

Or use npx without installing:

npx crawleo-mcp

Option 2: Clone Repository

git clone https://github.com/Crawleo/Crawleo-MCP.git
cd Crawleo-MCP
npm install
npm run build

Option 3: Docker

Build and run using Docker:

# Build the image
docker build -t crawleo-mcp .

# Run with your API key
docker run -e CRAWLEO_API_KEY=your_api_key crawleo-mcp

Docker configuration for MCP clients:

{
  "mcpServers": {
    "crawleo": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "-e", "CRAWLEO_API_KEY=YOUR_API_KEY_HERE", "crawleo-mcp"]
    }
  }
}

Option 4: Remote Server (No installation needed)

Use the hosted version at https://api.crawleo.dev/mcp - see configuration examples below.

Getting Your API Key

Visit crawleo.dev
Sign up for a free account
Navigate to your dashboard
Copy your API key (starts with sk_)

Setup Instructions

Using Local MCP Server (npm package)

After installing via npm, configure your MCP client to use the local server:

Claude Desktop / Cursor / Windsurf (Local):

{
  "mcpServers": {
    "crawleo": {
      "command": "npx",
      "args": ["crawleo-mcp"],
      "env": {
        "CRAWLEO_API_KEY": "YOUR_API_KEY_HERE"
      }
    }
  }
}

Or if installed globally:

{
  "mcpServers": {
    "crawleo": {
      "command": "crawleo-mcp",
      "env": {
        "CRAWLEO_API_KEY": "YOUR_API_KEY_HERE"
      }
    }
  }
}

From cloned repository:

{
  "mcpServers": {
    "crawleo": {
      "command": "node",
      "args": ["/path/to/Crawleo-MCP/dist/index.js"],
      "env": {
        "CRAWLEO_API_KEY": "YOUR_API_KEY_HERE"
      }
    }
  }
}

Using Remote Server (Hosted)

1. Claude Desktop

Location of config file:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json
Linux: ~/.config/Claude/claude_desktop_config.json

Configuration:

{
  "mcpServers": {
    "crawleo": {
      "url": "https://api.crawleo.dev/mcp",
      "transport": "http",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY_HERE"
      }
    }
  }
}

Replace YOUR_API_KEY_HERE with your actual API key from crawleo.dev.

Steps:

Open the config file in a text editor
Add the Crawleo MCP configuration
Save the file
Restart Claude Desktop completely (quit and reopen)
Start a new conversation and ask Claude to search the web!

Example usage:

"Search for the latest AI news and summarize the top 5 articles"
"Find Python web scraping tutorials and extract code examples"

2. Cursor IDE

Location of config file:

macOS: ~/.cursor/config.json or ~/Library/Application Support/Cursor/config.json
Windows: %APPDATA%\Cursor\config.json
Linux: ~/.config/Cursor/config.json

Configuration:

{
  "mcpServers": {
    "crawleo": {
      "url": "https://api.crawleo.dev/mcp",
      "transport": "http",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY_HERE"
      }
    }
  }
}

Steps:

Locate and open your Cursor config file
Add the Crawleo MCP configuration
Save the file
Restart Cursor
The MCP tools will be available in your AI assistant

Example usage in Cursor:

"Search for React best practices and add them to my code comments"
"Find the latest documentation for this API endpoint"

3. Windsurf IDE

Location of config file:

macOS: ~/Library/Application Support/Windsurf/config.json
Windows: %APPDATA%\Windsurf\config.json
Linux: ~/.config/Windsurf/config.json

Configuration:

{
  "mcpServers": {
    "crawleo": {
      "url": "https://api.crawleo.dev/mcp",
      "transport": "http",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY_HERE"
      }
    }
  }
}

Steps:

Open the Windsurf config file
Add the Crawleo MCP server configuration
Save and restart Windsurf
Start using web search in your coding workflow

4. GitHub Copilot

Location of config file:

For GitHub Copilot in VS Code or compatible editors, you need to configure MCP servers.

Configuration:

Create or edit your MCP config file and add:

{
  "servers": {
    "Crawleo": {
      "url": "https://api.crawleo.dev/mcp",
      "transport": "http",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY_HERE"
      }
    }
  }
}

Complete example with multiple servers:

{
  "servers": {
    "Crawleo": {
      "url": "https://api.crawleo.dev/mcp",
      "transport": "http",
      "headers": {
        "Authorization": "Bearer YOUR_API_KEY_HERE"
      }
    }
  }
}

Steps:

Open your GitHub Copilot MCP configuration
Add the Crawleo server configuration
Save the file
Restart VS Code or your IDE
GitHub Copilot can now use Crawleo for web searches!

Example usage:

Ask Copilot: "Search for the latest Python best practices"
Ask Copilot: "Find documentation for this library"

5. OpenAI Platform (Direct Integration)

OpenAI now supports MCP servers directly! Here's how to use Crawleo with OpenAI's API:

Python Example:

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-4",
    input=[
        {
            "role": "user",
            "content": [
                {
                    "type": "input_text",
                    "text": "search for latest news about openai models"
                }
            ]
        }
    ],
    text={
        "format": {
            "type": "text"
        },
        "verbosity": "medium"
    },
    reasoning={
        "effort": "medium"
    },
    tools=[
        {
            "type": "mcp",
            "server_label": "Crawleo",
            "server_url": "https://api.crawleo.dev/mcp",
            "server_description": "Crawleo MCP Server - Real-Time Web Knowledge for AI",
            "authorization": "YOUR_API_KEY_HERE",
            "allowed_tools": [
                "web.search",
                "web.crawl"
            ],
            "require_approval": "always"
        }
    ],
    store=True,
    include=[
        "reasoning.encrypted_content",
        "web_search_call.action.sources"
    ]
)

print(response)

Key Parameters:

server_url - Crawleo MCP endpoint
authorization - Your Crawleo API key
allowed_tools - Enable web.search and/or web.crawl
require_approval - Set to "always", "never", or "conditional"

Node.js Example:

import OpenAI from 'openai';

const client = new OpenAI();

const response = await client.responses.create({
  model: 'gpt-4',
  input: [
    {
      role: 'user',
      content: [
        {
          type: 'input_text',
          text: 'search for latest AI developments'
        }
      ]
    }
  ],
  tools: [
    {
      type: 'mcp',
      server_label: 'Crawleo',
      server_url: 'https://api.crawleo.dev/mcp',
      server_description: 'Crawleo MCP Server - Real-Time Web Knowledge for AI',
      authorization: 'YOUR_API_KEY_HERE',
      allowed_tools: ['web.search', 'web.crawl'],
      require_approval: 'always'
    }
  ]
});

console.log(response);

Available Tools

web.search

Search the web in real-time with customizable parameters.

Parameters:

query (required) - Search term
max_pages - Number of result pages (default: 1)
setLang - Language code (e.g., "en", "ar")
cc - Country code (e.g., "US", "EG")
device - Device type: "desktop", "mobile", "tablet" (default: "desktop")
enhanced_html - Get clean HTML (default: true)
raw_html - Get raw HTML (default: false)
markdown - Get Markdown format (default: true)
page_text - Get plain text (default: false)
auto_crawling - Auto-crawl result URLs (default: false)

Example:

Ask your AI: "Search for 'Python web scraping' and return results in Markdown"

web.crawl

Extract content from specific URLs.

Parameters:

urls (required) - List of URLs to crawl
rawHtml - Return raw HTML (default: false)
markdown - Convert to Markdown (default: false)
screenshot - Capture screenshot (optional)
country - Geographic location

Example:

Ask your AI: "Crawl https://example.com and extract the main content in Markdown"

Troubleshooting

MCP server not appearing

Check config file location - Make sure you're editing the correct file
Verify JSON syntax - Use a JSON validator to check for syntax errors
Restart the application - Completely quit and reopen (not just reload)
Check API key - Ensure your API key is valid and active at crawleo.dev

Authentication errors

Verify your API key is correct (should start with sk_)
Make sure the key is wrapped in quotes
Check that "Bearer " prefix is included in the Authorization header (for Claude/Cursor/Windsurf)
For OpenAI Platform, use the key directly in the authorization field
Confirm your account has available credits at crawleo.dev

No results returned

Check your internet connection
Verify the search query is not empty
Try a simpler search query first
Check API status at crawleo.dev

Tool names not recognized

Make sure you're using the correct tool names:

Use web.search (not search_web)
Use web.crawl (not crawl_web)

Usage Examples

Research Assistant

"Search for recent developments in quantum computing and summarize the key findings"

Content Analysis

"Search for competitor pricing pages and extract their pricing tiers"

Code Documentation

"Find the official documentation for FastAPI and extract the quickstart guide"

News Monitoring

"Search for today's news about artificial intelligence from US sources"

Market Research

"Search for customer reviews of iPhone 15 and analyze sentiment"

Pricing

Crawleo MCP uses the same affordable pricing as our API:

10,000 searches → $20
100,000 searches → $100
250,000 searches → $200

Check your usage and manage your subscription at crawleo.dev

Privacy & Security

✅ Zero data retention - We never store your search queries or results
✅ Secure authentication - API keys transmitted over HTTPS
✅ No tracking - Your usage patterns remain private

Support

Documentation: crawleo.dev/docs
API Status: crawleo.dev/status
Contact: support@crawleo.dev

Available Tools

2 tools

crawl_webCInspect

Crawl a specific webpage and extract its content in various formats including Markdown, raw HTML, and AI-enhanced HTML.

ParametersJSON Schema

Name	Required	Description
`url`	Yes	URL to crawl and extract content from
`markdown`	No	Return content in Markdown format
`raw_html`	No	Return original, unprocessed HTML
`enhanced_html`	No	Return AI-enhanced, cleaned HTML

TDQS

C2.9/5.0

Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the extraction of content in various formats but fails to disclose critical behavioral traits such as rate limits, authentication requirements, error handling, or what happens when crawling fails. For a web crawling tool with zero annotation coverage, this is a significant gap in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. Every part of the sentence contributes directly to understanding the tool's function, making it highly concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of web crawling (which involves network operations, potential failures, and format handling) and the lack of annotations and output schema, the description is incomplete. It doesn't address error conditions, return formats beyond naming them, or operational constraints, leaving significant gaps for the agent to infer behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description adds minimal value beyond the schema by mentioning 'various formats including Markdown, raw HTML, and AI-enhanced HTML,' which loosely corresponds to the boolean parameters but doesn't provide additional semantic context. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('crawl') and resource ('webpage'), and specifies the action ('extract its content in various formats'). It distinguishes from the sibling 'search_web' by focusing on content extraction from a specific URL rather than searching. However, it doesn't explicitly contrast with the sibling beyond the different action verbs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus the sibling 'search_web' or other alternatives. It mentions what the tool does but offers no context about appropriate use cases, prerequisites, or exclusions. This leaves the agent without clear direction on tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_webCInspect

Search the web using Crawleo's AI-powered search engine. Returns results with optional AI-enhanced HTML, markdown content, and structured data.

ParametersJSON Schema

Name	Required	Description	Default
`query`	Yes	Search query term. The main keyword or phrase to search for.
`max_pages`	No	Max result pages to crawl. Each page costs 1 credit. Min: 1
`setLang`	No	Language code for search interface (e.g., 'en', 'es', 'fr', 'ar')	en
`cc`	No	Country code for search results (e.g., 'US', 'GB', 'DE', 'EG')
`geolocation`	No	Geo location for search	random
`device`	No	Device simulation	desktop
`enhanced_html`	No	Return AI-enhanced, cleaned HTML optimized for processing
`raw_html`	No	Return original, unprocessed HTML of the page
`page_text`	No	Return extracted plain text without HTML tags
`markdown`	No	Return content in Markdown format for easy parsing

TDQS

C2.9/5.0

Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that results include 'optional AI-enhanced HTML, markdown content, and structured data,' which adds some context about output features. However, it fails to disclose critical behavioral traits such as rate limits, authentication needs, costs (implied by 'costs 1 credit' in the schema but not in the description), or error handling. For a search tool with 10 parameters, this is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and front-loaded, consisting of two sentences that directly state the tool's purpose and key output features. There's no wasted verbiage, and it efficiently communicates the core functionality. A slight deduction is made because it could integrate cost or sibling tool context more seamlessly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (10 parameters, no annotations, no output schema), the description is moderately complete. It covers the basic purpose and output formats but lacks details on behavioral aspects, usage guidelines, and output structure. Without annotations or an output schema, the agent must infer much from the input schema alone, leaving gaps in understanding the tool's full context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all 10 parameters. The description adds minimal value beyond the schema by hinting at 'AI-enhanced' outputs and 'structured data,' but it doesn't elaborate on parameter interactions or provide additional semantic context. This meets the baseline of 3 when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Search the web using Crawleo's AI-powered search engine.' It specifies the verb ('Search') and resource ('the web'), and mentions the engine's AI-powered nature. However, it doesn't explicitly differentiate from its sibling tool 'crawl_web' (e.g., by contrasting search vs. crawl operations), which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus its sibling 'crawl_web' or other alternatives. It lacks context about appropriate scenarios, exclusions, or prerequisites (e.g., when to prefer search over crawl). This leaves the agent without usage direction beyond the basic purpose.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

B3.1/5.0

Disambiguation5/5

The two tools have clearly distinct purposes: crawl_web targets a specific webpage for content extraction, while search_web performs broader web searches. There is no overlap in functionality, making it easy for an agent to choose the correct tool based on the need for focused crawling versus exploratory searching.

Naming Consistency5/5

Both tools follow a consistent verb_noun pattern (crawl_web, search_web) with clear, descriptive names that align with their functions. The naming is uniform and predictable, using snake_case throughout without any deviations.

Tool Count2/5

With only two tools, the server feels under-scoped for a web crawling and search domain. While the tools cover basic operations, the lack of additional tools (e.g., for managing crawl sessions, filtering results, or handling errors) limits functionality and may require agents to work around gaps in more complex workflows.

Completeness2/5

The tool surface is severely incomplete for a web crawling and search server. There are no tools for operations like configuring crawl parameters, managing search history, handling authentication, or processing batch requests, which are common in such domains. This will likely cause agent failures in extended or nuanced tasks.

Maintenance

ActivityInactive

ResponsivenessNo issues

Resources

Need Help?

Related Servers

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Fastcrawl
net.fastcrawl
Cloud scraping & crawling API for AI agents. Turn any URL into clean, LLM-ready markdown.
Browser Automation Web Scraping
1
agentsearch-mcp A
io.github.IsaiahDupree
LLM-ready web search + instant answers + URL-to-clean-text fetch for agents and RAG.
Search RAG Systems Browser Automation
31
Spider Cloud
cloud.spider
Crawl, scrape, search the web, and automate browsers at scale with anti-bot bypass.
Browser Automation Web Scraping
2
forage A
cash.forage
Web search and URL-to-markdown for agents. Pay per call in USDC on Base, no signup.
Browser Automation Web Scraping Search
21

Related MCP Servers

tavily-search
Search Web Scraping RAG Systems
Tomatio13
A
license
Not graded
quality
D
maintenance
Tavily AI search API
Updated a year ago (2025-08-19 14:04 UTC)
50
MIT
firecrawl-mcp-server
RAG Systems Web Scraping
firecrawl
A
license
A
quality
C
maintenance
A Model Context Protocol (MCP) server implementation that integrates with FireCrawl for advanced web scraping capabilities.
Updated 8 hours ago (2026-09-03 21:02 UTC)
26
40,139
7,386
MIT
tavily-mcp
Search Web Scraping Browser Automation
jackedelic
A
license
A
quality
D
maintenance
tavily-mcp
Updated a year ago (2025-05-29 12:27 UTC)
4
20,899
MIT
WaterCrawl MCP
Web Scraping Browser Automation Search
watercrawl
A
license
Not graded
quality
D
maintenance
Provides AI systems with web crawling, scraping, and search capabilities through WaterCrawl's API, enabling content extraction, site mapping, and web search with customizable options.
Updated 4 months ago (2026-05-01 05:31 UTC)
27
8
ISC

Appeared in Searches

A search for information about deep crawl web crawling and site auditing tools

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Crawleo/crawleo-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server

Crawleo MCP Server

Overview

Features

Installation

Option 1: NPM (Recommended for local usage)

Option 2: Clone Repository

Option 3: Docker

Option 4: Remote Server (No installation needed)

Getting Your API Key

Setup Instructions

Using Local MCP Server (npm package)

Using Remote Server (Hosted)

1. Claude Desktop

2. Cursor IDE

3. Windsurf IDE

4. GitHub Copilot

5. OpenAI Platform (Direct Integration)

Available Tools

web.search

web.crawl

Troubleshooting

MCP server not appearing

Authentication errors

No results returned

Tool names not recognized

Usage Examples

Research Assistant

Content Analysis

Code Documentation

News Monitoring

Market Research

Pricing

Privacy & Security

Support

Links

Available Tools

TDQS

TDQS

TDQS

Maintenance

Resources

Looking for Admin?

Related MCP Connectors

Related MCP Servers

tavily-search

firecrawl-mcp-server

tavily-mcp

WaterCrawl MCP

Appeared in Searches

Latest Blog Posts

MCP directory API