Skip to main content
Glama
ClearFetch - Unblocked Web & Anti-Bot MCP Server

ClearFetch - Unblocked Web & Anti-Bot MCP Server

README.md
# ClearFetch — Model Context Protocol (MCP) & SDKs for AI Agents

<div align="center">

[![PyPI version](https://img.shields.io/pypi/v/clearfetch.svg?color=blue)](https://pypi.org/project/clearfetch/)
[![npm version](https://img.shields.io/npm/v/clearfetch.svg?color=red)](https://www.npmjs.com/package/clearfetch)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![Node.js 18+](https://img.shields.io/badge/node-18+-green.svg)](https://nodejs.org/)

**The Official Model Context Protocol (MCP) Server & Client SDKs for ClearFetch.**  
*Bypass Cloudflare Turnstile, DataDome, and anti-bot challenges to deliver clean, token-optimized Markdown and structured JSON directly into your AI workflows.*

[Website](https://clearfetch.dev) • [API Documentation](https://clearfetch.dev/docs) • [PyPI SDK](https://pypi.org/project/clearfetch/) • [npm MCP Server](https://www.npmjs.com/package/clearfetch)

</div>

---

## 🚀 Quick Install & Distribution Packages

### 1. Model Context Protocol (MCP) Server for Cursor & Claude Desktop
Run instantly without cloning or local installation:
```bash
npx -y clearfetch
```

#### Claude Desktop Configuration (`claude_desktop_config.json`)
```json
{
  "mcpServers": {
    "clearfetch": {
      "command": "npx",
      "args": ["-y", "clearfetch"],
      "env": {
        "CLEARFETCH_API_KEY": "cf_live_your_api_key_here"
      }
    }
  }
}
```

#### Cursor IDE Configuration (`.cursor/mcp.json`)
```json
{
  "mcpServers": {
    "clearfetch": {
      "command": "npx",
      "args": ["-y", "clearfetch"],
      "env": {
        "CLEARFETCH_API_KEY": "cf_live_your_api_key_here"
      }
    }
  }
}
```

#### Available MCP Tools
- `clearfetch_read`: Fetches any protected URL and returns token-optimized, clean Markdown.
- `clearfetch_extract`: Extracts strictly-typed JSON matching your provided JSON schema.
- `clearfetch_search`: Performs concurrent web search and scrapes top results into a single context object.

---

### 2. Python SDK (`clearfetch`)

Install from PyPI:
```bash
pip install clearfetch
```

#### Basic Usage
```python
from clearfetch import ClearFetch

client = ClearFetch(api_key="cf_live_your_api_key_here")

# 1. Read protected webpage as clean Markdown
doc = client.read("https://www.bloomberg.com/markets", stealth="auto")
print(doc.title)
print(doc.markdown)

# 2. Extract structured JSON using JSON Schema
schema = {
    "type": "object",
    "properties": {
        "market_indices": {"type": "array", "items": {"type": "string"}},
        "top_stories": {"type": "array", "items": {"type": "string"}}
    },
    "required": ["market_indices"]
}
data = client.extract("https://www.bloomberg.com/markets", schema=schema)
print(data.data)
```

#### LangChain Integration
```python
from clearfetch.integrations.langchain import ClearFetchTool

tool = ClearFetchTool(api_key="cf_live_your_api_key_here")
response = tool.run("https://news.ycombinator.com")
print(response)
```

#### LlamaIndex Integration
```python
from clearfetch.integrations.llamaindex import ClearFetchReader

reader = ClearFetchReader(api_key="cf_live_your_api_key_here")
documents = reader.load_data(urls=["https://en.wikipedia.org/wiki/Artificial_intelligence"])
```

---

### 3. n8n Community Node (`n8n-nodes-clearfetch`)

In your n8n instance:
1. Navigate to **Settings → Community Nodes**.
2. Click **Install a community node**.
3. Enter `n8n-nodes-clearfetch` and confirm.

---

## 📡 REST API Reference

All requests authenticate using standard Bearer authorization: `Authorization: Bearer cf_live_...`.

### Read Webpage (`POST https://api.clearfetch.dev/v1/read`)
```bash
curl -X POST https://api.clearfetch.dev/v1/read \
  -H "Authorization: Bearer cf_live_your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/protected-page",
    "stealth": "auto",
    "only_main_content": true
  }'
```

### Extract Structured Data (`POST https://api.clearfetch.dev/v1/extract`)
```bash
curl -X POST https://api.clearfetch.dev/v1/extract \
  -H "Authorization: Bearer cf_live_your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://store.example.com/item/123",
    "json_schema": {
      "type": "object",
      "properties": {
        "product_name": { "type": "string" },
        "price_usd": { "type": "number" }
      },
      "required": ["product_name", "price_usd"]
    }
  }'
```

### Search & Scrape (`POST https://api.clearfetch.dev/v1/search`)
```bash
curl -X POST https://api.clearfetch.dev/v1/search \
  -H "Authorization: Bearer cf_live_your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Model Context Protocol servers 2026",
    "limit": 3,
    "scrape_results": true
  }'
```

---

## 📦 Packages in this Repository

- [`packages/mcp-server`](./packages/mcp-server): Model Context Protocol server published to npm as `clearfetch`.
- [`packages/python-sdk`](./packages/python-sdk): Python SDK with LangChain/LlamaIndex support published to PyPI as `clearfetch`.
- [`packages/n8n-nodes-clearfetch`](./packages/n8n-nodes-clearfetch): Official n8n community node.

---

## 🔑 Getting an API Key

Create a free account and get your API key at [clearfetch.dev](https://clearfetch.dev).

---

## 📄 License

MIT License. See [LICENSE](./LICENSE) for details.

TDQS

A3.8/5.0

Scored across 3 tools

Disambiguation5/5

Each tool has a distinct input signature and output shape: read_url takes a URL and returns Markdown, extract_structured_data takes a URL plus schema and returns typed JSON, and search_and_read takes a query and returns bundled results. An agent can pick unambiguously based on whether it has a URL or a query and whether it needs Markdown or schema-conforming JSON.

Naming Consistency5/5

All three names are snake_case and lead with a clear verb (read_, extract_, search_) followed by the object of the action. The pattern is predictable and consistent throughout.

Tool Count4/5

Three tools is on the lean side, but each maps to a genuinely different workflow (raw fetch, schema extraction, discovery+fetch) and none feels redundant. A focused anti-bot fetching server can justify a minimal surface, though one or two more convenience tools would not be out of place.

Completeness4/5

The fetch/extract/search triad covers the core content-acquisition lifecycle for agent context and RAG. The main gap is batch or multi-URL reading (and any crawl/site-traversal capability), which agents must work around by calling the tools repeatedly.

Maintenance

ActivityMaintained
ResponsivenessNo recent community issues