Skip to main content
Glama

⚡ x402-cleanweb-agent

Turn any messy webpage, YouTube video, or PDF paper into pure, LLM-ready clean Markdown on Polygon.
Zero Sign-up. Zero Subscriptions. True Machine-to-Machine HTTP 402 Micropayments for Autonomous AI Agents.

Live Web3 DApp Swagger API Python 3.10+ Polygon Web3 License: MIT


💡 Why x402-cleanweb-agent?

Traditional web scraping and data extraction APIs force expensive $49/month subscriptions and complex API key management.

x402-cleanweb-agent solves this for autonomous AI agents, scrapers, and developers:

  • No Monthly Subscriptions: Pay only for what you query ($0.005 ~ $0.05 per call in USDC).

  • No Sign-ups or API Keys: Native HTTP 402 Payment Required machine-to-machine protocol.

  • 🤖 Zero-Human AI Agent Ready: AI agents with a crypto wallet can autonomously buy data 24/7.

  • Sub-Second Speed: Strips ads, tracking scripts, and clutter, returning pure, structured Markdown.

  • 🪙 Ultra-Low Gas Fees: Powered by Polygon PoS (< $0.005 network gas).

  • 📊 Token Savings Engine: Calculates raw vs. cleaned token reduction (avg. 60~85% savings) and estimated LLM prompt cost savings ($).


Related MCP server: ToolSnap MCP

🚀 Live Demo & Service Endpoints

Service

Endpoint

Pricing

Output & Description

🌐 Clean Web

GET /api/v1/clean-web

0.01 USDC

Ad/Noise removal + AI-ready Markdown + Token Savings Analytics

🎬 YouTube Transcript

GET /api/v1/clean-youtube

0.02 USDC

Full video transcripts with timestamps formatted in Markdown

📑 PDF Paper & Report

GET /api/v1/clean-pdf

0.05 USDC

arXiv papers & earnings reports converted into structured Markdown

📝 Pure Plain Text

GET /api/v1/clean-text

0.005 USDC

Ultra-lightweight raw text extraction for fast vector indexing


🤖 Zero-Human Autonomous AI Agent Integration (Python SDK)

AI agents with a Polygon wallet (Private Key) can autonomously handle payment and data extraction with zero human intervention:

from autonomous_agent_client import AutonomousX402Agent

# 1. Initialize Autonomous Agent with Polygon Wallet
agent = AutonomousX402Agent(private_key="0xYOUR_AGENT_PRIVATE_KEY")

# 2. Autonomous Clean Web Extraction (0.01 USDC)
result = agent.clean_web("https://example.com/article")
print("AI-Ready Markdown:\n", result["markdown_content"])
print("Token Savings:", result["token_analytics"]["token_savings_percentage"])

# 3. Autonomous YouTube Transcript Extraction (0.02 USDC)
yt_result = agent.clean_youtube("https://www.youtube.com/watch?v=dQw4w9WgXcQ")
print("YouTube Transcripts:\n", yt_result["markdown_transcript"])

# 4. Autonomous PDF Paper Extraction (0.05 USDC)
pdf_result = agent.clean_pdf("https://arxiv.org/pdf/2301.00001.pdf")
print("PDF Paper Markdown:\n", pdf_result["markdown_content"])

🛠️ How It Works (M2M Architecture)

sequenceDiagram
    autonumber
    actor Agent as Autonomous AI Agent
    participant Server as x402 Gateway (FastAPI)
    participant Polygon as Polygon Mainnet (Bor RPC)
    participant Scraper as AI Data Cleaning Engine

    Agent->>Server: GET /api/v1/clean-web?url=https://example.com
    Note over Server: Check X-Payment-Tx header
    Server-->>Agent: 402 Payment Required (Chain ID: 137, Recipient, Amount)
    
    Agent->>Polygon: Send USDC Transfer (e.g. 0.01 USDC)
    Polygon-->>Agent: Return Tx Hash (0xabc...123)
    
    Agent->>Server: GET /api/v1/clean-web?url=... with Header [X-Payment-Tx: 0xabc...123]
    Server->>Polygon: Verify Receipt, Event Logs, Recipient & Nonce
    Polygon-->>Server: Tx Confirmed (Status: 1)
    
    Server->>Scraper: Sanitize and Structure to Clean Markdown
    Scraper-->>Server: Return Clean Markdown + Token Analytics
    Server-->>Agent: 200 OK (Clean Markdown & Analytics JSON)

⚡ Direct cURL Quickstart

# Step 1: Query without payment to inspect 402 payment requirements
curl -i -X GET "https://x402-cleanweb-agent.onrender.com/api/v1/clean-web?url=https://example.com"

# Step 2: After sending USDC on Polygon, query with transaction hash
curl -X GET "https://x402-cleanweb-agent.onrender.com/api/v1/clean-web?url=https://example.com" \
  -H "X-Payment-Tx: 0x<YOUR_POLYGON_TX_HASH>"

Sample Response

{
  "status": "success",
  "service": "clean-web",
  "source_url": "https://example.com",
  "title": "Example Domain",
  "markdown_content": "# Example Domain\n\nThis domain is for use in illustrative examples...",
  "token_analytics": {
    "raw_html_estimated_tokens": 1250,
    "clean_markdown_estimated_tokens": 280,
    "tokens_saved": 970,
    "token_savings_percentage": "77.6%",
    "estimated_llm_cost_saved_usd": "$0.0029"
  },
  "processing_time_seconds": 0.38
}

🔌 Model Context Protocol (MCP) Setup

Automatically configures Claude Desktop & Cursor without editing JSON files:

# Windows
install_mcp.bat

# macOS / Linux
python install_mcp.py

Option 2: Run via uvx (No installation needed)

Add directly to your claude_desktop_config.json or Cursor:

{
  "mcpServers": {
    "polygon-x402-cleanweb": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/nohosa001-pixel/x402-cleanweb-agent", "x402-agent"]
    }
  }
}

Option 3: Manual Local Configuration

{
  "mcpServers": {
    "polygon-x402-cleanweb": {
      "command": "python",
      "args": ["-u", "/absolute/path/to/x402-micro-agent/mcp_server.py"],
      "env": {
        "PYTHONUNBUFFERED": "1",
        "POLYGON_RPC_URL": "https://polygon-bor-rpc.publicnode.com",
        "SERVER_WALLET_ADDRESS": "0x255F9991233f86B29dB847c8d5b8CB9915e80dCf",
        "USDC_CONTRACT_ADDRESS": "0x3c499c542cEF5E3811e1192ce70d8cC03d5c3359"
      }
    }
  }
}

Exposed MCP Tools

  • get_payment_info(): Retrieve pricing tiers and recipient address.

  • fetch_clean_markdown(url, payment_tx_hash): Clean Web scraper (0.01 USDC).

  • fetch_youtube_transcript(url, language, payment_tx_hash): YouTube transcript extractor (0.02 USDC).

  • fetch_pdf_markdown(url, payment_tx_hash): PDF research paper converter (0.05 USDC).

  • fetch_plain_text(url, payment_tx_hash): Lightweight text scraper (0.005 USDC).


🛠️ Local Development & Running

# 1. Clone repository
git clone https://github.com/nohosa001-pixel/x402-cleanweb-agent.git
cd x402-cleanweb-agent

# 2. Setup virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Start local server
python main.py
# Server running at: http://localhost:8000

📜 On-Chain Contract & Network Details


🤝 Contributing & License

Contributions and suggestions are welcome!
Feel free to open an issue or pull request on GitHub: https://github.com/nohosa001-pixel/x402-cleanweb-agent/issues

Distributed under the MIT License.

Install Server
A
license - permissive license
A
quality
B
maintenance

Maintenance

Maintainers
Response time
Release cycle
Releases (12mo)
Commit activity

Related MCP Servers

  • A
    license
    A
    quality
    B
    maintenance
    Pay-per-use clean web reader for AI agents. URL in, markdown plus metadata out, in milliseconds. Settled per-call in USDC over x402 — no signup, no API keys.
    2
    1
    85
    2
    MIT
  • A
    license
    A
    quality
    A
    maintenance
    Context-efficient MCP server for AI agents. fetch_extract reduces token usage by a median 98.1% (53,820 → 2,001 tokens) vs raw HTML — saves ~$0.156/call at Sonnet pricing. Pay $0.02 USDC on Base via x402. First call free per wallet. 10 always-free utility tools included.
    33
    1
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    MCP server providing 17 keyless, pay-per-use web-data tools with signed-provenance receipts, enabling AI agents to autonomously fetch, extract, and verify web content on Base mainnet.
    44
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    MCP server that enables AI agents to fetch web content as clean, structured Markdown via NanoParse's hosted API, using x402 micropayments in USDC on Base.
    1
    253
    MIT

View all related MCP servers

Related MCP Connectors

  • Pay-per-call web scraping for AI agents via x402 on Base USDC. Six tools, no signup.

  • Web intelligence for AI agents: fetch, render, extract, research. x402 micropayments, no API keys.

  • Scrape, crawl, map and extract the web. Pay per call in USDC, no account or API key.

View all MCP Connectors

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/nohosa001-pixel/x402-cleanweb-agent'

If you have feedback or need assistance with the MCP directory API, please join our Discord server