Which integrations are available for this server?

Scans Codeforces public profiles and data as part of OSINT entity reconnaissance. Scans Dev.to public profiles and content as part of OSINT entity reconnaissance. Provides real-time web search through DuckDuckGo, including query results and dorking, with fallback search engines for resilience against rate limits. Scans GitHub public profiles and data as part of OSINT entity reconnaissance. Scans GitLab public profiles and data as part of OSINT entity reconnaissance. Scans Google Scholar public profiles and publications as part of OSINT entity reconnaissance. Provides Wayback Machine time-travel queries for URLs across custom date ranges, enabling retrieval of historical snapshots and deleted or changed pages. Scans Kaggle public profiles and data as part of OSINT entity reconnaissance. Scans LeetCode public profiles and data as part of OSINT entity reconnaissance. Scans Medium public profiles and content as part of OSINT entity reconnaissance. Provides GEOINT and location intelligence, including address resolution, GPS coordinates, and administrative boundary data via OpenStreetMap. Scans Reddit public profiles and activity as part of OSINT entity reconnaissance. Scans ResearchGate public profiles and publications as part of OSINT entity reconnaissance. Scans Substack public profiles and publications as part of OSINT entity reconnaissance. Provides instant transcription of YouTube videos, shorts, and live streams with timestamped transcripts via HTTP streams, without downloading video or requiring a local GPU.

How do I use InfinityScrape MCP?

1. Click on "Deploy Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@InfinityScrape MCP scrape https://example.com and summarize the page" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

InfinityScrape MCP

by Fearlessversee

Overview Schema Source Related Servers Discussions Admin

Python

Local

🌐 InfinityScrape MCP: World-Class Web Scraping, Dynamic SPA Rendering & 33-Tool OSINT Intelligence Suite

InfinityScrape MCP is a standalone, production-grade Model Context Protocol (MCP) server engineered to provide AI models (Open WebUI, Claude 3.7, DeepSeek-R1/V3, Antigravity AI, Cursor, LM Studio) with unlimited, high-speed, anti-bot resilient web scraping, dynamic SPA rendering, zero-selector contact/pricing/FAQ extraction, remote Word/Excel document streaming, XML sitemap crawling, DuckDuckGo search, Wayback Machine time-travel, instant YouTube transcription, and precision OSINT / GEOINT location intelligence.

📑 Table of Contents

Related MCP server: InfinityScrape MCP

🌟 Why InfinityScrape MCP?

Standard web scrapers often fail on modern websites due to Cloudflare challenges, heavy client-side JavaScript rendering, intrusive cookie consent modals, and rate limits. InfinityScrape solves these problems out-of-the-box:

Dual-Engine Scraping Architecture:
- Fast TLS Engine (primp + httpx): Mimics real Chrome/Safari browser TLS/JA3 fingerprints and HTTP/2 headers to bypass Cloudflare and Akamai challenges in <100ms.
- Dynamic Headless Browser (Playwright Chromium): Renders complex SPAs (React, Vue, Next.js, Angular), performs infinite scrolling, clicks elements, and executes custom JavaScript.
Singleton BrowserPool (~1.8s SPA Renders):
- Persistent Chromium process with ephemeral context isolation that completely eliminates cold-start latency and avoids memory leaks.
Military-Grade Anti-Bot Evasions (100% Undetectable):
- Cleanly deletes navigator.webdriver from prototype (get: () => undefined).
- Authentic WebGL Vendor & Renderer spoofing (Google Inc. (NVIDIA) & NVIDIA GeForce RTX 3080).
- Hardware concurrency (8 cores), device memory (8GB), and authentic window.chrome runtime emulation.
- Realistic Chrome PDF Viewer plugins and NaCl mimeTypes.
3-Tier Multi-Engine Search Failover (Anti-429 Resilience):
- Cascades automatically from DuckDuckGo API ➡️ DuckDuckGo HTML Lite ➡️ Bing HTML Fallback. Zero API keys, zero 429 rate-limits.
Cloudflare Turnstile & Interstitial Auto-Solver:
- Detects Turnstile challenge iframes and executes automated coordinate jitter to bypass interstitials.
Network-Level Ad & Tracker Elimination:
- Intercepts and aborts network calls to 35+ ad networks and tracking scripts (doubleclick, criteo, outbrain, google-analytics) before they download, cutting page load time by ~300% and memory usage by 70%.
- Automatically detects and decomposes OneTrust, Cookiebot, and sticky overlay popups.
Wayback Machine Time-Travel:
- Query internet archive history for any URL across custom date ranges to track competitor pricing changes, deleted pages, and historical copy.
Zero-GPU Instant YouTube Transcriber:
- Extracts complete video/shorts/live transcripts with timestamps ([MM:SS]) in <300ms directly via HTTP streams without downloading video or requiring local GPU Whisper models.
Deep Recursive Documentation Crawler:
- Asynchronous Breadth-First-Search (BFS) crawler with domain locking and path prefix filtering to aggregate entire documentation trees into unified Markdown.
State-of-the-Art Public OSINT & GEOINT Reconnaissance:
- Multi-Signal Confidence Scoring (0% - 100%): Evaluates Name + City + Street + PIN + Org + Role correlation to rank discovered dossiers.
- 25+ Global Platform Scanners: Scans GitHub, GitLab, StackOverflow, Kaggle, HuggingFace, LeetCode, Codeforces, Dev.to, Medium, Substack, Google Scholar, ResearchGate, Reddit, etc.
- OpenStreetMap GEOINT: Resolves global addresses down to street/postcode level with GPS coordinates and administrative boundaries.
SQLite Persistent Caching Layer:
- In-memory and SQLite-backed local cache for instant 0ms responses on repeat lookups with configurable TTL.

⚡ Competitive Comparison

Feature / Capability	Standard MCP Scrapers	Cloud Scraping APIs	InfinityScrape MCP
Cost & API Keys	Free (Basic)	Paid ($20 - $200/mo)	100% Free / Zero API Keys
Cloudflare / Akamai TLS Bypass	❌ Fails / 403	✅ Yes	✅ Built-in (`primp` JA3)
Dynamic SPAs & Infinite Scroll	❌ Limited	✅ Yes	✅ Built-in (`playwright`)
Real-Time Web Search & Dorking	❌ No	⚠️ Extra Cost	✅ Built-in (DuckDuckGo & Dorks)
Wayback Historical Snapshots	❌ No	❌ No	✅ Built-in (Archive API)
Network-Level Ad & Popup Stripping	❌ No	⚠️ Partial	✅ Built-in (35+ domains)
Zero-GPU YouTube Transcripts	❌ No	❌ No	✅ Built-in (<300ms)
Online PDF Page-by-Page Parser	❌ No	⚠️ Extra Cost	✅ Built-in (`pypdf`)
Deep Documentation Crawler	❌ No	⚠️ Extra Cost	✅ Built-in (Async BFS)
25+ Platform OSINT & Geocoding	❌ No	❌ No	✅ Built-in (0-100% Confidence)
OpenAPI 3.1.0 REST Bridge (Port 8000)	❌ No	⚠️ Proprietary	✅ Built-in (FastAPI /docs)

🏗️ Architectural Overview

                      ┌─────────────────────────────────────────────────────────┐
                      │    AI Clients: Open WebUI / Claude Desktop / Cursor     │
                      └────────────────────────────┬────────────────────────────┘
                                                   │
                  ┌────────────────────────────────┴────────────────────────────────┐
                  │                                                                 │
                  ▼                                                                 ▼
      [OpenAPI Bridge (Port 8000)]                                     [Stdio JSON-RPC 2.0 Server]
      FastAPI /docs & /openapi.json                                             (server.py)
                  │                                                                 │
                  └────────────────────────────────┬────────────────────────────────┘
                                                   │
                                                   ▼
            ┌──────────────────────┬──────────────────────┬──────────────────────┐
            ▼                      ▼                      ▼                      ▼
     [Fast TLS Engine]     [Playwright Engine]     [OSINT / GEOINT]     [Search & Media]
     • primp JA3/TLS       • Stealth Chromium      • 25+ Platform       • DuckDuckGo Search
     • HTTP/2 Headers      • Ad/Tracker Blocker      Scanners           • Wayback Snapshots
     • <100ms Execution    • Infinite Scroll       • OpenStreetMap      • YouTube (<300ms)
                           • Auto-Dismiss CMPs     • Reverse Geocoding  • Remote PDF Parser
                                                   • Match Confidence
                                                   │
                                                   ▼
                                    ┌──────────────────────────────┐
                                    │ SQLite Caching Layer (0ms)   │
                                    └──────────────────────────────┘

🚀 Quick Start & 1-Click Installation

1. Automated Setup

# Clone the repository
git clone https://github.com/virajverse/infinity-scraper.git
cd infinity-scraper

# Create virtual environment & install
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
pip install -e .
playwright install chromium

2. Launch FastAPI Bridge (Port 8000)

python openapi_bridge.py

Interactive Swagger Docs: http://127.0.0.1:8000/docs
OpenAPI 3.1.0 Schema: http://127.0.0.1:8000/openapi.json

🔌 AI Client Integration

1. Open WebUI (FastAPI Bridge on Port 8000)

Ensure the bridge is running (python openapi_bridge.py).
In Open WebUI, navigate to Workspace -> Tools -> Add Tool.
Import from URL: http://127.0.0.1:8000/openapi.json or use infinity_scraper_suite.
All 25 tools are instantly accessible to your agents!

2. Antigravity AI / Claude Desktop (Native Stdio)

Add to your mcp_config.json:

{
  "mcpServers": {
    "infinity-scraper": {
      "command": "python",
      "args": ["-m", "infinity_scraper.server"],
      "env": {
        "PYTHONUNBUFFERED": "1"
      }
    }
  }
}

🛠️ Complete 33-Tool Reference Catalog

1. Anti-Bot Web Scraping, Dynamic SPAs & Crawlers (6 Tools)

Tool	Description
`scrape_url`	Production-grade scraping into clean, ad-free Markdown with auto-engine switching (Fast TLS -> Playwright Chromium fallback). Automatically surfaces anti-bot challenge advisories for agent escalation.
`scrape_dynamic`	Dynamic SPA rendering via Singleton Playwright BrowserPool (~1.8s) with Cloudflare Turnstile auto-bypass and infinite scroll.
`extract_schema`	Zero-selector semantic extraction mapping custom JSON schemas with auto JSON-LD & OpenGraph meta fallback.
`deep_crawl`	Asynchronous recursive BFS documentation crawler with depth limits, domain locking, and path prefix filters.
`extract_structured`	Extracts HTML tables into structured Markdown/JSON datasets and aggregates rich page metadata.
`scrape_batch`	High-throughput concurrent scraping of multiple URLs with configurable concurrency limits.

2. Real-Time Web Search, Research & Archive OSINT (4 Tools)

Tool	Description
`search_web`	3-tier resilient real-time web search (DDGS API ➡️ DDG HTML Lite ➡️ Bing HTML Fallback) without API keys or 429 rate limits.
`search_and_scrape`	End-to-end autonomous web research pipeline: executes queries and scrapes top results into a synthesized research report.
`osint_search`	Advanced boolean dorking engine supporting `site:`, `filetype:`, `in_title:`, and negative keyword exclusions.
`osint_wayback_time_machine`	Queries historical archive snapshots, CDX timestamps, and past versions of deleted or updated web pages.

3. Media, Document & RAG Extraction (4 Tools)

Tool	Description
`extract_pdf`	Streams remote online PDF files page-by-page via TLS into clean Markdown text without local disk bloat.
`get_youtube_transcript`	Zero-GPU, sub-300ms transcript extraction with timestamps (`[MM:SS]`) from YouTube videos, shorts, and live streams.
`extract_image_exif`	Extracts camera hardware specifications, timestamps, and GPS coordinates with direct Google Maps navigation links.
`optimize_rag_chunks`	Semantic text and Markdown chunker with token boundary optimization for LLM RAG pipelines and vector stores.

4. Deep Public OSINT & Entity Reconnaissance (4 Tools)

Tool	Description
`osint_deep_public_recon`	Cross-correlates developer platforms, academic registries, and corporate filings into an entity intelligence dossier.
`osint_username_check`	Scans 25+ global developer, creator, and tech platforms to map digital handles and alias footprints.
`extract_reddit_thread_tool`	Extracts Reddit discussions, original post content, and nested comment trees into structured Markdown.
`extract_rss_feed_tool`	Ingests RSS and Atom feeds for real-time news tracking, competitor updates, and blog monitoring.

5. Infrastructure, Domain & Network Reconnaissance (5 Tools)

Tool	Description
`osint_domain_recon`	Audits domain WHOIS, RDAP records, registrar info, and SSL/TLS certificate chains.
`osint_tech_stack`	Deeply fingerprints website frontend frameworks, backend stacks, CDNs, and analytics trackers.
`osint_ip_lookup`	Gathers IP geolocation, Autonomous System Number (ASN), ISP, and network routing data.
`osint_subdomain_enumeration`	Discovers hidden subdomains, internal staging servers, and API routes via Certificate Transparency logs (crt.sh).
`osint_dns_audit`	Resolves and audits DNS records (A, AAAA, MX, TXT, NS, CNAME) including mail security SPF/DKIM verification.

6. Precision GEOINT & Location Intelligence (2 Tools)

Tool	Description
`osint_geoint_lookup`	Forward geocoding of landmarks, streets, and addresses via OpenStreetMap/Nominatim down to postal code and GPS coordinates.
`osint_location_entity_search`	Hierarchical entity location dorking correlating business entities with city, street, and postal landmarks.

7. Zero-Selector Semantics, Office Streaming & Change Radar (8 Tools)

Tool	Description
`extract_contacts`	Zero-selector extraction of verified emails, phone numbers, WhatsApp, GSTIN/CIN/EIN, social profiles, and postal codes.

| extract_pricing_table | Automatically identifies SaaS pricing cards, tier amounts ($/mo, $/yr, INR), billing cycles, feature matrices, and CTA buttons. | | extract_faqs | Heuristic and schema-based FAQ extractor pulling question-and-answer pairs from JSON-LD, <details>, and accordions. | | crawl_sitemap | High-speed XML sitemap parser with gzip decompression (.xml.gz) and recursive child sitemap index traversal. | | discover_sitemaps | Autonomously uncovers all active XML sitemaps for any domain via robots.txt inspection and standard endpoint probing. | | extract_docx_document | Streams remote online or local Word (.docx) documents into clean Markdown with full table matrices (zero local bloat). | | extract_xlsx_spreadsheet| Streams online or local Excel (.xlsx) spreadsheets, converting all worksheets into clean Markdown tables and JSON records. | | compare_page_diff | Temporal page diff & change radar detecting price adjustments, discontinued tiers, new features, and content similarity %. |

💻 Command-Line Interface (CLI)

InfinityScrape provides a built-in CLI (infinity-scraper-cli or python -m infinity_scraper.cli) for rapid terminal verification:

# 1. Scrape a webpage into clean Markdown (Fast TLS or Playwright with --browser)
infinity-scraper-cli scrape "https://news.ycombinator.com"
infinity-scraper-cli scrape "https://example.com/spa" --browser --scroll 2

# 2. Live Web Search & Instant Scraping
infinity-scraper-cli search "Generative Engine Optimization 2026" --max 5
infinity-scraper-cli search "Agentic Workflows" --max 3 --scrape

# 3. Recursive Documentation Tree Crawler
infinity-scraper-cli crawl "https://modelcontextprotocol.io" --pages 5 --depth 2

# 4. Stream & Extract Remote PDF Document
infinity-scraper-cli pdf "https://arxiv.org/pdf/1706.03762.pdf" --pages 10

# Note: All 25 tools (YouTube transcripts, OSINT, GEOINT, Wayback) are directly accessible via:
# - MCP Stdio: `python -m infinity_scraper.server`
# - OpenAPI Bridge (Port 8000): `http://127.0.0.1:8000/docs`

🧪 Running Automated Tests

# Run unit and integration tests
pytest tests/ -v

📄 License & Authors

Author: Viraj
License: MIT License. See LICENSE for details.

Available Tools

33 tools

compare_page_diffA

Temporal Page Diff & Change Radar: Compares two versions of a webpage or text, detecting pricing shifts, added/removed feature sections, and calculating overall content similarity percentage.

Args: old_content_or_url: Historical text/markdown or URL. new_content_or_url: Current live text/markdown or URL.

Inputs: new_content_or_url (required), old_content_or_url (required). Returns: result

crawl_sitemapB

High-Speed XML Sitemap & Index Parser: Streams and parses XML sitemaps, handling namespaces, .xml.gz gzip streams, and automatically crawls child sitemaps if a sitemap index is provided.

Args: sitemap_url: URL of the sitemap (e.g. 'https://example.com/sitemap.xml'). max_urls: Maximum URLs to extract (default 200). crawl_nested: If true, automatically follows nested sitemaps inside sitemapindex.

Inputs: sitemap_url (required), max_urls, crawl_nested. Returns: result

deep_crawlC

Recursively crawl a website or documentation tree up to max_depth and aggregate into unified documentation.

Inputs: start_url (required), max_depth, max_pages, path_prefix, +2 more. Returns: result

discover_sitemapsA

Autonomously discovers all active XML sitemaps for a website by reading robots.txt and testing standard location heuristics (/sitemap.xml, /sitemap_index.xml, etc.).

Args: base_url: The root domain or homepage URL (e.g. 'https://stripe.com').

Inputs: base_url (required). Returns: result

extract_contactsA

Zero-Selector Contact & Lead Intelligence: Heuristically extracts emails, phone numbers, WhatsApp links, social profiles, physical addresses, PIN/ZIP codes, and business registration IDs (GSTIN/CIN/EIN) from a webpage or raw text.

Args: url: The webpage URL to scrape and analyze (optional if html_text is given). html_text: Raw HTML or text to analyze directly (optional if url is given).

Inputs: url, html_text. Returns: result

extract_docx_documentA

Word Document (.docx) Streamer & Table Parser: Streams and converts remote or local Word .docx files into clean Markdown preserving headings (H1-H4), bullet points, and full table matrices without local disk bloat.

Args: url_or_path: Remote URL (http/https) or local absolute file path to the .docx file.

Inputs: url_or_path (required). Returns: result

extract_faqsB

Zero-Selector FAQ & Accordion Intelligence Extractor: Extracts structured question-and-answer pairs by inspecting JSON-LD FAQPage schemas, HTML5 tags, and accordion CSS heuristics.

Args: url: The webpage URL to extract FAQs from. html_text: Raw HTML to extract FAQs from.

Inputs: url, html_text. Returns: result

extract_image_exifA

Image EXIF & Geotag Recon Tool: Extracts camera make/model, creation timestamp, editing software, and exact GPS coordinates (with map link) from any online image.

Args: image_url: The direct HTTP/HTTPS URL of the image file (.jpg, .png, .tiff, .webp).

Inputs: image_url (required). Returns: result

extract_pdfB

Download and extract full text and metadata from an online PDF document.

Inputs: url (required), max_pages. Returns: result

extract_pricing_tableB

Zero-Selector SaaS & Product Pricing Table Extractor: Automatically identifies tier names (Free, Pro, Enterprise), price amounts ($/mo, $/yr, INR), billing cycles, feature bullet lists, popular plan badges, and CTA links.

Args: url: The pricing page URL to scrape and analyze. html_text: Raw HTML or text of the pricing section.

Inputs: url, html_text. Returns: result

extract_reddit_thread_toolA

Reddit Discussion Thread Extractor: Extracts full Reddit post content, upvote ratio, author, and top community comments into structured dialogue Markdown.

Args: url: Full Reddit post URL (e.g. 'https://www.reddit.com/r/programming/comments/...'). max_comments: Number of top comments to extract (default 25).

Inputs: url (required), max_comments. Returns: result

extract_rss_feed_toolA

RSS & Atom Feed Ingestion Tool: Extracts recent articles, publish dates, links, and summaries from any RSS or Atom feed (Substack, blogs, news).

Args: feed_url: The RSS or Atom feed URL (e.g. 'https://openai.com/news/rss.xml' or 'https://blog.bytebytego.com/feed'). max_items: Number of recent articles to extract (default 10).

Inputs: feed_url (required), max_items. Returns: result

extract_schemaB

Extract specific structured fields from a webpage using a key-to-CSS-selector dictionary map.

Inputs: url (required), schema (required), use_browser. Returns: result

extract_structuredC

Extract structured data (JSON-LD schema, HTML tables, metadata, and CSS-targeted nodes) as JSON.

Inputs: url (required), use_browser, css_selector, extract_tables, +1 more. Returns: result

extract_xlsx_spreadsheetA

Excel Spreadsheet (.xlsx) Streamer & Structured Table Extractor: Streams remote or local .xlsx spreadsheets and parses all worksheets into clean Markdown tables and structured JSON records.

Args: url_or_path: Remote URL (http/https) or local absolute file path to the .xlsx file. max_rows: Maximum rows to extract per worksheet (default 100).

Inputs: url_or_path (required), max_rows. Returns: result

get_youtube_transcriptC

Extract full text transcript and timestamps from any YouTube video URL in milliseconds (Zero GPU).

Inputs: url (required), with_timestamps. Returns: result

optimize_rag_chunksA

Semantic RAG Chunker & Token Optimizer: Splits large scraped web documents into header-aware semantic chunks with token counts and section headers for LLM embeddings and context efficiency.

Args: text_or_markdown: The large Markdown or text string to chunk. max_chunk_chars: Maximum characters per chunk (default 2000 ~500 tokens).

Inputs: text_or_markdown (required), max_chunk_chars. Returns: result

osint_deep_public_reconC

Ultra-Precision Public Intelligence Dossier Tool: Scrapes and correlates open web data across 25+ platforms with negative exclusions, PIN/Street drill-down, and multi-signal Confidence Scoring (0% - 100%).

Inputs: name (required), location, time_range, postal_code, +7 more. Returns: result

osint_dns_auditC

DNS & Infrastructure Audit Tool: Lookup IPv4, IPv6, and server infrastructure records for any domain.

Args: domain: Domain name to audit (e.g. 'openai.com').

Inputs: domain (required). Returns: result

osint_domain_reconC

OSINT Tool: Gather public domain intelligence including SSL Certificate validity/issuer, DNS resolution, and WHOIS/RDAP.

Inputs: domain_or_url (required). Returns: result

osint_geoint_lookupC

GEOINT Location Tool: Resolve any address, landmark, or street into a complete global administrative hierarchy.

Inputs: location_query (required), limit, country_code. Returns: result

osint_ip_lookupB

OSINT Tool: Lookup public IP Geolocation, ASN, ISP, and Organization information.

Inputs: ip_or_host (required). Returns: result

osint_location_entity_searchC

Hierarchical Location-based OSINT Search Matrix with Negative Exclusion Filters.

Inputs: entity_name (required), city, state, country, +5 more. Returns: result

osint_searchC

OSINT Tool: Targeted intelligence search / dorking using advanced boolean operators (site:, filetype:pdf/xlsx/docx, intitle:, -exclude).

Inputs: query (required), site, filetype, in_title, +3 more. Returns: result

osint_subdomain_enumerationB

Subdomain Recon Tool: Enumerate all public active and historical subdomains for any domain via Certificate Transparency Logs in <1 second.

Args: domain: The root domain to scan (e.g. 'target.com' or 'github.com'). limit: Maximum subdomains to return (default 50).

Inputs: domain (required), limit. Returns: result

osint_tech_stackC

OSINT Tool: Fingerprint web technology stack, frameworks (React, Vue, Next.js, Angular, Django, etc.), CMS, CDN, and servers.

Inputs: url (required). Returns: result

osint_username_checkB

OSINT Tool: Check public presence and profile existence for a handle/username across 25+ developer and creator platforms.

Inputs: username (required). Returns: result

osint_wayback_time_machineA

Wayback Machine Historical Recon Tool: Time-travel and retrieve archived snapshots of deleted or past webpages from the Internet Archive.

Args: url: The webpage URL to investigate. timestamp: Optional 14-digit timestamp (e.g. '20210515120000'). If empty, fetches most recent. list_snapshots: If True, returns a list of available historical snapshot dates instead of page content. limit: Number of snapshot dates to return (default 5).

Inputs: url (required), limit, timestamp, list_snapshots. Returns: result

scrape_batchC

Concurrently scrape multiple URLs in parallel and return their combined Markdown.

Inputs: urls (required), concurrency, use_browser. Returns: result

scrape_dynamicB

Scrape dynamic Single Page Applications (React, Vue, Next.js) with Playwright headless browser.

Inputs: url (required), block_ads, evaluate_js, css_selector, +5 more. Returns: result

scrape_urlC

Scrape any website and convert its content into clean, ad-free Markdown.

Inputs: url (required), engine, max_chars, strip_ads, +7 more. Returns: result

search_and_scrapeB

Autonomous research tool: searches the web and automatically scrapes the top results into a synthesized Markdown report.

Inputs: query (required), max_results, use_browser. Returns: result

search_webC

Perform a live web search to find relevant URLs, articles, and snippets for any topic.

Inputs: query (required), max_results, search_type. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

Deploy Server

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

C2.7/5.0

Scored 2026-11-04 03:35 across 33 tools

Disambiguation2/5

Several clusters of tools have unclear boundaries: scrape_url vs scrape_dynamic vs scrape_batch vs deep_crawl vs search_and_scrape all fetch page content, and extract_schema vs extract_structured both do field-level extraction. The OSINT cluster (osint_domain_recon, osint_dns_audit, osint_subdomain_enumeration, osint_tech_stack, osint_ip_lookup) and osint_geoint_lookup vs osint_location_entity_search overlap heavily, forcing guesswork about which to pick.

Naming Consistency3/5

Most tools follow snake_case verb_noun (scrape_url, extract_pdf, search_web), but there are deviations: two tools carry a redundant '_tool' suffix (extract_reddit_thread_tool, extract_rss_feed_tool), some use a domain prefix (osint_*), and others are bare noun_verb or non-verb forms (deep_crawl, scrape_batch). Readable overall but inconsistent conventions.

Tool Count2/5

33 tools is well above the comfortable range for a single server and includes near-duplicate scraping and OSINT entries that could be merged. The breadth (scraping, document parsing, OSINT, RAG chunking) makes the surface feel like several servers bundled together.

Completeness4/5

The scraping/extraction/OSINT domain is well covered: static and dynamic scraping, crawling, sitemaps, PDF/DOCX/XLSX, images, YouTube, Reddit, RSS, and a broad OSINT suite, plus RAG chunking and diffing. Minor gaps like screenshot capture, authenticated/form-based scraping, and a generic 'get robots.txt' helper are workable around.

Maintenance

ActivityInactive

ResponsivenessNo recent community issues

Resources

Related MCP Connectors

Webcrawl Anything
io.github.brainfish-ai
Scrape, map, crawl, search and monitor the web. Runs locally with no API key, cites every source.
Browser Automation Web Scraping Search
114
https://webcrawl.app/mcp
agentwebintelB
io.github.mlartab
Proxy-rotated web scraping and cross-platform intelligence for AI agents.
Web Scraping Browser Automation Agent Orchestration
21
https://agentwebintel.up.railway.app/mcp
Siftwright web dataA
com.siftwright
Web data for agents: YouTube transcripts, screenshots, Google News, WHOIS, jobs, tech stack, more.
Browser Automation Web Scraping Search
91
https://siftwright.com/mcp
Outcrawl
io.github.outcrawlai
Web infrastructure for AI agents: scrape, crawl and search the web, run browsers and agents.
Browser Automation Web Scraping Agent Orchestration
1
https://api.outcrawl.ai/mcp

Related MCP Servers

wigolo
Browser Automation Web Scraping Agent Orchestration
KnockOutEZ
A
license
A
quality
A
maintenance
Provides local-first web intelligence over MCP with tools for search, fetch, crawl, extract, cache, find-similar, research, and autonomous agent loops, requiring no API keys.
Updated a day ago (2026-11-13 13:03 UTC)
10
1,371 npm
5,543
AGPL 3.0
InfinityScrape MCP
Web Scraping Browser Automation Location Services
virajverse
F
license
Not graded
quality
B
maintenance
Enables AI agents to perform anti-bot web scraping, dynamic SPA rendering, YouTube transcript extraction, and deep OSINT/GEOINT intelligence gathering through 23 integrated tools.
Updated a month ago (2026-10-17 06:19 UTC)
-
web-search-server
Search Web Scraping Browser Automation
sanqiuyef
A
license
Not graded
quality
C
maintenance
Enables AI agents to run local multi-engine web searches, fetch pages through anti-bot fallback chains, and query site-specific sources such as GitHub, npm, PyPI, Docker Hub, and Twitter/X for structured results.
Updated a month ago (2026-10-18 10:50 UTC)
3
MIT
web-mcp
Browser Automation Search Web Scraping
panuhen
A
license
A
quality
B
maintenance
Gives AI agents local web search and page reading with no API keys, layering a SearXNG-backed search cache with browser-based fallbacks and returning clean page text from HTML, PDF, JSON, and archived snapshots. Fetches run in Docker alongside the search engine with SSRF protection, egress filtering, and untrusted-content marking so agents can safely retrieve live web data.
Updated a month ago (2026-10-07 20:23 UTC)
2
MIT

🌐 InfinityScrape MCP: World-Class Web Scraping, Dynamic SPA Rendering & 33-Tool OSINT Intelligence Suite

📑 Table of Contents

🌟 Why InfinityScrape MCP?

⚡ Competitive Comparison

🏗️ Architectural Overview

🚀 Quick Start & 1-Click Installation

1. Automated Setup

2. Launch FastAPI Bridge (Port 8000)

🔌 AI Client Integration

1. Open WebUI (FastAPI Bridge on Port 8000)

2. Antigravity AI / Claude Desktop (Native Stdio)

🛠️ Complete 33-Tool Reference Catalog

1. Anti-Bot Web Scraping, Dynamic SPAs & Crawlers (6 Tools)

2. Real-Time Web Search, Research & Archive OSINT (4 Tools)

3. Media, Document & RAG Extraction (4 Tools)

4. Deep Public OSINT & Entity Reconnaissance (4 Tools)

5. Infrastructure, Domain & Network Reconnaissance (5 Tools)

6. Precision GEOINT & Location Intelligence (2 Tools)

7. Zero-Selector Semantics, Office Streaming & Change Radar (8 Tools)

💻 Command-Line Interface (CLI)

🧪 Running Automated Tests

📄 License & Authors

Available Tools

Tool Schema Changelog

TDQS

Maintenance

Resources

Related MCP Connectors

Related MCP Servers

wigolo

InfinityScrape MCP

web-search-server

web-mcp