Skip to main content
Glama
willbohn

Spider Cloud MCP Server

by willbohn

Spider Cloud MCP Server

A high-performance Model Context Protocol (MCP) server that provides comprehensive web scraping, crawling, and data extraction capabilities through the Spider Cloud API. This server enables AI assistants like Claude to interact with web content using Spider Cloud's advanced scraping infrastructure.

🌟 Features

Core Tools

  • spider_scrape - Advanced single-page scraping with JavaScript rendering and anti-bot bypass

  • spider_crawl - Intelligent website crawling with depth control and filtering

  • spider_search - Google-like web search with content fetching capabilities

  • spider_links - Comprehensive link extraction and analysis

  • spider_screenshot - High-quality webpage screenshots with customization

  • spider_transform - HTML to markdown/text conversion with readability processing

Advanced Capabilities

  • πŸ›‘οΈ Anti-bot Detection Bypass - Stealth mode and advanced evasion techniques

  • 🌐 Premium Proxy Support - Geographic targeting with country-specific proxies

  • 🎭 JavaScript Rendering - Full browser emulation for dynamic content

  • πŸ“Š Metadata Extraction - Comprehensive page metadata and analytics

  • πŸ” CSS Selectors - Precise content targeting and extraction

  • πŸ’Ύ Cloud Storage - Optional data persistence in Spider Cloud

  • ⚑ High Performance - Optimized for speed with configurable timeouts

  • πŸ”’ Secure Authentication - Bearer token authentication with API key

  • πŸ“ˆ Cost Tracking - Real-time API usage cost monitoring

  • πŸ› Debug Mode - Comprehensive logging for troubleshooting

Related MCP server: FineData MCP Server

πŸ“‹ Prerequisites

πŸš€ Quick Start

# Global installation
npm install -g @willbohn/spider-mcp

# Or use with npx (no installation needed)
npx @willbohn/spider-mcp

Option 2: Clone from GitHub

Windows:

# Clone and install
git clone https://github.com/willbohn/spider-mcp.git
cd spider-mcp

# Run the Windows installer (PowerShell)
.\install-windows.ps1

# Or use the batch file (Command Prompt)
install-windows.bat

# Test the installation
$env:SPIDER_API_KEY="your_key"
node test.js

macOS/Linux:

# Clone and install
git clone https://github.com/willbohn/spider-mcp.git
cd spider-mcp
./install-local.sh

# Or manually:
npm install
npm link

# Test the installation
SPIDER_API_KEY=your_key node test.js

Option 3: Direct Path Configuration

Skip installation and point directly to the built files in your MCP client configuration.

βš™οΈ Configuration

Platform-Specific Setup Instructions

Claude Desktop (Windows)

  1. Find your configuration file:

    • Press Win + R, type %APPDATA%\Claude and press Enter

    • Open claude_desktop_config.json (create it if it doesn't exist)

  2. Add the Spider MCP configuration:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["@willbohn/spider-mcp"],
      "env": {
        "SPIDER_API_KEY": "your_spider_api_key_here"
      }
    }
  }
}
  1. Alternative: Using direct path (if npm doesn't work):

{
  "mcpServers": {
    "spider": {
      "command": "node",
      "args": ["C:\\Users\\YourName\\spider-mcp\\dist\\index.js"],
      "env": {
        "SPIDER_API_KEY": "your_spider_api_key_here"
      }
    }
  }
}

Note: On Windows, use double backslashes (\\) in paths or forward slashes (/).

Testing on Windows

# PowerShell
$env:SPIDER_API_KEY="your_key"
node test.js

# Command Prompt
set SPIDER_API_KEY=your_key
node test.js

Claude Desktop (macOS)

  1. Find your configuration file:

    open ~/Library/Application\ Support/Claude/

    Open claude_desktop_config.json (create it if it doesn't exist)

  2. Add the Spider MCP configuration:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["@willbohn/spider-mcp"],
      "env": {
        "SPIDER_API_KEY": "your_spider_api_key_here"
      }
    }
  }
}

Testing on macOS

export SPIDER_API_KEY="your_key"
node test.js

Claude Desktop (Linux)

  1. Find your configuration file:

    # Location varies by distribution, commonly:
    ~/.config/Claude/claude_desktop_config.json
    # or
    ~/.claude/claude_desktop_config.json
  2. Add the Spider MCP configuration:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["@willbohn/spider-mcp"],
      "env": {
        "SPIDER_API_KEY": "your_spider_api_key_here"
      }
    }
  }
}

Testing on Linux

export SPIDER_API_KEY="your_key"
node test.js

Other MCP Clients

Claude Code automatically detects MCP servers. Simply:

  1. Install the package globally:

    npm install -g @willbohn/spider-mcp
  2. Set your API key:

    • Windows (PowerShell): $env:SPIDER_API_KEY="your_key"

    • Windows (CMD): set SPIDER_API_KEY=your_key

    • macOS/Linux: export SPIDER_API_KEY="your_key"

  3. The server will be available in Claude Code

Add to your Cursor settings:

{
  "mcp.servers": {
    "spider": {
      "command": "npx",
      "args": ["@willbohn/spider-mcp"],
      "env": {
        "SPIDER_API_KEY": "your_spider_api_key_here"
      }
    }
  }
}

Add to your Continue configuration:

{
  "mcpServers": [
    {
      "name": "spider",
      "command": "npx",
      "args": ["@willbohn/spider-mcp"],
      "env": {
        "SPIDER_API_KEY": "your_spider_api_key_here"
      }
    }
  ]
}

Environment Variables

Variable

Required

Description

Default

SPIDER_API_KEY

Yes

Your Spider Cloud API key

-

SPIDER_API_BASE_URL

No

API endpoint URL

https://api.spider.cloud

SPIDER_REQUEST_TIMEOUT

No

Request timeout in milliseconds

60000

DEBUG

No

Enable debug logging

false

πŸ› οΈ Tool Documentation

spider_scrape

Scrape content from a single URL with advanced options.

Parameters:

  • url (required): Target URL to scrape

  • return_format: Output format (markdown, raw, text, html, screenshot, links)

  • js: Enable JavaScript rendering

  • wait_for: Wait time for page load (0-60000ms)

  • css_selector: CSS selector for specific content

  • proxy_enabled: Use premium proxy

  • proxy_country: Two-letter country code

  • stealth: Enable stealth mode

  • anti_bot: Advanced anti-bot bypass

  • headers: Custom HTTP headers

  • cookies: Cookie string

  • metadata: Include metadata

  • clean_html: Clean and sanitize HTML

  • media: Include media elements

Example:

{
  "url": "https://example.com",
  "return_format": "markdown",
  "js": true,
  "stealth": true,
  "css_selector": ".main-content"
}

spider_crawl

Crawl an entire website with intelligent navigation.

Parameters:

  • url (required): Starting URL

  • limit: Max pages to crawl (1-10000)

  • depth: Max crawl depth (0-10)

  • return_format: Output format

  • whitelist: URL patterns to include

  • blacklist: URL patterns to exclude

  • budget: Crawl budget configuration

  • subdomains: Include subdomains

  • sitemap: Use sitemap.xml

  • respect_robots: Respect robots.txt

  • Plus all proxy and rendering options from scrape

Example:

{
  "url": "https://docs.example.com",
  "limit": 50,
  "depth": 3,
  "whitelist": ["*/api/*"],
  "return_format": "markdown"
}

Search the web with Google-like results.

Parameters:

  • query (required): Search query

  • search_limit: Max results (1-100)

  • fetch_page_content: Fetch full content

  • tbs: Time-based search (qdr:d, qdr:w, qdr:m, qdr:y)

  • gl: Country code (e.g., us, uk)

  • hl: Language code (e.g., en, es)

  • safe: SafeSearch level (off, medium, high)

  • Plus content fetching options

Example:

{
  "query": "artificial intelligence news",
  "search_limit": 10,
  "tbs": "qdr:w",
  "gl": "us",
  "fetch_page_content": true
}

Extract and analyze links from a webpage.

Parameters:

  • url (required): Target URL

  • limit: Max links (1-5000)

  • depth: Extraction depth (0-5)

  • unique: Return only unique links

  • subdomains: Include subdomain links

  • external: Include external links

  • Plus standard options

spider_screenshot

Capture webpage screenshots.

Parameters:

  • url (required): Target URL

  • fullpage: Full page screenshot

  • viewport_width: Width in pixels (320-3840)

  • viewport_height: Height in pixels (240-2160)

  • format: Image format (png, jpeg, webp)

  • quality: JPEG/WebP quality (0-100)

  • omit_background: Transparent background (PNG only)

  • clip: Region to capture

spider_transform

Transform HTML to clean, readable formats.

Parameters:

  • data (required): HTML/text to transform

  • return_format (required): Target format (markdown, text, raw, clean_html)

  • readability: Apply readability processing

  • clean: Remove unnecessary elements

  • include_links: Include hyperlinks

  • include_images: Include images

πŸ§ͺ Testing

Run the comprehensive test suite:

Windows

# PowerShell
$env:SPIDER_API_KEY="your_api_key_here"
node test.js

# With debug output
$env:DEBUG="true"
$env:SPIDER_API_KEY="your_api_key_here"
node test.js

# Command Prompt
set SPIDER_API_KEY=your_api_key_here
node test.js

macOS/Linux

# Set your API key
export SPIDER_API_KEY=your_api_key_here

# Run tests
node test.js

# With debug output
DEBUG=true SPIDER_API_KEY=your_api_key_here node test.js

Test Suites

# Quick smoke tests
npm run test:quick

# Full comprehensive suite (100+ tests)
npm run test:full

# LinkedIn-specific tests
npm run test:linkedin

# Run specific category
npm run test:category -- --category scraping

πŸ“Š API Response Format

All tools return responses in a consistent format:

{
  "success": true,
  "results": [...],
  "count": 10,
  "costs": {
    "total_cost": 0.00012,
    "compute_cost": 0.00008,
    "bandwidth_cost": 0.00004
  },
  "metadata": {
    "duration": 1234,
    "status": 200
  }
}

πŸ”§ Development

Building from Source

npm install
npm run build

Running in Development Mode

npm run dev

Project Structure

spider-mcp/
β”œβ”€β”€ src/
β”‚   └── index.ts        # Main server implementation
β”œβ”€β”€ dist/               # Compiled JavaScript
β”œβ”€β”€ examples/           # Configuration examples
β”œβ”€β”€ package.json        # Dependencies and scripts
β”œβ”€β”€ tsconfig.json       # TypeScript configuration
└── README.md          # This file

πŸ› Troubleshooting

Common Issues

"SPIDER_API_KEY environment variable is required"

  • Ensure your API key is set in the environment or configuration

  • Check the key is valid at spider.cloud

"Payment required" error

"Rate limit exceeded"

  • You've hit the API rate limit

  • Wait a few minutes or upgrade your plan

Search tool timeout

  • Search operations can take 15-30 seconds

  • This is normal behavior for comprehensive searches

Debug Mode

Enable detailed logging:

Windows (PowerShell):

$env:DEBUG="true"
$env:SPIDER_API_KEY="your_key"
node dist/index.js

Windows (Command Prompt):

set DEBUG=true
set SPIDER_API_KEY=your_key
node dist/index.js

macOS/Linux:

DEBUG=true SPIDER_API_KEY=your_key node dist/index.js

πŸ“ Error Handling

The server provides detailed error messages:

  • 401: Invalid API key

  • 402: Payment required (add credits)

  • 429: Rate limit exceeded

  • 500+: Server errors (contact support)

πŸ”’ Security

  • API keys are never logged or stored

  • All requests use HTTPS

  • Bearer token authentication

  • Input validation on all parameters

  • Sanitized error messages

πŸ“ˆ Performance

  • Configurable timeouts (default: 60s)

  • Automatic retry logic for transient failures

  • Connection pooling for efficiency

  • Response caching at API level

  • Optimized for concurrent requests

🀝 Contributing

Contributions are welcome! Please:

  1. Fork the repository

  2. Create a feature branch

  3. Commit your changes

  4. Push to the branch

  5. Open a Pull Request

πŸ“„ License

MIT License - see LICENSE file for details

πŸ”— Resources

πŸ’¬ Support


Built with ❀️ for the MCP ecosystem

Available Tools

6 tools
spider_crawlC

Crawl an entire website with configurable depth, limits, and filtering options

ParametersJSON Schema
NameRequiredDescriptionDefault
jsNo
urlYes
depthNo
limitNo
budgetNo
headersNo
sitemapNo
stealthNo
anti_botNo
metadataNo
blacklistNo
whitelistNo
store_dataNo
subdomainsNo
proxy_countryNo
proxy_enabledNo
return_formatNo
respect_robotsNo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It doesn't disclose that this is a long-running, high-cost operation, rate limiting, permission requirements, or what 'store_data' actually mutates in Spider Cloud – significant gaps for an 18-parameter mutation-capable crawl.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no waste, though it is arguably too terse for an 18-parameter tool. Efficient phrasing that earns its place but under-delivers given the complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an 18-parameter crawl with no annotations and no output schema, the description covers almost none of what an agent needs: cost/time expectations, parameter interactions, or the effect of store_data. Inadequate for the tool's complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is reported 0% (raw schema has empty descriptions), so the description must compensate. It mentions 'depth, limits, and filtering options' generically, leaving 15+ parameters – budget, blacklist/whitelist wildcards, stealth/anti_bot, proxy, store_data – completely unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Crawl) and resource (entire website) and lists the dimensions of configurability. It distinguishes itself from spider_scrape (single page) and spider_search (query-based) by scope, though it doesn't name those siblings explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implied usage: crawl an 'entire website' when you need multi-page coverage versus a single scrape. No explicit when-to-use, when-not-to-use, or named alternatives, so the agent must infer routing among the five siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_scrapeC

Scrape content from a single URL with advanced options for JavaScript rendering, anti-bot bypass, and content extraction

ParametersJSON Schema
NameRequiredDescriptionDefault
jsNo
urlYes
mediaNo
cookiesNo
headersNo
stealthNo
timeoutNo
anti_botNo
metadataNo
wait_forNo
clean_htmlNo
store_dataNo
css_selectorNo
proxy_countryNo
proxy_enabledNo
return_formatNo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden, and it discloses almost nothing: no note that premium proxy/stealth modes may carry cost or latency, no authentication or Spider Cloud credential requirement, no rate limits, and no statement of what happens to the page content when store_data persists it remotely. The three 'advanced options' named are capabilities rather than behaviors.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One front-loaded sentence with no filler; the core action leads and the qualifiers follow. It is efficient, though it spends its back half listing three features that the schema already documents, which is space that could have bought usage guidance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 16-parameter, no-annotation, no-output-schema tool, this is thin: nothing about the default markdown return, nothing about cost or credential prerequisites, nothing about how it differs from the five sibling tools. An agent can call it, but cannot confidently choose it over spider_crawl or predict what comes back.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema itself carries a per-parameter description for all 16 fields (js, anti_bot, css_selector, timeout bounds, return_format default markdown, etc.), so the schema does the heavy lifting even though the reported coverage signal reads 0%. The description adds essentially no parameter meaning beyond echoing the js, anti_bot, and extraction options already named in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb+resource ('Scrape content from a single URL') and enumerates the capability areas: JavaScript rendering, anti-bot bypass, content extraction. This separates it from the plural-scope siblings (crawl/search), but it never names those siblings, so an agent must infer the boundary itself.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisites, and no mention of alternatives such as spider_crawl for multi-page work or spider_screenshot when an image is wanted. The word 'single URL' is the only implicit routing signal, and it is embedded in the purpose statement rather than framed as a decision rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_screenshotC

Capture high-quality screenshots of webpages with customizable viewport and options

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes
clipNo
formatNo
qualityNo
timeoutNo
fullpageNo
wait_forNo
proxy_countryNo
proxy_enabledNo
viewport_widthNo
omit_backgroundNo
viewport_heightNo

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full disclosure burden, yet it only says 'high-quality' and 'customizable'. It omits whether proxies are required for some sites, how rendering/waiting behaves, rate limits, or that output is an image rather than content.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler or repetition. It is efficient, though its brevity reflects under-specification rather than disciplined selection of detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parameter tool with no annotations and no output schema, the description is far too thin. It says nothing about proxy usage, timing behavior, or the nature of the returned image, so an agent must infer everything from the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds only the vague phrase 'customizable viewport and options' and explains no parameter. Although the input schema itself carries per-parameter descriptions, the description contributes no semantics beyond it, leaving the description-level meaning at a minimum.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Capture ... screenshots of webpages') with a hint at configurable scope. It is clearly distinct from siblings like spider_scrape and spider_links, though it never names or contrasts them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this versus spider_scrape/spider_crawl, no prerequisites, and no mention of when a screenshot is inappropriate. The only implied usage is taking a visual capture of a page.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

spider_transformC

Transform HTML content to clean markdown, text, or formatted output

ParametersJSON Schema
NameRequiredDescriptionDefault
dataYes
cleanNo
readabilityNo
include_linksNo
return_formatYes
include_imagesNo

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden, yet it discloses only the transformation targets. It does not state that this is a pure, side-effect-free operation, nor does it explain what 'clean', 'readability', or 'formatted output' actually do to the input.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no wasted words. It is efficient, though its brevity is partly the cause of the coverage gaps rather than a deliberate trade-off.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter tool with no annotations and no output schema, the description should explain transformation behavior and the optional flags. Instead it only restates the high-level purpose, leaving the agent with no guidance on how the options interact or what the result looks like.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is reported at 0%, and the description adds no parameter-level meaning. It mentions markdown and text (partially mapping to return_format) but says nothing about the other options (raw, clean_html) or the clean/readability/include_links/include_images toggles.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Transform'), a specific resource ('HTML content'), and names the output target formats (markdown, text, formatted output). An agent can tell it is a conversion utility, though it never explicitly contrasts itself with siblings like spider_scrape or spider_crawl.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives, no prerequisites, and no mention of when it should not be used. The agent must infer that this is for post-scrape content conversion rather than retrieval.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 6 tool updatesv1.1.0
    • First observedspider_crawl
    • First observedspider_links
    • First observedspider_scrape
    • First observedspider_screenshot
    • First observedspider_search
    • First observedspider_transform

TDQS

B3.3/5.0

Scored across 6 tools

Disambiguation5/5

Each tool targets a distinct web operation: scrape single URL, crawl site, search web, extract links, screenshot, and transform HTML. Overlap is minimal and descriptions clarify boundaries.

Naming Consistency5/5

All tools use the same 'spider_' prefix followed by a clear verb/noun, creating a fully predictable naming pattern.

Tool Count5/5

Six tools are well-scoped for a web scraping and content processing server, covering common workflows without bloat.

Completeness4/5

Core lifecycle is covered: discovery/search, extraction/scraping, crawling, link analysis, screenshots, and transformation. Minor gaps exist, such as batch operations or structured data extraction, but agents can work around them.

Maintenance

ActivityInactive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    D
    maintenance
    Enables AI assistants to access real-time web data through search, markdown scraping, and browser automation while bypassing anti-bot protections. It provides tools for web research, e-commerce monitoring, and data extraction from across the globe.
    4
    7,023 npm
    5
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables AI agents to scrape any website by providing tools for JavaScript rendering, antibot bypass, and automatic captcha solving. It supports synchronous, asynchronous, and batch scraping operations with built-in proxy rotation.
    5
    207 npm
    MIT