Skip to main content
Glama

Crawlee MCP

로컬 Model Context Protocol 서버로, Crawlee를 사용하여 웹 가져오기, 크롤링, 스크린샷을 제공합니다. stdio를 통해 실행되므로 Claude Code, Claude Desktop, Codex, Cursor 및 모든 MCP 호환 클라이언트에서 사용할 수 있습니다.

도구

도구

설명

fetch_url

단일 URL을 가져와 깨끗한 마크다운 반환

fetch_multiple

여러 URL을 병렬로 가져오기

crawl

사이트를 크롤링하고 링크를 발견하여 페이지 콘텐츠 반환

screenshot

페이지의 PNG 스크린샷 촬영

ping

상태 확인

Related MCP server: weblens-mcp

설치

Node.js 20 이상이 필요합니다. JavaScript 렌더링 또는 스크린샷을 처음 사용할 때는 Chromium 브라우저도 필요합니다:

npx playwright install chromium

npm에 게시되면 모든 클라이언트가 다음 명령어로 시작할 수 있습니다:

npx -y @patrikmichi/crawlee-mcp

npm 패키지가 게시될 때까지 GitHub에서 직접 설치하세요:

npx -y github:patrikmichi/crawlee-mcp

클라이언트 설정

Claude Code

claude mcp add --transport stdio crawlee -- npx -y @patrikmichi/crawlee-mcp

Claude Desktop

Claude Desktop 설정 파일에 다음을 추가하세요:

{
  "mcpServers": {
    "crawlee": {
      "command": "npx",
      "args": ["-y", "@patrikmichi/crawlee-mcp"]
    }
  }
}

Codex

~/.codex/config.toml에 다음을 추가하세요:

[mcp_servers.crawlee]
command = "npx"
args = ["-y", "@patrikmichi/crawlee-mcp"]

기타 MCP 클라이언트

동일한 명령어와 인수를 사용하세요: npx -y @patrikmichi/crawlee-mcp.

설정

cp env.example .env

변수

필수

설명

CRAWLEE_PROXY_URLS

아니오

교체를 위한 쉼표로 구분된 프록시 URL

CRAWLEE_TIMEOUT_MS

아니오

요청 제한 시간 (ms) (기본값: 30000)

CRAWLEE_STEALTH

아니오

스텔스 모드를 비활성화하려면 false로 설정 (기본값: true)

개발

npm install
npm start          # Run with tsx (no build needed)
npm run build      # Compile TypeScript
npm run typecheck  # Type check only

이 서버는 의도적으로 로컬 및 stdio 전용입니다. JavaScript 렌더링과 스크린샷에는 로컬 Playwright 브라우저가 필요하기 때문입니다.

Available Tools

5 tools
crawlA

Crawl a website starting from a seed URL, following discovered links across multiple pages. Returns an array of {url, title, content} objects. Use link_pattern to restrict crawling to specific URL patterns (e.g. "https://example.com/blog/**").

Inputs: url (required), label, max_pages, render_js, +3 more.

fetch_multipleA

Fetch multiple URLs in parallel and return an array of {url, content, error} objects. Useful for bulk research or comparison tasks.

Inputs: urls (required), render_js.

fetch_urlB

Fetch a web page and return its content as clean markdown. Use render_js=true for JavaScript-heavy pages (SPAs, Twitter/X, etc). Use render_js=false for fast HTML-only fetching.

Inputs: url (required), render_js, extract_content.

pingB

Check that the crawlee MCP server is running.

Inputs: no named parameters.

screenshotB

Take a screenshot of a web page and return it as a base64-encoded PNG image.

Inputs: url (required), full_page.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.8/5.0

    Scored across 5 tools

    Disambiguation5/5

    Each tool has a clearly distinct purpose: fetching a single page, fetching multiple pages in parallel, taking a screenshot, health checking, and crawling a website. There is no overlap or ambiguity between the tools.

    Naming Consistency5/5

    All tool names follow a consistent verb-based pattern using lowercase and underscores where needed (e.g., fetch_url, fetch_multiple). No mixing of styles like camelCase or inconsistent verb forms.

    Tool Count5/5

    With 5 tools, the server is well-scoped for web crawling and scraping. Each tool earns its place: single fetch, batch fetch, screenshot, full crawl, and a health check. The count is neither too thin nor excessive.

    Completeness4/5

    The tool set covers the core workflows of fetching pages and crawling sites, with parallel fetching and screenshot support. Minor gaps exist, such as advanced extraction capabilities or session management, but the surface is practical for typical use cases.

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables web scraping and crawling capabilities for LLM clients, supporting single-page scraping, multi-page website crawling, and web search with multiple engines (Playwright, Cheerio, Puppeteer) and flexible output formats including markdown, HTML, text, and screenshots.
      7
      MIT
    • A
      license
      A
      quality
      D
      maintenance
      Enables AI agents to fetch and render web pages (including JavaScript-heavy SPAs) with headless Chromium, extract readable content with Mozilla Readability, capture navigation links, download images, and return a clean markdown file path.
      1
      17 npm
      ISC