crawl4ai_mcp_server.cpython-311.pyc•24.3 kB
�
�0�h�M � �
� d Z ddlZddlZddlmZmZ ddlmZ ddlm Z m
Z
ddlmZ ddl
mZ ddlmZ ej ej d ej d
�� � ej e� � Zd� Z e d
dd�� � Ze� � � de
deeef fd�� � Ze� � � d%dee ed�� � f dee edd�� � f de
defd�� � Ze� � � d&dee ed�� � f dee ed�� � f de
defd �� � Ze� � � d&dee ed!�� � f de
defd"�� � Zd#� Zed$k r e� � dS dS )'a�
Crawl4AI MCP Server
A FastMCP server that provides web scraping and crawling capabilities using Crawl4AI.
This server exposes three core tools:
1. get_page_structure - The "eyes" for analyzing webpage structure
2. crawl_with_schema - The "hands" for executing precise extraction schemas
3. take_screenshot - Media capture for visual representation
This server is designed to work with client-side AI that acts as the "brain"
to analyze and command the scraping operations.
Architecture:
- FastMCP handles MCP protocol and tool registration
- AsyncWebCrawler provides web scraping capabilities
- Proper logging to stderr prevents MCP stdio corruption
- All tools use async/await patterns for non-blocking operation
� N)�Dict�Any)�Path)�FastMCP�Context)�Field)� Annotated)�AsyncWebCrawlerz4%(asctime)s - %(name)s - %(levelname)s - %(message)sT)�level�format�stream�forcec �6 � t | dt j i|�� dS )z4Safe printing that goes to stderr instead of stdout.�fileN)�print�sys�stderr)�args�kwargss �V/Users/jeremyparker/Desktop/Claude Coding Projects/crawl4ai-mcp/crawl4ai_mcp_server.py�
safe_printr , s"