Claude Desktop / Cursor IDE
        │
        │  MCP (JSON-RPC over stdio)
        ▼
  scraper_mcp.py   ◄── FastMCP server
        │
        ├── Playwright  (headless browser, JS rendering)
        └── lxml        (HTML parsing / XPath / CSS selectors)

# 1. Clone this repo
git clone https://github.com/RebaubierWilliam/web-scraping-with-mcp.git
cd web-scraping-with-mcp

# 2. Create and activate a virtual environment
python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS / Linux
source .venv/bin/activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Install Playwright browsers
python -m playwright install chromium

{
  "mcpServers": {
    "web-scraper": {
      "command": "python",
      "args": ["C:/Users/YourName/Source/web-scraping-with-mcp/scraper_mcp.py"]
    }
  }
}

Fetch the page https://www.amazon.fr/dp/B09V3KXJPB and extract the product details.

Go to https://news.ycombinator.com and give me all the links.

Scrape the prices table from https://example.com/pricing

python scraper_mcp.py