Competitor Hunter
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Competitor Hunteranalyze Notion's pricing page"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
๐ฏ Competitor Hunter
AI-Powered Competitor Analysis Agent | Automated web scraping and structured data extraction using MCP, LangGraph, and Playwright
Language | ่ฏญ่จ: English | ไธญๆ
๐ Introduction
Competitor Hunter is a production-ready AI agent that automates competitor analysis by scraping product pages and extracting structured information using Large Language Models. Built on the Model Context Protocol (MCP), it seamlessly integrates with Claude Desktop and other MCP-compatible clients.
Key Capabilities
๐ Intelligent Web Scraping: Automated browser-based content extraction with anti-detection features
๐ค LLM-Powered Extraction: Structured data extraction using OpenAI-compatible APIs
๐ Structured Output: Pydantic-validated product information (pricing, features, SWOT analysis)
๐ LangGraph Workflow: Robust state management and error handling
๐ MCP Integration: Native support for Claude Desktop and MCP clients
Related MCP server: merch-connector
๐๏ธ Architecture
The system follows Hexagonal Architecture with clear separation of concerns. Workflow: User Request โ MCP Server โ LangGraph Workflow โ Browser Scraping โ LLM Extraction โ Structured Data Response.
โจ Core Features
๐ค AI-Powered: Intelligent extraction using LLM with automatic SWOT analysis
๐ Structured Output: Pydantic-validated data models (pricing, features, summary)
๐ก๏ธ Anti-Detection: Random User-Agents, intelligent scrolling, auto-screenshots
๐ MCP Native: Seamless integration with Claude Desktop and Cursor IDE
๐ฆ CLI Tool: Professional command-line interface via
competitor-huntercommandAsync/Await: Full asynchronous programming for optimal performance
๐ Quick Start
Prerequisites
Python 3.10+ (3.11 or 3.12 recommended)
UV or Poetry (dependency manager)
Playwright browsers (installed automatically)
Installation
Clone the repository:
git clone https://github.com/your-username/competitor-hunter.git cd competitor-hunterInstall dependencies (using UV):
uv syncOr using Poetry:
poetry installOr using pip:
python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate pip install -e ".[dev]"Install Playwright browsers:
playwright install chromium
Configuration
Create a .env file in the project root:
# OpenAI API Configuration
OPENAI_API_KEY=your_openai_api_key_here
OPENAI_BASE_URL=https://api.openai.com/v1 # Optional: for custom endpoints
OPENAI_MODEL_NAME=gpt-4o # Optional: default is gpt-4o
# Browser Configuration
HEADLESS_MODE=true # Set to false for debugging
# Database Configuration
DB_PATH=data/competitors.db # SQLite database path๐ก Tip: Copy .env.example to .env and fill in your values:
cp .env.example .env๐ธ Screenshots & Examples
Analysis Results

Screenshot of Notion pricing page analysis
CLI Output Example
$ competitor-hunter https://www.notion.so/pricing
๐ ๆญฃๅจๅๆ: https://www.notion.so/pricing
โ
ๅๆๅฎๆ๏ผ
======================================================================
๐ฆ ไบงๅๅ็งฐ: Notion
๐ URL: https://www.notion.so/pricing
๐ ๆดๆฐๆถ้ด: 2024-06-13 00:00:00+00:00
======================================================================
๐ฐ ๅฎไปทๆนๆก (4 ไธช):
โข Free: 0 USD / monthly
โข Plus: 10 USD / monthly
โข Business: 20 USD / monthly
โข Enterprise: Custom USD / custom
โจ ๆ ธๅฟๅ่ฝ (13 ไธช):
1. AI automation
2. Enterprise search
3. Meeting notes
...
๐พ ็ปๆๅทฒไฟๅญๅฐ: reports/product_Notion.jsonJSON Output Structure
The analysis results are saved as structured JSON files:
{
"product_name": "Notion",
"url": "https://www.notion.so/pricing",
"pricing_tiers": [
{
"name": "Free",
"price": "0",
"currency": "USD",
"billing_cycle": "monthly"
},
{
"name": "Plus",
"price": "10",
"currency": "USD",
"billing_cycle": "monthly"
}
],
"core_features": [
"AI automation",
"Docs",
"Knowledge Base"
],
"summary": "## ไบงๅๆฆ่ฟฐ\nNotion ๆฏไธๆฌพ้ๆๆกฃ็ผ่พ...",
"last_updated": "2024-06-13T00:00:00Z"
}๐ Usage
Method 1: CLI Command (Easiest)
After installation, use the competitor-hunter command:
# Analyze a single website
competitor-hunter https://www.notion.so/pricing
# Specify output file
competitor-hunter https://example.com output.json
# Batch analysis
competitor-hunter https://site1.com https://site2.com https://site3.comResults are automatically saved to the reports/ directory with proper UTF-8 encoding.
Method 2: MCP Server Mode (Recommended for AI Assistants)
Run the MCP server to enable integration with Claude Desktop or Cursor:
python -m src.competitor_hunter.interface.mcp_server.serverClaude Desktop Integration
Add the following configuration to your Claude Desktop claude_desktop_config.json:
macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"competitor-hunter": {
"command": "python",
"args": [
"-m",
"src.competitor_hunter.interface.mcp_server.server"
],
"cwd": "/path/to/competitor-hunter"
}
}
}Cursor IDE Integration
Create .cursor/mcp.json in your project root:
{
"mcpServers": {
"competitor-hunter": {
"command": "python",
"args": [
"-m",
"src.competitor_hunter.interface.mcp_server.server"
],
"cwd": "${workspaceFolder}"
}
}
}After restarting, you can use the tool directly in chat:
Analyze this competitor: https://www.notion.so/pricingMethod 3: Python Library
Use the LangGraph workflow directly in your Python code:
import asyncio
from competitor_hunter.core import graph, AgentState, cleanup_resources
async def analyze(url: str):
# Initialize state
initial_state: AgentState = {
"url": url,
"scraped_content": None,
"product": None,
"error": None,
}
# Run workflow
result = await graph.ainvoke(initial_state)
# Check results
if result.get("error"):
print(f"Error: {result['error']}")
return None
product = result["product"]
print(f"Product: {product.product_name}")
print(f"Pricing Tiers: {len(product.pricing_tiers)}")
print(f"Features: {product.core_features}")
return product
# Use
product = await analyze("https://www.notion.so/pricing")
await cleanup_resources()Output Structure
All analysis results are saved to the reports/ directory:
reports/
โโโ product_Notion.json
โโโ product_Example_Domain.json
โโโ ...Each JSON file contains:
Product name and URL
Pricing tiers (name, price, currency, billing cycle)
Core features list
Markdown-formatted summary with SWOT analysis
Last updated timestamp
๐งช Development
Running Tests
# Run all tests
pytest tests/ -v
# Run specific test file
pytest tests/test_crawler.py -v
# Run with coverage
pytest tests/ --cov=src/competitor_hunter --cov-report=htmlCode Quality
# Format code
black src/ tests/
# Lint code
ruff check src/ tests/
# Type checking (if using mypy)
mypy src/Project Structure
competitor-hunter/
โโโ src/
โ โโโ competitor_hunter/
โ โโโ cli.py # CLI command-line interface
โ โโโ main.py # Application entry point
โ โโโ config.py # Configuration management
โ โโโ core/ # Domain models & LangGraph workflow
โ โ โโโ models.py # Pydantic models (CompetitorProduct, etc.)
โ โ โโโ graph.py # LangGraph workflow definition
โ โโโ infrastructure/ # External services
โ โ โโโ browser/ # Playwright browser service
โ โ โโโ llm/ # LLM extractor service
โ โโโ interface/ # Entry points
โ โโโ mcp_server/ # MCP server implementation
โโโ config/ # Configuration files
โ โโโ app.yaml.example # Configuration template
โโโ docker/ # Docker configuration
โ โโโ Dockerfile # Docker image definition
โ โโโ docker-compose.yml # Docker Compose configuration
โโโ examples/ # Example scripts
โโโ tests/ # Test suite
โโโ reports/ # Analysis results (gitignored)
โโโ data/ # SQLite database (gitignored)
โโโ logs/ # Screenshots & logs (gitignored)
โโโ pyproject.toml # Project dependencies & CLI entry points
โโโ README.md # This file๐ฆ Dependencies
Core Dependencies
mcp: Model Context Protocol server implementation
langgraph: Workflow orchestration
langchain: LLM integration framework
playwright: Browser automation
pydantic: Data validation and serialization
html2text: HTML to Markdown conversion
loguru: Structured logging
Development Dependencies
pytest: Testing framework
pytest-asyncio: Async test support
ruff: Fast Python linter
black: Code formatter
๐ค Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
Fork the repository
Create your feature branch (
git checkout -b feature/amazing-feature)Commit your changes (
git commit -m 'Add some amazing feature')Push to the branch (
git push origin feature/amazing-feature)Open a Pull Request
๐ License
This project is licensed under the MIT License - see the LICENSE file for details.
๐ Acknowledgments
Built with LangGraph for workflow orchestration
Powered by Playwright for browser automation
Integrated with Model Context Protocol (MCP) for AI agent communication
๐ Support
For issues, questions, or contributions, please open an issue on GitHub.
Made with โค๏ธ for competitive intelligence
Related MCP Connectors
Social media analytics, video analysis, and competitor intel for any MCP-compatible AI agent.
AMZScout Skill + MCP gives AI agents live access to real Amazon marketplace data across 14 Amazon marketplaces. Analyze any ASIN, validate product ideas, research niches, compare competitors, discover profitable keywords, and build data-driven PPC strategies using trusted Amazon insights instead of AI assumptions. Works with Claude, ChatGPT, Cursor, and any other MCP-compatible AI client. To connect, you'll need an AMZScout API plan and authorize your account. Get access and view pricing here: https://learn.amzscout.net/amazon-product-api-for-ai-agents
The Google for AI agents โ company intel, competitor tracking, market research via MCP. JSON output
Your agent needs marketplace data โ what a product costs on Amazon and Google Shopping, who the sellers are, what reviewers actually complain about. **What you can ask for** โข "What is this ASIN's price history, rating and seller list?" โข "Who else sells this product, and at what price?" โข "Pull the reviews for this product and group the complaints." โข "What comes up on Google Shopping for this query in the UK?" โข "Compare these products across both marketplaces." **How to use it** Point any MCP client at https://mcp.aisa.one/seo-merchant/mcp and sign in with OAuth โ there is no key to create or paste. 22 tools: Amazon products, ASIN detail and sellers; Google Shopping products, product info, sellers and reviews; live and queued forms, with raw HTML where you need it. **It is also a door to the rest** The same login reaches 26 sources and 580+ operations. Price the product here, then ask the same agent what the brand's site traffic or ad spend looks like โ without adding a second server. **What it costs** Finding and inspecting an operation is free. Running one is billed per call at API prices, with no seat and no monthly minimum, and every call takes max_price_usd so an agent cannot overspend by accident. **Where else it reaches** https://mcp.aisa.one/seo/mcp for all of it at once โ rankings, keywords, backlinks, site health and AI-answer visibility across DataForSEO, Semrush and Ahrefs.
Related MCP Servers
- AlicenseAqualityBmaintenanceA Model Context Protocol (MCP) integration that provides Claude Desktop with autonomous browser automation capabilities. This agent enables Claude to interact with web content, manipulate DOM elements, execute JavaScript, and perform API requests.133 npm41TypeScriptMozilla Public 2.0
- AlicenseAqualityFmaintenanceAn MCP server that gives AI agents eyes on any e-commerce storefront, enabling scraping, analysis, and comparison through the Model Context Protocol.1450 npmMIT
- FlicenseNot gradedqualityDmaintenanceEnables monitoring competitor websites for price drops, new products, promotions, and content changes directly from Claude or any MCP-compatible AI client.-
- FlicenseNot gradedqualityCmaintenanceRemote MCP server for AI-powered competitor intelligence, enabling real-time analysis of any product URL through tools like analyze_competitor and report retrieval.110-