The MCP Web Scrape server is a comprehensive web content extraction and analysis tool that converts web pages into clean, agent-friendly formats with smart caching and ethical compliance.

Content Extraction & Transformation: Convert HTML to clean Markdown/text/JSON with citations (extract_content), extract plain text (extract_text_only), summarize content with customizable formats (summarize_content), translate content (translate_content), convert pages to PDF (convert_to_pdf), and generate word clouds (generate_word_cloud).

Structured Data Extraction: Extract links with filtering (extract_links), images with metadata (extract_images), forms with validation rules (extract_forms), tables with export options (extract_tables), social media links (extract_social_media), contact information (extract_contact_info), heading hierarchy (extract_headings), RSS/Atom feeds (extract_feeds), and structured data including JSON-LD, microdata, RDFa, OpenGraph, and schema.org markup (extract_structured_data, extract_schema_markup).

Content Analysis: Search within pages with regex support (search_content), extract keywords (extract_keywords), analyze readability using multiple metrics (analyze_readability), detect language with confidence scores (detect_language), extract named entities (extract_entities), perform sentiment analysis at document/paragraph/sentence level (sentiment_analysis), classify content into categories (classify_content), and compare content between URLs (compare_content).

SEO & Marketing Tools: Analyze competitors for insights (analyze_competitors), generate optimized meta tags (generate_meta_tags), check broken links and redirects (check_broken_links), analyze page speed with Core Web Vitals (analyze_page_speed), validate HTML structure (validate_html), analyze overall performance including SEO and accessibility (analyze_performance), and generate sitemaps (generate_sitemap).

Security & Privacy: Scan for vulnerabilities including XSS and CSRF (scan_vulnerabilities), check SSL certificates with chain details (check_ssl_certificate), analyze cookies for security flags (analyze_cookies), detect tracking scripts (detect_tracking), and check privacy policy compliance with GDPR, CCPA, COPPA, and PIPEDA (check_privacy_policy).

Monitoring & Tracking: Monitor uptime with configurable intervals (monitor_uptime), track content changes with similarity analysis (monitor_changes, track_changes_detailed), analyze traffic patterns (analyze_traffic_patterns), and benchmark performance against competitors (benchmark_performance).

Utility & Management: Process multiple URLs efficiently in batches (batch_extract), validate robots.txt compliance (validate_robots), check URL accessibility (check_url_status), manage cache with statistics and selective clearing (clear_cache, get_cache_stats), and generate comprehensive reports in JSON, HTML, or Markdown formats (generate_reports).

Key Benefits: Provides clean Markdown output optimized for AI agents, citation links for fact verification, deterministic and cached results with ETag/304 support, ethical scraping with robots.txt respect and rate limiting, and supports both STDIO and HTTP/SSE transports.

Transforms web content into clean, agent-ready Markdown with automatic citations, removing clutter while preserving content hierarchy.

Provides specialized tools to discover and parse RSS and Atom feeds from websites for automated content syndication and extraction.

1. Click on "Install Server".

2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.

3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@MCP Web Scrape scrape the main text from https://news.ycombinator.com and format it as markdown"

That's it! The server will respond to your query, and you can continue using it as needed.

Here is a step-by-step guide with screenshots.