MCP Web Scrape
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| MCP_WEB_SCRAPE_CACHE_DIR | No | Directory for caching scraped content. | ./cache |
| MCP_WEB_SCRAPE_RATE_LIMIT | No | Rate limiting threshold to prevent server overload. | 1000 |
| MCP_WEB_SCRAPE_USER_AGENT | No | Custom User-Agent string for identifying the scraper. | MyBot/1.0 |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {} |
| resources | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| extract_contentB | Extract and clean content from a web page, returning Markdown with citation |
| summarize_contentC | Generate a summary of already extracted content |
| clear_cacheC | Clear cached content entries |
| get_page_metadataB | Extract meta tags, title, description, keywords from web pages |
| check_url_statusC | Check if URL is accessible and get HTTP status codes |
| extract_linksC | Extract all links from a web page with filtering options |
| extract_imagesC | Extract all images from a web page with metadata |
| search_contentB | Search for specific text patterns within extracted content |
| get_cache_statsC | Get detailed cache statistics and usage information |
| validate_robotsC | Check robots.txt compliance for specific URLs |
| extract_structured_dataC | Extract JSON-LD, microdata, and schema.org data |
| compare_contentC | Compare content between two URLs or cached versions |
| batch_extractC | Extract content from multiple URLs in a single operation |
| extract_formsC | Extract form elements and their structure from web pages |
| extract_tablesC | Extract and parse HTML tables with optional CSV export |
| extract_social_mediaC | Extract social media links and metadata from web pages |
| extract_contact_infoC | Extract contact information like emails, phones, addresses from web pages |
| extract_headingsB | Extract document structure and heading hierarchy from web pages |
| extract_feedsB | Discover and parse RSS/Atom feeds from web pages |
| monitor_changesC | Monitor web page content changes over time |
| analyze_performanceC | Analyze web page performance metrics |
| generate_sitemapC | Generate sitemap by crawling website pages |
| validate_htmlC | Validate HTML structure, accessibility, and SEO |
| convert_to_pdfB | Convert web page content to PDF format |
| extract_text_onlyB | Extract plain text content without any formatting or HTML |
| generate_word_cloudB | Generate word frequency analysis and word cloud data from web content |
| translate_contentB | Translate web page content to different languages |
| extract_keywordsB | Extract important keywords and phrases from web content |
| analyze_readabilityC | Analyze text readability using various metrics |
| detect_languageB | Detect the primary language of web page content |
| extract_entitiesB | Extract named entities (people, places, organizations) from web content |
| sentiment_analysisC | Analyze sentiment and emotional tone of web content |
| classify_contentC | Classify web content into categories and topics |
| analyze_competitorsC | Analyze competitor websites for SEO and content insights |
| extract_schema_markupC | Extract and validate schema.org structured data markup |
| check_broken_linksC | Check for broken links and redirects on web pages |
| analyze_page_speedC | Analyze page loading speed and performance metrics |
| generate_meta_tagsC | Generate optimized meta tags for SEO based on content analysis |
| scan_vulnerabilitiesC | Scan web pages for common security vulnerabilities |
| check_ssl_certificateC | Check SSL certificate validity and security details |
| analyze_cookiesB | Analyze cookies set by web pages for privacy and security |
| detect_trackingC | Detect tracking scripts and privacy-related elements |
| check_privacy_policyC | Analyze privacy policy content and compliance |
| monitor_uptimeC | Monitor website uptime and availability |
| track_changes_detailedC | Track detailed changes in web page content with diff analysis |
| analyze_traffic_patternsC | Analyze traffic patterns and user behavior indicators |
| benchmark_performanceC | Benchmark website performance against competitors and industry standards |
| generate_reportsC | Generate comprehensive reports combining multiple analysis tools |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 48 tools
Most tools have distinct purposes, but there is notable overlap in some areas, such as analyze_performance and analyze_page_speed both focusing on performance metrics, and extract_structured_data and extract_schema_markup both handling structured data extraction. The descriptions help differentiate them, but agents might still face confusion in selecting the right tool for specific tasks.
Tool names follow a highly consistent verb_noun pattern throughout, such as analyze_competitors, extract_content, and validate_html. There are no deviations in naming conventions, making the set predictable and easy to navigate for agents.
With 48 tools, the count is excessive for a web scraping server, leading to potential bloat and complexity. A more focused set of 10-20 tools could cover the domain effectively without overwhelming agents, as many tools could be consolidated or omitted.
The tool set provides comprehensive coverage for web scraping and analysis, including extraction, analysis, monitoring, validation, and reporting. There are no obvious gaps; it supports full workflows from data retrieval to insights, ensuring agents can handle diverse tasks without dead ends.