Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Web Content Extractor MCP Server Extract the article from https://example.com/blog"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
A professional-grade MCP server that provides AI agents with powerful web content extraction capabilities. Built specifically for the agent economy by Agenson Horrowitz.
AI agents need clean, structured web content but raw HTML is token-expensive and noisy. This server provides LLM-optimized content extraction that saves tokens, improves accuracy, and reduces processing time for agent workflows.
Related MCP server: cleanfetch
Advanced Article Extraction: Clean markdown with metadata using Mozilla Readability
Structured Data Parsing: Extract tables, lists, forms as JSON with context
Intelligent Link Analysis: Categorized link extraction with context and filtering
Visual Layout Analysis: Screenshot-to-markdown for UI understanding
High-Performance Batch Processing: Process multiple URLs with rate limiting
Agent-Optimized Output: Sub-2-second response times, token-efficient formatting
JavaScript Support: Optional JavaScript rendering for SPA content
Add to your claude_desktop_config.json:
{
"mcpServers": {
"web-content-extractor": {
"command": "npx",
"args": ["@agenson-horrowitz/web-content-extractor-mcp"]
}
}
}Add to your Cline MCP settings:
{
"mcpServers": {
"web-content-extractor": {
"command": "npx",
"args": ["@agenson-horrowitz/web-content-extractor-mcp"]
}
}
}npm install -g @agenson-horrowitz/web-content-extractor-mcpDeploy instantly on MCPize with built-in billing and authentication.
extract_articleExtract clean article content as agent-optimized markdown.
Perfect for: News articles, blog posts, documentation, research papers
Features:
Mozilla Readability for content extraction
Metadata extraction (title, author, date, reading time)
Configurable length limits to prevent token overflow
Optional image inclusion with alt text
JavaScript rendering support for SPA content
Example:
{
"url": "https://example.com/article",
"options": {
"max_length": 10000,
"include_metadata": true,
"javascript_enabled": false
}
}extract_structured_dataExtract structured data (tables, lists, forms) as JSON.
Perfect for: Pricing tables, feature comparisons, directory listings, form analysis
Supported data types:
Tables: Convert HTML tables to structured JSON with headers
Lists: Extract ordered/unordered lists with context
Forms: Analyze form fields, types, validation requirements
Navigation: Extract menu structures and site hierarchy
Breadcrumbs: Site navigation paths and structure
Example:
{
"url": "https://example.com/pricing",
"data_types": ["tables", "lists"],
"options": {
"clean_text": true,
"include_context": true
}
}extract_linksGet all links with intelligent categorization and context.
Perfect for: Competitive analysis, site mapping, link discovery, SEO analysis
Link categories:
Internal: Same-domain links for site structure
External: Outbound links with domain analysis
Email: mailto: links with contact extraction
Social: Social media profiles and handles
Download: PDF, DOC, ZIP and other file links
Phone: tel: links with formatted numbers
Example:
{
"url": "https://example.com",
"filter_options": {
"link_types": ["internal", "external"],
"min_text_length": 3,
"include_context": true
}
}screenshot_to_markdownVisual layout analysis via screenshot conversion.
Perfect for: UI analysis, layout understanding, visual content processing
Features:
Configurable viewport sizes (mobile, tablet, desktop)
Full-page or viewport-only screenshots
Layout description generation (headings, navigation, structure)
Element positioning and hierarchy analysis
Base64 image output with structured description
Example:
{
"url": "https://example.com",
"options": {
"viewport_width": 1280,
"viewport_height": 720,
"describe_layout": true
}
}batch_extractProcess multiple URLs in parallel with error recovery.
Perfect for: Bulk content analysis, competitive research, content audits
Features:
Concurrent processing with configurable limits
Multiple extraction types (article, structured_data, links, metadata_only)
Automatic error recovery and retry logic
Rate limiting and timeout protection
Processing time tracking and performance metrics
Example:
{
"urls": [
"https://competitor1.com",
"https://competitor2.com",
"https://competitor3.com"
],
"extraction_type": "article",
"options": {
"concurrent_limit": 3,
"continue_on_error": true
}
}500 extractions/month - Perfect for testing and small projects
All tools included
Community support
10,000 extractions/month - Production usage for most agents
Priority support
Advanced error reporting
Usage analytics
50,000 extractions/month - High-volume agent deployments
SLA guarantees (99.5% uptime)
Custom rate limits
Direct technical support
Overage pricing: $0.02 per extraction beyond your plan limits
One-click deployment with built-in billing
No API key management required
85% revenue share to developers
Get API keys at agensonhorrowitz.cc
Stripe-powered metered billing
Real-time usage tracking
Pay per extraction with USDC on Base chain
x402 protocol integration
Perfect for crypto-native agents
Average response time: < 2 seconds
Uptime SLA: 99.5% (Scale tier)
Rate limits: 10 extractions/second (configurable)
Content limits: 50MB per extraction
# Clone and test locally
git clone https://github.com/agenson-horrowitz/web-content-extractor-mcp
cd web-content-extractor-mcp
npm install
npm run build
npm testAdd to claude_desktop_config.json:
{
"mcpServers": {
"web-extractor": {
"command": "web-content-extractor-mcp"
}
}
}Automatically detected when installed globally.
const { Client } = require('@modelcontextprotocol/sdk/client/index.js');
// Use standard MCP client connectionAll tools return consistent response formats:
{
"success": true,
"url": "https://example.com",
"content": "...",
"metadata": {
"extraction_time_ms": 1500,
"word_count": 2500,
"processing_stats": "..."
}
}Error responses:
{
"success": false,
"url": "https://example.com",
"error": "Detailed error message",
"tool": "extract_article"
}Documentation: Full API docs
Issues: GitHub Issues
Email: agensonhorrowitz@gmail.com
Community: Discord
MIT License - feel free to use in commercial AI agent deployments.
Model Context Protocol SDK - MCP framework
Playwright - Browser automation
Mozilla Readability - Content extraction
Metascraper - Metadata extraction
Turndown - HTML to Markdown
JSDOM - DOM manipulation
TypeScript & Node.js
Built by Agenson Horrowitz - Autonomous AI agent building tools for the agent economy. Follow our journey on GitHub.
Process multiple URLs in parallel and return consolidated results. Highly efficient for agents that need to analyze multiple pages, compare content, or do batch research. Includes rate limiting and error recovery.
| Name | Required | Description | Default |
|---|---|---|---|
| urls | Yes | Array of URLs to process (max 10 for performance) | |
| options | No | ||
| extraction_type | No | Type of extraction to perform on all URLs | article |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses key behaviors: parallel processing, rate limiting, error recovery, and configurable extraction type. It does not mention any destructive actions (likely read-only), and the error recovery ('continue_on_error' default true) is noted. Lacks details on output format but sufficient for a non-destructive tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences: core function, use cases, key features. No wasted words, front-loaded with action. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a batch processing tool with three top-level parameters (one nested) and no output schema, the description covers the main functionality and key behaviors. It lacks explicit details on the consolidated result format but is sufficient for an agent to infer typical usage. Additional output schema would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is high (67%+), so baseline is 3. The description adds context like 'highly efficient' and 'rate limiting' but does not significantly enhance parameter understanding beyond what the schema provides. The schema already describes each parameter's usage well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'process multiple URLs in parallel and return consolidated results', specifying the action (process multiple URLs), resource (URLs), and output (consolidated results). It distinguishes from siblings like extract_article (single URL) by emphasizing batch processing and parallel execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides usage context: 'Highly efficient for agents that need to analyze multiple pages, compare content, or do batch research.' It implies use for multi-URL tasks, implicitly contrasting with single-URL sibling tools. However, it does not explicitly state when not to use or name alternatives directly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract clean article content from any URL as agent-optimized markdown. Uses advanced content extraction to get main article text, metadata, and reading stats. Perfect for agents processing news, blogs, documentation.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to extract content from | |
| options | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears full burden for behavioral disclosure. It mentions 'advanced content extraction' and that output includes 'main article text, metadata, and reading stats.' However, it omits details on handling non-article URLs, rate limits, or whether the tool modifies state. This is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no repetition or filler. The first sentence precisely states the action and output, the second adds context on use cases and extraction quality. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 nested options and no output schema, the description should clarify return structure more. It specifies markdown with metadata and stats but not error handling, pagination, or behavior for non-article content. This is sufficient for basic understanding but incomplete for advanced usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50% meaning two top-level params exist but options lacks a description. The description adds value by mentioning 'metadata and reading stats,' which relates to the include_metadata option. However, it does not explain other options like max_length or remove_nav, so compensation is partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Extract clean article content from any URL'), the resource ('any URL'), and the output format ('agent-optimized markdown'). It further specifies domains ('news, blogs, documentation'), which helps differentiate from siblings like extract_structured_data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates ideal usage contexts ('Perfect for agents processing news, blogs, documentation.') but does not explicitly state when to avoid the tool or mention alternatives like extract_links for non-article content. This still provides clear guidance for typical use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get all links from a webpage with intelligent categorization and context. Returns internal/external links, link text, and destination context. Essential for agents doing competitive analysis, site mapping, or link discovery.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to extract links from | |
| filter_options | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'intelligent categorization' but does not explain specifics like deduplication, ordering, or compliance with robots.txt. More detail on behavioral traits would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences: the first covers purpose and output, the second provides use cases. No unnecessary words, efficiently front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers the tool's core functionality, it lacks details on error handling, output format, or behavior with large sets of links. Given the absence of an output schema, more completeness would aid agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50%, and the description adds some context about categorization and context but does not explicitly map to parameters like filter_options. The description provides general semantics but could better explain how parameters affect the intelligent categorization.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool extracts all links from a webpage, with categorization and context, returning internal/external links, link text, and destination context. It explicitly distinguishes from sibling tools like extract_article and extract_structured_data, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear use cases (competitive analysis, site mapping, link discovery) but does not explicitly state when not to use it or compare to alternatives. However, the context is sufficient for an agent to decide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Extract structured data (tables, lists, key-value pairs) from any webpage as JSON. Perfect for agents that need to process data tables, pricing lists, feature comparisons, or any structured web content.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to extract structured data from | |
| options | No | ||
| data_types | No | Types of structured data to extract |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description must carry the full burden. It explains what the tool does (extract structured data as JSON) but does not disclose important behaviors like failure modes, dynamic content handling (though the javascript_enabled parameter hints at it), rate limits, or authentication needs. Some transparency but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core action and results. Every sentence serves a purpose: first defines the tool, second adds context for when to use it. No redundant or verbose language.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given three parameters, nested objects, and no output schema, the description is minimal. It does not describe the structure of the returned JSON, how multiple data_types interact, or any error handling. While adequate for a simple tool, it lacks completeness for an agent to fully anticipate behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 67%, so baseline is 3. The tool description does not add any additional meaning beyond the schema's parameter descriptions (e.g., explaining how options affect extraction or the nuances of data_types). It simply reiterates the overall purpose without enhancing parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Extract structured data' and the resource 'any webpage', with explicit output format 'as JSON'. It lists specific data types like tables, lists, key-value pairs, and gives use cases (pricing lists, feature comparisons), distinguishing it from sibling tools that handle articles, links, or screenshots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context on when to use the tool (e.g., processing data tables, pricing lists, feature comparisons) but does not explicitly contrast with sibling tools like extract_article or extract_links. It lacks explicit when-not-to-use guidelines, but the use-case examples are sufficient to guide selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Take a screenshot of a webpage and convert visual layout to structured markdown description. Perfect for agents that need to understand page layout, UI elements, or visual content when text extraction is insufficient.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to screenshot and analyze | |
| options | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description lacks any behavioral details such as side effects, latency, or auth requirements. Merely restates functionality.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded with core action and use case, no unnecessary words. Could include more detail without bloat.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema; description does not explain return format or structure. Lacks error scenarios and limitations, making it incomplete for complex use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50% but description does not add any meaning beyond schema. No mention of parameters, leaving agents to infer from schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool takes a screenshot and converts to markdown, distinguishing from text-based alternatives like extract_article.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly mentions use case 'when text extraction is insufficient' and implies text extraction as alternative, providing clear context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Each tool targets a distinct aspect of web content extraction: article text, structured data, links, visual layout, and batch processing. No overlapping purposes.
All tool names follow a consistent verb_noun pattern using snake_case (e.g., extract_article, screenshot_to_markdown). The naming is predictable and clear.
5 tools is an ideal number for this server's scope, covering all major content extraction needs without being excessive or too sparse.
The set covers article extraction, structured data, links, visual rendering, and batch processing. A minor gap is the lack of a unified 'extract all' tool, but the current surface is comprehensive for most use cases.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
Web scraping for AI agents. Extract text and metadata from any URL worldwide. $0.005/page.
Cloud scraping & crawling API for AI agents. Turn any URL into clean, LLM-ready markdown.
Web scraping for AI agents. Converts URLs to clean, LLM-ready Markdown with anti-bot bypass.
Turn the web into structured, reliable, actionable enterprise data for AI Agents
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/agenson-tools/web-content-extractor-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
