mcp-trafilatura-server
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-trafilatura-serverExtract the article from https://www.bbc.com/news"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP Trafilatura Server
A Model Context Protocol (MCP) server that provides web content extraction capabilities using Trafilatura's Python API. This server allows MCP clients to extract clean, readable content from web pages and HTML documents.
Features
Clean Content Extraction: Uses Trafilatura's advanced algorithms to extract main article content while filtering out navigation, ads, and boilerplate
Multiple Output Formats: Supports Markdown, plain text, and XML output
Flexible Input: Accept either URLs (with automatic fetching) or raw HTML content
Configurable Extraction: Fine-tune extraction behavior with precision, inclusion of tables, images, links, and comments
Async Implementation: Non-blocking operations with proper timeout handling
Robust Error Handling: Comprehensive error handling with informative error messages
Type Safety: Full type hints and Pydantic validation
Related MCP server: WebforAI Text Extractor
Installation
Quick Install via pip
pip install mcp-trafilatura-serverOption 1: Using uvx (Recommended - No Installation)
The easiest way to run the server with any MCP client:
# Run directly with uvx (no installation needed)
uvx mcp-trafilatura-server
# Or specify from local directory during development
uvx --from . mcp-trafilatura-serverFor use with Claude Desktop, add to your configuration:
{
"mcpServers": {
"trafilatura": {
"command": "uvx",
"args": ["mcp-trafilatura-server"]
}
}
}Or for local development:
{
"mcpServers": {
"trafilatura": {
"command": "uvx",
"args": ["--from", "/path/to/mcp-trafilatura", "mcp-trafilatura-server"]
}
}
}Option 2: Using uv for Development
# Clone the repository
git clone https://github.com/achieveai/mcp-web-extractor.git
cd mcp-trafilatura
# Create virtual environment and install
uv venv
uv pip install -e .
# Run the server
uv run mcp-trafilatura-serverOption 3: Traditional pip Installation
# Install from PyPI
pip install mcp-trafilatura-server
# Or for development:
git clone https://github.com/achieveai/mcp-web-extractor.git
cd mcp-trafilatura
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
pip install -e .Option 4: Using requirements.txt
# Install dependencies directly
pip install -r requirements.txt
# Then install the package
pip install -e .Usage
As an MCP Server
The server can be used with any MCP-compatible client. Register it in your client configuration:
{
"mcpServers": {
"web-extractor": {
"command": "mcp-trafilatura-server",
"args": []
}
}
}Direct Usage
You can also run the server directly:
# Run the server (listens on stdio)
mcp-trafilatura-server
# Or run the module directly
python -m trafilatura_mcp.serverTool: extract_markdown
The server provides a single tool called extract_markdown that extracts content from web pages or HTML.
Parameters
Required (one of):
url(string): URL to fetch and extract content from (http/https only)html(string): Raw HTML content to extract from
Optional:
precision(boolean, default: true): Favor precision over recall in extractioninclude_comments(boolean, default: false): Include HTML comments in outputinclude_tables(boolean, default: true): Include tables in extracted contentinclude_images(boolean, default: true): Include images in extracted contentinclude_links(boolean, default: true): Include links in extracted contenttimeout(integer, default: 30): Request timeout in seconds for URL fetching (5-120)output_format(string, default: "markdown"): Output format - "markdown", "txt", or "xml"
Example Usage
Extracting from a URL:
{
"name": "extract_markdown",
"arguments": {
"url": "https://example.com/article",
"precision": true,
"include_tables": true,
"output_format": "markdown"
}
}Extracting from HTML:
{
"name": "extract_markdown",
"arguments": {
"html": "<html><body><h1>Title</h1><p>Content...</p></body></html>",
"include_comments": false,
"output_format": "txt"
}
}Minimal usage:
{
"name": "extract_markdown",
"arguments": {
"url": "https://news.ycombinator.com/"
}
}Configuration for Popular MCP Clients
Claude Desktop
Add to your Claude Desktop configuration file:
Windows: %APPDATA%\Claude\claude_desktop_config.json
macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Using uvx (recommended - no installation needed):
{
"mcpServers": {
"trafilatura": {
"command": "uvx",
"args": ["mcp-trafilatura-server"]
}
}
}Or if installed via pip:
{
"mcpServers": {
"web-extractor": {
"command": "mcp-trafilatura-server",
"args": []
}
}
}VS Code with Continue
Add to your Continue configuration:
{
"mcpServers": [
{
"name": "trafilatura",
"command": "uvx",
"args": ["mcp-trafilatura-server"]
}
]
}Development
Project Structure
mcp-trafilatura/
├── src/
│ └── trafilatura_mcp/
│ ├── __init__.py
│ ├── server.py # Main server implementation
│ └── py.typed # Type hints marker
├── pyproject.toml # Package configuration
├── requirements.txt # Dependencies
├── LICENSE # GPL-3.0 license
├── MANIFEST.in # Package data files
└── README.md # This fileDevelopment Setup
# Clone the repository
git clone https://github.com/achieveai/mcp-web-extractor.git
cd mcp-trafilatura
# Create virtual environment
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
# Install development dependencies
pip install -e ".[dev]"
# Run type checking
mypy src/
# Run linting
ruff check src/
# Format code
black src/
isort src/Testing
# Install test dependencies
pip install -e ".[dev]"
# Run tests
pytest
# Run tests with coverage
pytest --cov=trafilatura_mcpError Handling
The server provides comprehensive error handling:
Input Validation: Invalid URLs, missing required parameters, or invalid parameter values
Network Errors: Connection timeouts, HTTP errors, or unreachable URLs
Extraction Errors: Empty content, parsing failures, or unsupported content types
Server Errors: Internal errors with detailed logging
All errors are returned as proper MCP error responses with descriptive messages.
Logging
The server uses Python's built-in logging with INFO level by default. Logs include:
Server startup and shutdown
URL fetching attempts
Content extraction operations
Error conditions with details
Dependencies
modelcontextprotocol: MCP protocol implementation
pydantic: Data validation and settings management
trafilatura: Core content extraction functionality
httpx: Async HTTP client for URL fetching
typing-extensions: Additional type hints support
License
This project is licensed under the GNU General Public License v3.0 (GPL-3.0) because it directly imports and uses Trafilatura's Python API, which is GPL-3.0 licensed. See the LICENSE file for details.
Important: Since this server uses Trafilatura's Python API (Option B from docs), it constitutes a derivative work under GPL-3.0. If you need different licensing terms, consider:
Using Option A (CLI subprocess approach) for more licensing flexibility
Implementing your own extraction logic
Contacting Trafilatura maintainers for commercial licensing
Contributing
Fork the repository
Create a feature branch
Make your changes
Add tests for new functionality
Ensure all tests pass
Submit a pull request
Support
For issues, questions, or contributions, please visit the project repository or open an issue.
Changelog
v0.1.0
Initial release
Basic content extraction from URLs and HTML
Support for multiple output formats
Comprehensive error handling and validation
Async implementation with timeout support
This server cannot be installed
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Latest Blog Posts
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/achieveai/mcp-web-extractor'
If you have feedback or need assistance with the MCP directory API, please join our Discord server