Which integrations are available for this server?

Utilizes Mozilla's Readability library (the same engine used in Firefox's Reader View) to extract meaningful content from web pages for conversion to Markdown Converts clean HTML to high-quality Markdown with TurndownService, supporting both web scraping and direct conversion of local HTML files Leverages Mozilla's Readability library to extract the main content from web pages while removing clutter and navigation elements

How do I use Website Scraper MCP Server?

1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Website Scraper MCP Server scrape the latest blog post from example.com" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

Website Scraper

A command-line tool and MCP server for scraping websites and converting HTML to Markdown.

Features

Extracts meaningful content from web pages using Mozilla's Readability library (the same engine used in Firefox's Reader View)
Converts clean HTML to high-quality Markdown with TurndownService
Securely handles HTML by removing potentially harmful script tags
Works as both a command-line tool and an MCP server
Supports direct conversion of local HTML files to Markdown

Related MCP server: MCP Server Fetch Python

Installation

# Install dependencies
npm install

# Build the project
npm run build

# Optionally, install globally
npm install -g .

Usage

CLI Mode

# Print output to console
scrape https://example.com

# Save output to a file
scrape https://example.com output.md

# Convert a local HTML file to Markdown
scrape --html-file input.html

# Convert a local HTML file and save output to a file
scrape --html-file input.html output.md

# Show help
scrape --help

# Or run via npm script
npm run start:cli -- https://example.com

MCP Server Mode

This tool can be used as a Model Context Protocol (MCP) server:

# Start in MCP server mode
npm start

Code Structure

src/index.ts - Core functionality and MCP server implementation
src/cli.ts - Command-line interface implementation
src/data_processing.ts - HTML to Markdown conversion functionality

API

The tool exports the following functions:

// Scrape a website and convert to Markdown
import { scrapeToMarkdown } from './build/index.js';

// Convert HTML string to Markdown directly
import { htmlToMarkdown } from './build/data_processing.js';

async function example() {
  // Web scraping
  const markdown = await scrapeToMarkdown('https://example.com');
  console.log(markdown);
  
  // Direct HTML conversion
  const html = '<h1>Hello World</h1><p>This is <strong>bold</strong> text.</p>';
  const md = htmlToMarkdown(html);
  console.log(md);
}