Skip to main content
Glama
Sign In
enesjakozh

Website Scraper MCP Server

by tolik-unicornrider
npmGitHub
JavaScript
11,993
2
OverviewInspectNewSchemaRelated ServersReviewsScore
Need Help?View Source CodeReport Issue

Integrations

  • Utilizes Mozilla's Readability library (the same engine used in Firefox's Reader View) to extract meaningful content from web pages for conversion to Markdown

  • Converts clean HTML to high-quality Markdown with TurndownService, supporting both web scraping and direct conversion of local HTML files

  • Leverages Mozilla's Readability library to extract the main content from web pages while removing clutter and navigation elements

Website Scraper

A command-line tool and MCP server for scraping websites and converting HTML to Markdown.

Features

  • Extracts meaningful content from web pages using Mozilla's Readability library (the same engine used in Firefox's Reader View)
  • Converts clean HTML to high-quality Markdown with TurndownService
  • Securely handles HTML by removing potentially harmful script tags
  • Works as both a command-line tool and an MCP server
  • Supports direct conversion of local HTML files to Markdown

Installation

# Install dependencies npm install # Build the project npm run build # Optionally, install globally npm install -g .

Usage

CLI Mode

# Print output to console scrape https://example.com # Save output to a file scrape https://example.com output.md # Convert a local HTML file to Markdown scrape --html-file input.html # Convert a local HTML file and save output to a file scrape --html-file input.html output.md # Show help scrape --help # Or run via npm script npm run start:cli -- https://example.com

MCP Server Mode

This tool can be used as a Model Context Protocol (MCP) server:

# Start in MCP server mode npm start

Code Structure

  • src/index.ts - Core functionality and MCP server implementation
  • src/cli.ts - Command-line interface implementation
  • src/data_processing.ts - HTML to Markdown conversion functionality

API

The tool exports the following functions:

// Scrape a website and convert to Markdown import { scrapeToMarkdown } from './build/index.js'; // Convert HTML string to Markdown directly import { htmlToMarkdown } from './build/data_processing.js'; async function example() { // Web scraping const markdown = await scrapeToMarkdown('https://example.com'); console.log(markdown); // Direct HTML conversion const html = '<h1>Hello World</h1><p>This is <strong>bold</strong> text.</p>'; const md = htmlToMarkdown(html); console.log(md); }

License

ISC

This server cannot be installed

-
security - not tested
F
license - not found
-
quality - not tested

hybrid server

The server is able to function both locally and remotely, depending on the configuration or use case.

An MCP server that extracts meaningful content from websites and converts HTML to high-quality Markdown, using Mozilla's Readability engine.

  1. Features
    1. Installation
      1. Usage
        1. CLI Mode
        2. MCP Server Mode
      2. Code Structure
        1. API
          1. License

            Related MCP Servers

            • MCP NPX Fetch

              tokenizin-agency
              A
              security
              A
              license
              A
              quality
              A powerful MCP server for fetching and transforming web content into various formats (HTML, JSON, Markdown, Plain Text) with ease.
              Last updated -
              4
              146
              12
              TypeScript
              MIT License
              • Apple
              • Linux

            • MCP Server Fetch Python

              tatn
              A
              security
              A
              license
              A
              quality
              An MCP server for fetching and transforming web content into various formats.
              Last updated -
              4
              4
              Python
              MIT License
              • Apple

            • MD Webcrawl MCP

              jmh108
              -
              security
              A
              license
              -
              quality
              A Python-based MCP server that crawls websites to extract and save content as markdown files, with features for mapping website structure and links.
              Last updated -
              1
              Python
              MIT License

            • Mozilla Readability Parser MCP Server

              jmh108
              -
              security
              A
              license
              -
              quality
              A Python implementation of an MCP server that extracts webpage content, removes ads and non-essential elements, and transforms it into clean, LLM-optimized Markdown.
              Last updated -
              1
              Python
              MIT License
              • Linux
              • Apple

            View all related MCP servers

            New MCP Servers

            ID: 1o02okzu2w