Skip to main content
Glama
enesjakozh

Website Scraper MCP Server

by tolik-unicornrider
npmGitHub
JavaScript
7,866
6
OverviewInspectNewSchemaRelated ServersReviewsScore
Need Help?View Source CodeReport Issue

网站抓取工具

用于抓取网站并将 HTML 转换为 Markdown 的命令行工具和 MCP 服务器。

特征

  • 使用 Mozilla 的Readability库（与 Firefox 的 Reader View 使用的引擎相同）从网页中提取有意义的内容
  • 使用 TurndownService 将干净的 HTML 转换为高质量的 Markdown
  • 通过删除潜在有害的脚本标签来安全地处理 HTML
  • 既可用作命令行工具，又可用作 MCP 服务器
  • 支持本地HTML文件直接转换为Markdown

安装

# Install dependencies npm install # Build the project npm run build # Optionally, install globally npm install -g .

用法

CLI模式

# Print output to console scrape https://example.com # Save output to a file scrape https://example.com output.md # Convert a local HTML file to Markdown scrape --html-file input.html # Convert a local HTML file and save output to a file scrape --html-file input.html output.md # Show help scrape --help # Or run via npm script npm run start:cli -- https://example.com

MCP 服务器模式

该工具可用作模型上下文协议（MCP）服务器：

# Start in MCP server mode npm start

代码结构

  • src/index.ts - 核心功能和 MCP 服务器实现
  • src/cli.ts - 命令行界面实现
  • src/data_processing.ts - HTML 到 Markdown 的转换功能

API

该工具导出以下功能：

// Scrape a website and convert to Markdown import { scrapeToMarkdown } from './build/index.js'; // Convert HTML string to Markdown directly import { htmlToMarkdown } from './build/data_processing.js'; async function example() { // Web scraping const markdown = await scrapeToMarkdown('https://example.com'); console.log(markdown); // Direct HTML conversion const html = '<h1>Hello World</h1><p>This is <strong>bold</strong> text.</p>'; const md = htmlToMarkdown(html); console.log(md); }

执照

国际学习中心

Install Server
A
security – no known vulnerabilities
F
license - not found
A
quality - confirmed to work

How are these scores calculated?

hybrid server

The server is able to function both locally and remotely, depending on the configuration or use case.

Tools

MCP 服务器使用 Mozilla 的 Readability 引擎从网站中提取有意义的内容并将 HTML 转换为高质量的 Markdown。

  1. 特征
    1. 安装
      1. 用法
        1. CLI模式
        2. MCP 服务器模式
      2. 代码结构
        1. API
          1. 执照

            Related MCP Servers

            • MCP NPX Fetch

              tokenizin-agency
              A
              security
              A
              license
              A
              quality
              A powerful MCP server for fetching and transforming web content into various formats (HTML, JSON, Markdown, Plain Text) with ease.
              Last updated -
              4
              1,285
              32
              TypeScript
              MIT License
              • Apple
              • Linux

            • MCP Server Fetch Python

              tatn
              A
              security
              A
              license
              A
              quality
              An MCP server for fetching and transforming web content into various formats.
              Last updated -
              4
              6
              Python
              MIT License
              • Apple

            • Fetch MCP

              jae-jae
              A
              security
              A
              license
              A
              quality
              An MCP server that retrieves web page content using Playwright headless browser, capable of extracting main content and converting to Markdown format.
              Last updated -
              2
              658
              727
              TypeScript
              MIT License
              • Apple

            • markdown2pdf-mcp

              Serendipity-AI
              A
              security
              A
              license
              A
              quality
              MCP server that converts Markdown to high-quality PDF documents using LaTeX, enabling AI agents like Claude to generate professional PDFs without requiring sign-ups or credit cards.
              Last updated -
              1
              600
              6
              JavaScript
              MIT License

            View all related MCP servers

            Appeared in Searches

            New MCP Servers

            MCP directory API

            We provide all the information about MCP servers via our MCP API.

            curl -X GET 'https://glama.ai/api/mcp/v1/servers/tolik-unicornrider/mcp_scraper'

            If you have feedback or need assistance with the MCP directory API, please join our Discord server