Skip to main content
Glama
Website Scraper MCP Server

Website Scraper MCP Server

ウェブサイトスクレーパー

ウェブサイトをスクレイピングし、HTML を Markdown に変換するためのコマンドライン ツールと MCP サーバー。

特徴

  • Mozilla のReadabilityライブラリ (Firefox のリーダー ビューで使用されているのと同じエンジン) を使用して、Web ページから意味のあるコンテンツを抽出します。

  • TurndownService を使用して、クリーンな HTML を高品質の Markdown に変換します。

  • 潜在的に有害なスクリプトタグを削除してHTMLを安全に処理します

  • コマンドラインツールとMCPサーバーの両方として機能します

  • ローカル HTML ファイルを Markdown に直接変換できます

Related MCP server: superFetch MCP Server

インストール

# Install dependencies
npm install

# Build the project
npm run build

# Optionally, install globally
npm install -g .

使用法

CLIモード

# Print output to console
scrape https://example.com

# Save output to a file
scrape https://example.com output.md

# Convert a local HTML file to Markdown
scrape --html-file input.html

# Convert a local HTML file and save output to a file
scrape --html-file input.html output.md

# Show help
scrape --help

# Or run via npm script
npm run start:cli -- https://example.com

MCP サーバーモード

このツールは、モデル コンテキスト プロトコル (MCP) サーバーとして使用できます。

# Start in MCP server mode
npm start

コード構造

  • src/index.ts - コア機能とMCPサーバーの実装

  • src/cli.ts - コマンドラインインターフェースの実装

  • src/data_processing.ts - HTML から Markdown への変換機能

API

このツールは次の機能をエクスポートします。

// Scrape a website and convert to Markdown
import { scrapeToMarkdown } from './build/index.js';

// Convert HTML string to Markdown directly
import { htmlToMarkdown } from './build/data_processing.js';

async function example() {
  // Web scraping
  const markdown = await scrapeToMarkdown('https://example.com');
  console.log(markdown);
  
  // Direct HTML conversion
  const html = '<h1>Hello World</h1><p>This is <strong>bold</strong> text.</p>';
  const md = htmlToMarkdown(html);
  console.log(md);
}

ライセンス

ISC

Available Tools

1 tool
scrape-to-markdownD

Inputs: url (required).

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    D1.7/5.0

    Scored across 1 tool

    Disambiguation5/5

    With only one tool, there is no possibility of ambiguity or overlap between tools. The single tool 'scrape-to-markdown' stands alone with no other tools to confuse it with.

    Naming Consistency5/5

    Since there is only one tool, naming consistency is inherently perfect. The tool name 'scrape-to-markdown' follows a clear verb-noun pattern, and there are no other tools to deviate from this pattern.

    Tool Count2/5

    A single tool for a 'Website Scraper MCP Server' is too few for the apparent scope, as scraping typically involves multiple operations like configuring, parsing, or handling errors. This minimal set feels incomplete and thin for the domain.

    Completeness1/5

    The tool set is severely incomplete for a scraping server. It lacks essential operations such as fetching URLs, handling different content types, error management, or data extraction beyond markdown. This single tool creates significant gaps that will cause agent failures in real-world scraping tasks.

    Maintenance

    ActivityInactive
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      Not graded
      maintenance
      An MCP server for web content extraction that converts HTML pages into clean, LLM-optimized Markdown using Mozilla's Readability. It supports batch processing, intelligent multi-page crawling, and configurable caching while respecting robots.txt standards.
      48 npm
      -
    • A
      license
      A
      quality
      D
      maintenance
      An MCP server that fetches web pages and extracts clean, AI-friendly Markdown content using Mozilla Readability. It provides secure web access for LLMs with built-in SSRF protection and automated content cleaning for improved context retrieval and summarization.
      1
      80 npm
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      An MCP server that scrapes content from web pages, including JavaScript-heavy sites, and converts it into high-quality Markdown. It leverages Playwright for headless browser automation and Pypandoc for clean content conversion.
      Apache 2.0
    • A
      license
      Not graded
      quality
      F
      maintenance
      An MCP server that extracts clean Markdown or HTML content from web pages by stripping away ads, navigation, and clutter. It offers tools to process URLs or raw HTML, returning structured metadata alongside the main article content.
      2
      MIT