Skip to main content
Glama
tolik-unicornrider

Website Scraper MCP Server

웹사이트 스크래퍼

웹사이트를 스크래핑하고 HTML을 마크다운으로 변환하는 명령줄 도구이자 MCP 서버입니다.

특징

  • Mozilla의 Readability 라이브러리(Firefox의 Reader View에서 사용되는 것과 동일한 엔진)를 사용하여 웹 페이지에서 의미 있는 콘텐츠를 추출합니다.

  • TurndownService를 사용하여 깨끗한 HTML을 고품질 Markdown으로 변환합니다.

  • 잠재적으로 유해한 스크립트 태그를 제거하여 HTML을 안전하게 처리합니다.

  • 명령줄 도구와 MCP 서버로 모두 작동합니다.

  • 로컬 HTML 파일을 Markdown으로 직접 변환하는 기능을 지원합니다.

Related MCP server: superFetch MCP Server

설치

지엑스피1

용법

CLI 모드

# Print output to console
scrape https://example.com

# Save output to a file
scrape https://example.com output.md

# Convert a local HTML file to Markdown
scrape --html-file input.html

# Convert a local HTML file and save output to a file
scrape --html-file input.html output.md

# Show help
scrape --help

# Or run via npm script
npm run start:cli -- https://example.com

MCP 서버 모드

이 도구는 MCP(Model Context Protocol) 서버로 사용할 수 있습니다.

# Start in MCP server mode
npm start

코드 구조

  • src/index.ts - 핵심 기능 및 MCP 서버 구현

  • src/cli.ts - 명령줄 인터페이스 구현

  • src/data_processing.ts - HTML을 마크다운으로 변환하는 기능

API

이 도구는 다음 기능을 내보냅니다.

// Scrape a website and convert to Markdown
import { scrapeToMarkdown } from './build/index.js';

// Convert HTML string to Markdown directly
import { htmlToMarkdown } from './build/data_processing.js';

async function example() {
  // Web scraping
  const markdown = await scrapeToMarkdown('https://example.com');
  console.log(markdown);
  
  // Direct HTML conversion
  const html = '<h1>Hello World</h1><p>This is <strong>bold</strong> text.</p>';
  const md = htmlToMarkdown(html);
  console.log(md);
}

특허

아이에스씨

Available Tools

1 tool
scrape-to-markdownD
ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes

TDQS

D1/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Tool has no description.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness1/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Tool has no description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Tool has no description.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has no description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose1/5

Does the description clearly state what the tool does and how it differs from similar tools?

Tool has no description.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines1/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Tool has no description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool update
    • First observedscrape-to-markdown

TDQS

D1.7/5.0

Scored across 1 tool

Disambiguation5/5

With only one tool, there is no possibility of ambiguity or overlap between tools. The single tool 'scrape-to-markdown' stands alone with no other tools to confuse it with.

Naming Consistency5/5

Since there is only one tool, naming consistency is inherently perfect. The tool name 'scrape-to-markdown' follows a clear verb-noun pattern, and there are no other tools to deviate from this pattern.

Tool Count2/5

A single tool for a 'Website Scraper MCP Server' is too few for the apparent scope, as scraping typically involves multiple operations like configuring, parsing, or handling errors. This minimal set feels incomplete and thin for the domain.

Completeness1/5

The tool set is severely incomplete for a scraping server. It lacks essential operations such as fetching URLs, handling different content types, error management, or data extraction beyond markdown. This single tool creates significant gaps that will cause agent failures in real-world scraping tasks.

Maintenance

ActivityInactive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • F
    license
    Not graded
    quality
    Not graded
    maintenance
    An MCP server for web content extraction that converts HTML pages into clean, LLM-optimized Markdown using Mozilla's Readability. It supports batch processing, intelligent multi-page crawling, and configurable caching while respecting robots.txt standards.
    69 npm
    -
  • A
    license
    A
    quality
    C
    maintenance
    An MCP server that fetches web pages and extracts clean, AI-friendly Markdown content using Mozilla Readability. It provides secure web access for LLMs with built-in SSRF protection and automated content cleaning for improved context retrieval and summarization.
    1
    90 npm
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    An MCP server that scrapes content from web pages, including JavaScript-heavy sites, and converts it into high-quality Markdown. It leverages Playwright for headless browser automation and Pypandoc for clean content conversion.
    Apache 2.0
  • A
    license
    Not graded
    quality
    F
    maintenance
    An MCP server that extracts clean Markdown or HTML content from web pages by stripping away ads, navigation, and clutter. It offers tools to process URLs or raw HTML, returning structured metadata alongside the main article content.
    2
    MIT