Doc Scraper MCP Server

Integrations

  • Converts HTML documentation to markdown format and saves the converted documentation to a specified output path

Doc Scraper MCP Server

A Model Context Protocol (MCP) server that provides documentation scraping functionality. This server converts web-based documentation into markdown format using jina.ai's conversion service.

Features

  • Scrapes documentation from any web URL
  • Converts HTML documentation to markdown format
  • Saves the converted documentation to a specified output path
  • Integrates with the Model Context Protocol (MCP)

Installation

Installing via Smithery

To install Doc Scraper for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @askjohngeorge/mcp-doc-scraper --client claude
  1. Clone the repository:
git clone https://github.com/askjohngeorge/mcp-doc-scraper.git cd mcp-doc-scraper
  1. Create and activate a virtual environment:
python -m venv venv source venv/bin/activate # On Windows, use: venv\Scripts\activate
  1. Install the dependencies:
pip install -e .

Usage

The server can be run using Python:

python -m mcp_doc_scraper

Tool Description

The server provides a single tool:

  • Name: scrape_docs
  • Description: Scrape documentation from a URL and save as markdown
  • Input Parameters:
    • url: The URL of the documentation to scrape
    • output_path: The path where the markdown file should be saved

Project Structure

doc_scraper/ ├── __init__.py ├── __main__.py └── server.py

Dependencies

  • aiohttp
  • mcp
  • pydantic

Development

To set up the development environment:

  1. Install development dependencies:
pip install -r requirements.txt
  1. The server uses the Model Context Protocol. Make sure to familiarize yourself with MCP documentation.

License

MIT License

-
security - not tested
F
license - not found
-
quality - not tested

local-only server

The server can only run on the client's local machine because it depends on local resources.

Converts web-based documentation into markdown format using jina.ai's conversion service, allowing users to scrape documentation from any URL and save it as markdown files.

  1. Features
    1. Installation
      1. Installing via Smithery
    2. Usage
      1. Tool Description
    3. Project Structure
      1. Dependencies
        1. Development
          1. License

            Related MCP Servers

            • A
              security
              A
              license
              A
              quality
              Integrates Jina.ai's Reader API with LLMs for efficient and structured web content extraction, optimized for documentation and web content analysis.
              Last updated -
              1
              24
              24
              JavaScript
              MIT License
              • Linux
            • A
              security
              A
              license
              A
              quality
              Converts various file types and web content to Markdown format. It provides a set of tools to transform PDFs, images, audio files, web pages, and more into easily readable and shareable Markdown text.
              Last updated -
              10
              16
              987
              TypeScript
              MIT License
            • A
              security
              A
              license
              A
              quality
              Converts various file formats to Markdown using the MarkItDown utility and can be integrated with MCP clients for seamless document processing and conversion.
              Last updated -
              26
              Python
              MIT License
            • A
              security
              A
              license
              A
              quality
              An MCP server that enables users to download webpages as markdown files using r.jina.ai service, with features for configurable download directories and automatic date-stamped filenames.
              Last updated -
              5
              11
              23
              JavaScript
              MIT License

            View all related MCP servers

            ID: a5s4bbasd2