Doc Scraper MCP Server

A Model Context Protocol (MCP) server that provides documentation scraping functionality. This server converts web-based documentation into markdown format using jina.ai's conversion service.

Features

Scrapes documentation from any web URL
Converts HTML documentation to markdown format
Saves the converted documentation to a specified output path
Integrates with the Model Context Protocol (MCP)

Installation

Installing via Smithery

To install Doc Scraper for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @askjohngeorge/mcp-doc-scraper --client claude

Clone the repository:

git clone https://github.com/askjohngeorge/mcp-doc-scraper.git
cd mcp-doc-scraper

Create and activate a virtual environment:

python -m venv venv
source venv/bin/activate  # On Windows, use: venv\Scripts\activate

Install the dependencies:

pip install -e .

Usage

The server can be run using Python:

python -m mcp_doc_scraper

Tool Description

The server provides a single tool:

Name: scrape_docs
Description: Scrape documentation from a URL and save as markdown
Input Parameters:
- url: The URL of the documentation to scrape
- output_path: The path where the markdown file should be saved

Project Structure

doc_scraper/
├── __init__.py
├── __main__.py
└── server.py

Dependencies

aiohttp
mcp
pydantic

Development

To set up the development environment:

Install development dependencies:

pip install -r requirements.txt

The server uses the Model Context Protocol. Make sure to familiarize yourself with MCP documentation.

License

MIT License

This server cannot be installed

security - not tested

license - not found

quality - not tested

How are these scores calculated?

hybrid server

The server is able to function both locally and remotely, depending on the configuration or use case.

Converts web-based documentation into markdown format using jina.ai's conversion service, allowing users to scrape documentation from any URL and save it as markdown files.

Related Resources

Reddit Discussion about this server

Related MCP Servers

Markdownify MCP Server
zcaceres
A
security
A
license
A
quality
Converts various file types and web content to Markdown format. It provides a set of tools to transform PDFs, images, audio files, web pages, and more into easily readable and shareable Markdown text.
Last updated -
10
10
2,019
TypeScript
MIT License
Markdownify MCP Server - UTF-8 Enhanced
JDJR2024
A
security
A
license
A
quality
A document conversion server that transforms various file formats (PDFs, documents, images, audio, web content) to Markdown with improved multilingual and UTF-8 support.
Last updated -
10
0
9
TypeScript
MIT License
MCP Markdown Conversion Server
FradSer
A
security
F
license
A
quality
A server that converts various file formats (PDF, images, Office documents, etc.) to Markdown descriptions using Cloudflare AI services.
Last updated -
1
8
33
JavaScript
read-website-fast
just-every
A
security
A
license
A
quality
Fast, token-efficient web content extraction tool that converts websites to clean Markdown for AI agents, featuring smart caching, content extraction with Mozilla Readability, and polite crawling capabilities.
Last updated -
1
662
67
JavaScript
MIT License

View all related MCP servers

Doc Scraper MCP Server