Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@redup.mcp-web-parser Parse https://example.com into markdown"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP Streamable HTTP service that parses web pages into cleaned markdown via
Crawl4AI POST /crawl (0.8.x).
Thin MCP façade over Crawl4AI HTTP API — no browser in this image.
Crawl requests set exclude_all_images (and disable screenshot/pdf) so
upstream payloads stay small; markdown and links are unchanged for tools.
upstream_base_url is required at runtime (config or
McpWebParser___upstream_base_url). Defaults ship empty (OSS-safe: no
cluster hostnames or internal proxies in the repo).
Optional egress proxy for Crawl4AI IP substitution via server config
default_proxy → crawler_config.proxy_config.server. Empty = direct fetch.
Proxy is not a tool argument (deploy/runtime only).
Tool results are JSON (success / markdown / status_code / …),
not a concatenated text dump.
Targeted at Crawl4AI 0.8.x (per-request proxy_config works). On 0.9+
Docker API may reject proxy / proxy_config in the request body.
Contract: MCP tools parse_page, fetch_binary.
Endpoint: POST http://<host>:8000/mcp (stateless Streamable HTTP, JSON).
Metrics: GET http://<host>:9999/metrics (Prometheus via redup-servicekit).
Tool args:
parse_page — HTML pages → markdown JSON. Not for PDF/DOCX/ZIP/images
(is_binary=true → switch to fetch_binary).
fetch_binary — download binary files (pdf/docx/zip/images/…). Download only
(no OCR/unzip). Bytes only in JSON content_base64 (no shared disk path).
Downstream tools must accept those bytes via their own input contract.
Not a fallback when HTML parse_page fails.
Agent registration example: {"id":"web-parser","url":"http://…:8000/mcp"}
→ LLM names mcp__web-parser__parse_page / mcp__web-parser__fetch_binary.
Related MCP server: pulldown
config/config.yaml:
service:
console_log_level: INFO
host: "0.0.0.0"
port: 8000
path: /mcp
max_workers: 4
hpa_max_workers: 2
McpWebParser:
upstream_base_url: ""
upstream_token: ""
default_proxy: ""
request_timeout_seconds: 120
max_timeout_seconds: 300
max_markdown_chars: 100000
max_binary_bytes: 15728640
delay_before_return_html: 2.5
json_response: true
stateless_http: trueOverride via servicekit env substitution (section___key):
export McpWebParser___upstream_base_url=https://crawl4ai.example.com
export McpWebParser___default_proxy=http://user:pass@proxy.example:3128
export McpWebParser___upstream_token=
export service___port=8000Startup fails fast if upstream_base_url is empty.
docker run --rm -p 8000:8000 -p 9999:9999 \
-e McpWebParser___upstream_base_url=https://crawl4ai.example.com \
-e McpWebParser___default_proxy=http://proxy.example:3128 \
redup4ai/redup.mcp-web-parser:0.1.0-3.13-slimMCP URL: http://127.0.0.1:8000/mcp. Metrics: http://127.0.0.1:9999/metrics.
GitHub Release publishes {VERSION}-3.13-slim to Docker Hub (DOCKERHUB_USER /
DOCKERHUB_PASSWORD secrets).
Requires Python 3.13+ and a reachable Crawl4AI base URL:
export McpWebParser___upstream_base_url=https://crawl4ai.example.com
uv sync
uv run python -m redup_mcp_web_parser.service config/config.yamlDesktop MCP clients (stdio):
uv run redup-mcp-web-parser \
--transport stdio \
--upstream-base-url https://crawl4ai.example.comuv sync --dev
uv run pytest tests -q -m "not live"Optional live smoke (needs a real Crawl4AI):
export McpWebParser___upstream_base_url=https://crawl4ai.example.com
uv run pytest tests -m live -qMIT — see LICENSE and NOTICE.
Download a binary file and return metadata + base64 bytes as JSON.
WHEN TO USE: the URL itself is a file download — pdf, docx/xlsx/pptx, odt/epub, zip/tar/gz/7z/rar, png/jpeg/gif/webp, or similar. Also use when parse_page returned is_binary=true / hint to call fetch_binary. WHEN NOT TO USE: normal HTML pages (example.com, wiki, /abs, blogs) — use parse_page. Never switch to this tool only because parse_page failed on an HTML URL (anti-bot, timeout, empty markdown).
Download only: no text extraction, no OCR, no unzip. Bytes live only in
JSON content_base64 (last; may be large) — this server does not write
a shared filesystem path. Downstream tools that need the bytes must use
their own input contract. Prefer metadata fields kind, size, filename.
Returns JSON: success, url, status_code, media_type, kind, size, filename, truncated, error_message, used_proxy, content_base64.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | http(s) URL of a binary file to download (pdf, docx, zip, png, … — not an HTML page). | |
| timeout | No | Request timeout in seconds (server may clamp). |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses key behavioral aspects: it performs a download only (no text extraction, OCR, or unzip), places bytes in JSON 'content_base64' rather than writing to a filesystem, and notes potential large size. It complements the annotations (readOnly, idempotent, non-destructive) with concrete details, exceeding the minimum required transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with distinct sections (description, when to use, when not to use, download only, returns) and every sentence adds value. It avoids redundancy while covering necessary details, maintaining a concise yet complete structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Beyond the annotations and schema, the description includes the list of return fields, explains the tool's role relative to parse_page, and specifies that it does not interact with a filesystem. This fully contextualizes the tool within its environment, making it self-sufficient for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides clear descriptions for both parameters: 'url' is specified as an http(s) URL for a binary file (non-HTML), and 'timeout' is described with units and the clamping behavior. The tool description reinforces the URL semantics, ensuring full coverage of parameter meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: downloading a binary file and returning metadata and base64 bytes as JSON. It uses specific verbs ('Download', 'return') and specifies the resource type and output format, effectively distinguishing it from the sibling tool parse_page.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'WHEN TO USE' and 'WHEN NOT TO USE' sections provide explicit conditions, including types of content (pdfs, zips, images) and when to prefer parse_page. It also advises against using this tool as a fallback for parse_page failures, covering both use and non-use scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read an HTML web page and return cleaned markdown as JSON.
WHEN TO USE: HTML articles, docs, wiki, blog posts, /abs pages. WHEN NOT TO USE: PDF, DOCX, XLSX, ZIP, images, or other file downloads — call fetch_binary instead. If this tool returns is_binary=true, switch to fetch_binary (do not retry parse_page in a loop).
Returns JSON fields: success, url, status_code, markdown, error_message, links_internal, links_external, truncated, used_proxy, content_type, is_binary, binary_kind, hint.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | http(s) URL of an HTML page (not a PDF/ZIP/DOCX file). | |
| timeout | No | Request timeout in seconds (server may clamp). |
| Name | Required | Description |
|---|---|---|
| result | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds behavioral context beyond annotations. It discloses the is_binary detection mechanism and the instruction to switch tools without retry loops. It also mentions the timeout clamping behavior. These are beyond what readOnlyHint and openWorldHint provide, and nothing contradicts the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-organized with clear sections and a bullet-like list of return fields. The main purpose is front-loaded, and every sentence adds value—usage instructions, output fields, and binary handling—without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lists all return JSON fields, covers edge cases (binary detection and switching), mentions the timeout behavior, and provides clear usage guidance. With the output schema present, the description still adds valuable operational context, making it complete for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of the parameter descriptions (url and timeout). The tool description does not add extra parameter semantics beyond the schema, but it reinforces the URL type by mentioning 'HTML articles' and excluding binary files. Since schema coverage is high, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool reads an HTML web page and returns cleaned markdown as JSON. It distinguishes itself from the sibling tool fetch_binary by specifying when to use it (HTML pages) and when not (binary files). The verb 'read' plus the resource 'HTML web page' makes the purpose very specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit WHEN TO USE and WHEN NOT TO USE sections are provided, naming the alternative tool fetch_binary. It also gives a specific fallback instruction: if is_binary=true, switch to fetch_binary and do not retry parse_page in a loop. This completely covers when and when not to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
The two tools are cleanly separated: parse_page handles HTML pages and returns markdown, while fetch_binary handles file downloads and returns metadata plus bytes. They also include when-to-use and when-not-to-use guidance, plus a clear fallback path when parse_page detects binary content.
Both tools use the verb_noun pattern: parse_page and fetch_binary. Naming is regular, consistent, and immediately communicates what each tool does.
Two tools is lower than the typical 3-15 range, but it maps exactly to the server's intended scope: HTML parsing and binary fetching. The set is slightly thin but not excessive or insufficient for such a narrow purpose.
The tool set covers the main web resource categories: HTML pages are parsed to markdown, and binaries are downloaded with metadata and bytes. The is_binary fallback closes the biggest edge case, so there are no obvious dead ends.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
MCP server (stdio): fetch web pages as clean readable markdown via the AgentForge API
Document-to-Markdown MCP server — convert PDF, Office and HTML into LLM-ready Markdown.
Free remote MCP server for fetching public web pages through a rotating proxy pool.
Hosted MCP server: convert PDFs to clean, LLM-ready Markdown with tables, formulas and OCR.
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/redup-ai/redup.mcp-web-parser'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
