Digest MCP Server
Uses puppeteer-core to connect to browserless.io cloud browsers for extracting fully rendered DOM content from dynamic web pages, including SPAs and infinite scroll sites, with configurable wait times and scrolling behavior.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Digest MCP Server get the full article from https://news.example.com/long-read with cleanup"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Digest MCP Server
MCP server for web content digestion using browserless.io via puppeteer-core. Extracts fully rendered DOM content from dynamic web pages including SPAs and infinite scroll sites.
Features
Connect to browserless.io cloud browsers
Load web pages with configurable wait times
Scroll down pages multiple times with delays
Extract complete page content (HTML)
Related MCP server: MCP Fetch Page
Installation
npm install
npm run buildConfiguration
Set your browserless.io API key using one of these methods:
Option 1: Using .env file (recommended)
Create a .env file in the project root:
cp .env.example .envThen edit .env and add your API key:
BROWSERLESS_API_KEY=your_api_key_hereOption 2: Using environment variable
export BROWSERLESS_API_KEY=your_api_key_hereUsage
Running the Server
The server uses stdio transport for MCP communication:
node build/index.jsTool: web_content
Fetches web page content with optional scrolling and HTML cleanup.
Parameters:
url(string, required): The URL to fetchinitialWaitTime(number, optional): Time to wait in milliseconds after loading the page. Default: 3000scrolls(number, optional): Number of times to scroll down the page. Default: 5scrollWaitTime(number, optional): Time to wait in milliseconds between each scroll. Default: 1000cleanup(boolean, optional): Whether to clean up HTML (remove scripts, styles, SVG, forms, etc.) and keep only meaningful text content. Default: false
Returns:
size(number): Size of the content in bytescontent(string): The fetched HTML content
Example:
{
"url": "https://example.com",
"initialWaitTime": 2000,
"scrolls": 3,
"scrollWaitTime": 1000,
"cleanup": true
}How It Works
Connects to browserless.io using your API key via WebSocket
Creates a new page in the remote browser
Navigates to the specified URL (waits for DOM content loaded)
Waits 1 second for page stabilization
Waits for the initial wait time (default: 3 seconds)
Scrolls to the bottom of the page the specified number of times
After each scroll, intelligently waits for new content to load by:
Monitoring page height changes
Detecting dynamically loaded content
Waiting up to scrollWaitTime for new content (default: 3 seconds)
Waits for network to idle (AJAX requests complete)
Waits 1 additional second for JavaScript rendering
Returns the fully RENDERED DOM (not raw HTML source)
Includes all JavaScript-generated content
Includes all AJAX-loaded content
Includes all dynamically inserted elements
Uses
document.documentElement.outerHTMLfor complete rendered state
Dynamic Content & Infinite Scroll
The tool is specifically designed for modern web applications with dynamic content:
AJAX/JavaScript Handling:
✅ Waits for network idle: Ensures all AJAX requests complete
✅ Returns rendered DOM: Gets actual content after JavaScript execution
✅ Not raw HTML source: Uses browser's rendered output
✅ Includes dynamic elements: Captures content inserted by React, Vue, Angular, etc.
Infinite Scroll Support:
✅ Scrolls to bottom: Triggers lazy-loading mechanisms
✅ Detects new content: Monitors page height changes
✅ Smart waiting: Exits early when content loads
✅ Multiple fallbacks: Keyboard scroll if JavaScript fails
Perfect for:
Single Page Applications (React, Vue, Angular)
Infinite scroll feeds (Twitter, Facebook, LinkedIn)
Lazy-loaded images and content
AJAX-powered content (search results, filters)
Dynamic dashboards and admin panels
Tips for best results:
Default
scrolls: 5works well for most pages with lazy-loaded contentIncrease
scrollsto 10-15 for very long infinite scroll pagesSet
scrolls: 0to disable scrolling for static pagesUse
scrollWaitTimeof 1000-3000ms for slow-loading content (default: 1000ms)Increase
initialWaitTimeto 5000+ if page has heavy initializationFor SPAs, allow time for initial JavaScript bootstrap
Use
cleanup: trueto extract only meaningful text content without scripts, styles, and visual elementsUse
cleanup: false(default) to get the full rendered HTML
MCP Client Configuration
Add to your MCP client configuration (e.g., Claude Desktop):
{
"mcpServers": {
"digest": {
"command": "node",
"args": ["/path/to/digest-mcp/build/index.js"],
"env": {
"BROWSERLESS_API_KEY": "your_api_key_here"
}
}
}
}License
ISC
Available Tools
1 toolweb_contentFetch Web ContentA
Fetch fully rendered DOM content using browserless.io. Handles AJAX/JavaScript dynamic loading. Optimized for SPAs and infinite scroll pages. Returns the complete rendered HTML after all JavaScript execution, including dynamically loaded content. Each scroll waits for page height changes and network activity to settle.
Inputs: url (required), initialWaitTime, scrolls, scrollWaitTime, +1 more. Returns: size, content
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 1 tool
With only one tool, there is no possibility of confusion or overlap between tools. The single tool 'web_content' has a clearly defined and distinct purpose of fetching fully rendered DOM content from web pages.
Since there is only one tool, naming consistency is inherently perfect. The tool name 'web_content' follows a clear and descriptive pattern, though no pattern can be established across multiple tools.
A single tool is too few for a server named 'Digest MCP Server', which suggests a broader scope of digesting or processing content. While the tool is well-described, the server's purpose likely requires more operations (e.g., parsing, summarizing, or managing multiple sources) to be effectively useful.
The tool surface is severely incomplete for a digest server. It only provides fetching of web content, with no tools for processing, analyzing, summarizing, or managing the fetched data. This creates significant gaps that will hinder agents from performing comprehensive digest-related tasks.
Related MCP Connectors
Fetch, crawl, and browse protected pages with anti-bot handling - renders in a real browser and
Fetch and extract data from any public web page, even JS-rendered or anti-bot protected
Crawl, scrape, search the web, and automate browsers at scale with anti-bot bypass.
Fetch any public web page through managed proxies, with optional JS rendering and extraction rules.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables web scraping and document processing with JavaScript execution, anti-detection measures, batch processing, and structured data extraction. Supports multiple formats including markdown, HTML, screenshots, and handles PDFs with OCR capabilities.44 PyPI4MIT
- AlicenseAqualityDmaintenanceEnables browser-based web page fetching with automatic cookie management and CSS selector extraction, supporting JavaScript rendering and authenticated sessions through a Chrome extension.12MIT
- AlicenseNot gradedqualityDmaintenanceFetches web pages with JavaScript rendering, pierces Shadow DOM, and enables interactive actions like clicking and form filling using a real Chrome browser.2MIT
- AlicenseNot gradedqualityFmaintenanceEnables intelligent web scraping with support for static pages, JavaScript-rendered SPAs, and natural language data extraction.MIT