Skip to main content
Glama
Digest MCP Server

Digest MCP Server

by bakhtiyork

Digest MCP Server

MCP server for web content digestion using browserless.io via puppeteer-core. Extracts fully rendered DOM content from dynamic web pages including SPAs and infinite scroll sites.

Features

  • Connect to browserless.io cloud browsers

  • Load web pages with configurable wait times

  • Scroll down pages multiple times with delays

  • Extract complete page content (HTML)

Related MCP server: MCP Fetch Page

Installation

npm install
npm run build

Configuration

Set your browserless.io API key using one of these methods:

Create a .env file in the project root:

cp .env.example .env

Then edit .env and add your API key:

BROWSERLESS_API_KEY=your_api_key_here

Option 2: Using environment variable

export BROWSERLESS_API_KEY=your_api_key_here

Usage

Running the Server

The server uses stdio transport for MCP communication:

node build/index.js

Tool: web_content

Fetches web page content with optional scrolling and HTML cleanup.

Parameters:

  • url (string, required): The URL to fetch

  • initialWaitTime (number, optional): Time to wait in milliseconds after loading the page. Default: 3000

  • scrolls (number, optional): Number of times to scroll down the page. Default: 5

  • scrollWaitTime (number, optional): Time to wait in milliseconds between each scroll. Default: 1000

  • cleanup (boolean, optional): Whether to clean up HTML (remove scripts, styles, SVG, forms, etc.) and keep only meaningful text content. Default: false

Returns:

  • size (number): Size of the content in bytes

  • content (string): The fetched HTML content

Example:

{
  "url": "https://example.com",
  "initialWaitTime": 2000,
  "scrolls": 3,
  "scrollWaitTime": 1000,
  "cleanup": true
}

How It Works

  1. Connects to browserless.io using your API key via WebSocket

  2. Creates a new page in the remote browser

  3. Navigates to the specified URL (waits for DOM content loaded)

  4. Waits 1 second for page stabilization

  5. Waits for the initial wait time (default: 3 seconds)

  6. Scrolls to the bottom of the page the specified number of times

  7. After each scroll, intelligently waits for new content to load by:

    • Monitoring page height changes

    • Detecting dynamically loaded content

    • Waiting up to scrollWaitTime for new content (default: 3 seconds)

  8. Waits for network to idle (AJAX requests complete)

  9. Waits 1 additional second for JavaScript rendering

  10. Returns the fully RENDERED DOM (not raw HTML source)

    • Includes all JavaScript-generated content

    • Includes all AJAX-loaded content

    • Includes all dynamically inserted elements

    • Uses document.documentElement.outerHTML for complete rendered state

Dynamic Content & Infinite Scroll

The tool is specifically designed for modern web applications with dynamic content:

AJAX/JavaScript Handling:

  • ✅ Waits for network idle: Ensures all AJAX requests complete

  • ✅ Returns rendered DOM: Gets actual content after JavaScript execution

  • ✅ Not raw HTML source: Uses browser's rendered output

  • ✅ Includes dynamic elements: Captures content inserted by React, Vue, Angular, etc.

Infinite Scroll Support:

  • ✅ Scrolls to bottom: Triggers lazy-loading mechanisms

  • ✅ Detects new content: Monitors page height changes

  • ✅ Smart waiting: Exits early when content loads

  • ✅ Multiple fallbacks: Keyboard scroll if JavaScript fails

Perfect for:

  • Single Page Applications (React, Vue, Angular)

  • Infinite scroll feeds (Twitter, Facebook, LinkedIn)

  • Lazy-loaded images and content

  • AJAX-powered content (search results, filters)

  • Dynamic dashboards and admin panels

Tips for best results:

  • Default scrolls: 5 works well for most pages with lazy-loaded content

  • Increase scrolls to 10-15 for very long infinite scroll pages

  • Set scrolls: 0 to disable scrolling for static pages

  • Use scrollWaitTime of 1000-3000ms for slow-loading content (default: 1000ms)

  • Increase initialWaitTime to 5000+ if page has heavy initialization

  • For SPAs, allow time for initial JavaScript bootstrap

  • Use cleanup: true to extract only meaningful text content without scripts, styles, and visual elements

  • Use cleanup: false (default) to get the full rendered HTML

MCP Client Configuration

Add to your MCP client configuration (e.g., Claude Desktop):

{
  "mcpServers": {
    "digest": {
      "command": "node",
      "args": ["/path/to/digest-mcp/build/index.js"],
      "env": {
        "BROWSERLESS_API_KEY": "your_api_key_here"
      }
    }
  }
}

License

ISC

Available Tools

1 tool
web_contentFetch Web ContentA

Fetch fully rendered DOM content using browserless.io. Handles AJAX/JavaScript dynamic loading. Optimized for SPAs and infinite scroll pages. Returns the complete rendered HTML after all JavaScript execution, including dynamically loaded content. Each scroll waits for page height changes and network activity to settle.

Inputs: url (required), initialWaitTime, scrolls, scrollWaitTime, +1 more. Returns: size, content

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.9/5.0

    Scored across 1 tool

    Disambiguation5/5

    With only one tool, there is no possibility of confusion or overlap between tools. The single tool 'web_content' has a clearly defined and distinct purpose of fetching fully rendered DOM content from web pages.

    Naming Consistency5/5

    Since there is only one tool, naming consistency is inherently perfect. The tool name 'web_content' follows a clear and descriptive pattern, though no pattern can be established across multiple tools.

    Tool Count2/5

    A single tool is too few for a server named 'Digest MCP Server', which suggests a broader scope of digesting or processing content. While the tool is well-described, the server's purpose likely requires more operations (e.g., parsing, summarizing, or managing multiple sources) to be effectively useful.

    Completeness2/5

    The tool surface is severely incomplete for a digest server. It only provides fetching of web content, with no tools for processing, analyzing, summarizing, or managing the fetched data. This creates significant gaps that will hinder agents from performing comprehensive digest-related tasks.

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables web scraping and document processing with JavaScript execution, anti-detection measures, batch processing, and structured data extraction. Supports multiple formats including markdown, HTML, screenshots, and handles PDFs with OCR capabilities.
      44 PyPI
      4
      MIT
    • A
      license
      A
      quality
      D
      maintenance
      Enables browser-based web page fetching with automatic cookie management and CSS selector extraction, supporting JavaScript rendering and authenticated sessions through a Chrome extension.
      1
      2
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      Fetches web pages with JavaScript rendering, pierces Shadow DOM, and enables interactive actions like clicking and form filling using a real Chrome browser.
      2
      MIT