Skip to main content
Glama

WebDataTools Web content for AI & RAG MCP server

webdatatools-rag-mcp

An MCP server with 12 web content for ai & rag tools for AI agents — Claude Desktop, Cursor, Cline or any MCP client. Web search with page text, website-to-Markdown crawling, clean article extraction, JSON-LD/structured data, Google News and press-release monitoring — ready for LLM context and RAG pipelines.

This server uses your own Apify API token. Every tool call runs a WebDataTools Actor under your Apify account and is billed to your Apify credit — pay per result, the price is in each tool description. Your token is only sent to Apify's API.

Quick start

Requires Node.js 18+.

APIFY_TOKEN=apify_api_... npx -y github:paulet4a-commits/webdatatools-rag-mcp

Get a free token (the free plan includes monthly credit): https://console.apify.com/settings/integrations

Related MCP server: Firecrawl MCP Server

Claude Desktop / Cursor

Add this to claude_desktop_config.json (Claude Desktop) or .cursor/mcp.json (Cursor):

{
  "mcpServers": {
    "webdatatools-rag": {
      "command": "npx",
      "args": [
        "-y",
        "github:paulet4a-commits/webdatatools-rag-mcp"
      ],
      "env": {
        "APIFY_TOKEN": "apify_api_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
      }
    }
  }
}

Tools (12)

Tool

What it does

Price (free plan)

Backing Actor

ai_web_search

AI Web Search & Read: Google results as clean Markdown

$0.005 / result

Actor

website_to_markdown

Website to Markdown — Content Crawler for LLM & RAG

$0.001 / result

Actor

llms_txt_generator

llms.txt Generator (Website to llms.txt & llms-full.txt)

$0.001 / page

Actor

pdf_to_markdown

PDF to Markdown Converter (Text, Headings, Metadata)

$0.002 / pdf

Actor

rag_text_chunker

RAG Text Chunker (Markdown to Chunks, Token Counts)

$0.0002 / chunk

Actor

article_extractor

Article & News Extractor (clean text, author, date, markdown)

$0.002 / result

Actor

structured_data_extractor

Structured Data & JSON-LD Extractor (Schema.org, Open Graph)

$0.002 / result

Actor

google_news_scraper

Google News Scraper (Keyword, Topic, Site, Date Range)

$0.0005 / article

Actor

press_release_monitor

Press Release Monitor: PR Newswire, BusinessWire, GlobeNewswire

$0.001 / result

Actor

medium_scraper

Medium Scraper (Articles, Claps, Tags, Reading Time)

$0.004 / article

Actor

google_ai_overviews_scraper

Google AI Overviews Scraper (AI Overview Answers and Sources)

$0.0125 / AI Overview

Actor

rss_feed_reader

RSS Feed Reader (RSS and Atom Items)

$0.0005 / item

Actor

More WebDataTools MCP servers

License

MIT

Available Tools

6 tools
article_extractorA

Article & News Extractor returns clean article text, title, author(s), publish/modified date, tags and images from any news or blog URL — one row per URL, as Markdown, plain text or HTML. Billed to your own Apify account: ~$0.002 per result (Apify free-plan price, lower on paid plans).

Inputs: urls (required), outputFormat, includeImages.

google_news_scraperA

Google News Scraper returns headlines from Google News RSS search and topic feeds for any keyword, site: or when: query, and resolves each article's real publisher URL — one row per article. Billed to your own Apify account: ~$0.0005 per article (Apify free-plan price, lower on paid plans).

Inputs: queries (required), topics, country, language, +3 more.

press_release_monitorA

Press Release Monitor returns keyword-matched press releases from PR Newswire, Business Wire and GlobeNewswire RSS feeds — company, publish date, category and summary, one row per release. Billed to your own Apify account: ~$0.001 per result (Apify free-plan price, lower on paid plans).

Inputs: sources, keywords, sinceHours, includeBody, +1 more.

structured_data_extractorA

Structured Data & JSON-LD Extractor reads every Schema.org JSON-LD block and Open Graph tag on a page and returns one clean row per URL — product, price, rating, article, job posting, event, recipe, FAQ and breadcrumb data, ready for RAG pipelines and SEO rich-result audits. Billed to your own Apify account: ~$0.002 per result (Apify free-plan price, lower on paid plans).

Inputs: urls (required), followRedirects, includeOpenGraph, includeRawJsonLd.

website_to_markdownB

Content Crawler turns any site into clean Markdown per page for LLMs, RAG pipelines and vector DBs — no headless browser, $1 per 1,000 pages. Billed to your own Apify account: ~$0.001 per result (Apify free-plan price, lower on paid plans).

Inputs: startUrls (required), maxDepth, maxPages, useSitemap, +6 more.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.8/5.0

    Scored across 6 tools

    Disambiguation4/5

    Most tools are clearly differentiated by source or output type: general web search, site-to-Markdown crawl, article extraction, structured data, Google News, and press releases. The main overlap is between website_to_markdown and article_extractor when processing a single news/blog article, but their broader purposes are still distinguishable.

    Naming Consistency4/5

    All names use snake_case and are descriptive, which makes them readable and predictable. The patterns vary slightly (action-oriented names like ai_web_search vs. noun-extractor names like article_extractor), but the set remains consistent overall.

    Tool Count5/5

    Six tools is a well-scoped size for a web-data RAG server. Each tool has a distinct source or extraction focus, so the surface does not feel bloated or thin.

    Completeness4/5

    The set covers core web-data acquisition workflows: search, crawling, article extraction, structured data, news, and press releases. Minor gaps such as PDF extraction, social media sources, or advanced crawl controls exist, but the core RAG pipeline needs are well represented.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      B
      maintenance
      Enables MCP-compatible AI agents to search, scrape, crawl, map, and interact with live web pages, returning clean, structured, agent-ready content. It also supports recurring page monitoring, autonomous deep research, and credit-usage checks.
      26
      181,410 npm
      1
      MIT
    • F
      license
      Not graded
      quality
      A
      maintenance
      Enables AI agents to run web searches across configured providers and crawl individual pages or whole sites into clean Markdown or JSON, including authenticated crawls and local or HTTP MCP deployment.
      -
    • F
      license
      Not graded
      quality
      A
      maintenance
      Enables AI agents to search the web through configured providers and crawl single pages or whole sites into clean Markdown or JSON, including content behind logins, over MCP STDIO or HTTP.
      -