Skip to main content
Glama
nikolaymokh-dev

common-crawl-mcp

by nikolaymokh-dev
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Remote

Related Servers

Alternatives to common-crawl-mcp

No user-submitted related servers found.

    Related Servers

    • mcp-common-crawl

      arturseo-geo
      F
      license
      A
      quality
      D
      maintenance
      MCP server for Common Crawl CDX that enables backlink discovery, expired domain finding, and competitor gap analysis without requiring API keys.
      4
      -

    • MCP Wayback Machine Server

      Mearman
      A
      license
      A
      quality
      B
      maintenance
      MCP server and CLI tool for interacting with the Internet Archive's Wayback Machine, supporting full CDX search, snapshot retrieval, screenshot listing, snapshot comparison, and optional authentication.
      8
      694 npm
      52
      Creative Commons Attribution Non Commercial Share Alike 4.0 International

    • mcp-server-wayback

      lakshyamehta03
      A
      license
      A
      quality
      C
      maintenance
      MCP server for the Internet Archive's Wayback Machine. Search archived snapshots, extract page text from a specific date, track how a site has changed over time, check if broken links are recoverable, and perform research across Internet Archive collections.
      6
      3
      MIT

    • mcp-server-webcrawl

      pragmar
      F
      license
      Not graded
      quality
      D
      maintenance
      Bridge the gap between your web crawl and AI language models. With mcp-server-webcrawl, your AI client filters and analyzes web content under your direction or autonomously, extracting insights from your web content. Supports WARC, wget, InterroBot, Katana, and SiteOne crawlers.
      46
      Python
      -
    • seosiri-search-governance-mcp by SEOSiri-Official

      seosiri-search-governance-mcpofficial

      SEOSiri-Official
      A
      license
      Not graded
      quality
      A
      maintenance
      MCP server for AI search crawler governance, brand safety, and search infrastructure auditing. Provides tools to audit robots.txt, canonical links, sitemaps, redirects, and send IndexNow notifications.
      MIT

    • crawlgraph-mcp

      pucilpet
      A
      license
      A
      quality
      B
      maintenance
      MCP server for the CrawlGraph backlink-intelligence API. Gives any MCP client - Claude Desktop, Claude Code, Cursor, Cline, Zed, Windsurf - backlink lookups and competitor gap analysis built on the public Common Crawl webgraph (4.4B edges, 120M domains).
      4
      9 npm
      5
      MIT
    Deploy Server
    One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

    TDQS

    A3.7/5.0

    Scored across 13 tools

    Disambiguation5/5

    Each tool targets a distinct piece of Common Crawl data: crawls, presence, dynamics, comparisons, locales, sections, URLs, duplicates, errors, robots, discovery, suffix enumeration, and text fetching. There is no overlap; even related tools like cc_presence and cc_dynamics are clearly differentiated by scope (single crawl vs. trend).

    Naming Consistency4/5

    All tool names share the cc_ prefix, providing a strong family resemblance. The second part mixes nouns (cc_crawls, cc_urls) and verbs (cc_compare, cc_fetch_text), but the pattern is predictable and readable. Minor inconsistency: some names are plural nouns, some are singular abstract concepts, but the prefix makes the set coherent.

    Tool Count5/5

    With 13 tools, the set is well-scoped for a Common Crawl analysis server. Each tool addresses a specific analytical need without redundancy or bloat. The count fits comfortably in the ideal 3-15 range.

    Completeness5/5

    The toolset covers the full lifecycle of Common Crawl exploration: discovering crawls, checking domain presence, tracking dynamics, comparing domains, drilling into URLs/content, diagnosing errors/robots, and identifying duplicates. It also supports domain discovery and suffix analysis, leaving no obvious dead ends for typical use cases.

    Maintenance

    ActivitySlowing
    ResponsivenessNo issues

    Resources