Skip to main content
Glama
coolaigit

site-crawler-mcp

by coolaigit
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

Related Servers

Alternatives to site-crawler-mcp

No user-submitted related servers found.

    Related Servers

    • read-website-fast

      ashleynikolaidis52
      A
      license
      Not graded
      quality
      A
      maintenance
      Fetches webpages and crawls sites locally, stripping noise and converting content into clean, link-preserving Markdown with a minimal token footprint. Supports configurable crawl depth, concurrency, robots.txt compliance, and caching for efficient AI agent and LLM pipelines.
      MIT

    • mtscraper

      mazprotech
      A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to crawl websites, follow internal links, render JavaScript pages, and extract clean main text and linked PDF tables into structured records. It exposes a crawl tool over stdio that returns records inline and writes a complete JSONL file.
      MIT

    • read-website-fast

      gokhanalvarez
      A
      license
      Not graded
      quality
      A
      maintenance
      Enables AI agents and MCP clients to fetch webpages locally, strip noise, preserve links, and convert content into clean, token-efficient Markdown, with optional depth crawling, caching, and robots.txt-aware rate limiting.
      MIT

    • MCP Web Research Agent

      SnotacusNexus
      A
      license
      Not graded
      quality
      D
      maintenance
      Enables automated web research and intelligence gathering through recursive web crawling, multi-engine search integration, and persistent SQLite storage with support for keyword filtering and multiple export formats.
      MIT

    • read-website-fast

      fcoigvozst
      F
      license
      Not graded
      quality
      A
      maintenance
      Fetches webpages locally, extracts readable content, and converts it to clean Markdown with preserved links, optional multi-page crawling, caching, and robots.txt support for token-efficient use by AI agents and LLM pipelines.
      -
    Deploy Server
    Let your AI agent see the webWeb-to-screenshot MCP for Claude, Cursor and Codex. No cookie banners. Free to start.snap-render.comAd

    TDQS

    A3.5/5.0

    Scored across 6 tools

    Disambiguation5/5

    Each tool has a clearly distinct purpose: full site crawling, single page scraping, summarization, link clicking, file downloading, and querying. No overlap or ambiguity.

    Naming Consistency5/5

    All tools follow a consistent verb_noun pattern in snake_case (e.g., crawl_site, scrape_page, query_crawls), making the tool set predictable and easy to navigate.

    Tool Count5/5

    With 6 tools, the server is well-scoped. It covers the essential operations for a site crawler without unnecessary bloat or missing functionality.

    Completeness4/5

    The tool set covers core workflows: crawling, scraping, summarizing, interacting, downloading, and querying. Minor gaps like explicit crawl session management or update/delete operations exist, but the surface is largely complete for typical use cases.

    Maintenance

    ActivityStale
    ResponsivenessNo issues

    Resources