Skip to main content
Glama
Sign In
Web Crawler MCP Server

Web Crawler MCP Server

by Insightful-Pipe
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Remote

Related Servers

Alternatives to Web Crawler MCP Server

No user-submitted related servers found.

    Related Servers

    • seo-audit-mcp

      cyber-kani
      A
      license
      Not graded
      quality
      D
      maintenance
      Enables AI agents to perform instant SEO audits, check robots.txt, sitemaps, and AI crawler access for any URL without API keys.
      MIT

    • Crawl Readiness MCP Server

      Grubbomatic
      A
      license
      A
      quality
      B
      maintenance
      Checks what GPTBot, ClaudeBot, PerplexityBot and 50+ other AI crawlers can read on a site and what its access policy allows, checks whether a page permits search, answers, training or agent use (with a signed receipt), generates llms.txt, robots.txt and JSON-LD, and reads your Answer Analysis. Nine tools for Claude Desktop, Cursor and other MCP clients.
      9
      460 npm
      MIT

    • SEO Audit MCP

      Nexariza-AI
      A
      license
      Not graded
      quality
      C
      maintenance
      Enables AI assistants to run read-only technical SEO and AI-search-readiness audits on any URL via GET/HEAD requests. Checks on-page elements like titles, meta descriptions, headings, canonical and Open Graph tags, JSON-LD validity, alt text and link health, plus robots.txt, sitemaps and llms.txt for crawler access.
      12 npm
      MIT

    • @didyouseo/mcp

      Sathish27
      A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI assistants to run SEO audits and use site tools for meta tags, robots.txt, schema validation, SERP previews, Core Web Vitals, Open Graph, hreflang, Search Console, GA4, keywords, and backlinks via the Model Context Protocol.
      69 npm
      MIT

    • Rampify MCP Server

      rampify-dev
      A
      license
      A
      quality
      B
      maintenance
      Integrates SEO analysis and Google Search Console data directly into Claude Code and Cursor. Performs real-time site audits, detects technical SEO issues, validates meta tags, generates structured data, and provides AI-powered recommendations for both production sites and local development servers.
      19
      17 npm
      3
      MIT

    • aeo-graph-engine

      1nc0gn30
      A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to crawl live websites, audit AEO readiness, generate Schema.org @graph JSON-LD, llms.txt, ai.txt, and robots.txt, inject structured data into HTML, validate optimizations, and retrieve framework-specific code snippets.
      MIT

    This server cannot be deployed

    One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

    Maintenance

    ActivitySlowing
    ResponsivenessOutdated data

    Resources