A
licenseNot graded
qualityD
maintenanceEnables AI agents to perform instant SEO audits, check robots.txt, sitemaps, and AI crawler access for any URL without API keys.
MIT
Web Crawler MCP Server
Related Servers
Alternatives to Web Crawler MCP Server
No user-submitted related servers found.
Related Servers
- AlicenseAqualityAmaintenanceChecks a website's AI readiness: can ChatGPT, Claude, Perplexity, Google AI and 50+ other AI crawlers (GPTBot, ClaudeBot, PerplexityBot) access it? Scores the site, then generates llms.txt, an AI-crawler robots.txt and JSON-LD, and reads LLM Monitor trends, from Claude Desktop, Cursor and other MCP clients.8206 npmMIT
- AlicenseNot gradedqualityBmaintenanceEnables AI assistants to run SEO audits and use site tools for meta tags, robots.txt, schema validation, SERP previews, Core Web Vitals, Open Graph, hreflang, Search Console, GA4, keywords, and backlinks via the Model Context Protocol.MIT
- AlicenseAqualityCmaintenanceIntegrates SEO analysis and Google Search Console data directly into Claude Code and Cursor. Performs real-time site audits, detects technical SEO issues, validates meta tags, generates structured data, and provides AI-powered recommendations for both production sites and local development servers.191 npm3MIT
- AlicenseNot gradedqualityAmaintenanceEnables AI agents to crawl live websites, audit AEO readiness, generate Schema.org @graph JSON-LD, llms.txt, ai.txt, and robots.txt, inject structured data into HTML, validate optimizations, and retrieve framework-specific code snippets.MIT
- AlicenseAqualityAmaintenanceEnables AI assistants to run evidence-first technical SEO and GEO audits, inspect HTML, structured data, robots policies and sitemaps, and compare before/after reports to verify fixes.51MIT
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo issues