Related Servers

No user-submitted related servers found.

Related Servers

• crawlerpeek

  crawlerpeek

  A

  license

  A

  quality

  B

  maintenance

  Exposes three read-only MCP tools that let an assistant check whether specified AI crawlers are allowed by a site's robots.txt and how a URL responds when fetched with each bot's user-agent, fetch a page as a single named bot, and generate a robots.txt that welcomes AI crawlers.

  Updated 10 days ago (2026-10-29 21:19 UTC)

  6

  237 npm

  3

  MIT

• mcp-geo

  josezuma

  A

  license

  -Not graded

  quality

  D

  maintenance

  MCP server exposing Generative Engine Optimization tools to any AI agent, enabling checking of llms.txt, auditing robots.txt for AI crawlers, and validating JSON-LD schema.

  Updated 4 months ago (2026-07-11 14:28 UTC)

  MIT

• robotstxt-ai-mcp

  sharozdawa

  A

  license

  A

  quality

  D

  maintenance

  Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.

  Updated 8 months ago (2026-03-22 09:30 UTC)

  5

  1

  MIT
• 

  llmstxtgenerator-mcpofficial

  llms-txt-generator

  A

  license

  -Not graded

  quality

  C

  maintenance

  Enables AI agents to create, validate, and audit llms.txt files for websites, including checking against Google Lighthouse's Agentic Browsing requirements, via Streamable HTTP with no API key needed.

  Updated 2 months ago (2026-08-26 03:28 UTC)

  8 npm

  2

  MIT
• 

  seosiri-search-governance-mcpofficial

  SEOSiri-Official

  A

  license

  -Not graded

  quality

  B

  maintenance

  MCP server for AI search crawler governance, brand safety, and search infrastructure auditing. Provides tools to audit robots.txt, canonical links, sitemaps, redirects, and send IndexNow notifications.

  Updated a month ago (2026-10-02 03:44 UTC)

  MIT

• Crawl Readiness MCP Server

  Grubbomatic

  A

  license

  A

  quality

  B

  maintenance

  Checks what GPTBot, ClaudeBot, PerplexityBot and 50+ other AI crawlers can read on a site and what its access policy allows, checks whether a page permits search, answers, training or agent use (with a signed receipt), generates llms.txt, robots.txt and JSON-LD, and reads your Answer Analysis. Nine tools for Claude Desktop, Cursor and other MCP clients.

  Updated 10 days ago (2026-10-29 22:07 UTC)

  9

  460 npm

  MIT