Skip to main content
Glama

Server Details

Paste access-log lines: every crawler identified, impersonators named, robots.txt and WAF rules.

If you are the author of this connector, you can claim ownership with GitHub, an HTTP challenge, or a DNS record. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
Status
Not tested
Last Tested
Never
Transport
Streamable HTTP
URL

Tool Definition Quality

Score is being calculated. Check back soon.

Frequently Asked Questions

Discussions

No comments yet. Be the first to start the discussion!

Related MCP Servers

  • A
    license
    A
    quality
    C
    maintenance
    Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.
    5
    1
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    Audits AI-bot visibility: robots.txt per-bot for 22 AI user-agents (GPTBot/ClaudeBot/PerplexityBot/etc), Cloudflare flags, JSON-LD, sitemap, llms.txt, SPA shell, plus cross-model brand mentions via Perplexity + OpenRouter. 0-100 score. SSRF-guarded, spend-capped.
    4
    1
    MIT
  • F
    license
    A
    quality
    D
    maintenance
    Simulates Googlebot crawling to verify bot detection and analytics tracking by capturing events and providing detailed reports.
    1
  • A
    license
    Not graded
    quality
    C
    maintenance
    Open-source technical SEO crawler MCP server built on LibreCrawl. Runs full audits inside Claude, Cursor, or Codex — 50+ checks (hreflang, schema.org, security headers, WAF detection on 200-OK pages), chunked-progressive engine for large sites, ephemeral by design (server forgets every audit after download).
    38
    MIT

View all MCP Servers

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources