Skip to main content
Glama
RichardDillman

Googlebot Simulator MCP

Googlebot Simulator MCP

An MCP server that simulates Googlebot crawling pages and captures analytics events. Use this to verify your site's bot detection and analytics tracking work correctly.

Features

  • Authentic Googlebot User-Agents: Uses real Googlebot desktop and mobile user-agent strings

  • Idle Detection Simulation: Mimics Googlebot's requestIdleCallback-based idle detection

  • Event Interception: Captures analytics/tracking requests matching your pattern

  • Detailed Reporting: Returns timing info, captured payloads, and screenshots

Related MCP server: gsc-mcp

Installation

Add to Claude Code

claude mcp add googlebot-sim -- npx googlebot-simulator-mcp

Or add to .claude.json

{
  "mcpServers": {
    "googlebot-sim": {
      "type": "stdio",
      "command": "npx",
      "args": ["googlebot-simulator-mcp"]
    }
  }
}

From source

git clone https://github.com/RichardDillman/googlebot-simulator-mcp.git
cd googlebot-simulator-mcp
npm install
npm run build

Usage

Once added to Claude, you can use natural language:

"Test https://example.com/jobs as Googlebot, watching for requests containing 'page_viewed'"

Or be more specific:

"Simulate mobile Googlebot on https://talent.com/jobs, capture events to **/serp-event-producer/send"

Tool: simulate_googlebot

Input

{
  // Required: URL(s) to test
  urls: string | string[];

  // Required: Pattern to match analytics events
  eventPattern: {
    urlPattern?: string;    // e.g., "**/analytics/**"
    bodyContains?: string;  // e.g., "page_viewed"
  };

  // Optional configuration
  options?: {
    userAgent?: "desktop" | "mobile";  // Default: "desktop"
    idleTimeout?: number;              // Default: 5000ms
    maxWaitTime?: number;              // Default: 10000ms
    captureScreenshot?: boolean;       // Default: true
    screenshotDir?: string;            // Default: cwd
  };
}

Output

{
  results: Array<{
    url: string;
    success: boolean;              // true if matching event was captured
    eventsFired: Array<{
      timestamp: number;
      url: string;
      method: string;
      payload: object;
      matchedPattern: boolean;
      responseStatus?: number;
    }>;
    timing: {
      navigationStart: number;
      domContentLoaded: number;
      idleDetected: number;
      totalTime: number;
      firstIdleCallback?: number;
    };
    screenshot?: string;           // Path to screenshot file
    errors: string[];
    idleCallbacksObserved: number;
  }>;
  summary: {
    totalUrls: number;
    passed: number;
    failed: number;
    totalEventsCaptured: number;
  };
}

Example Output

{
  "results": [
    {
      "url": "https://talent.com/jobs",
      "success": true,
      "eventsFired": [
        {
          "timestamp": 1704825600000,
          "url": "https://events.talent.com/serp-event-producer/send",
          "method": "POST",
          "payload": {
            "event_type": "page_viewed_bot",
            "bot_type": "google",
            "page_name": "serp"
          },
          "matchedPattern": true,
          "responseStatus": 200
        }
      ],
      "timing": {
        "navigationStart": 1704825595000,
        "domContentLoaded": 1704825596500,
        "idleDetected": 1704825600000,
        "totalTime": 5000,
        "firstIdleCallback": 3200
      },
      "screenshot": "/tmp/googlebot_desktop_jobs_1704825600000.png",
      "errors": [],
      "idleCallbacksObserved": 3
    }
  ],
  "summary": {
    "totalUrls": 1,
    "passed": 1,
    "failed": 0,
    "totalEventsCaptured": 1
  }
}

How It Works

  1. Launches headless Chromium with Googlebot user-agent and viewport settings

  2. Injects tracking script to monitor requestIdleCallback usage

  3. Sets up network interception to capture matching requests

  4. Navigates to URL and waits for page to reach idle state

  5. Captures screenshot at the "idle" moment (what Googlebot sees)

  6. Returns detailed report with all captured events and timing

Use Cases

  • Verify bot detection: Ensure your analytics correctly identifies Googlebot

  • Test event timing: Confirm events fire before Googlebot's idle timeout

  • Debug rendering: See what content is visible when Googlebot considers the page "done"

  • Compare desktop vs mobile: Test both Googlebot variants

License

MIT

Install Server
F
license - not found
A
quality
D
maintenance

Maintenance

Maintainers
Response time
Release cycle
Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Servers

  • A
    license
    A
    quality
    A
    maintenance
    MCP server for Google Search Console, URL Inspection & Indexing API — search analytics, sitemap management, and batch indexing
    13
    1,069
    6
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.
    5
    MIT
  • A
    license
    A
    quality
    A
    maintenance
    AI-powered exploratory QA agent. Explores web apps like a real user — 18 MCP tools for clicking, filling forms, and navigating. Automatically verifies that actions persist (fake deletes, failed edits). Runs 16 detection types including dead links, SEO, accessibility, and performance checks.
    29
    2
    MIT

View all related MCP servers

Related MCP Connectors

  • AI QA tester — real browsers scan sites for bugs, SEO, perf, and accessibility issues via chat.

  • Website performance monitoring: scans, Core Web Vitals, RUM data and alerts.

  • Technical SEO + GEO (AI-search) site audits: hosted crawls, prioritized fixes, report diffs.

View all MCP Connectors

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/RichardDillman/googlebot-simulator-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server