"Information about crawer or crawler" matching MCP servers:

An MCP (Model Context Protocol) server that gives AI agents live, structured ad intelligence across Facebook, Google, and Instagram — data that no base model can produce from training alone. Powered by Apify actors. Works with any MCP-compatible client: Cursor, Claude, etc.

An MCP server for crawling websites and extracting comprehensive data including images, SEO metadata, security headers, and business intelligence. It features twelve distinct extraction modes to perform detailed audits for e-commerce sites and general web analysis.

An MCP server that detects affiliate links and provides information about Tracklix, enabling AI assistants to answer affiliate-link-related questions without any network calls or API keys.

Provides four free tools to assess AI visibility: crawler access, entity recognition, off-page gaps, and shopping agent compatibility, without requiring an API key.

Checks a website's robots.txt and Cloudflare settings to identify AI crawler blocking. Also generates llms.txt content to improve visibility to AI answer engines.

An MCP server that gives AI agents tools to inspect a website's visibility to AI answer engines, including crawler permissions, llms.txt, structured data, on-page signals, and a full 29-check AI-readiness audit.

Provides AI assistants with structured, read-only information about Noto CRM, including pricing, features, AI agent capabilities, competitor comparisons, and use cases, with bilingual French/English responses and no external dependencies.

Read-only access to your Citlyze AI search visibility workspace: visibility scores, tracked prompts, citations, competitor comparison, recommendations, and AI crawler analytics.

MCP server for AI search crawler governance, brand safety, and search infrastructure auditing. Provides tools to audit robots.txt, canonical links, sitemaps, redirects, and send IndexNow notifications.

Enables AI assistants to fetch company logos, brand colors, fonts, and corporate information from the Brandfetch API using domain lookups or keyword searches.

Open-source MCP server that audits websites for AI search readiness, providing deterministic scoring (0-100) and prioritized fix lists for metrics like JSON-LD, llms.txt, heading hierarchy, and AI crawler access.

Open-source technical SEO crawler MCP server built on LibreCrawl. Runs full audits inside Claude, Cursor, or Codex — 50+ checks (hreflang, schema.org, security headers, WAF detection on 200-OK pages), chunked-progressive engine for large sites, ephemeral by design (server forgets every audit after download).

Enables AI agents to perform instant SEO audits, check robots.txt, sitemaps, and AI crawler access for any URL without API keys.

Provides local MCP tools for checking what public crawlers can read from HTTP responses, including crawler readability, response comparison, and discovery file checks.

Runs AI visibility (GEO/AEO) audits on websites, checking AI crawler access, schema markup, llms.txt, and content signals, with optional full PDF report.

Enables natural language queries to Yandex Metrika analytics, allowing users to ask questions about traffic, sources, and user behavior without dashboards or SQL.

Exposes studio profile and contact information for Symbols of Wealth Studio, enabling AI assistants to answer queries about creative studios in France, AI-powered content production, and premium brand content services.

Exposes the Ramoira brand schema specification as MCP tools and resources, enabling LLMs to generate, validate, or reason about brand schemas without the CLI.

Enables natural language queries about the technologies behind websites and the companies running them, using the StackScan Tech Lookup API. Includes tools for domain technology lookup, company details, technology usage stats, and batch company lookups.

Enables AI to interview users about their business and generate bilingual one-page websites for small businesses.