"Information about crawer or crawler" matching MCP servers:

A comprehensive website crawler and SEO analyzer that stores site data in a local SQLite database for AI-driven auditing. It enables users to detect technical SEO issues, broken links, and security vulnerabilities through natural language queries or terminal commands.

Enables LLM clients to index Java, Python, TypeScript, and JavaScript projects into a persistent property graph and query callers, callees, execution paths, virtual dispatch, aspect advice, framework wiring, annotations, and external dependencies through MCP tools.

A specialized web crawler and scraper for the US Patent and Trademark Office (USPTO) website, built with Crawl4AI and MCP integration, enabling patent and trademark search, extraction, and status checking.

This server crawls documentation sites and provides search and page-retrieval tools through MCP, enabling users to query indexed documentation content.

Enables site-wide crawling of internal links with BFS traversal, time/title/URL filtering, and page operations like summarization and link clicking, with results persisted to SQLite.

Enables keyword, creator, or item-based image and video retrieval and downloading from Douyin, Xiaohongshu, Weibo, Bilibili, X, and Instagram. It adds intent parsing, semantic and visual re-ranking, quality filtering, and deduplication so results are relevant rather than every image on a page.

Enables crawling news articles by category (politics, economy, society, etc.) and generates blog-ready markdown summaries. Provides MCP tools for listing categories, crawling news, getting article details, and saving to files.

Provides comprehensive system diagnostics and hardware analysis through 10 specialized tools for troubleshooting and environment monitoring. Offers targeted information gathering for CPU, memory, network, storage, processes, and security analysis across Windows, macOS, and Linux platforms.

A simple MCP server implementation in TypeScript that communicates over stdio, allowing users to ask questions that end with 'yes or no' to trigger the MCP tool in Cursor.

Discovers, identifies, and governs bots on your website with MCP tools for scanning, analyzing, and managing bot traffic.

Enables AI clients to query financial news from Yahoo Finance, CNBC, configurable RSS feeds, and 鉅亨網 (cnYES), and to fetch historical OHLCV market data from Yahoo and Investing.com. Supports stock and keyword search, category subscriptions, time filtering, article body extraction, and stdio integration with Cursor, Claude Desktop, and Spring AI.

Enables users to retrieve and structure shipment tracking information from DB Schenker's website using natural language queries.

Enables detection of AI-generated content in images, videos, audio, and text via the AI or Not API. Supports media analysis tools for deepfakes, synthetic voices, and AI-written text.

Provides tools to monitor host system health including CPU load, disk usage, and network status while enabling file system management tasks like searching and moving files. It includes built-in safety guards to prevent operations on critical system directories.

Enables LLM clients to access crop yield predictions from RGB imagery, topic-based PDF literature, and soil lab/moisture CSV data.

Free, open-source desktop SEO crawler with a built-in MCP server. 250 technical SEO, GEO and AI-search checks across 23 areas, run locally; MCP tools list crawls, inspect issues, run Lighthouse, check AI-crawler access, and apply confirmation-gated WordPress fixes with rollback.

Toolset that crawls websites, generates Markdown documentation, and makes that documentation searchable via a Model Context Protocol (MCP) server for integration with tools like Cursor.

MCP server for audio transcription using OpenRouter models, supporting verbatim, cleaned, and custom transcription modes.

Enables a bot to inspect, query, create, edit, and script Roblox Studio instances in token-cheap batches, plus run Luau, playtests, screenshots, and inputs through Roblox's official Studio MCP connection. It runs locally over stdio or an authenticated local HTTP endpoint, returning small structured answers instead of dumping whole games into context.

Enables Claude and other MCP clients to read, extract, crawl, and export data from open browser tabs and specified URLs locally, including logged-in sessions, structured page data, contacts, images, sitemaps, and Shopify catalogues.

Lets an AI assistant check any URL to see how much of a page's main content is present in the raw HTML versus only after JavaScript runs, showing which text blocks, meta fields, canonical tags, JSON-LD and links AI crawlers like GPTBot and ClaudeBot cannot read. It also reports what robots.txt allows for 22 AI bot tokens and how the site responds to each bot's user-agent.

Enables users to check which page content AI crawlers like GPTBot and ClaudeBot cannot read because it only appears after JavaScript runs, by comparing raw HTML with rendered HTML in headless Chrome.

Enables AI assistants to crawl and extract data from Chinese social media platforms like Bilibili, Xiaohongshu, and Douyin. Provides search, content detail retrieval, and creator information tools with persistent browser sessions and QR code login support.

An MCP server that detects affiliate links and provides information about Tracklix, enabling AI assistants to answer affiliate-link-related questions without any network calls or API keys.

Checks a website's robots.txt and Cloudflare settings to identify AI crawler blocking. Also generates llms.txt content to improve visibility to AI answer engines.

Audit a site for AI readiness: AI-crawler access, schema, FAQ and content structure. Generates the fixes.

Exposes canonical knowledge about DeepSeek FR (models, pricing, FAQ, official links) to MCP-compatible AI clients, enabling querying of site information without API keys.

Provides curated, meaningful information about Voi ecosystem protocols, applications, assets, and names, enabling identification and search through natural language.

An MCP server providing comprehensive HubSpot CMS development documentation, enabling users to get contextual information about building HubSpot websites through natural language.

Get information about the current operating environment