"Travis CI" matching MCP servers:

Provides agent-friendly web search via MCP with zero-config free engines, region-aware failover, freshness filtering, and a one-call research tool that searches, fetches, and summarizes pages.

Checks a website's robots.txt and Cloudflare settings to identify AI crawler blocking. Also generates llms.txt content to improve visibility to AI answer engines.

Fetches and analyzes website content to provide business context to AI agents, designed for Voice AI in GoHighLevel.

Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.

Enables web search on Chinese internet content using Bocha AI, returning results as Markdown or JSON with configurable freshness and summary options.

Provides Bing Chinese search and webpage crawling tools, enabling AI assistants to fetch real-time web information without API keys.

Enables AI agents to check whether a public website is crawlable, understandable, and ready for AI search workflows through local-only audits of robots.txt, sitemaps, metadata, and llms.txt.

B2B lead generation MCP server with 20+ lead generation tools - Apollo scraping, Google Maps, email finder, email validator, mobile finder, skip trace, ecommerce store data, and more.

An MCP server that provides access to Jina AI's powerful web services (page reading, web search, fact checking) through Claude.

CloakBrowser MCP is a Python MCP server that lets agents control a CloakBrowser-backed browser from Linux servers, CI jobs, and other environments where a normal desktop browser is not available.

Enables access to LinkedIn data through the Li Data Scraper API, supporting profile enrichment, company details, people search, post interactions, and activity tracking.

Interact with WebScraping.AI API for web data extraction and scraping

Enables AI agents to research FOMO.family through a local browser session, including discovering trending and most-held coins, reading trader theses, viewing holders, and managing watchlists.

An MCP server optimized for Chinese users that enables AI models to perform Bing searches and fetch webpage content without requiring an API key. It provides specialized tools for searching and scraping web data directly through supported AI environments like Claude or Cursor.

Enables web search using Bing and DuckDuckGo combined, and fetching readable content from web pages, with no API key required.

Provides read-only MCP tools to search YouTube, retrieve video metadata, transcripts, comments, channel information, and popular videos.

Enables AI agents to fully control a terminal browser via MCP, navigating pages, filling forms, logging in, and managing cookies with deterministic numbered links and forms. Also provides an interactive human CLI.

HTML Parser AI - MCP server providing AI-powered tools and automation by MEOK AI Labs

Enables intelligent web scraping with support for static pages, JavaScript-rendered SPAs, and natural language data extraction.

Automates cross-platform e-commerce sourcing between JD and Taobao using AI-driven browser automation to search, extract product details, compare prices, and identify profitable arbitrage opportunities.