Information about web crawling or spiders

Glama

Search for:

Information about web crawling or spiders

View all MCP Servers

Why this server?
This server provides web research capabilities, allowing users to search Google, extract webpage content, and capture screenshots, which are all components of web crawling.
MCP Web Research Server
jevy
-
security
A
license
-
quality
A server that integrates with Claude Desktop to enable real-time web research capabilities, allowing users to search Google, extract webpage content, and capture screenshots directly from conversations.
Last updated -
854
MIT License
Why this server?
This server provides web research capabilities, allowing users to search Google, extract webpage content, and capture screenshots, which are all components of web crawling.
MCP Web Research Server
chuanmingliu
-
security
A
license
-
quality
A Model Context Protocol server that enables Claude to perform web research by integrating Google search, extracting webpage content, and capturing screenshots.
Last updated -
854
4
MIT License
Why this server?
This server is designed for web research, allowing users to search Google and extract webpage content, key components of web crawling.
MCP Web Research Server
mzxrai
A
security
A
license
A
quality
A Model Context Protocol (MCP) server for web research. Bring real-time info into Claude and easily research any topic.
Last updated -
3
854
189
JavaScript
MIT License
Why this server?
This server performs web searches and retrieves up-to-date information using a Crawler API, enabling web crawling functionality.
WebSearch-MCP
mnhlt
A
security
F
license
A
quality
A Model Context Protocol server that enables AI assistants to perform real-time web searches, retrieving up-to-date information from the internet via a Crawler API.
Last updated -
1
44
8
JavaScript
Why this server?
This server enables interaction with web pages, takes screenshots, and executes JavaScript, useful for advanced web crawling and scraping.
MCP-Server-Playwright
Automata-Labs-team
A
security
A
license
A
quality
Enables LLMs to interact with web pages, take screenshots, and execute JavaScript in a real browser environment
Last updated -
10
327
85
JavaScript
MIT License
Why this server?
This server is designed for web scraping of difficult-to-access websites affected by bot detection, captchas, or geolocation restrictions.
ScrAPI MCP Server
DevEnterpriseSoftware
-
security
-
license
-
quality
A server that enables web scraping of difficult-to-access websites affected by bot detection, captchas, or geolocation restrictions, returning results in either HTML or Markdown format.
Last updated -
1
JavaScript
Why this server?
This server provides access to Jina AI's web services including page reading and web search, useful for gathering data from the internet.
mcp-jina-ai
JoeBuildsStuff
A
security
A
license
A
quality
An MCP server that provides access to Jina AI's powerful web services (page reading, web search, fact checking) through Claude.
Last updated -
3
189
12
JavaScript
MIT License
Why this server?
This server extracts and transforms webpage content into clean, LLM-optimized Markdown, which is useful for web crawling and data extraction.
Mozilla Readability Parser MCP Server
emzimmer
A
security
A
license
A
quality
Extracts and transforms webpage content into clean, LLM-optimized Markdown. Returns article title, main content, excerpt, byline and site name. Uses Mozilla's Readability algorithm to remove ads, navigation, footers and non-essential elements while preserving the core content structure.
Last updated -
1
4
11
MIT License
Why this server?
Gitee API integration, repository, issue, and pull request management, and more.
Code Review MCP Server
crazyrabbitLTC
-
security
F
license
-
quality
Analyzes codebases using Repomix and LLMs to provide structured code reviews with specific issues and recommendations, supporting multiple LLM providers including OpenAI, Anthropic, and Gemini.
Last updated -
2
JavaScript
Why this server?
A powerful MCP server for fetching and transforming web content into various formats (HTML, JSON, Markdown, Plain Text) with ease.
MCP NPX Fetch
tokenizin-agency
A
security
A
license
A
quality
A powerful MCP server for fetching and transforming web content into various formats (HTML, JSON, Markdown, Plain Text) with ease.
Last updated -
4
146
12
TypeScript
MIT License