Skip to main content
Glama
123,555 servers. Updated

Matching MCP tools:

Matching MCP Connectors:

"Information about web crawlers or related technology" matching MCP servers:

GET /v1/servers – MCP directory API reference
  • F
    license
    A
    quality
    B
    maintenance
    Odysseus Web MCP is a standalone Model Context Protocol (MCP) server for safe public-web search and URL fetching.
    2
    -
  • A
    license
    A
    quality
    B
    maintenance
    A fast web reader for AI assistants, served over MCP. It reads web pages and PDFs and returns only the passages that answer the question, not whole pages. It runs locally and keeps nothing.
    3
    7
    Apache 2.0
  • A
    license
    A
    quality
    C
    maintenance
    MCP server for web scraping with natural language, enabling site crawling, page fetching, and extraction of emails, phones, links, and social profiles. Returns clean markdown, handles JavaScript rendering, and respects rate limits.
    24
    1
    MIT
  • A
    license
    B
    quality
    B
    maintenance
    Enables agents to fetch web pages as clean Markdown, inspect metadata, extract links, analyze tech stacks, and audit or compare SEO, with per-call billing and automatic refunds for failed fetches.
    8
    169 PyPI
    Business Source 1.1
  • A
    license
    A
    quality
    A
    maintenance
    Connect AI assistants to your Gramps Web family tree. Search and edit genealogy records, explore ancestors, descendants, relationships and timelines, and read document scans with vision-capable models. Includes guided research workflows, source citations, read-only mode, and privacy-aware media previews. Install with Docker or a Claude Desktop bundle.
    2
    35
    23
    AGPL 3.0
  • A
    license
    A
    quality
    B
    maintenance
    Enables MCP-capable agents to perform reliable web search and content fetching through a single server that automatically fails over across multiple providers. It handles keyed or keyless authentication automatically and returns clean Markdown or text results.
    3
    1
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    An MCP server that lets an AI agent probe a live URL and confirm whether sensitive files (e.g., .git, .env, source maps) are genuinely served by fetching and validating the content, avoiding false positives.
    2
    16 npm
    1
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables reading web pages and PDFs as clean Markdown, extracting main content and letting agents query for relevant passages, outline sections, search in-page, and list links while respecting robots.txt and blocking private addresses.
    4
    1
    MIT
  • A
    license
    A
    quality
    A
    maintenance
    MCP server for AI-powered web research — search (Google, Brave, Serper, SearXNG), scrape any page, extract PDFs/DOCX/YouTube transcripts, academic & patent search. Single Go binary. Works with Claude, Cursor, Copilot, and any MCP client.
    25
    79
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables an assistant to search and read AnkiWeb's shared-deck catalogue, download decks as .apkg files or Markdown, list the decks synced to your account, and publish a deck after a confirmed preview, all through your own AnkiWeb session. It can reuse a signed-in session from a local Chromium browser or open a login window, keeping credentials out of the server.
    11
    365 npm
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Provides a unified web search tool for AI agents by running multiple search providers concurrently and returning normalized results. It integrates with Claude Code, Codex CLI, OpenCode, and Hermes.
    1
    379 PyPI
    17
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Self-hostable MCP server providing web search through an operator-configured SearXNG instance, public-page fetching, and HTML-to-Markdown extraction. Search requires SearXNG with JSON output enabled.
    3
    163 npm
    3,534 PyPI
    MIT
  • A
    license
    B
    quality
    B
    maintenance
    A local Python WebSocket bridge and browser extension that securely connects web AI chats to local Model Context Protocol (MCP) servers. It enables web-based AIs to execute local tools with granular permission controls, providing the same powerful tool and system access typically exclusive to desktop agent harnesses like DeepSeek Harness, Claude Code, Codex and more.
    4
    16
    Apache 2.0
  • A
    license
    B
    quality
    C
    maintenance
    MCP server exposing 17 pay-per-call web, finance, legal and onchain utilities for AI agents: crypto readers and Base deployer intelligence, SEC EDGAR filings, OFAC sanctions checks, insider transactions, 13F holdings, and multi-jurisdiction caselaw and legislation search. Pay per call in USDC on Base via x402 or MPP.
    17
    MIT
  • A
    license
    C
    quality
    C
    maintenance
    Enables AI agents to perform web searches via DuckDuckGo without API keys and automate browser tasks with headless Chromium. Supports multi-step research workflows that automatically search and extract content from multiple pages.
    7
    61 PyPI
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables users to audit personal reading streams by quantifying category diversity with Shannon entropy, detecting lexical redundancy and echo chambers via Jaccard similarity, and ranking content sources by decibel signal-to-noise ratio for pruning recommendations.
    7
    MIT
  • F
    license
    Not graded
    quality
    B
    maintenance
    📇 ☁️ - Capture web pages as cryptographically signed, tamper-evident evidence. Ed25519 signatures, RFC 3161 timestamps, and WACZ archives. Four tools: capture_url, get_capture, list_captures, verify_capture.
    2
    -
  • A
    license
    A
    quality
    A
    maintenance
    Enables AI agents to check a list of websites by running an Apify Actor through one tool, returning per-site tech stack, on-page SEO issues and score, robots.txt rules for AI crawlers, sitemap, llms.txt presence, and an AI-readiness score. Supports local stdio operation with an Apify token for tool calls.
    8
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables AI agents to extract clean LLM-ready markdown and readable articles from web pages, parse metadata, tables, links, feeds, and contact info, audit paywalls, SEO, and XSS, and access these tools via Streamable HTTP or a stdio bridge.
    1
    12
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables querying Shopify App Store apps and Chrome Web Store extensions by category, keyword, or listing URL, returning ratings, reviews, pricing or user counts, and developer website, domain, address, and email. Filters by minimum rating or reviews and produces one row per listing ready for outreach or enrichment.
    1
    122 npm
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables autonomous agents to inspect live web pages, prune noisy DOM content down to semantic essentials, and bind each form field to agent state by matching input attributes, autocomplete hints, and regex validations. It also converts HTML/Markdown tables into type-inferred JSON records and scores agent action trajectories for precision, recall, and edit distance.
    7
    MIT
  • F
    license
    Not graded
    quality
    B
    maintenance
    Enables MCP-compatible agents to scrape web pages and distill them into clean Markdown by removing cookie banners, navbars, and ads for efficient consumption.
    7
    -
  • A
    license
    A
    quality
    B
    maintenance
    Buy at cash-only or unsupported shops by paying crypto (BTC signet, USDC) to a proxy shopper through a 2-of-3 escrow with timelocks over Nostr, or earn as a proxy shopper. Tools that move money act only with confirm: true.
    19
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Serpex is a web search engine built for AI agents. serpex_search returns ranked web results as JSON, optionally with the full content of the top 5 or 10 pages as Markdown. serpex_extract turns up to 10 URLs into clean Markdown or HTML. Use it to ground answers and enrich company and lead data across legal, hiring, finance, product, news, research and B2B enrichment. 1 credit per search.
    2
    53 npm
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    MCP server for Drop Beacon (edc4me.com) — search and explore 80,000+ EDC products across 1,000+ brands. Track new drops, compare prices across retailers, check real-time stock availability, and analyze market trends for knives, flashlights, fidget toys, wallets, pens, and more.
    4
    58 npm
    MIT
  • A
    license
    B
    quality
    B
    maintenance
    Evidence-governed coding agent memory and knowledge lifecycle MCP server for DeepSeek Harness (DSH), OpenAI Codex, and Google Antigravity. Provides local SQLite FTS5 + semantic hybrid retrieval, automatic session/compaction checkpointing, counter-example demotion, and a hard 1,200-char context budget guardrail with zero third-party Python dependencies.
    11
    1
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Free web search and page fetching for AI agents. No account or API key required. Connect to https://search.parallel.ai/mcp over Streamable HTTP. Tools: web_search and web_fetch. Free-tier rate limits apply.
    33
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Website Intelligence MCP Server exposes 10 tools for web scraping and analysis, enabling AI agents to convert URLs to Markdown, extract metadata, detect technologies, find contacts, and perform SEO checks. It wraps a REST API and supports optional API keys.
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables targeted company discovery, real-time DNS MX deliverability diagnostics, and clean web Markdown extraction for AI agents.
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables agents to convert any URL into clean, structured, agent-optimized content with a single paid call, paying $0.02 USDC per successful extraction via x402 on Base without API keys or accounts.
    MIT
  • F
    license
    Not graded
    quality
    B
    maintenance
    Enables autonomous agents to establish persistent identities, interact and play supported games, manage social and memory data, and read evidence-backed reputation and discovery catalogs. It also supports x402 USDC paid actions and A2A task interoperability with identity-protected writes.
    -
  • A
    license
    A
    quality
    B
    maintenance
    MCP server for searching French and European law, providing natural language access to jurisprudence, consolidated legal texts, and related legal information via tools like search_decisions and search_legal_texts.
    4
    36
    Apache 2.0
  • A
    license
    A
    quality
    B
    maintenance
    Enables natural language queries about the technologies behind websites and the companies running them, using the StackScan Tech Lookup API. Includes tools for domain technology lookup, company details, technology usage stats, and batch company lookups.
    6
    69 npm
    3
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    Investigate any website's technology stack: CMS, eCommerce, frameworks, analytics, CRM, payments, and hosting behind any URL.
    3
    52 npm
    1
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Exposes three read-only MCP tools that let an assistant check whether specified AI crawlers are allowed by a site's robots.txt and how a URL responds when fetched with each bot's user-agent, fetch a page as a single named bot, and generate a robots.txt that welcomes AI crawlers.
    6
    141 npm
    3
    MIT