Skip to main content
Glama
123,900 servers. Updated
Browse by product

Matching MCP tools:

Matching MCP Connectors:

"Information about web crawlers or related technology" matching MCP servers:

GET /v1/servers – MCP directory API reference
  • A
    license
    Not graded
    quality
    C
    maintenance
    Provides live wait times and office details for all 60 Oregon DMV field offices by scraping the ODOT website, with no authentication required.
    70 npm
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables AI coding agents to set up the kit and build pages with it by serving each release's own docs data: searching and describing components, producing install code for a given setup, returning class and content conventions, and checking pages for unknown classes, hooks and missing files. It reads only public docs, with no sign-in or stored data.
    5
    BSD 3-Clause
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables web research through multi-provider search, documentation lookup, public code search, and clean Markdown extraction from static or JavaScript-rendered pages via five read-only tools.
    1
    Apache 2.0
  • F
    license
    Not graded
    quality
    B
    maintenance
    Ultra-lightweight headless browser for AI agents. Provides MCP tools for navigating URLs, extracting structured content, and building autonomous agent workflows.
    3
    -
  • A
    license
    A
    quality
    C
    maintenance
    MCP server that parses web pages into clean markdown via Crawl4AI's HTTP API, exposing parse_page and health-check tools with configurable proxy and timeout.
    2
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables AI assistants to fetch public web pages as readable text or HTML, with caching, live refresh, and parallel job handling.
    3
    17 npm
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables users to audit personal reading streams by quantifying category diversity with Shannon entropy, detecting lexical redundancy and echo chambers via Jaccard similarity, and ranking content sources by decibel signal-to-noise ratio for pruning recommendations.
    7
    MIT
  • A
    license
    Not graded
    quality
    F
    maintenance
    Implementation of an MCP server for the RAG Web Browser Actor. This Actor serves as a web browser for large language models (LLMs) and RAG pipelines, similar to a web search in ChatGPT.
    41 npm
    206
    Apache 2.0
  • A
    license
    A
    quality
    A
    maintenance
    Enables AI agents to check a list of websites by running an Apify Actor through one tool, returning per-site tech stack, on-page SEO issues and score, robots.txt rules for AI crawlers, sitemap, llms.txt presence, and an AI-readiness score. Supports local stdio operation with an Apify token for tool calls.
    8
    MIT
  • F
    license
    A
    quality
    C
    maintenance
    Enables coding agents to drive a real Playwright-backed browser session, navigating, observing, interacting, and capturing proof on web pages via MCP tools across stdio and Streamable HTTP transports.
    15
    2
    -
  • A
    license
    Not graded
    quality
    B
    maintenance
    DocPull v6 is a local-first MCP server that turns public web sources into cited, versioned context packs for AI agents, RAG systems, and developer workflows. It can fetch and sync public static and server-rendered sources such as changelogs, standards, API references, OpenAPI specs, feeds, repo/package metadata, papers, filings, pricing pages, blogs, and other web sources.
    212 PyPI
    29
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables autonomous agents to inspect live web pages, prune noisy DOM content down to semantic essentials, and bind each form field to agent state by matching input attributes, autocomplete hints, and regex validations. It also converts HTML/Markdown tables into type-inferred JSON records and scores agent action trajectories for precision, recall, and edit distance.
    7
    MIT
  • A
    license
    Not graded
    quality
    Not graded
    maintenance
    Enables autonomous web form inspection and schema auto-mapping by matching input attributes, autocomplete hints, and regex validations to agent state, while also pruning DOMs and converting tables into structured JSON for web agents.
    MIT
  • F
    license
    Not graded
    quality
    B
    maintenance
    Enables MCP-compatible agents to scrape web pages and distill them into clean Markdown by removing cookie banners, navbars, and ads for efficient consumption.
    7
    -
  • F
    license
    Not graded
    quality
    C
    maintenance
    Remote MCP server of Aureum Technology GmbH (IT service provider, Rodewisch/Vogtland, Germany): company and contact data, service catalogue, current prices and full-text search across aureum-tech.com for AI assistants. Streamable HTTP at https://aureum-tech.com/mcp, read-only, no authentication.
    -
  • A
    license
    A
    quality
    D
    maintenance
    Fact-checks and fixes AI outputs by catching hallucinations, repairing broken JSON, and correcting errors before they reach users, with tools for verification, validation, and correction.
    4
    46 npm
    MIT
  • A
    license
    A
    quality
    F
    maintenance
    Unleashes LLM-powered agents to autonomously execute and debug web apps directly in your code editor, with features like webapp navigation, network traffic capture, and console error collection.
    2
    1,232
    Apache 2.0
  • A
    license
    A
    quality
    B
    maintenance
    Enables assistants to submit and monitor durable web scraping jobs, extract public pages into Markdown, CSS-defined fields, and source receipts, and map sites or search indexed content through MCP tools.
    11
    MIT
  • A
    license
    C
    quality
    B
    maintenance
    Connects AI coding assistants to Snyk API & Web for onboarding scan targets, configuring authentication, running DAST scans, and triaging findings through natural language.
    51
    61 PyPI
    8
    Apache 2.0
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables agents to find relevant documentation with source URLs, evidence, and freshness, search and browse an llms.txt directory, validate or draft llms.txt files, and manage site registration, verification, and change watches.
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables web search for AI agents with pay-per-search in USDC, no API keys needed.
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Written in Go, MOB³ features its own HTML rendering engine with JavaScript support, an ANSI renderer, and inline image support. No GUI toolkit or persistent browser profile required. Built as a true terminal web browser for humans and AI agents.
    7
    AGPL 3.0
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables a local AI agent to investigate service incidents by querying read-only telemetry such as service health, metrics, logs, and traces, then produce verifiable assessments with evidence links and uncertainty.
    Apache 2.0
  • A
    license
    A
    quality
    B
    maintenance
    MCP server for searching French and European law, providing natural language access to jurisprudence, consolidated legal texts, and related legal information via tools like search_decisions and search_legal_texts.
    4
    36
    Apache 2.0
  • A
    license
    A
    quality
    B
    maintenance
    Enables natural language queries about the technologies behind websites and the companies running them, using the StackScan Tech Lookup API. Includes tools for domain technology lookup, company details, technology usage stats, and batch company lookups.
    6
    69 npm
    3
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Screen-reader navigation cost analyzer. Measures how hard it is for assistive-technology users to discover, reach, and operate interactive targets on the web.
    9
    48 npm
    1
    Apache 2.0
  • A
    license
    A
    quality
    F
    maintenance
    A robust Model Control Protocol server that enables AI agents to access real-time cyber threat intelligence and detailed information about vulnerabilities, threat actors, malware, and other cyber-security entities.
    10
    16 PyPI
    8
    Apache 2.0
  • A
    license
    A
    quality
    B
    maintenance
    Enables AI assistants to rewrite AI-styled English prose into more natural human writing, with tools for humanizing text and retrieving information about the underlying model.
    2
    133 npm
    1
    Apache 2.0
  • A
    license
    A
    quality
    D
    maintenance
    Enables AI assistants to interact with the Open Targets Platform API to search and retrieve comprehensive data about target-disease associations, drugs, genes, and drug discovery information through GraphQL queries.
    5
    37
    Apache 2.0
  • A
    license
    A
    quality
    D
    maintenance
    Enables AI agents to search and retrieve detailed information about US golf courses, including course details, tees, climate, and nearby points of interest.
    6
    79 npm
    4
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Checks a site for AI crawlers: which of 11 AI crawlers its robots.txt lets in, whether its llms.txt follows llmstxt.org, and what its home page says without scripts. Findings with evidence; no scores, no model calls, no telemetry.
    5
    49 npm
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    A Model Context Protocol server that allows AI assistants to search for and retrieve information about OpenTofu providers, modules, resources, and data sources from the OpenTofu Registry.
    5
    591 npm
    116
    Mozilla Public 2.0
  • A
    license
    A
    quality
    B
    maintenance
    Provides access to the SonoVault music metadata API, enabling search and retrieval of information about 90M+ tracks, artists, labels, and releases, including ISRC, ISWC, and cross-platform IDs.
    15
    56 npm
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Enables web content extraction, screenshot capture, web search, arXiv paper search, and image search through Jina AI's APIs. Provides tools for reading URLs as markdown, searching the web for current information, and finding academic papers or images.
    19
    913
    Apache 2.0
  • A
    license
    A
    quality
    B
    maintenance
    Enables AI assistants to answer questions about nixpkgs packages using nixkeeper's public data, covering outdated versions, Hydra build status and failures, update-bot attempts, related PRs and issues, CVEs, and what needs attention for a maintainer or team. It runs locally over stdio, reads only public data, and requires no account or token.
    10
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    Enables access to Hong Kong government's official open data portal (DATA.GOV.HK) through natural language queries. Supports searching datasets, browsing categories, and retrieving detailed information about Hong Kong's public data resources.
    8
    22 PyPI
    9
    MIT
  • A
    license
    B
    quality
    F
    maintenance
    Enables an assistant to verify whether claims about AI systems hold against supplied evidence bytes, replay conformance vectors, and inspect related terms and artifact digests offline without fetching, writing, or sending data.
    9
    Apache 2.0
  • A
    license
    C
    quality
    D
    maintenance
    Enables large language models to retrieve up-to-date and correct information about dlt pipelines, datasets, and schemas, improving AI-assisted development.
    8
    68 PyPI
    10
    Apache 2.0
  • A
    license
    Not graded
    quality
    Not graded
    maintenance
    Provides large language models with access to news and information about renewable energy projects in Germany, allowing filtering by location, topic (solar, wind, hydrogen), and date range.
    Apache 2.0
  • A
    license
    Not graded
    quality
    C
    maintenance
    A local MCP server that wraps the Wappalyzer API to identify web technologies, subdomains, and site metadata. It enables users to perform site lookups and access technology categories through natural language interfaces.
    4
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Provides AI coding agents with a live web search tool that returns extracted, answer-ready text from web pages. Enables real-time information retrieval without any local installation or maintenance.
    4 npm
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables AI assistants to query BuiltWith's technology detection data directly, supporting natural language questions about website technologies.
    248 npm
    43
    MIT