Skip to main content
Glama
459,100 tools. Updated 2026-08-17 05:10

"Web crawling techniques and tools" matching MCP tools:

Matching MCP Servers

  • A
    license
    A
    quality
    B
    maintenance
    Provides web search and content extraction tools for AI assistants, with zero configuration and swappable backends.
    6
    29
    1
    MIT
  • F
    license
    A
    quality
    D
    maintenance
    Provides local web search and content fetching capabilities for AI assistants, enabling them to search DuckDuckGo and extract clean text from web pages. All requests originate from the user's machine to ensure direct network control and bypass external proxies.
    2

Matching MCP Connectors

  • Domain/IP intelligence, web page capture and search APIs

  • Pay-per-call AI tools over x402: web research, summarization, structured extraction (USDC, Base).

  • Extract new data from a completed crawl without re-crawling pages. Submit a new prompt and receive a jobId to track progress, paying only for AI tokens.
    MIT
  • Perform full reconnaissance on a target URL using HTTP headers, TLS, DNS, and web technology detection. Adjust depth for quick, standard, or comprehensive analysis.
    AGPL 3.0
  • Retrieve a full malware analysis overview for a SHA256 hash, including MITRE ATT&CK techniques, network indicators, processes, and extracted files from Hybrid Analysis.
    MIT
  • Test web application file upload security by attempting to upload PHP shells using Content-Type bypass techniques and alternative extensions to identify vulnerabilities.
    MIT
  • Browse and filter AI-generated images and videos from Civitai by tags, models, tools, techniques, and content ratings. Sort results and apply NSFW filters to find specific visual content.
    MIT
  • Enrich IPs and domains with Shodan, Censys, WHOIS, reverse IP, Wayback Machine, and VirusTotal data. Auto-detects target type for 6 techniques in a single call.
    AGPL 3.0
  • Enumerate subdomains, detect wildcard DNS, discover related domains via certificate transparency, and summarize attack surface using 8 techniques across 3 phases.
    AGPL 3.0
  • Fingerprint web servers and detect technologies, analytics, CMS, and APIs by analyzing HTTP headers, cookies, caching, compression, and redirects.
    AGPL 3.0
  • Check which censorship evasion techniques (DoH, ECH, domain fronting, Tor bridges) currently work or fail in any country.
    MIT
  • Search the MITRE ATLAS catalog of AI/ML attack techniques by keyword, tactic, or maturity to discover techniques matching your threat model.
    MIT
  • Enrich a threat by ID with malware/tools used, targeted sectors, affected OSes, campaigns, ATLAS techniques, mitigations, and detection data. Offers the "what was used and who was targeted" view.
    MIT
  • Search MITRE ATT&CK techniques by keyword in their name or description. Returns matching techniques with IDs and optional descriptions for enterprise, mobile, or ICS domains.
    Apache 2.0