Skip to main content
Glama
459,153 tools. Updated 2026-08-17 06:10

"Web crawling techniques and tools" matching MCP tools:

Matching MCP Servers

  • A
    license
    A
    quality
    B
    maintenance
    Provides web search and content extraction tools for AI assistants, with zero configuration and swappable backends.
    6
    29
    1
    MIT

Matching MCP Connectors

  • Domain/IP intelligence, web page capture and search APIs

  • Pay-per-call AI tools over x402: web research, summarization, structured extraction (USDC, Base).

  • Extract new data from a completed crawl without re-crawling pages. Submit a new prompt and receive a jobId to track progress, paying only for AI tokens.
    MIT
  • Perform full reconnaissance on a target URL using HTTP headers, TLS, DNS, and web technology detection. Adjust depth for quick, standard, or comprehensive analysis.
    AGPL 3.0
  • Retrieve a full malware analysis overview for a SHA256 hash, including MITRE ATT&CK techniques, network indicators, processes, and extracted files from Hybrid Analysis.
    MIT
  • Test web application file upload security by attempting to upload PHP shells using Content-Type bypass techniques and alternative extensions to identify vulnerabilities.
    MIT
  • Browse and filter AI-generated images and videos from Civitai by tags, models, tools, techniques, and content ratings. Sort results and apply NSFW filters to find specific visual content.
    MIT
  • Enrich IPs and domains with Shodan, Censys, WHOIS, reverse IP, Wayback Machine, and VirusTotal data. Auto-detects target type for 6 techniques in a single call.
    AGPL 3.0
  • Enumerate subdomains, detect wildcard DNS, discover related domains via certificate transparency, and summarize attack surface using 8 techniques across 3 phases.
    AGPL 3.0
  • Fingerprint web servers and detect technologies, analytics, CMS, and APIs by analyzing HTTP headers, cookies, caching, compression, and redirects.
    AGPL 3.0
  • Check which censorship evasion techniques (DoH, ECH, domain fronting, Tor bridges) currently work or fail in any country.
    MIT
  • Search the MITRE ATLAS catalog of AI/ML attack techniques by keyword, tactic, or maturity to discover techniques matching your threat model.
    MIT
  • Enrich a threat by ID with malware/tools used, targeted sectors, affected OSes, campaigns, ATLAS techniques, mitigations, and detection data. Offers the "what was used and who was targeted" view.
    MIT
  • Search MITRE ATT&CK techniques by keyword in their name or description. Returns matching techniques with IDs and optional descriptions for enterprise, mobile, or ICS domains.
    Apache 2.0