"Information about web scraping or crawling" matching MCP servers:

An MCP server that lets an AI agent probe a live URL and confirm whether sensitive files (e.g., .git, .env, source maps) are genuinely served by fetching and validating the content, avoiding false positives.

MCP server that exposes 31 OSINT checks from Lissy93/web-check as tools for website analysis, including SSL, DNS, headers, WHOIS, and security presets. Enables natural-language-driven web recon and health checks.

Enables AI agents to audit URLs for security headers, generate CSP Level 3 policies, compute SRI hashes, evaluate WCAG 2.2 contrast, and remediate headers.

Connects AI coding assistants to Snyk API & Web for onboarding scan targets, configuring authentication, running DAST scans, and triaging findings through natural language.

Enables proof-backed code audit via MCP tools for AST-based code maps, symbol lookup, callers, audit commands, and persistent verdict memory.

MCP server that wraps URLFinder and JSFinder to extract URLs, JavaScript files, subdomains, and sensitive information from web pages, with support for batch scanning and deep crawling.

Enables agents to inspect web pages for Cloudflare Turnstile challenges without an API key, and clear Turnstile or WAF challenges using a SolveGate API key.

Enables scanning of Claude Code skills, plugins, or MCP servers for malware before installation via static analysis.

Enables AI agents to access VirusTotal intelligence through MCP, supporting file, URL, domain, and IP lookups, plus analysis, via remote HTTP or local stdio.

Free deterministic security scan of a public git repository (GitHub, GitLab, Codeberg, Bitbucket): vulnerable dependencies via OSV.dev, secret patterns, and config lint, returned as structured JSON. Tools: scan_repository(url), audit_pricing(). Runs locally over stdio (python3 web/mcp_stdio.py or the Dockerfile) and is also hosted at https://project-feldspar.com/mcp. MIT licence, stdlib-only. Buil

A read-only MCP server for the Qualys PCI Merchant API that lets LLM assistants answer questions about PCI compliance posture, such as which hosts are failing PCI or listing high findings.

MCP server for Android APK triage, providing tools to parse APK headers, list DEX classes, and decode AndroidManifest.xml using apktool or androguard backends.

Enables coding agents to perform network diagnostics and lookups, including subnet calculations, port and MAC vendor information, DNS and blocklist checks, TLS certificate inspection, and public IP discovery, all without needing an account or sending telemetry.

Enables agents and CI pipelines to audit MCP servers and agent tool-chains by statically scanning repositories, local checkouts, tools/list exports, or live endpoints for risks such as destructive actions without confirmation, mismatched safety annotations, injection surfaces, credential or PII exposure, and unguarded command, path, or URL sinks. All checks are read-only and never execute the scanned code or call tools/call.

Enables querying MITRE's Common Weakness Enumeration database for vulnerability information, supporting CWE lookup, search, taxonomy navigation, and relationship mapping through MCP.

Runs WCAG accessibility, SEO, performance, and security audits on URLs or raw HTML via SiteLint Auditor. Enables LLM agents to audit web pages and check WCAG criteria through MCP tools.

Enables users to retrieve and display CVE vulnerability information from the National Vulnerability Database (NVD) with support for keyword search and detailed lookup.

Enables MCP clients to work with DefAudit, supporting project and scan listing, starting scans, retrieving results, and marking or unmarking false positives.

Enables policy-first defensive security operations for MCP, providing repository and web-security analysis with controlled authorization, scoped execution, and auditability.

A security gate MCP server that audits agent extensions (skills, MCP servers, tools) by scanning for risks, adversarial analysis, and sandbox execution, returning a trust verdict of allow, quarantine, or block.