Skip to main content
Glama
270,522 tools. Last updated 2026-07-07 21:14

"Web scraping tools and techniques for data extraction" matching MCP tools:

  • Execute a single call that `consult` handed you, and bill on success. Used for any external capability (image/video/audio generation, web search, scraping, email, document parsing, code sandbox, browser automation, embeddings, etc.). The server validates params against a registered schema and proxies to the upstream — you never pass URLs or API keys. Always get the exact (service, action, params, max_cost_cents) from `consult` first — don't guess them.
    Connector
  • Executes a Strale capability by slug and returns the result. Use this when you need to perform any verification, validation, lookup, or data extraction from the 271-capability registry. Call strale_search first to find the right slug and required input fields. Returns a result object with the capability output, quality score (SQS), latency, price charged, and data provenance. Five free capabilities work without an API key (10/day limit). Paid capabilities debit from the wallet — check strale_balance first for high-value calls.
    Connector
  • Look up a MITRE ATLAS technique — the AI/ML adversarial attack catalog. ATLAS catalogues TTPs targeting machine learning systems: prompt injection, model evasion, training data poisoning, model theft, etc. Roughly 80% of ATLAS techniques are AI/ML-specific (no ATT&CK bridge); 20% mirror an enterprise ATT&CK technique via attack_reference_id — use that to pivot to D3FEND defenses (d3fend_defense_for_attack) and CVE search. Sub-techniques inherit `tactics` from the parent (inherited_tactics=true flag) when ATLAS upstream leaves them empty. Use this tool when the user asks about AI/ML threats, LLM red-teaming, or adversarial ML; for multiple techniques in one call (e.g. drilling into a case study's techniques_used), prefer bulk_atlas_technique_lookup. Returns 404 when the id is not in the synced ATLAS catalog. Free: 30/hr, Pro: 500/hr. Returns {technique_id, name, description, tactics, inherited_tactics, maturity (demonstrated|feasible|realized), attack_reference_id, attack_reference_url, subtechnique_of, created_date, modified_date, next_calls}.
    Connector
  • Bulk ATLAS technique lookup — retrieve full records for up to 50 techniques in a single request instead of N separate atlas_technique_lookup calls. Designed as the natural follow-up to atlas_case_study_lookup, whose techniques_used array can be passed directly. Each item is the same shape as atlas_technique_lookup, including parent-tactics inheritance for sub-techniques (inherited_tactics=true flag) and per-item next_calls (D3FEND bridge when attack_reference_id present, sibling-technique search by tactic, parent lookup for sub-techniques). Free: 30/hr (1 per item), Pro: 500/hr. Returns {results [{technique_id, status (ok|not_found|invalid_format), technique, error}], total, successful, failed, partial, summary}.
    Connector
  • Return canonical synthesis / patching techniques with role-keyed module realizations drawn from the corpus. Use this when the user asks "how do I do X?" with X being a recognisable technique (low-pass-gate plucks, pinged-filter percussion, parallel multiband processing, complex-oscillator FM, karplus-strong pluck, clocked-delay feedback, modal-resonator excitation, wavefolder harmonics, envelope-follower ducking, Maths-style function-generator omnibus). It's also the right tool when the user has a module and asks "what's this good for?" — pass filter.module_id to retrieve every technique that references the module via its role_realizations. Each technique declares role_definitions (the roles the technique uses, each with required and optional affordances) and role_realizations (concrete modules that fill each role, with the affordances they provide). The model substitutes modules from the user's rack into roles by affordance match — DO NOT treat the realization list as exhaustive or as a recipe. Args: - filter (optional): { capability?, module_id?, text? } - capability: kebab-case capability id (see search_modules _meta.taxonomy). Returns techniques whose required *or* optional capability list includes this id. - module_id: "<manufacturer>/<module-slug>". Returns techniques that have a role_realization referencing this module. - text: free-text phrase. Substring-matches against technique id/label/description AND a curated alias table (technique_aliases) — that's the right surface when a user types evocative prose like "stuttering delay", "plucked string", "source of uncertainty" that doesn't grep against any kebab-case id. Two-way alias match: long alias ("source of uncertainty") matches short query ("uncertainty"), and vice versa. - When multiple filters supplied, AND-intersects. - Omit filter entirely to list all techniques. Returns: { "techniques": [ { "id": "low-pass-gate-pluck", "label": "Low-Pass Gate Pluck", "description": "Send a short envelope...", "required_capabilities": ["lowpass-gate"], "optional_capabilities": ["envelope-generator", "function-generator"], "role_definitions": [ { "role_id": "lpg", "description": "The vactrol-based or vactrol-emulating element. Strictly required...", "required_affordances": ["lowpass-gate"], "optional_affordances": [] }, ... ], "role_realizations": [ { "role_id": "lpg", "module_id": "make-noise/optomix", "affordances_provided": ["lowpass-gate"], "notes": "Two-channel vactrol-based LPG..." }, ... ], "canonical_instance": { "rationale": "...", "lineage": [ { "position": 1, "label": "Buchla 292 (1970)", "module_id": null, "notes": "..." }, { "position": 2, "label": "Tiptop Audio Buchla 292t", "module_id": "tiptop-audio/buchla-292t" }, ... ] }, "counter_canonical_notes": [ { "claim_pushed_back_against": "Optomix is the canonical pairing with Plaits...", "evidence": "The corpus catalogs 19 LPG-capable modules..." } ], "coverage": [ { "role_id": "voice", "realizations_count": 3 }, { "role_id": "lpg", "realizations_count": 19 }, { "role_id": "env", "realizations_count": 6 }, { "role_id": "clock", "realizations_count": 2 } ] } ], "_meta": { "filter": {...}, "feedback_hint"?: string } } How to use role data: - role_realizations are CURATORIAL SAMPLES, not exhaustive lists. The coverage[].realizations_count tells you how many are documented; other modules may fill the same role. - To find modules in the user's rack that can fill a role, use find_role_realizations(technique_id, role_id, available_modules). - canonical_instance is opt-in and sparse. Most techniques don't have one; that absence is information. When present, it documents a documented historical lineage (e.g., Buchla 292 → 292t → MMG → Optomix for low-pass-gate-pluck) — NOT a prescription. - counter_canonical_notes push back on likely training-data priors. When the user invokes a canonical-sounding claim that has a counter_canonical_note, surface the pushback. Errors: - "Module not found: <id>" if filter.module_id is supplied and unknown. - Empty techniques[] with a feedback_hint when filters produce no matches — call report_gap if the user expected coverage.
    Connector
  • Check the extraction status of one or more parts. Free. Each entry includes the current extraction step, elapsed seconds, and document ID. Use after prefetch_datasheets or after read_datasheet triggers a new extraction. Recommended polling cadence: every 5-10 seconds. Extraction typically takes 30s-2min for new parts, so polling faster than every 5s wastes calls. Stop polling once status is 'ready', 'failed', 'no_source', or 'unsupported'. DATASHEET STATUS VALUES: - 'ready' — extracted and indexed; call read_datasheet, search_datasheets, or analyze_image. - 'extracting' / 'in_progress' / 'queued' / 'pending' — extraction running or scheduled. Poll check_extraction_status every 5-10s until 'ready' or 'failed'. Typical time: 30s-2min. - 'not_extracted' — known part but datasheet hasn't been fetched yet. Trigger it via prefetch_datasheets (cheapest) or by calling read_datasheet (auto-triggers on first read). - 'no_source' — we couldn't find a public datasheet URL for this MPN. First, retry prefetch_datasheets in 10-30s (the URL resolver re-runs and often finds a source on the second pass). If still 'no_source', the agent can upload the PDF manually via request_datasheet_upload + confirm_datasheet_upload (see those tools). Org-uploaded datasheets are private to the org. - 'unsupported' — PDF exists but can't be extracted (scanned image-only, encrypted, or corrupted). Upload a clean text-based PDF via request_datasheet_upload to override. - 'failed' / 'error' — extraction errored. The response includes the error reason. Retry via prefetch_datasheets or escalate to support. - 'rejected' — input wasn't a real MPN (bare value like '100nF', description, or reference designator). Fix the input and re-call. - 'deduplicated' — another part in the family already has this datasheet; same content is returned under the primary MPN.
    Connector

Matching MCP Servers

  • F
    license
    A
    quality
    D
    maintenance
    Provides local web search and content fetching capabilities for AI assistants, enabling them to search DuckDuckGo and extract clean text from web pages. All requests originate from the user's machine to ensure direct network control and bypass external proxies.
    Last updated
    2

Matching MCP Connectors

  • Generic URL crawl + HTML extraction — fallback for sites without dedicated MCPs.

  • 40+ web scraping tools from Firecrawl, Bright Data, Jina, Olostep, ScrapeGraph, Notte, and Riveter. Scrape, crawl, screenshot, and extract from any website. Starts at $0.01/call. Get your API key at app.xpay.sh or xpay.tools

  • Find tools by describing the data or task. Use when you need to browse, search, look up, or discover what tools exist for: SEC filings, financials, revenue, profit, FDA drugs, adverse events, FRED economic data, Census demographics, BLS jobs/unemployment/inflation, ATTOM real estate, ClinicalTrials, USPTO patents, weather, news, crypto, stocks. Returns the top-N most relevant tools with names, descriptions, and full input schemas (with curated examples) — each result is ready to call directly, no second schema lookup needed. Call this FIRST when you have many tools available and want to see the option set (not just one answer).
    Connector
  • Find tools by describing the data or task. Use when you need to browse, search, look up, or discover what tools exist for: SEC filings, financials, revenue, profit, FDA drugs, adverse events, FRED economic data, Census demographics, BLS jobs/unemployment/inflation, ATTOM real estate, ClinicalTrials, USPTO patents, weather, news, crypto, stocks. Returns the top-N most relevant tools with names, descriptions, and full input schemas (with curated examples) — each result is ready to call directly, no second schema lookup needed. Call this FIRST when you have many tools available and want to see the option set (not just one answer).
    Connector
  • Upload an image (base64 encoded) and extract its dominant colour palette, with each colour matched to its nearest named archive entry with full cultural provenance. Uses K-means++ extraction plus Bradford chromatic adaptation for accuracy. Returns up to 5 dominant colours, each with archive name, cultural story, nearest RAL standard, and WCAG accessibility data. Works for product photography, interior photos, artwork, brand assets, and mood boards. The image is never stored — processed in memory only.
    Connector
  • Extract structured information from web pages using LLM capabilities. Supports both cloud AI and self-hosted LLM extraction. **Best for:** Extracting specific structured data like prices, names, details from web pages. **Not recommended for:** When you need the full content of a page (use scrape); when you're not looking for specific structured data. **Arguments:** - urls: Array of URLs to extract information from - prompt: Custom prompt for the LLM extraction - schema: JSON schema for structured data extraction - allowExternalLinks: Allow extraction from external links - enableWebSearch: Enable web search for additional context - includeSubdomains: Include subdomains in extraction **Prompt Example:** "Extract the product name, price, and description from these product pages." **Usage Example:** ```json { "name": "firecrawl_extract", "arguments": { "urls": ["https://example.com/page1", "https://example.com/page2"], "prompt": "Extract product information including name, price, and description", "schema": { "type": "object", "properties": { "name": { "type": "string" }, "price": { "type": "number" }, "description": { "type": "string" } }, "required": ["name", "price"] }, "allowExternalLinks": false, "enableWebSearch": false, "includeSubdomains": false } } ``` **Returns:** Extracted structured data as defined by your schema.
    Connector
  • Autonomous web research agent. This is a separate AI agent layer that independently browses the internet, searches for information, navigates through pages, and extracts structured data based on your query. You describe what you need, and the agent figures out where to find it. **How it works:** The agent performs web searches, follows links, reads pages, and gathers data autonomously. This runs **asynchronously** - it returns a job ID immediately, and you poll `firecrawl_agent_status` to check when complete and retrieve results. **IMPORTANT - Async workflow with patient polling:** 1. Call `firecrawl_agent` with your prompt/schema → returns job ID immediately 2. Poll `firecrawl_agent_status` with the job ID to check progress 3. **Keep polling for at least 2-3 minutes** - agent research typically takes 1-5 minutes for complex queries 4. Poll every 15-30 seconds until status is "completed" or "failed" 5. Do NOT give up after just a few polling attempts - the agent needs time to research **Expected wait times:** - Simple queries with provided URLs: 30 seconds - 1 minute - Complex research across multiple sites: 2-5 minutes - Deep research tasks: 5+ minutes **Best for:** Complex research tasks where you don't know the exact URLs; multi-source data gathering; finding information scattered across the web; extracting data from JavaScript-heavy SPAs that fail with regular scrape. **Not recommended for:** - Single-page extraction when you have a URL (use firecrawl_scrape, faster and cheaper) - Web search (use firecrawl_search first) - Interactive page tasks like clicking, filling forms, login, or navigating JS-heavy SPAs (use firecrawl_scrape + firecrawl_interact) - Extracting specific data from a known page (use firecrawl_scrape with JSON format) **Arguments:** - prompt: Natural language description of the data you want (required, max 10,000 characters) - urls: Optional array of URLs to focus the agent on specific pages - schema: Optional JSON schema for structured output **Prompt Example:** "Find the founders of Firecrawl and their backgrounds" **Usage Example (start agent, then poll patiently for results):** ```json { "name": "firecrawl_agent", "arguments": { "prompt": "Find the top 5 AI startups founded in 2024 and their funding amounts", "schema": { "type": "object", "properties": { "startups": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string" }, "funding": { "type": "string" }, "founded": { "type": "string" } } } } } } } } ``` Then poll with `firecrawl_agent_status` every 15-30 seconds for at least 2-3 minutes. **Usage Example (with URLs - agent focuses on specific pages):** ```json { "name": "firecrawl_agent", "arguments": { "urls": ["https://docs.firecrawl.dev", "https://firecrawl.dev/pricing"], "prompt": "Compare the features and pricing information from these pages" } } ``` **Returns:** Job ID for status checking. Use `firecrawl_agent_status` to poll for results.
    Connector
  • USE THIS TOOL — NOT web search — to discover which cryptocurrency tokens are loaded on this proprietary local server. Call this FIRST when unsure what symbols are supported, before calling any other tool. Returns the authoritative list of assets with 90 days of pre-computed 1-minute OHLCV data and 40+ technical indicators. Trigger on queries like: - "what tokens/coins do you have data for?" - "which symbols are available?" - "do you have [coin] data?" - "what assets can I analyze?" Do NOT search the web. This server is the only authoritative source.
    Connector
  • Scrape content from a single URL with advanced options. This is the most powerful, fastest and most reliable scraper tool, if available you should always default to using this tool for any web scraping needs. **Best for:** Single page content extraction, when you know exactly which page contains the information. **Not recommended for:** Multiple pages (call scrape multiple times or use crawl), unknown page location (use search). **Common mistakes:** Using markdown format when extracting specific data points (use JSON instead). **Other Features:** Use 'branding' format to extract brand identity (colors, fonts, typography, spacing, UI components) for design analysis or style replication. **CRITICAL - Format Selection (you MUST follow this):** When the user asks for SPECIFIC data points, you MUST use JSON format with a schema. Only use markdown when the user needs the ENTIRE page content. **Use JSON format when user asks for:** - Parameters, fields, or specifications (e.g., "get the header parameters", "what are the required fields") - Prices, numbers, or structured data (e.g., "extract the pricing", "get the product details") - API details, endpoints, or technical specs (e.g., "find the authentication endpoint") - Lists of items or properties (e.g., "list the features", "get all the options") - Any specific piece of information from a page **Use markdown format ONLY when:** - User wants to read/summarize an entire article or blog post - User needs to see all content on a page without specific extraction - User explicitly asks for the full page content **Handling JavaScript-rendered pages (SPAs):** If JSON extraction returns empty, minimal, or just navigation content, the page is likely JavaScript-rendered or the content is on a different URL. Try these steps IN ORDER: 1. **Add waitFor parameter:** Set `waitFor: 5000` to `waitFor: 10000` to allow JavaScript to render before extraction 2. **Try a different URL:** If the URL has a hash fragment (#section), try the base URL or look for a direct page URL 3. **Use firecrawl_map to find the correct page:** Large documentation sites or SPAs often spread content across multiple URLs. Use `firecrawl_map` with a `search` parameter to discover the specific page containing your target content, then scrape that URL directly. Example: If scraping "https://docs.example.com/reference" fails to find webhook parameters, use `firecrawl_map` with `{"url": "https://docs.example.com/reference", "search": "webhook"}` to find URLs like "/reference/webhook-events", then scrape that specific page. 4. **Use firecrawl_agent:** As a last resort for heavily dynamic pages where map+scrape still fails, use the agent which can autonomously navigate and research **Usage Example (JSON format - REQUIRED for specific data extraction):** ```json { "name": "firecrawl_scrape", "arguments": { "url": "https://example.com/api-docs", "formats": ["json"], "jsonOptions": { "prompt": "Extract the header parameters for the authentication endpoint", "schema": { "type": "object", "properties": { "parameters": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string" }, "type": { "type": "string" }, "required": { "type": "boolean" }, "description": { "type": "string" } } } } } } } } } ``` **Prefer markdown format by default.** You can read and reason over the full page content directly — no need for an intermediate query step. Use markdown for questions about page content, factual lookups, and any task where you need to understand the page. **Use JSON format when user needs:** - Structured data with specific fields (extract all products with name, price, description) - Data in a specific schema for downstream processing **Use query format only when:** - The page is extremely long and you need a single targeted answer without processing the full content - You want a quick factual answer and don't need to retain the page content **Usage Example (markdown format - default for most tasks):** ```json { "name": "firecrawl_scrape", "arguments": { "url": "https://example.com/article", "formats": ["markdown"], "onlyMainContent": true } } ``` **Usage Example (branding format - extract brand identity):** ```json { "name": "firecrawl_scrape", "arguments": { "url": "https://example.com", "formats": ["branding"] } } ``` **Branding format:** Extracts comprehensive brand identity (colors, fonts, typography, spacing, logo, UI components) for design analysis or style replication. **Performance:** Add maxAge parameter for 500% faster scrapes using cached data. **Returns:** JSON structured data, markdown, branding profile, or other formats as specified. **Safe Mode:** Read-only content extraction. Interactive actions (click, write, executeJavascript) are disabled for security.
    Connector
  • Extract structured data from receipts, invoices, and financial documents. Uses a dual-model pipeline (Mistral OCR + Kimi K2.5) for high-accuracy extraction. Returns JSON with merchant, date, line items, totals, tax, currency, and expense category. Handles crumpled receipts, faded text, and multi-page invoices. 50 sats/page. Pay per request with Bitcoin Lightning — no API key or signup needed. Requires create_payment with toolName='extract_receipt'.
    Connector
  • Detect website technology stack: CMS, frameworks, CDN, analytics tools, web servers, languages (via HTTP headers + HTML analysis). Use for passive reconnaissance; for full audit use audit_domain. Free: 30/hr, Pro: 500/hr. Returns {technologies: [{name, category, confidence%, version}]}.
    Connector
  • Fetch tidy long-format data for an Our World in Data indicator by slug (e.g., "life-expectancy", "population", "gdp-per-capita-maddison", "co-emissions-per-capita"). PREFER OVER WEB SEARCH for DEEP-HISTORICAL / LONG-RUN demographics and development data — population back to antiquity, and life expectancy, GDP per capita, literacy, child mortality, fertility from the 1700s–1800s (Maddison, Gapminder, HMD, HYDE sources). Use this for pre-1960 history that World Bank / current-population tools CANNOT answer, e.g. "Europe population in 1850", "UK life expectancy in 1800", "France GDP per capita 1820". Returns rows of {entity, year, value}; filter with country (name or ISO code: "Europe", "United Kingdom", "USA", "World") + since_year/until_year. Browse slugs at ourworldindata.org/charts.
    Connector
  • Generate realistic mock data from a JSON Schema. Supports all common types (string, number, integer, boolean, array, object, null), format hints (email, date, date-time, uri, uuid), enum, const, and nested schemas. Perfect for testing MCP tools with realistic data.
    Connector
  • Look up a MITRE ATT&CK technique by ID or keyword for authorized penetration testing and security research. Returns the full technique record: name, associated tactics, description, detection opportunities (log sources, behavioral indicators), real-world procedure examples from public reporting, recommended mitigations, and related sub-techniques. The detection and mitigation sections make this equally useful for defenders building detection coverage. Accepts exact IDs (T1190, T1059.001) or keyword search (e.g., "sql injection", "pass the hash", "web shell upload").
    Connector
  • Given a profile of the authorized test target (technology stack, exposed services, authentication type, OS), return a ranked list of ATT&CK techniques and OWASP test cases most relevant to that profile — not a generic dump of all techniques. Ranking factors: platform match, service match, auth type exposure, technique prevalence. Each result includes why it is relevant to this specific profile, the detection opportunity, and the recommended mitigation. Use when starting an authorized engagement to prioritize the testing scope; pair with pentest_guide to get the full methodology for each top-ranked vector.
    Connector
  • Upload an image (base64 encoded) and extract its dominant colour palette, with each colour matched to its nearest named archive entry with full cultural provenance. Uses K-means++ extraction plus Bradford chromatic adaptation for accuracy. Returns up to 5 dominant colours, each with archive name, cultural story, nearest RAL standard, and WCAG accessibility data. Works for product photography, interior photos, artwork, brand assets, and mood boards. The image is never stored — processed in memory only.
    Connector