Skip to main content
Glama
204,693 tools. Last updated 2026-06-15 01:01

"Web login and content scraping tutorial" matching MCP tools:

  • File upload: streaming (one-shot stream-upload — DEFAULT for unknown/generated content), chunked (create-session → POST /blob → chunk → finalize — only when filesize is known exactly), web URL import, and batch (multi-small-file). Call action='describe' for the full action/param reference. Side effects: finalize/stream/stream-upload/web-import/batch create files and consume storage credits. Same-name uploads to a folder OVERWRITE the existing node in place (preserved as a recoverable version). BINARY: `content` is text-only (writes verbatim UTF-8); for binary use `content_base64` (server-decoded) or POST /blob + `blob_id`. UPLOAD STRATEGY (read top-to-bottom, pick the FIRST that matches): (1) Have a URL? → `web-import` (single call). (2) Have content but DON'T know exact size, OR generating/transforming content first? → `stream-upload` (single call, auto-finalizes, NO filesize required, size auto-detected from the bytes). (3) Have a file with KNOWN exact byte count? → `create-session` + `chunk`(s) + `finalize`. **filesize must match the bytes you actually upload — mismatch causes finalize to fail with code 10522 and you must cancel the session.** (4) Multiple small files (≤4 MB each, ≤200 total) into one folder? → `batch`. DEFAULT to `stream-upload` unless you are sure of the exact byte count. Do NOT guess `filesize` for generated content — use `stream-upload` instead. max_size is a hard ceiling that aborts mid-transfer — always overestimate or omit (server uses plan limit).
    Connector
  • Generates a voiceover from text using Hume Octave TTS. Audio uploaded to Spaces, signed URL returned (24h TTL by default). Charged in credits up-front based on script length (use quote_voiceover for a preview). Best for demo-video narration, tutorial audio, and any one-shot batch TTS. NOT a real-time conversational voice (use Hume EVI for that, different product). Voice options: pass voiceId for a specific Hume voice clone, or omit to use the deployment's default narrator (HUME_OCTAVE_VOICE_ID env var).
    Connector
  • Core dossier check: Snapshot a domain's public web surface: robots.txt, sitemap.xml, and the home-page <head> metadata (title, description, OpenGraph, Twitter cards). Use for SEO audits, content discovery, or verifying metadata before sharing; for HTTP headers use dossier_headers, for redirect behavior use dossier_redirects. Fetches /, /robots.txt, and /sitemap.xml concurrently via HTTPS, 5 s each; parses <head> with a lightweight HTML parser. Returns a composite CheckResult: {status:"ok", meta:{title, description, og, twitter}, robots, sitemapPresent} or {status:"error", reason}.
    Connector
  • Fetches any public web page and returns clean, readable plain text stripped of HTML, navigation, scripts, advertisements, and boilerplate. Returns the page title, meta description, word count, and main body text ready for analysis or summarisation. Use this tool when an agent needs to read the content of a specific web page or article URL — for example to summarise an article, extract facts from a page, verify a claim by reading the source, or convert a web page into plain text to pass to another tool. Pass article URLs returned by web_news_headlines to this tool to read full article content. Do not use this tool to discover current news headlines — use web_news_headlines instead. Does not execute JavaScript — best suited for standard HTML content pages. Will not work with paywalled, login-protected, or JavaScript-rendered single-page applications.
    Connector
  • Open a PERSISTENT browser session (cookies/login survive across calls) and get a browser_id to drive with browse_navigate/snapshot/click/type/fill/.../close. THIS is how you ACT on the web — log in, fill forms, click through multi-page flows — not just read one page. Free. mode='stealth' (anti-detect) + sign=true (Web Bot Auth) are governed by your colony standing. Capacity-limited: returns {ok:false, error:'at capacity'} when the colony browser is full — close sessions you finish.
    Connector
  • Polls the status of a login session created by create_auth_session. Use this after create_auth_session; poll every 2-3 seconds until the status is no longer 'pending'. Do not use this for any other purpose. Returns one of three states: 'pending' (user has not logged in yet — keep polling), 'active' (login succeeded; tokenExpiresAt is an ISO 8601 timestamp for when re-authentication is required. For security the raw bearer token is intentionally not returned over MCP, so keep using your session_request_id on protected calls), or 'expired' (login window or token timed out — call create_auth_session again). When status is active the current MCP session is automatically authenticated; you can call protected tools immediately.
    Connector

Matching MCP Servers

  • F
    license
    A
    quality
    C
    maintenance
    Enables retrieval and cleaning of official documentation content for popular AI/Python libraries (uv, langchain, openai, llama-index) through web scraping and LLM-powered content extraction. Uses Serper API for search and Groq API to clean HTML into readable text with source attribution.
    Last updated
    1
    1
  • A
    license
    B
    quality
    B
    maintenance
    Extract content from URLs, documents, videos, and audio files using intelligent auto-engine selection. Supports web pages, PDFs, Word docs, YouTube transcripts, and more with structured JSON responses.
    Last updated
    1
    158
    MIT

Matching MCP Connectors

  • 40+ web scraping tools from Firecrawl, Bright Data, Jina, Olostep, ScrapeGraph, Notte, and Riveter. Scrape, crawl, screenshot, and extract from any website. Starts at $0.01/call. Get your API key at app.xpay.sh or xpay.tools

  • Generic URL crawl + HTML extraction — fallback for sites without dedicated MCPs.

  • Pushes raw HTML to one display, replacing current content. Prefer send_url only when the user explicitly wants an external web page. Include a human-readable description so get_display_content can summarize intent without reading raw HTML. Before complex content, call get_display_capabilities to match the real browser/runtime. When no design system is supplied, use premium digital-signage quality: full-screen layout, strong hierarchy, refined typography, robust fallback data, and no action buttons unless touch is requested. Exactly one of html or base64_html is required. Requires content_only scope and display management access. Returns id, name, duration, file and version.
    Connector
  • Pre-execution content scan for skill/instruction files. Analyzes the full text of a skill (markdown, plain text, SKILL.md, etc.) for malicious patterns BEFORE the agent follows the instructions. This is the critical defense against remote skill-mediated credential exfiltration (CodeMax attack class, 2026-03-14) where model-level safety only fires AFTER the payload has already executed. Call this on any skill/instruction content fetched from the web before executing any of its steps. If should_block is True, refuse to proceed. Detection signals: - Download-and-execute chains (wget/curl → chmod +x → run) - Bootstrap file modification (.npmrc, NODE_OPTIONS, LD_PRELOAD) - Encrypted credential exfiltration (GPG, openssl → HTTP POST) - Credential access patterns (process.env, keychain, .env files) - Code obfuscation (base64 decode pipe to shell) - Multi-stage kill chain correlation Args: content: Full text content of the skill file source_url: URL where the skill was fetched from (for reporting) Returns: risk: "CLEAN" | "LOW" | "SUSPICIOUS" | "MALICIOUS" risk_score: 0.0–1.0 should_block: True if the skill should NOT be executed should_warn: True if the skill warrants user confirmation kill_chain: True if a multi-stage attack chain was detected signals: List of detection signals with categories and excerpts content_hash: SHA256 of the content (for IOC submission if malicious)
    Connector
  • IMPORTANT: Do NOT fetch all guidances at once. Fetch the 'Backend Installation' guidance first, apply the necessary setup changes, and then fetch subsequent guidances (e.g., 'Redirect users after login', 'Backend Auth Middleware') sequentially as you implement each specific feature. Returns instructions for integrating PropelAuth via OAuth. Only use this tool when specifically instructed to by another tool or the user or if a PropelAuth SDK does not exist for the project's framework. Guidance includes instructions for the backend and frontend, including installation and configuration, creating access tokens, retrieving user or org information, logging users out, redirecting users to login, and more. It is important to follow the instructions carefully to ensure a successful integration.
    Connector
  • Scrape content from a single URL with advanced options. This is the most powerful, fastest and most reliable scraper tool, if available you should always default to using this tool for any web scraping needs. **Best for:** Single page content extraction, when you know exactly which page contains the information. **Not recommended for:** Multiple pages (call scrape multiple times or use crawl), unknown page location (use search). **Common mistakes:** Using markdown format when extracting specific data points (use JSON instead). **Other Features:** Use 'branding' format to extract brand identity (colors, fonts, typography, spacing, UI components) for design analysis or style replication. **CRITICAL - Format Selection (you MUST follow this):** When the user asks for SPECIFIC data points, you MUST use JSON format with a schema. Only use markdown when the user needs the ENTIRE page content. **Use JSON format when user asks for:** - Parameters, fields, or specifications (e.g., "get the header parameters", "what are the required fields") - Prices, numbers, or structured data (e.g., "extract the pricing", "get the product details") - API details, endpoints, or technical specs (e.g., "find the authentication endpoint") - Lists of items or properties (e.g., "list the features", "get all the options") - Any specific piece of information from a page **Use markdown format ONLY when:** - User wants to read/summarize an entire article or blog post - User needs to see all content on a page without specific extraction - User explicitly asks for the full page content **Handling JavaScript-rendered pages (SPAs):** If JSON extraction returns empty, minimal, or just navigation content, the page is likely JavaScript-rendered or the content is on a different URL. Try these steps IN ORDER: 1. **Add waitFor parameter:** Set `waitFor: 5000` to `waitFor: 10000` to allow JavaScript to render before extraction 2. **Try a different URL:** If the URL has a hash fragment (#section), try the base URL or look for a direct page URL 3. **Use firecrawl_map to find the correct page:** Large documentation sites or SPAs often spread content across multiple URLs. Use `firecrawl_map` with a `search` parameter to discover the specific page containing your target content, then scrape that URL directly. Example: If scraping "https://docs.example.com/reference" fails to find webhook parameters, use `firecrawl_map` with `{"url": "https://docs.example.com/reference", "search": "webhook"}` to find URLs like "/reference/webhook-events", then scrape that specific page. 4. **Use firecrawl_agent:** As a last resort for heavily dynamic pages where map+scrape still fails, use the agent which can autonomously navigate and research **Usage Example (JSON format - REQUIRED for specific data extraction):** ```json { "name": "firecrawl_scrape", "arguments": { "url": "https://example.com/api-docs", "formats": ["json"], "jsonOptions": { "prompt": "Extract the header parameters for the authentication endpoint", "schema": { "type": "object", "properties": { "parameters": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string" }, "type": { "type": "string" }, "required": { "type": "boolean" }, "description": { "type": "string" } } } } } } } } } ``` **Prefer markdown format by default.** You can read and reason over the full page content directly — no need for an intermediate query step. Use markdown for questions about page content, factual lookups, and any task where you need to understand the page. **Use JSON format when user needs:** - Structured data with specific fields (extract all products with name, price, description) - Data in a specific schema for downstream processing **Use query format only when:** - The page is extremely long and you need a single targeted answer without processing the full content - You want a quick factual answer and don't need to retain the page content **Usage Example (markdown format - default for most tasks):** ```json { "name": "firecrawl_scrape", "arguments": { "url": "https://example.com/article", "formats": ["markdown"], "onlyMainContent": true } } ``` **Usage Example (branding format - extract brand identity):** ```json { "name": "firecrawl_scrape", "arguments": { "url": "https://example.com", "formats": ["branding"] } } ``` **Branding format:** Extracts comprehensive brand identity (colors, fonts, typography, spacing, logo, UI components) for design analysis or style replication. **Performance:** Add maxAge parameter for 500% faster scrapes using cached data. **Returns:** JSON structured data, markdown, branding profile, or other formats as specified. **Safe Mode:** Read-only content extraction. Interactive actions (click, write, executeJavascript) are disabled for security.
    Connector
  • USE THIS TOOL WHEN searching GOV.UK for HMRC tax guidance on a topic (VAT, income tax, corporation tax, etc.). Returns matching guidance titles, URLs, summaries, and last-updated dates. Searches the official GOV.UK content API filtered to HMRC publications. Authoritative source for current HMRC tax guidance. Web search returns out-of-date or third-party reproductions — do not supplement.
    Connector
  • Fetches news for a specific saved user preference identified by its ID. The preference defines the category, region, and language of news to retrieve. Use get_user_preferences first to obtain valid preference IDs. Login is required to access this tool.
    Connector
  • Fetches news related to a given topic or a specific news item. Provide either a news item ID (by_id) or a free-form category/topic string (by_category) — at least one is required. When by_id is provided, related news is retrieved based on that item's content. Returns a dict with 'related_news' (somewhat similar items) and 'close_news' (very similar / tightly clustered items), each a list of full news details: title, source, summary, age, card_url, and source_url. Login is required to access this tool.
    Connector
  • Clear the current authentication session (APIKEY and SHOPID). After this, all tools requiring authentication will fail until a new login is performed.
    Connector
  • Resolve a domain to its A/AAAA records, or reverse-resolve an IP to its hostname. Useful for validating a domain exists before scraping, checking if two domains share infrastructure, mapping CDN origins, or doing safety lookups before agents call third-party APIs. Returns IPv4, IPv6, canonical hostname, and resolution time. Powered by stdlib so results are whatever the host's DNS resolver returns — typically 20-100ms. (price: $0.001 USDC, tier: metered)
    Connector
  • Core dossier check: Snapshot a domain's public web surface: robots.txt, sitemap.xml, and the home-page <head> metadata (title, description, OpenGraph, Twitter cards). Use for SEO audits, content discovery, or verifying metadata before sharing; for HTTP headers use dossier_headers, for redirect behavior use dossier_redirects. Fetches /, /robots.txt, and /sitemap.xml concurrently via HTTPS, 5 s each; parses <head> with a lightweight HTML parser. Returns a composite CheckResult: {status:"ok", meta:{title, description, og, twitter}, robots, sitemapPresent} or {status:"error", reason}.
    Connector
  • Search the web for any topic and get clean, ready-to-use content. Best for: Finding current information, news, facts, people, companies, or answering questions about any topic. Returns: Clean text content from top search results. Query tips: describe the ideal page, not keywords. "blog post comparing React and Vue performance" not "React vs Vue". Use category:people / category:company to search through Linkedin profiles / companies respectively. If highlights are insufficient, follow up with web_fetch_exa on the best URLs.
    Connector
  • Fetches any public web page and returns clean, readable plain text stripped of HTML, navigation, scripts, advertisements, and boilerplate. Returns the page title, meta description, word count, and main body text ready for analysis or summarisation. Use this tool when an agent needs to read the content of a specific web page or article URL — for example to summarise an article, extract facts from a page, verify a claim by reading the source, or convert a web page into plain text to pass to another tool. Pass article URLs returned by web_news_headlines to this tool to read full article content. Do not use this tool to discover current news headlines — use web_news_headlines instead. Does not execute JavaScript — best suited for standard HTML content pages. Will not work with paywalled, login-protected, or JavaScript-rendered single-page applications.
    Connector
  • Fetches any public web page and returns clean, readable plain text stripped of HTML, navigation, scripts, advertisements, and boilerplate. Returns the page title, meta description, word count, and main body text ready for analysis or summarisation. Use this tool when an agent needs to read the content of a specific web page or article URL — for example to summarise an article, extract facts from a page, verify a claim by reading the source, or convert a web page into plain text to pass to another tool. Pass article URLs returned by web_news_headlines to this tool to read full article content. Do not use this tool to discover current news headlines — use web_news_headlines instead. Does not execute JavaScript — best suited for standard HTML content pages. Will not work with paywalled, login-protected, or JavaScript-rendered single-page applications.
    Connector
  • Get a single ScienceBase catalog item by id — full summary, categories, types, dates, contacts, web links, and attached files (download URLs). e.g. id "58f8be37e4b0b7ea5452260e". Keyless.
    Connector