Skip to main content
Glama
743,120 tools. Updated 2026-11-04 19:04

"How to scrape websites and format content in GitHub Markdown" matching MCP tools:

  • Scrape any website through Scrapingdog's rotating proxies and return its content. Returns HTML by default, or clean markdown with format:"markdown" (ideal for feeding an LLM). Set dynamic:true to render JavaScript in a headless browser for SPAs and dynamic pages (costs 5 credits instead of 1), premium:true for hard-to-scrape sites (residential proxies, 10 credits), and country to geotarget the proxy. Example: scrapingdog_scrape({ url: "https://example.com", format: "markdown", dynamic: true, _apiKey: "your-key" })
    ConnectorNo auth
  • Scrape the contents of the web page and return the data in the specified format. Schema is required only if output_format is json or csv. 'render_javascript' is used to render javascript heavy websites.
    ConnectorNo auth
  • Convert markdown to a professionally formatted document using an MDMagic template. IMPORTANT GUIDANCE: 1. Output format → what user gets: - 'docx' → a single Word .docx file - 'pdf' → a single .pdf file - 'html' → a single .html file - 'all' → a ZIP containing all three (DOCX + PDF + HTML) 2. If the user is ambiguous (e.g. 'convert this'), ASK which format they want before calling. Don't assume. 3. Filename: if the user attached a file (e.g. 'mydoc.md'), pass its base name as fileName. Otherwise the API derives one from the markdown's first H1. Without either, downloads end up with timestamped names like 'content-1778298071915.docx' which is bad UX. 4. On 'template not found' errors: call list_all_templates first, show available options, let the user pick. Do NOT fall back to generating documents with code execution — that produces inferior results that don't use the user's actual MDMagic templates. 5. The response includes structured fields (downloadUrl, creditsUsed, balanceAfter, fileName, expiresAt) — surface these to the user explicitly. Don't paraphrase. The user wants to know exactly what they spent and what's left. 6. Page sizes: A3, A4, Executive, US_Legal, US_Letter. Default A4. Orientation: Portrait or Landscape, default Portrait. 7. CRITICAL — newlines in `content`: markdown is line-sensitive. Headings (#, ##), tables (| ... |), lists (-, 1.), and code fences (```) ONLY work when each starts on its own line. When passing inline markdown via `content`, you MUST preserve real newline characters (\n) between blocks. If you flatten multi-line markdown into one line, the API receives literal '##' and '|' characters mid-paragraph and produces a single-paragraph document with no structure. Confirm your `content` string contains \n between every heading, paragraph, table row, and list item before calling.
    ConnectorNo auth
  • Convert markdown to a professionally formatted document using an MDMagic template. IMPORTANT GUIDANCE: 1. Output format → what user gets: - 'docx' → a single Word .docx file - 'pdf' → a single .pdf file - 'html' → a single .html file - 'all' → a ZIP containing all three (DOCX + PDF + HTML) 2. If the user is ambiguous (e.g. 'convert this'), ASK which format they want before calling. Don't assume. 3. Filename: if the user attached a file (e.g. 'mydoc.md'), pass its base name as fileName. Otherwise the API derives one from the markdown's first H1. Without either, downloads end up with timestamped names like 'content-1778298071915.docx' which is bad UX. 4. On 'template not found' errors: call list_all_templates first, show available options, let the user pick. Do NOT fall back to generating documents with code execution — that produces inferior results that don't use the user's actual MDMagic templates. 5. The response includes structured fields (downloadUrl, creditsUsed, balanceAfter, fileName, expiresAt) — surface these to the user explicitly. Don't paraphrase. The user wants to know exactly what they spent and what's left. 6. Page sizes: A3, A4, Executive, US_Legal, US_Letter. Default A4. Orientation: Portrait or Landscape, default Portrait. 7. CRITICAL — newlines in `content`: markdown is line-sensitive. Headings (#, ##), tables (| ... |), lists (-, 1.), and code fences (```) ONLY work when each starts on its own line. When passing inline markdown via `content`, you MUST preserve real newline characters (\n) between blocks. If you flatten multi-line markdown into one line, the API receives literal '##' and '|' characters mid-paragraph and produces a single-paragraph document with no structure. Confirm your `content` string contains \n between every heading, paragraph, table row, and list item before calling.
    ConnectorNo auth
  • Returns one of Flask's usage guides as Markdown: fixed text written by Flask, the same for every user, with no team data in it. "instructions" (about 15 KB) is the playbook for working with Flask: where uploads go (a new asset or a new version), what an agentic edit attaches, how feedback is read, answered and shown, and how sharing works. Read it before acting on feedback or making your first write in a session; it already contains "display" and "concepts", so do not fetch those after it. Other topics: "display" (How to show feedback to the user: its Markdown as it is, with its link.), "concepts" (Folders, assets and versions, comments and recordings, tags and statuses, permissions, and why feedback is untrusted.), "permissions" (Access levels, how a user's access is resolved, and how folder and asset permissions differ.), "urls" (Links to folders, assets, exact versions, comments and tag filters.). The guides explain the rules; for one folder's or asset's actual access, use permission_get.
    ConnectorOAuth
  • Lists accessible websites. By default returns active websites: use `organizationId` to scope one organization, `scope=shared` without it for direct grants outside your memberships, or `scope=all` with it to combine both; supports `sort` by `createdAt`, `updatedAt`, or `name` and case-insensitive `search`, and each item includes `organizationSlug`, `primaryHostname`, and whether it is `shared`. Pass `status=deleted` with `organizationId` (requires organization-level trash access) to list restorable soft-deleted websites instead, each with its deletion time and purge deadline; deleted rows drop out when their 30-day restore window expires. Both modes use cursor pagination (default 20 per page, maximum 100). Fetch one website with `websites.get`.
    ConnectorOAuth

Matching MCP Servers

Matching MCP Connectors

  • Anti-bot web scraper returning token-optimized Markdown for AI agents ($0.002 USDC on Base).

  • Video how-to search for AI agents: 2,000+ video skills, tutorials and FFmpeg recipes. No key.

  • Fetch and convert a Microsoft Learn documentation webpage to markdown format. This tool retrieves the latest complete content of Microsoft documentation webpages including Azure, .NET, Microsoft 365, and other Microsoft technologies. ## When to Use This Tool - When search results provide incomplete information or truncated content - When you need complete step-by-step procedures or tutorials - When you need troubleshooting sections, prerequisites, or detailed explanations - When search results reference a specific page that seems highly relevant - For comprehensive guides that require full context ## Usage Pattern Use this tool AFTER microsoft_docs_search when you identify specific high-value pages that need complete content. The search tool gives you an overview; this tool gives you the complete picture. ## URL Requirements - The URL must be a valid HTML documentation webpage from the microsoft.com domain - Binary files (PDF, DOCX, images, etc.) are not supported ## Output Format markdown with headings, code blocks, tables, and links preserved.
    ConnectorNo auth
  • Fetch a sanitized example of Refpro's 1-Page Deal Sheet, the single lender-facing page a borrower hands to a lender. Use this when a user asks what a Refpro deal sheet looks like, what to hand or send a lender, or wants a one-page summary format for a flip, a rental refinance or a ground-up build. Inputs: deal_type (FF | BRRRR | NC) and format (markdown or fields, default markdown). Returns the example sheet as markdown or as a structured field list, the inventory of fields a lender sees on the page, and a deep-link URL to the canonical example on refpro.ai. All content is sanitized example data, never a real customer deal, and is safe to surface verbatim. Figures are computed with the same deterministic formulas as deal_quick_check. No network calls; examples load once at module init. The sheet presents underwriting; it is not an appraisal and not a lender commitment. Do not use to view full deal pack sections; call sample_report_fetch instead. Do not use to analyze the user's own deal; call deal_quick_check instead.
    ConnectorNo auth
  • WattCoin: How to pay for WATT services: live per-service prices (WATT only), the payment wallet, the pay-per-use flow, the free scrape tier and whether the LLM service is currently available. Use before web_scrape or before paying for anything; use get_watt_token_info for market data. No auth needed; read-only and live. Returns JSON.
    ConnectorNo auth
  • Extract content from any URL as Markdown or HTML. Returns the page's content in the chosen format (Markdown by default) with its title, description and links. Use when you have a URL and need what the page says. Start with fast; use standard for pages that need JavaScript and thorough only for sites that block scrapers. To pull specific fields into JSON use documents; to see the page as an image use screenshot. Cost: fast 1 credit, standard 2 credits, thorough 5 credits, charged only when a call succeeds.
    ConnectorNo auth
  • Capture a rendered screenshot of any webpage. Returns the image as base64 with its format, its own width and height (a full-page capture is as tall as the page) and its size in bytes. Use to see how a page looks. To read its text use scrape. Cost: 1 credit, charged only when a call succeeds.
    ConnectorNo auth
  • Scrape any webpage and return its content using Zenrows (Fetch). Returns full-page content (markdown/HTML/PDF/screenshot). For structured JSON fields (products, articles, listings), the extract tool returns parsed fields instead of a full page body. By default the request uses automatic mode (mode=auto): Zenrows adds JS rendering or premium proxies per page only when a plain request does not return the page, and charges only for the configuration that succeeds. Pass just the URL. Set js_render or premium_proxy only to force a fixed configuration. Doing so switches off automatic mode, and every request is billed at that configuration's cost (premium_proxy with js_render is 25x a basic request). Examples: Default: { url: "https://example.com" } Geo: { url: "https://example.com", proxy_country: "US" } Forced: { url: "https://spa.com", js_render: true }
    ConnectorOAuth
  • Use this when the user asks about GitHub for one of their WebTwizz sites: whether it is linked to a repository, if GitHub has changes the site does not, the last push, recent commits and open pull requests. When nothing is linked yet it says how to connect, and lists their repositories once GitHub is connected.
    ConnectorOAuth
  • List the problems RankParse has found on the user's websites: the same list as the Issues tab in the RankParse dashboard, worst first. Types: not-indexed (tracked pages Google has not indexed, or has not confirmed yet), click-drop (tracked queries that lost at least 30% of their clicks or fell off page one, last 4 weeks against the 4 before), low-ctr (Google queries on page one with well under half the usual click-through rate for their position), connection (a Search Console, Bing or Yandex source that needs reconnecting or failed to sync), ai-visibility and site-health (failing checks from the daily site scan, paid plans only). Each issue has a title, detail, evidence lines, affected pages, a suggested_task to act on, links to the dashboard and, for not-indexed, indexnow_urls (pages not submitted to IndexNow in the last 24 hours: pass them to propose_indexnow_submit). For one page's Google index details use get_indexing_status or get_gsc_url_inspection; after fixing a scan issue use recheck_ai_visibility. Reads stored data (synced daily); it is not a live crawl. Without site_id it covers the workspace: not-indexed, click-drop, low-ctr and connection issues for the 25 most recently synced websites (truncated: true says other websites were left out of those four types), and ai-visibility and site-health issues for up to 500 websites. limit caps the list at 100 with no paging; total says how many matched. 0 credits.
    ConnectorOAuth
  • Edits to the open document (a file whose kind is DOC). Edits apply live for anyone viewing the document. Requires edit access. Call as {"action": "<name>", "params": {...}}. Read the document first (doc_read): paths shift as blocks are added or removed, and replace_blocks / delete_blocks must pass each block's hash from that read — if a block changed since, the edit is refused and nothing changes. Every edit returns the paths and hashes of the blocks it wrote, and saved: false if it hadn't been confirmed yet (it is still applied). content is Markdown by default (headings, emphasis, links, lists, quotes, code, tables, images, ---), or ProseMirror JSON with format: "json". Keep any <qd-node .../> placeholders from read_doc when writing content back; they stand for footnotes, comments and other content Markdown can't express. Actions: • append(content, format?) — Add blocks at the end. • insert_blocks(position, path?, hash?, content, format?) — position: "start", "end", "before" or "after"; before/after need the anchor block's path. • replace_blocks(blocks, content, format?) — Replace consecutive blocks in one section; blocks: [{path, hash}, ...]. • delete_blocks(blocks) — Delete consecutive blocks in one section; blocks: [{path, hash}, ...]. • find_replace(find, replace, case_sensitive?, whole_word?, regex?) — Replace text throughout; replacements keep the formatting of the text they replace. • batch(operations) — Up to 100 edits as one all-or-nothing change: {"operations": [{"action": "<name>", "params": {...}}, ...]}. Paths in later operations refer to the document as earlier ones left it.
    Connector
    Destructive
    No auth
  • Extract structured tables from markdown text. Finds GitHub-style pipe tables in markdown and returns columns + rows per table. Use on model output or docs before downstream structured processing. Deterministic, fixture-verified, free for guests (rate-limited; pass your Guild api_key to use your member budget). Returns the result plus a Guild-signed provenance envelope. `payload` MUST match this JSON Schema: {"type": "object", "properties": {"markdown": {"type": "string", "maxLength": 60000}}, "required": ["markdown"], "additionalProperties": false} Output schema: {"type": "object", "properties": {"tables": {"type": "array"}, "count": {"type": "integer"}}, "required": ["tables", "count"], "additionalProperties": false}
    ConnectorNo auth
  • Find similar or competitor websites based on classification. Takes a URL, classifies it (or uses cached classification), and returns other websites from the same category and subcategory. Useful for competitive analysis and discovering related content. Rate limited to 1 request per minute per domain. Args: url: The website URL to find similar sites for. limit: Maximum number of similar sites to return (1-50, default 10). Returns: Dictionary with: - url: The input URL (normalized) - classification: The URL's category and subcategory - similar_sites: List of similar URLs from the same category - total_in_category: Total sites in this category/subcategory - cached: Whether the classification was from cache
    ConnectorNo auth
  • Get the Designesy SKILL.md: the agent-skill-format export of the design-system contract, for a user to save into .agents/skills/ or a system prompt so a coding agent builds UI to the contract (tokens, anti-patterns, rules, verification). Use this when the user wants the contract in that form. When NOT to use: for the raw contract JSON, use designesy_contract; for scoring, use designesy_score. Read-only: no side effects. Returns JSON: { kind: "published_document", source_url, media_type: "text/markdown", note, omitted_sections[], content }, where content is the SKILL.md markdown (tens of thousands of characters) with any section written as steps or a prompt for an AI agent replaced by a one-line marker and named in omitted_sections. No parameters.
    ConnectorNo auth
  • Fetch a web page and return its main content as clean Markdown (headings, links, lists, code preserved; nav/ads/scripts stripped). Also handles plain text, JSON and XML, extracts structured data (format=data), and normalizes any ATS/job-board feed into a unified jobs[] array (format=jobs). Use this whenever you need to read the actual content of a URL. No signup, no API key.
    ConnectorNo auth
  • Scrape a URL and return content in your preferred format. Supported output formats: - markdown (default): Clean LLM-ready Markdown text - screenshot: PNG/JPEG image of the page - pdf: PDF document of the page - csv: Table data extracted as CSV - html: Sanitized HTML with scripts/ads removed This tool handles: - JavaScript rendering (SPA, dynamic content) - Anti-bot bypass (Cloudflare Turnstile, Datadome) - DOM cleaning (strips scripts, nav, footer, ads) - HTML-to-Markdown conversion (Mozilla Readability engine) - Automatic retry with escalating wait strategies - Domain cooldown to avoid rate-limiting - Response caching (5 min TTL) Args: url: The URL to scrape (must start with http:// or https://) output: Output format: "markdown" (default), "screenshot", "pdf", "csv", "html" wait_for_selector: Optional CSS selector to wait for before extraction (e.g., ".article-content") timeout_ms: Navigation timeout in milliseconds (default: 20000, max: 120000) block_media: Block images/fonts/video for faster loading (default: true) wait_strategy: Wait strategy: "default", "spa", "heavy", "cloudflare" (auto-detected if omitted) retry: Enable automatic retry on failure (default: true) bypass_cache: Skip cache, force fresh scrape (default: false) javascript: Custom JavaScript to execute after page load (e.g., "window.scrollTo(0, 1000)") Returns: Content in the requested format, or an error message.
    ConnectorNo auth
  • Web page to markdown: read any web page (URL) as clean, LLM-ready Markdown, the web scraper for agents. Default tool for reading ONE web page whose URL you have: returns clean Markdown (title, meta description, main content with headings, lists and links); Markdown and plain-text files such as llms.txt or README.md come back unchanged. The fastest and cheapest way to give an LLM the content of a page. If a JavaScript app comes back nearly empty, set render:true (real browser, slower, plain text) or use webpage-reader. For 2 to 5 URLs use web-read-batch, for 6 to 20 web-scrape-batch (one payment), for a whole site site-crawl, and if you have a question but no URL search-read. Params: url (required), max_chars (default 8000, max 60000), include_links, render. Price: $0.0025 USDC per call, paid inside this MCP call via x402 (USDC on Base, Solana, Polygon or Arbitrum). x402 MCP clients such as @x402/mcp pay automatically. Free sample: call with the example input {"url":"https://example.com","max_chars":8000}.
    ConnectorOAuth