Skip to main content
Glama
510,000 tools. Updated 2026-09-03 14:58

"How to scrape websites and format content in GitHub Markdown" matching MCP tools:

  • Scrape any website through Scrapingdog's rotating proxies and return its content. Returns HTML by default, or clean markdown with format:"markdown" (ideal for feeding an LLM). Set dynamic:true to render JavaScript in a headless browser for SPAs and dynamic pages (costs 5 credits instead of 1), premium:true for hard-to-scrape sites (residential proxies, 10 credits), and country to geotarget the proxy. Example: scrapingdog_scrape({ url: "https://example.com", format: "markdown", dynamic: true, _apiKey: "your-key" })
    Connector
  • Scrape the contents of the web page and return the data in the specified format. Schema is required only if output_format is json or csv. 'render_javascript' is used to render javascript heavy websites.
    Connector
  • Convert markdown to a professionally formatted document using an MDMagic template. IMPORTANT GUIDANCE: 1. Output format → what user gets: - 'docx' → a single Word .docx file - 'pdf' → a single .pdf file - 'html' → a single .html file - 'all' → a ZIP containing all three (DOCX + PDF + HTML) 2. If the user is ambiguous (e.g. 'convert this'), ASK which format they want before calling. Don't assume. 3. Filename: if the user attached a file (e.g. 'mydoc.md'), pass its base name as fileName. Otherwise the API derives one from the markdown's first H1. Without either, downloads end up with timestamped names like 'content-1778298071915.docx' which is bad UX. 4. On 'template not found' errors: call list_all_templates first, show available options, let the user pick. Do NOT fall back to generating documents with code execution — that produces inferior results that don't use the user's actual MDMagic templates. 5. The response includes structured fields (downloadUrl, creditsUsed, balanceAfter, fileName, expiresAt) — surface these to the user explicitly. Don't paraphrase. The user wants to know exactly what they spent and what's left. 6. Page sizes: A3, A4, Executive, US_Legal, US_Letter. Default A4. Orientation: Portrait or Landscape, default Portrait. 7. CRITICAL — newlines in `content`: markdown is line-sensitive. Headings (#, ##), tables (| ... |), lists (-, 1.), and code fences (```) ONLY work when each starts on its own line. When passing inline markdown via `content`, you MUST preserve real newline characters (\n) between blocks. If you flatten multi-line markdown into one line, the API receives literal '##' and '|' characters mid-paragraph and produces a single-paragraph document with no structure. Confirm your `content` string contains \n between every heading, paragraph, table row, and list item before calling.
    Connector
  • Read a workspace's doc (TipTap rich-text) body. Format is negotiable via `format`: `markdown` (default — CommonMark + GFM, ready to feed to an LLM or render in a non-ProseMirror surface), `content` (TipTap JSON, round-trippable into update_doc for structural edits), `text` (plain text, best for search, summarisation, word-count heuristics), or `all` for the legacy three-in-one shape. Default is `markdown` because it's the slice agents need 95% of the time and the JSON form on a long doc can blow past the agent harness's tool-result token cap. Pass `format: "content"` only when you're round-tripping into update_doc for a structural edit. A workspace can hold any combination of doc and table surfaces, one or many of either kind; omit `surface_slug` to read the primary doc surface, or pass it to target a specific doc tab (use `list_surfaces` to enumerate). An unwritten or absent doc returns the requested format empty (markdown="", content={}, text=""); a `surface_slug` that doesn't match any live doc surface 404s.
    Connector
  • Convert markdown to a professionally formatted document using an MDMagic template. IMPORTANT GUIDANCE: 1. Output format → what user gets: - 'docx' → a single Word .docx file - 'pdf' → a single .pdf file - 'html' → a single .html file - 'all' → a ZIP containing all three (DOCX + PDF + HTML) 2. If the user is ambiguous (e.g. 'convert this'), ASK which format they want before calling. Don't assume. 3. Filename: if the user attached a file (e.g. 'mydoc.md'), pass its base name as fileName. Otherwise the API derives one from the markdown's first H1. Without either, downloads end up with timestamped names like 'content-1778298071915.docx' which is bad UX. 4. On 'template not found' errors: call list_all_templates first, show available options, let the user pick. Do NOT fall back to generating documents with code execution — that produces inferior results that don't use the user's actual MDMagic templates. 5. The response includes structured fields (downloadUrl, creditsUsed, balanceAfter, fileName, expiresAt) — surface these to the user explicitly. Don't paraphrase. The user wants to know exactly what they spent and what's left. 6. Page sizes: A3, A4, Executive, US_Legal, US_Letter. Default A4. Orientation: Portrait or Landscape, default Portrait. 7. CRITICAL — newlines in `content`: markdown is line-sensitive. Headings (#, ##), tables (| ... |), lists (-, 1.), and code fences (```) ONLY work when each starts on its own line. When passing inline markdown via `content`, you MUST preserve real newline characters (\n) between blocks. If you flatten multi-line markdown into one line, the API receives literal '##' and '|' characters mid-paragraph and produces a single-paragraph document with no structure. Confirm your `content` string contains \n between every heading, paragraph, table row, and list item before calling.
    Connector
  • Fetch and convert a Microsoft Learn documentation webpage to markdown format. This tool retrieves the latest complete content of Microsoft documentation webpages including Azure, .NET, Microsoft 365, and other Microsoft technologies. ## When to Use This Tool - When search results provide incomplete information or truncated content - When you need complete step-by-step procedures or tutorials - When you need troubleshooting sections, prerequisites, or detailed explanations - When search results reference a specific page that seems highly relevant - For comprehensive guides that require full context ## Usage Pattern Use this tool AFTER microsoft_docs_search when you identify specific high-value pages that need complete content. The search tool gives you an overview; this tool gives you the complete picture. ## URL Requirements - The URL must be a valid HTML documentation webpage from the microsoft.com domain - Binary files (PDF, DOCX, images, etc.) are not supported ## Output Format markdown with headings, code blocks, tables, and links preserved.
    Connector

Matching MCP Servers

Matching MCP Connectors

  • Transform any blog post or article URL into ready-to-post social media content for Twitter/X threads, LinkedIn posts, Instagram captions, Facebook posts, and email newsletters. Pay-per-event: $0.07 for all 5 platforms, $0.03 for single platform.

  • Markdown utilities MCP.

  • Extract structured tables from markdown text. Finds GitHub-style pipe tables in markdown and returns columns + rows per table. Use on model output or docs before downstream structured processing. Deterministic, fixture-verified, free for guests (rate-limited; pass your Guild api_key to use your member budget). Returns the result plus a Guild-signed provenance envelope. `payload` MUST match this JSON Schema: {"type": "object", "properties": {"markdown": {"type": "string", "maxLength": 60000}}, "required": ["markdown"], "additionalProperties": false} Output schema: {"type": "object", "properties": {"tables": {"type": "array"}, "count": {"type": "integer"}}, "required": ["tables", "count"], "additionalProperties": false}
    Connector
  • Find similar or competitor websites based on classification. Takes a URL, classifies it (or uses cached classification), and returns other websites from the same category and subcategory. Useful for competitive analysis and discovering related content. Rate limited to 1 request per minute per domain. Args: url: The website URL to find similar sites for. limit: Maximum number of similar sites to return (1-50, default 10). Returns: Dictionary with: - url: The input URL (normalized) - classification: The URL's category and subcategory - similar_sites: List of similar URLs from the same category - total_in_category: Total sites in this category/subcategory - cached: Whether the classification was from cache
    Connector
  • Fetch a sanitized public sample section from Refpro's reference deal library. Inputs: deal_type (FF | BRRRR | NC) and section (summary | financials | risk_notes | full). Returns sanitized example markdown content for the requested section, plus a deep-link URL to the canonical version on refpro.ai. The 'full' section stitches summary, financials, and risk_notes in order. All content is sanitized example data — not a real customer deal — and is safe to surface verbatim to end users. No network calls; samples are loaded once at module init.
    Connector
  • Get the Designesy SKILL.md — the agent-skill-format export of the design-system contract, written as behavioral rules an AI coding agent can drop into .agents/skills/ or a system prompt. Use this when you want the contract in a form that steers how an agent *builds* UI (tokens, anti-patterns, behavioral rules, verification). When NOT to use: for the raw contract JSON, use designesy_contract; for scoring, use designesy_score. Read-only — no side effects. Returns markdown text (SKILL.md format) — drop into .agents/skills/ or paste into a system prompt. No parameters.
    Connector
  • Fetch a web page and return its main content as clean Markdown (headings, links, lists, code preserved; nav/ads/scripts stripped). Also handles plain text, JSON and XML, extracts structured data (format=data), and normalizes any ATS/job-board feed into a unified jobs[] array (format=jobs). Use this whenever you need to read the actual content of a URL. No signup, no API key.
    Connector
  • Scrape a URL and return content in your preferred format. Supported output formats: - markdown (default): Clean LLM-ready Markdown text - screenshot: PNG/JPEG image of the page - pdf: PDF document of the page - csv: Table data extracted as CSV - html: Sanitized HTML with scripts/ads removed This tool handles: - JavaScript rendering (SPA, dynamic content) - Anti-bot bypass (Cloudflare Turnstile, Datadome) - DOM cleaning (strips scripts, nav, footer, ads) - HTML-to-Markdown conversion (Mozilla Readability engine) - Automatic retry with escalating wait strategies - Domain cooldown to avoid rate-limiting - Response caching (5 min TTL) Args: url: The URL to scrape (must start with http:// or https://) output: Output format: "markdown" (default), "screenshot", "pdf", "csv", "html" wait_for_selector: Optional CSS selector to wait for before extraction (e.g., ".article-content") timeout_ms: Navigation timeout in milliseconds (default: 20000, max: 120000) block_media: Block images/fonts/video for faster loading (default: true) wait_strategy: Wait strategy: "default", "spa", "heavy", "cloudflare" (auto-detected if omitted) retry: Enable automatic retry on failure (default: true) bypass_cache: Skip cache, force fresh scrape (default: false) javascript: Custom JavaScript to execute after page load (e.g., "window.scrollTo(0, 1000)") Returns: Content in the requested format, or an error message.
    Connector
  • Submit a clinical encounter letter to the Appendix physician team for review. The letter should be in Markdown format following the Appendix letter structure. Returns feedback on completeness or a checkout URL when ready. One of our board-certified physicians will personally review the submission and provide clinical guidance and a prescription if appropriate.
    Connector
  • [SDK Docs] Fetch the full markdown content of a specific documentation page from Docs. Use this when you have a page URL and want to read its content. Accepts full URLs (e.g. https://docs.sodax.com//getting-started). Since `searchDocumentation` returns partial content, use `getPage` to retrieve the complete page when you need more details. The content includes links you can follow to navigate to related pages.
    Connector
  • Return the complete UploadKit quickstart walkthrough for Next.js — install, API key env, route handler, provider, first component, optional BYOS — in one markdown document. When to use: the user is brand new to UploadKit and asks "how do I get started?", "set this up for me", or any variation that signals zero prior context. Prefer scaffold_route_handler + scaffold_provider + get_install_command when you already know which specific step they need. Returns: a plain-text markdown document. Takes no parameters. Read-only, static content, idempotent.
    Connector
  • Search the web Search the web. Two modes governed by ``scrapeOptions``. - **Omit ``scrapeOptions``** → SERP-only: returns the search engine's raw snippets (``url`` + ``meta`` with ``title`` / ``description`` / ``source`` / ``publishedAt`` / ``imageUrl*``). No per-page fetch, fast and cheap. - **Pass ``scrapeOptions: {}``** → deep-scrape every result, return page-faithful Markdown under ``markdown``. - **Pass ``scrapeOptions: {"format": "json"}``** → deep-scrape every result, return the structured page summary under ``json`` (same shape as ``/webtools/scrape``'s ``json`` field). In deep-scrape mode, results where the chosen format produced no content are dropped from the response, so the response may hold fewer than ``limit`` results. ``meta.statusCode`` carries the fetched page's HTTP status when deep-scraped. ``query`` is compatible with common Google search-operator syntax: ``site:``, ``intitle:``, ``filetype:``, ``"exact phrase"``, ``-exclude``. To filter by whole domains, prefer the structured ``includeDomains`` / ``excludeDomains`` — they are folded into the matching ``site:`` / ``-site:`` operators (and may be combined, e.g. include a parent domain while excluding one subdomain). Use ``sources`` to pick the result bucket — ``"web"`` (default), ``"news"``, or ``"images"`` (combinable); ``tbs`` for a time filter (``qdr:d`` / ``qdr:w`` / ``qdr:m`` / ``qdr:y``); ``limit`` (1-20, default 10) to cap results. Billing scales with the number of results returned, with a minimum of 1 credit per call (an empty result set still bills the minimum). ### Responses: **200**: Successful Response (Success Response) Content-Type: application/json **Example Response:** ```json { "success": true, "meta": { "requestId": "Requestid", "timestamp": "Timestamp" } } ``` **Output Schema:** ```json { "properties": { "success": { "type": "boolean", "title": "Success", "description": "Whether the request was successful", "default": true }, "data": { "description": "Response data payload" }, "error": { "description": "Error details if request failed" }, "meta": { "description": "Metadata for API responses.\n\nCredit fields follow the ADR-0003 parallel-fields strategy (Option 3):\n- `credits_remaining` / `credits_consumed` (int): legacy fields, rounded\n to whole credits, kept for zero-breaking-change to existing SDK clients.\n- `credits_remaining_exact` / `credits_consumed_exact` (float): new\n precision-aware fields for clients that opt in to decimal credits.\n\nSee ADR-0003 decision 5 and the \u00a78 deprecation timeline.\n\nTODO(2026-11, ADR-0003 \u00a78 +6mo): mark `credits_remaining` /\n`credits_consumed` as `deprecated=True` in their Field() definitions\nand announce in customer changelog.\nTODO(2027-05, ADR-0003 \u00a78 +12mo): remove the legacy int fields via a\nmajor-version bump of the OpenAPI surface.", "properties": { "requestId": { "type": "string", "title": "Requestid", "description": "Unique request identifier" }, "timestamp": { "type": "string", "title": "Timestamp", "description": "Response timestamp in ISO 8601 format" }, "total": { "title": "Total", "description": "Total number of records" }, "page": { "title": "Page", "description": "Current page number" }, "pageSize": { "title": "Pagesize", "description": "Number of records per page" }, "totalPages": { "title": "Totalpages", "description": "Total number of pages" }, "creditsRemaining": { "title": "Creditsremaining", "description": "Remaining API credits (rounded to whole credits; see creditsRemainingExact for precise value)" }, "creditsConsumed": { "title": "Creditsconsumed", "description": "Credits consumed by this request (rounded; see creditsConsumedExact for precise value)" }, "creditsRemainingExact": { "title": "Creditsremainingexact", "description": "Remaining API credits, precise to 1 decimal place" }, "creditsConsumedExact": { "title": "Creditsconsumedexact", "description": "Credits consumed by this request, precise to 1 decimal place" }, "tokensUsage": { "description": "Provider token-usage block \u2014 populated on terminal video polls only, null on every non-video endpoint. See TokensUsage for its fields." } }, "type": "object", "required": [ "requestId", "timestamp" ], "title": "ResponseMeta" } }, "type": "object", "required": [ "meta" ], "title": "OpenApiResponse[CrawlerSearch]", "examples": [] } ``` **422**: Validation Error Content-Type: application/json **Example Response:** ```json { "detail": [ { "loc": [], "msg": "Message", "type": "Error Type", "ctx": {} } ] } ``` **Output Schema:** ```json { "properties": { "detail": { "items": { "properties": { "loc": { "items": {}, "type": "array", "title": "Location" }, "msg": { "type": "string", "title": "Message" }, "type": { "type": "string", "title": "Error Type" }, "input": { "title": "Input" }, "ctx": { "type": "object", "title": "Context" } }, "type": "object", "required": [ "loc", "msg", "type" ], "title": "ValidationError" }, "type": "array", "title": "Detail" } }, "type": "object", "title": "HTTPValidationError" } ```
    Connector
  • Returns Reality Graph's free fill-in template (v0) for a verifiable task contract: goal, non-goals, boundaries (may change / must not change / forbidden), 3-7 yes/no acceptance criteria, validation plan, expected evidence, assumptions, open questions — with a filled example and fill-in guidance. Write the contract before an AI agent runs; verify the result against it after. format='json' returns a machine-fillable JSON structure; default is a compact markdown skeleton. Set lang='de' for German. Static content, nothing stored.
    Connector
  • Get CCPEDIA's CONTENT-CATALOG metrics: how many Canton items CCPEDIA has indexed. Counts of CIPs (total/in-review/final), forum topics & posts, GitHub issues/PRs, dev-fund proposals, docs pages, videos, blog posts, mailing threads, and radar items. This is CCPEDIA coverage/inventory, NOT live Canton on-chain stats. For on-chain validators/rounds use get_network_state, for price/TVL use get_token_market.
    Connector
  • Fetch the full markdown content of a specific documentation page from BlockRazor. Use this when you have a page URL and want to read its content. Accepts full URLs (e.g. https://docs.blockrazor.io//getting-started). Since `searchDocumentation` returns partial content, use `getPage` to retrieve the complete page when you need more details. The content includes links you can follow to navigate to related pages.
    Connector
  • Convert Markdown or HTML you already have into Word, PDF, HTML, LaTeX, EPUB and other formats. Pass the content directly; the document does not need to exist in mdedit. Conversion is asynchronous: pass the returned opaque jobId to get_conversion_status until it completes, and never show that identifier to the user. To export a document already stored in mdedit, use start_article_export instead.
    Connector