"How to scrape websites and format content in GitHub Markdown" matching MCP tools:

• web-scrape

  arch-tools-mcp

  Scrape and extract content from websites with optional CSS selector

  Connector

• get_quickstart

  Uploadkit

  Return the complete UploadKit quickstart walkthrough for Next.js — install, API key env, route handler, provider, first component, optional BYOS — in one markdown document. When to use: the user is brand new to UploadKit and asks "how do I get started?", "set this up for me", or any variation that signals zero prior context. Prefer scaffold_route_handler + scaffold_provider + get_install_command when you already know which specific step they need. Returns: a plain-text markdown document. Takes no parameters. Read-only, static content, idempotent.

  Connector

• get_doc

  Dock

  Read a workspace's doc (TipTap rich-text) body. Format is negotiable via `format`: `markdown` (default — CommonMark + GFM, ready to feed to an LLM or render in a non-ProseMirror surface), `content` (TipTap JSON, round-trippable into update_doc for structural edits), `text` (plain text, best for search, summarisation, word-count heuristics), or `all` for the legacy three-in-one shape. Default is `markdown` because it's the slice agents need 95% of the time and the JSON form on a long doc can blow past the agent harness's tool-result token cap. Pass `format: "content"` only when you're round-tripping into update_doc for a structural edit. A workspace can hold any combination of doc and table surfaces, one or many of either kind; omit `surface_slug` to read the primary doc surface, or pass it to target a specific doc tab (use `list_surfaces` to enumerate). An unwritten or absent doc returns the requested format empty (markdown="", content={}, text=""); a `surface_slug` that doesn't match any live doc surface 404s.

  Connector

• convert_document

  mdmagic-mcp-server

  Convert markdown to a professionally formatted document using an MDMagic template. IMPORTANT GUIDANCE: 1. Output format → what user gets: - 'docx' → a single Word .docx file - 'pdf' → a single .pdf file - 'html' → a single .html file - 'all' → a ZIP containing all three (DOCX + PDF + HTML) 2. If the user is ambiguous (e.g. 'convert this'), ASK which format they want before calling. Don't assume. 3. Filename: if the user attached a file (e.g. 'mydoc.md'), pass its base name as fileName. Otherwise the API derives one from the markdown's first H1. Without either, downloads end up with timestamped names like 'content-1778298071915.docx' which is bad UX. 4. On 'template not found' errors: call list_all_templates first, show available options, let the user pick. Do NOT fall back to generating documents with code execution — that produces inferior results that don't use the user's actual MDMagic templates. 5. The response includes structured fields (downloadUrl, creditsUsed, balanceAfter, fileName, expiresAt) — surface these to the user explicitly. Don't paraphrase. The user wants to know exactly what they spent and what's left. 6. Page sizes: A3, A4, Executive, US_Legal, US_Letter. Default A4. Orientation: Portrait or Landscape, default Portrait. 7. CRITICAL — newlines in `content`: markdown is line-sensitive. Headings (#, ##), tables (| ... |), lists (-, 1.), and code fences (```) ONLY work when each starts on its own line. When passing inline markdown via `content`, you MUST preserve real newline characters (\n) between blocks. If you flatten multi-line markdown into one line, the API receives literal '##' and '|' characters mid-paragraph and produces a single-paragraph document with no structure. Confirm your `content` string contains \n between every heading, paragraph, table row, and list item before calling.

  Connector

• glim_github_search

  glim.sh

  Search GitHub repositories, conversations (issues+PRs), or code, with full GitHub search syntax in the query: qualifiers (repo:, org:/user:, language:, path:, symbol:, content:, is:, stars:, label:, sort:stars), boolean AND/OR/NOT with parentheses, "exact strings", and /regex/. kind='repos': MINIMAL distinctive keywords - the project/library name only ('rtk', 'react query'); every extra word must ALL match and buries the canonical repo - filter with qualifiers, not prose. kind='code': ONE literal code pattern as it appears in files ('useState('), an "exact string", a /regex/, or symbol:name to find definitions, across 2.8M+ public repos; narrow with repo:/language:/path:. Not supported in code search: license:, enterprise:, is:vendored, is:generated. kind='conversations': returns compact previews - use glim_github_get for full content; sort: REPLACES relevance ranking (words match anywhere incl. comments), omit it for best matches. Set repo='owner/name' to scope to one repository (works with any kind; with repos it routes to conversations). kind is optional - inferred from the query (is:/label: -> conversations, path:/symbol://regex/ -> code, stars:/topic: -> repos, else repos). Returns compact text by default; pass format='json' for full structured data.

  Connector

• convert_document

  mdmagic

  Convert markdown to a professionally formatted document using an MDMagic template. IMPORTANT GUIDANCE: 1. Output format → what user gets: - 'docx' → a single Word .docx file - 'pdf' → a single .pdf file - 'html' → a single .html file - 'all' → a ZIP containing all three (DOCX + PDF + HTML) 2. If the user is ambiguous (e.g. 'convert this'), ASK which format they want before calling. Don't assume. 3. Filename: if the user attached a file (e.g. 'mydoc.md'), pass its base name as fileName. Otherwise the API derives one from the markdown's first H1. Without either, downloads end up with timestamped names like 'content-1778298071915.docx' which is bad UX. 4. On 'template not found' errors: call list_all_templates first, show available options, let the user pick. Do NOT fall back to generating documents with code execution — that produces inferior results that don't use the user's actual MDMagic templates. 5. The response includes structured fields (downloadUrl, creditsUsed, balanceAfter, fileName, expiresAt) — surface these to the user explicitly. Don't paraphrase. The user wants to know exactly what they spent and what's left. 6. Page sizes: A3, A4, Executive, US_Legal, US_Letter. Default A4. Orientation: Portrait or Landscape, default Portrait. 7. CRITICAL — newlines in `content`: markdown is line-sensitive. Headings (#, ##), tables (| ... |), lists (-, 1.), and code fences (```) ONLY work when each starts on its own line. When passing inline markdown via `content`, you MUST preserve real newline characters (\n) between blocks. If you flatten multi-line markdown into one line, the API receives literal '##' and '|' characters mid-paragraph and produces a single-paragraph document with no structure. Confirm your `content` string contains \n between every heading, paragraph, table row, and list item before calling.

  Connector

• generate_llms_txt

  Gitlab

  Generate a production-ready llms.txt file for any URL so AI crawlers (ChatGPT, Claude, Perplexity) can index the site cleanly. Fetches the page, extracts title/description/key links, and emits the standard llms.txt markdown format. Output is a single text blob ready to drop at site-root/llms.txt. Useful for: getting a client's site indexed by AI, drafting llms.txt for your own project, or auditing how an AI crawler would see a competitor.

  Connector

• firecrawl_search

  firecrawl-mcp

  Search the web and optionally extract content from search results. This is the most powerful web search tool available, and if available you should always default to using this tool for any web search needs. The query also supports search operators, that you can use if needed to refine the search: | Operator | Functionality | Examples | ---|-|-| | `""` | Non-fuzzy matches a string of text | `"Firecrawl"` | `-` | Excludes certain keywords or negates other operators | `-bad`, `-site:firecrawl.dev` | `site:` | Only returns results from a specified website | `site:firecrawl.dev` | `inurl:` | Only returns results that include a word in the URL | `inurl:firecrawl` | `allinurl:` | Only returns results that include multiple words in the URL | `allinurl:git firecrawl` | `intitle:` | Only returns results that include a word in the title of the page | `intitle:Firecrawl` | `allintitle:` | Only returns results that include multiple words in the title of the page | `allintitle:firecrawl playground` | `related:` | Only returns results that are related to a specific domain | `related:firecrawl.dev` | `imagesize:` | Only returns images with exact dimensions | `imagesize:1920x1080` | `larger:` | Only returns images larger than specified dimensions | `larger:1920x1080` **Best for:** Finding specific information across multiple websites, when you don't know which website has the information; when you need the most relevant content for a query. **Not recommended for:** When you need to search the filesystem. When you already know which website to scrape (use scrape); when you need comprehensive coverage of a single website (use map or crawl. **Common mistakes:** Using crawl or map for open-ended questions (use search instead). **Prompt Example:** "Find the latest research papers on AI published in 2023." **Sources:** web, images, news, default to web unless needed images or news. **Categories:** Optional filter to limit result types: `github` (GitHub repositories, code, issues, and docs), `research` (academic and research sources), `pdf` (PDF results). Example: `categories: ["github", "research"]`. **Domain filters:** Use includeDomains to restrict results to specific domains, or excludeDomains to remove domains. Do not use both in the same request. Domains must be hostnames only, without protocol or path. **Scrape Options:** Only use scrapeOptions when you think it is absolutely necessary. When you do so default to a lower limit to avoid timeouts, 5 or lower. **Optimal Workflow:** Search first using firecrawl_search without formats, then after fetching the results, use the scrape tool to get the content of the relevantpage(s) that you want to scrape **After the search:** Once you have processed the results (or decided they were not useful), call `firecrawl_search_feedback` with the `id` from this response. The first feedback per search refunds 1 credit and helps Firecrawl improve search quality. **Usage Example without formats (Preferred):** ```json { "name": "firecrawl_search", "arguments": { "query": "top AI companies", "limit": 5, "includeDomains": ["example.com"], "sources": [ { "type": "web" } ] } } ``` **Usage Example with formats:** ```json { "name": "firecrawl_search", "arguments": { "query": "latest AI research papers 2023", "limit": 5, "categories": ["github", "research"], "lang": "en", "country": "us", "sources": [ { "type": "web" }, { "type": "images" }, { "type": "news" } ], "scrapeOptions": { "formats": ["markdown"], "onlyMainContent": true } } } ``` **Returns:** A JSON envelope of the form `{ success, data: { web?, images?, news? }, id, creditsUsed }`. Each result array contains the search results (with optional scraped content). Pass the top-level `id` to `firecrawl_search_feedback` after you've used the results.

  Connector

• firecrawl_parse

  firecrawl-mcp

  Parse a file using Firecrawl's /v2/parse endpoint. In local/non-cloud MCP mode, this tool reads filePath from the MCP server filesystem and posts multipart data to the configured self-hosted FIRECRAWL_API_URL, preserving the existing direct-read behavior. In hosted CLOUD_SERVICE mode, this tool is a two-call flow because hosted MCP cannot read your local filesystem: 1. Call with filePath, contentType, parse options, and optional declaredSizeBytes. The hosted server mints a short-lived upload URL and returns a safe local curl PUT command plus nextToolCall. 2. Run the returned curl command locally, then call firecrawl_parse again with uploadRef and the desired parse options. The hosted server calls /v2/parse server-side with your session credential. **Best for:** Extracting content from a local document (PDF, Word, Excel, HTML, etc.); pulling structured data out of a file with JSON format; converting binary documents into markdown for downstream reasoning. **Not recommended for:** Remote URLs (use firecrawl_scrape); multiple files at once (call parse multiple times); documents that require interactive actions, screenshots, or change tracking — those aren't supported by the parse endpoint. **Common mistakes:** In hosted mode, do not pass both filePath and uploadRef. Phase 1 uses filePath only to generate upload instructions; phase 2 uses uploadRef only to parse server-side. **Supported file types:** .html, .htm, .xhtml, .pdf, .docx, .doc, .odt, .rtf, .xlsx, .xls **Unsupported options:** actions, screenshot/branding/changeTracking formats, waitFor > 0, location, mobile, proxy values other than "auto" or "basic". **Privacy:** Set `redactPII: true` to return content with personally identifiable information redacted. **CRITICAL - Format Selection (same rules as firecrawl_scrape):** When the user asks for SPECIFIC data points from a document, you MUST use JSON format with a schema. Only use markdown when the user needs the ENTIRE document content. **Handling PDFs:** Add `"parsers": ["pdf"]` (optionally with `pdfOptions.maxPages`) when parsing a PDF so the PDF engine is invoked explicitly. For very long documents, cap `maxPages` to keep the response within token limits. **Hosted phase 1 example:** ```json { "name": "firecrawl_parse", "arguments": { "filePath": "/absolute/path/to/document.pdf", "contentType": "application/pdf", "formats": ["markdown"], "parsers": ["pdf"], "zeroDataRetention": true } } ``` **Hosted phase 2 example:** ```json { "name": "firecrawl_parse", "arguments": { "uploadRef": "upload-ref-from-phase-1", "formats": ["markdown"], "parsers": ["pdf"], "zeroDataRetention": true } } ``` **Returns:** Phase 1 hosted upload instructions or a parsed document with markdown, html, links, summary, json, or query results depending on the requested formats.

  Connector

• read_kb_doc

  fixflow

  SECOND STEP in the troubleshooting workflow. Read the full content and solution of a specific Knowledge Base card. Returns the card content WITH reliability metrics and related cards so you can assess trustworthiness and explore connected issues. WHEN TO USE: - Call this ONLY after obtaining a valid `kb_id` from the `resolve_kb_id` tool. INPUT: - `kb_id`: The exact ID of the card (e.g., 'CROSS_DOCKER_001'). OUTPUT: - Returns reliability metrics followed by the full Markdown content of the card, plus related cards. - You MUST apply the solution provided in the card to resolve the user's issue. - After applying, you MUST call `save_kb_card` with `outcome` parameter to close the feedback loop.

  Connector

• firecrawl_scrape

  xpay✦ Web Scraping Collection

  Scrape content from a single URL with advanced options. This is the most powerful, fastest and most reliable scraper tool, if available you should always default to using this tool for any web scraping needs. **Best for:** Single page content extraction, when you know exactly which page contains the information. **Not recommended for:** Multiple pages (call scrape multiple times or use crawl), unknown page location (use search). **Common mistakes:** Using markdown format when extracting specific data points (use JSON instead). **Other Features:** Use 'branding' format to extract brand identity (colors, fonts, typography, spacing, UI components) for design analysis or style replication. **CRITICAL - Format Selection (you MUST follow this):** When the user asks for SPECIFIC data points, you MUST use JSON format with a schema. Only use markdown when the user needs the ENTIRE page content. **Use JSON format when user asks for:** - Parameters, fields, or specifications (e.g., "get the header parameters", "what are the required fields") - Prices, numbers, or structured data (e.g., "extract the pricing", "get the product details") - API details, endpoints, or technical specs (e.g., "find the authentication endpoint") - Lists of items or properties (e.g., "list the features", "get all the options") - Any specific piece of information from a page **Use markdown format ONLY when:** - User wants to read/summarize an entire article or blog post - User needs to see all content on a page without specific extraction - User explicitly asks for the full page content **Handling JavaScript-rendered pages (SPAs):** If JSON extraction returns empty, minimal, or just navigation content, the page is likely JavaScript-rendered or the content is on a different URL. Try these steps IN ORDER: 1. **Add waitFor parameter:** Set `waitFor: 5000` to `waitFor: 10000` to allow JavaScript to render before extraction 2. **Try a different URL:** If the URL has a hash fragment (#section), try the base URL or look for a direct page URL 3. **Use firecrawl_map to find the correct page:** Large documentation sites or SPAs often spread content across multiple URLs. Use `firecrawl_map` with a `search` parameter to discover the specific page containing your target content, then scrape that URL directly. Example: If scraping "https://docs.example.com/reference" fails to find webhook parameters, use `firecrawl_map` with `{"url": "https://docs.example.com/reference", "search": "webhook"}` to find URLs like "/reference/webhook-events", then scrape that specific page. 4. **Use firecrawl_agent:** As a last resort for heavily dynamic pages where map+scrape still fails, use the agent which can autonomously navigate and research **Usage Example (JSON format - REQUIRED for specific data extraction):** ```json { "name": "firecrawl_scrape", "arguments": { "url": "https://example.com/api-docs", "formats": ["json"], "jsonOptions": { "prompt": "Extract the header parameters for the authentication endpoint", "schema": { "type": "object", "properties": { "parameters": { "type": "array", "items": { "type": "object", "properties": { "name": { "type": "string" }, "type": { "type": "string" }, "required": { "type": "boolean" }, "description": { "type": "string" } } } } } } } } } ``` **Prefer markdown format by default.** You can read and reason over the full page content directly — no need for an intermediate query step. Use markdown for questions about page content, factual lookups, and any task where you need to understand the page. **Use JSON format when user needs:** - Structured data with specific fields (extract all products with name, price, description) - Data in a specific schema for downstream processing **Use query format only when:** - The page is extremely long and you need a single targeted answer without processing the full content - You want a quick factual answer and don't need to retain the page content **Usage Example (markdown format - default for most tasks):** ```json { "name": "firecrawl_scrape", "arguments": { "url": "https://example.com/article", "formats": ["markdown"], "onlyMainContent": true } } ``` **Usage Example (branding format - extract brand identity):** ```json { "name": "firecrawl_scrape", "arguments": { "url": "https://example.com", "formats": ["branding"] } } ``` **Branding format:** Extracts comprehensive brand identity (colors, fonts, typography, spacing, logo, UI components) for design analysis or style replication. **Performance:** Add maxAge parameter for 500% faster scrapes using cached data. **Returns:** JSON structured data, markdown, branding profile, or other formats as specified. **Safe Mode:** Read-only content extraction. Interactive actions (click, write, executeJavascript) are disabled for security.

  Connector

• microsoft_docs_fetch

  xpay✦ DevTools Collection

  Fetch and convert a Microsoft Learn documentation webpage to markdown format. This tool retrieves the latest complete content of Microsoft documentation webpages including Azure, .NET, Microsoft 365, and other Microsoft technologies. ## When to Use This Tool - When search results provide incomplete information or truncated content - When you need complete step-by-step procedures or tutorials - When you need troubleshooting sections, prerequisites, or detailed explanations - When search results reference a specific page that seems highly relevant - For comprehensive guides that require full context ## Usage Pattern Use this tool AFTER microsoft_docs_search when you identify specific high-value pages that need complete content. The search tool gives you an overview; this tool gives you the complete picture. ## URL Requirements - The URL must be a valid HTML documentation webpage from the microsoft.com domain - Binary files (PDF, DOCX, images, etc.) are not supported ## Output Format markdown with headings, code blocks, tables, and links preserved.

  Connector

• create_support_ticket

  Dock

  File a support ticket. Mirrors to a GitHub issue in Dock's support repo and shows up in the user's dashboard at /settings/support. Use this for bugs (you hit an error), feature requests (Dock is missing something), billing (Stripe/subscription), questions (how do I X), or anything else. Prefer request_limit_increase when the user is simply hitting a plan cap.

  Connector

• render_report

  Valuein — SEC EDGAR Fundamentals & Smart-Money Data

  Return a 15-minute presigned download URL for a report in the requested binary format. `format=md` presigns the cached markdown — instant, no compute. `format=docx` returns a branded Word document with a cover page (logo, title, ticker, tier badge), the report body (abstract, sections, citations table with clickable SEC EDGAR links), and a back page (methodology, sources, disclaimer). The DOCX is cached in R2 alongside the markdown after first build so repeat downloads are instant; pass `force_regenerate: true` to bust the cache (e.g. right after `update_report`). Tier gate mirrors `get_report`: authors always see their own reports; non-authors below the report's required tier get an upgrade prompt.

  Connector

• iliad_web_research_crawl

  AXIS Toolbox — Agentic Commerce Codebase Intelligence

  Crawl a domain and scrape multiple pages using Firecrawl. Returns array of scraped pages with markdown content. Best for site mapping, content audits, or bulk research. Requires Authorization: Bearer <api_key>. Pricing: $0.25 standard, $0.12 lite per page crawled (up to 100 pages per request). Use iliad_web_research for single-page scrapes.

  Connector

• generate_llms_txt

  airquality

  Generate a production-ready llms.txt file for any URL so AI crawlers (ChatGPT, Claude, Perplexity) can index the site cleanly. Fetches the page, extracts title/description/key links, and emits the standard llms.txt markdown format. Output is a single text blob ready to drop at site-root/llms.txt. Useful for: getting a client's site indexed by AI, drafting llms.txt for your own project, or auditing how an AI crawler would see a competitor.

  Connector

• generate_llms_txt

  microlink

  Generate a production-ready llms.txt file for any URL so AI crawlers (ChatGPT, Claude, Perplexity) can index the site cleanly. Fetches the page, extracts title/description/key links, and emits the standard llms.txt markdown format. Output is a single text blob ready to drop at site-root/llms.txt. Useful for: getting a client's site indexed by AI, drafting llms.txt for your own project, or auditing how an AI crawler would see a competitor.

  Connector

• fetch

  Sofya

  Fetch one or more URLs and return their content as clean markdown. Use this to read articles, documentation, blog posts, or any page where you need the complete text, not just a snippet from search. Also supports PDF, DOCX, and other document formats. Costs 1 credit per URL. Max 10 URLs per request. Failed URLs are not charged. Set include_raw_html=true to also get the raw HTML source in each result. Useful for inspecting embedded URLs, data attributes, iframes, or script tags that are stripped during markdown conversion. Returns null for non-HTML content (PDF, DOCX, etc.). Same cost. Returns: results (array of {title, url, content, raw_html, published_time, success, error}), credits_used, credits_remaining. Args: urls: List of URLs to fetch (max 10) include_raw_html: Include raw HTML source in each result (default false)

  Connector

• how_to_give_feedback

  life. scored.

  How to suggest a better weight, a fresh source, or a new rule via GitHub, so improvements from many people aggregate in the open.

  Connector

• get_task_contract_template

  Reality Graph Verification Tools

  Returns Reality Graph's free fill-in template (v0) for a verifiable task contract: goal, non-goals, boundaries (may change / must not change / forbidden), 3-7 yes/no acceptance criteria, validation plan, expected evidence, assumptions, open questions — with a filled example and fill-in guidance. Write the contract before an AI agent runs; verify the result against it after. format='json' returns a machine-fillable JSON structure; default is a compact markdown skeleton. Set lang='de' for German. Static content, nothing stored.

  Connector