OmniExtract MCP
Extracts native and auto-generated subtitles from Bilibili videos via a 3-layer subtitle engine, returning deduplicated full-text transcripts and timestamped segments without downloading video files.
Fast-path fetching of GitHub raw README and blob content, converting it into clean LLM-ready Markdown.
Extracts watermark-free media URLs and metadata from TikTok links by resolving shortlinks and computing signing parameters, without requiring a headless browser.
Extracts native and auto-generated VTT/SRT captions from YouTube videos, producing deduplicated full-text transcripts and timestamped segments in seconds without downloading the video.
OmniExtract MCP & Apify Actor (omni-extract-mcp)
Universal, Stateless, Cookie-Free Content & Transcript Extractor for AI Agents (Claude Desktop, Cursor, Windsurf, OpenClaw, n8n, Make, and Apify x402).
👉 Run Instantly in the Cloud (Zero Setup, Pay-Per-Result / x402 Ready): https://apify.com/poweragy/omni-extract-mcp
Why OmniExtract?
Standard WebFetch and browser tools fail with 403 Forbidden, login walls, or empty JavaScript shells when given social media or video URLs. OmniExtract converts complex links into clean, LLM-ready Markdown + Timestamped Transcripts + Watermark-Free Media URLs in one call:
X (Twitter) Posts & Long-Form X Articles: Full Draft.js block rendering (
# / ## / ###, ordered/unordered lists, code blocks, blockquotes, quoted posts) and media extraction with zero personal cookies required.Douyin & TikTok Watermark-Free Extraction: Pure-algorithm
x-secsdk-web-signaturesigning (ttwid+uifid+ MD5 salt) + 302 shortlink resolution + binary MIME magic sniffing (WebP/JPEG/PNG) — zero headless browser overhead.YouTube & Bilibili 3-Layer Subtitle Engine: Extracts native & auto-generated VTT/SRT captions into deduplicated full-text transcripts and timestamped segments in ~2 seconds without downloading heavy video files.
Anti-Bot Web & GitHub Markdown: Automatic GitHub raw README/blob fast-path, semantic HTML-to-Markdown conversion, bot-wall detection, and Jina Reader fallback.
Related MCP server: @hauntapi/mcp-server
Option 1: Hosted Cloud API & MCP via Apify Store (Recommended)
No local dependencies (ffmpeg / yt-dlp) or server maintenance required:
Apify Store Page: https://apify.com/poweragy/omni-extract-mcp
REST API Call:
curl -X POST "https://api.apify.com/v2/acts/poweragy~omni-extract-mcp/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "urls": [ "https://x.com/ai_chuhai/status/2037437653071835440", "https://v.douyin.com/0kH2cQVrf0o/" ], "include_transcript": true }'
Option 2: Local MCP Server (Claude Desktop / Cursor / Windsurf)
pip install git+https://github.com/solarfans/omni-extract-mcp.gitAdd to your claude_desktop_config.json or .cursor/mcp.json:
{
"mcpServers": {
"omni-extract": {
"command": "python3",
"args": ["-m", "omni_extract.mcp_server", "--mode", "stdio"]
}
}
}Exposed MCP Tools
omni_extract_url(url, include_transcript=True, output_format="markdown"): Extract a single URL into structured Markdown or JSON.omni_extract_batch(urls, include_transcript=True): Concurrently extract up to 10 URLs in parallel.omni_check_quota(api_key=""): Check remaining API key credits or daily quota.
Option 3: Self-Hosted HTTP / Remote MCP Gateway
python3 -m omni_extract.mcp_server --mode http --port 8080GET /v1/health: Health & capability checkPOST /mcp: Streamable HTTP MCP JSON-RPC 2.0 endpointPOST /v1/extract: Direct REST single URL extractionPOST /v1/extract/batch: Direct REST concurrent batch extractionGET /v1/quota: Check remaining API key credits or daily anonymous free quota
Option 4: Quick Directory Integration (Smithery / mcpservers.org / Cursor / Claude Desktop)
Hosted Apify MCP Server (Zero Local Install)
Connect directly using Apify's hosted MCP endpoint (https://mcp.apify.com/?actors=poweragy/omni-extract-mcp):
{
"mcpServers": {
"omni-extract-cloud": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://mcp.apify.com/?actors=poweragy/omni-extract-mcp"],
"env": {
"APIFY_TOKEN": "YOUR_APIFY_TOKEN"
}
}
}
}Troubleshooting & Common Error Fixes (FAQ)
Fix: X (Twitter) Articles Truncated / JS Shell: Standard HTTP scrapers stop at the X JS shell or truncated preview. Pass the full
x.com/<user>/status/<id>URL toomni_extract_url— OmniExtract resolves the underlying Draft.js blocks (article.content.blocks) and entity media automatically without cookies.Fix: Douyin Watermark & Cookie Expired Errors: Pass
v.douyin.comordouyin.com/video/...links directly. OmniExtract resolves 302 redirects and computesx-secsdk-web-signaturenatively, returning clean Markdown + direct media URLs without headless browser overhead.Fix: YouTube Transcript Blocked on Cloud IPs: Runs a 3-layer native +
yt-dlpcaption pipeline without downloading full MP4 streams.Empty
{}health checks vs billed items: Empty input{}runs execute a read-only connectivity check and skip PPE item billing (apify_ppe_skipped: true), while real extraction runs emit transparentcharged/charge_failedcounters in the Actor summary.Registry manifests included:
smithery.yamlandserver.json.
Available Tools
3 toolsomni_check_quotaA
Check remaining API key credits or daily anonymous free quota.
Inputs: api_key.
omni_extract_batchC
Concurrently extract up to 10 URLs (X Articles, Douyin, TikTok, YouTube, GitHub, Web) in parallel.
Inputs: urls (required), api_key, include_transcript.
omni_extract_urlB
Universal, cookie-free content & transcript extractor for AI Agents. Extracts full long-form X (Twitter) Articles/Tweets, watermark-free Douyin/TikTok videos & photo galleries, YouTube/Bilibili video subtitles/transcripts with timestamps, GitHub READMEs/code, and anti-bot web pages into clean Markdown.
Inputs: url (required), api_key, output_format, include_transcript.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 3 tools
Each tool targets a distinct operation: single-URL extraction, batched extraction, and quota checking. There is no realistic way to confuse batch extraction with a single extraction or with a credit check, and the descriptions reinforce the boundary.
All three names follow a strict server-prefixed verb_noun pattern: omni_extract_url, omni_extract_batch, omni_check_quota. The shared 'omni_' namespace plus consistent verb forms makes the set fully predictable.
Three tools is a tight, well-scoped surface for a stateless extraction service, and each one earns its place. Nothing redundant, nothing gratuitous.
The core workflow is fully covered: extract one, extract many, and monitor credits. Minor gaps exist, such as no way to select output format/depth, inspect supported-platform info, or check the status of a long-running batch, but agents can work around these.
Maintenance
Related MCP Connectors
Hosted X (Twitter) MCP server for Claude, ChatGPT and Cursor, with each action switched per account.
Web scraping MCP server: any URL to Markdown or JSON, with JavaScript rendering and batch jobs.
Hosted Web Crawler MCP server for Claude, ChatGPT and Cursor, with each action switched per account.
Unified social scraper MCP — profiles, posts, videos across ~40 platforms via Streamable HTTP.
Related MCP Servers
- AlicenseAqualityCmaintenanceMCP server for web scraping — extract clean markdown, links, and metadata from any URL. Free Firecrawl alternative.5119 npm6MIT
- AlicenseAqualityBmaintenanceWeb extraction MCP server for AI agents. Extract structured data from any URL with built-in Cloudflare bypass, JavaScript rendering, and intelligent parsing. Returns clean markdown or JSON.5794 npm2MIT
- AlicenseAqualityBmaintenanceA local-first MCP server that lets AI agents read any webpage as clean Markdown, crawl whole sites within configured limits, search without API keys, and solve supported captchas locally — all without cloud services or third-party keys.1222AGPL 3.0
- AlicenseNot gradedqualityCmaintenanceThis self-hosted MCP server enables users to read public Xiaohongshu and Douyin content, including post text, images, video frames, and optional audio transcription, from compatible Streamable HTTP MCP clients like Claude and ChatGPT.MIT