Skip to main content
Glama
OmniExtract MCP

OmniExtract MCP

OmniExtract MCP & Apify Actor (omni-extract-mcp)

Apify Store MCP Protocol License: MIT

Universal, Stateless, Cookie-Free Content & Transcript Extractor for AI Agents (Claude Desktop, Cursor, Windsurf, OpenClaw, n8n, Make, and Apify x402).

👉 Run Instantly in the Cloud (Zero Setup, Pay-Per-Result / x402 Ready): https://apify.com/poweragy/omni-extract-mcp


Why OmniExtract?

Standard WebFetch and browser tools fail with 403 Forbidden, login walls, or empty JavaScript shells when given social media or video URLs. OmniExtract converts complex links into clean, LLM-ready Markdown + Timestamped Transcripts + Watermark-Free Media URLs in one call:

  1. X (Twitter) Posts & Long-Form X Articles: Full Draft.js block rendering (# / ## / ###, ordered/unordered lists, code blocks, blockquotes, quoted posts) and media extraction with zero personal cookies required.

  2. Douyin & TikTok Watermark-Free Extraction: Pure-algorithm x-secsdk-web-signature signing (ttwid + uifid + MD5 salt) + 302 shortlink resolution + binary MIME magic sniffing (WebP/JPEG/PNG) — zero headless browser overhead.

  3. YouTube & Bilibili 3-Layer Subtitle Engine: Extracts native & auto-generated VTT/SRT captions into deduplicated full-text transcripts and timestamped segments in ~2 seconds without downloading heavy video files.

  4. Anti-Bot Web & GitHub Markdown: Automatic GitHub raw README/blob fast-path, semantic HTML-to-Markdown conversion, bot-wall detection, and Jina Reader fallback.


Related MCP server: @hauntapi/mcp-server

No local dependencies (ffmpeg / yt-dlp) or server maintenance required:

  • Apify Store Page: https://apify.com/poweragy/omni-extract-mcp

  • REST API Call:

    curl -X POST "https://api.apify.com/v2/acts/poweragy~omni-extract-mcp/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
      -H "Content-Type: application/json" \
      -d '{
        "urls": [
          "https://x.com/ai_chuhai/status/2037437653071835440",
          "https://v.douyin.com/0kH2cQVrf0o/"
        ],
        "include_transcript": true
      }'

Option 2: Local MCP Server (Claude Desktop / Cursor / Windsurf)

pip install git+https://github.com/solarfans/omni-extract-mcp.git

Add to your claude_desktop_config.json or .cursor/mcp.json:

{
  "mcpServers": {
    "omni-extract": {
      "command": "python3",
      "args": ["-m", "omni_extract.mcp_server", "--mode", "stdio"]
    }
  }
}

Exposed MCP Tools

  • omni_extract_url(url, include_transcript=True, output_format="markdown"): Extract a single URL into structured Markdown or JSON.

  • omni_extract_batch(urls, include_transcript=True): Concurrently extract up to 10 URLs in parallel.

  • omni_check_quota(api_key=""): Check remaining API key credits or daily quota.


Option 3: Self-Hosted HTTP / Remote MCP Gateway

python3 -m omni_extract.mcp_server --mode http --port 8080
  • GET /v1/health: Health & capability check

  • POST /mcp: Streamable HTTP MCP JSON-RPC 2.0 endpoint

  • POST /v1/extract: Direct REST single URL extraction

  • POST /v1/extract/batch: Direct REST concurrent batch extraction

  • GET /v1/quota: Check remaining API key credits or daily anonymous free quota


Option 4: Quick Directory Integration (Smithery / mcpservers.org / Cursor / Claude Desktop)

Hosted Apify MCP Server (Zero Local Install)

Connect directly using Apify's hosted MCP endpoint (https://mcp.apify.com/?actors=poweragy/omni-extract-mcp):

{
  "mcpServers": {
    "omni-extract-cloud": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://mcp.apify.com/?actors=poweragy/omni-extract-mcp"],
      "env": {
        "APIFY_TOKEN": "YOUR_APIFY_TOKEN"
      }
    }
  }
}

Troubleshooting & Common Error Fixes (FAQ)

  • Fix: X (Twitter) Articles Truncated / JS Shell: Standard HTTP scrapers stop at the X JS shell or truncated preview. Pass the full x.com/<user>/status/<id> URL to omni_extract_url — OmniExtract resolves the underlying Draft.js blocks (article.content.blocks) and entity media automatically without cookies.

  • Fix: Douyin Watermark & Cookie Expired Errors: Pass v.douyin.com or douyin.com/video/... links directly. OmniExtract resolves 302 redirects and computes x-secsdk-web-signature natively, returning clean Markdown + direct media URLs without headless browser overhead.

  • Fix: YouTube Transcript Blocked on Cloud IPs: Runs a 3-layer native + yt-dlp caption pipeline without downloading full MP4 streams.

  • Empty {} health checks vs billed items: Empty input {} runs execute a read-only connectivity check and skip PPE item billing (apify_ppe_skipped: true), while real extraction runs emit transparent charged / charge_failed counters in the Actor summary.

  • Registry manifests included: smithery.yaml and server.json.

Available Tools

3 tools
omni_check_quotaA

Check remaining API key credits or daily anonymous free quota.

Inputs: api_key.

omni_extract_batchC

Concurrently extract up to 10 URLs (X Articles, Douyin, TikTok, YouTube, GitHub, Web) in parallel.

Inputs: urls (required), api_key, include_transcript.

omni_extract_urlB

Universal, cookie-free content & transcript extractor for AI Agents. Extracts full long-form X (Twitter) Articles/Tweets, watermark-free Douyin/TikTok videos & photo galleries, YouTube/Bilibili video subtitles/transcripts with timestamps, GitHub READMEs/code, and anti-bot web pages into clean Markdown.

Inputs: url (required), api_key, output_format, include_transcript.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.6/5.0

    Scored across 3 tools

    Disambiguation5/5

    Each tool targets a distinct operation: single-URL extraction, batched extraction, and quota checking. There is no realistic way to confuse batch extraction with a single extraction or with a credit check, and the descriptions reinforce the boundary.

    Naming Consistency5/5

    All three names follow a strict server-prefixed verb_noun pattern: omni_extract_url, omni_extract_batch, omni_check_quota. The shared 'omni_' namespace plus consistent verb forms makes the set fully predictable.

    Tool Count5/5

    Three tools is a tight, well-scoped surface for a stateless extraction service, and each one earns its place. Nothing redundant, nothing gratuitous.

    Completeness4/5

    The core workflow is fully covered: extract one, extract many, and monitor credits. Minor gaps exist, such as no way to select output format/depth, inspect supported-platform info, or check the status of a long-running batch, but agents can work around these.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      B
      maintenance
      A local-first MCP server that lets AI agents read any webpage as clean Markdown, crawl whole sites within configured limits, search without API keys, and solve supported captchas locally — all without cloud services or third-party keys.
      12
      22
      AGPL 3.0
    • A
      license
      Not graded
      quality
      C
      maintenance
      This self-hosted MCP server enables users to read public Xiaohongshu and Douyin content, including post text, images, video frames, and optional audio transcription, from compatible Streamable HTTP MCP clients like Claude and ChatGPT.
      MIT