# SocialMCP

Unified Model Context Protocol (MCP) server for **YouTube**, **TikTok**, **Instagram**, and **Threads**.

It extracts video and post metadata, creator feeds, comments, direct media streams, and full transcripts. For videos without captions or videos made of screenshots and text slides, SocialMCP combines local **Faster-Whisper** audio transcription with **keyframe & carousel OCR** (`ffmpeg` + `Pillow` + `Tesseract`).

## What it does

- **TikTok** (no API key or login needed): video and photo-carousel metadata, view/like/comment/share/save counts, creator feeds, closed captions, Whisper audio transcription, and OCR for screenshot videos and slide carousels.
- **Instagram** (no API key needed; anonymous embed fallback included): Reels, photo posts, multi-slide carousels, creator feeds, direct CDN links, Whisper speech transcription, and OCR for screenshot Reels and image slides.
- **Threads** (no login needed): post content, replies/comments, topic/hashtag search, creator feeds, direct media links, and optional video/image transcription.
- **YouTube**:
  - Transcripts and caption tracks work without any API key (multi-tier caption fetcher + Whisper + keyframe OCR fallback).
  - Search, batch video details, and playlist management use the YouTube Data API v3 (OAuth credentials optional, only needed for these tools).

## System requirements

- Python 3.11+
- `ffmpeg` (for audio extraction and video keyframe sampling)
- `tesseract` with Russian and English language packs (`tesseract-ocr`, `tesseract-ocr-rus`, `tesseract-ocr-eng`) for transcribing screenshots and on-screen text

### Installing system dependencies

**macOS (Homebrew):**
```bash
brew install ffmpeg tesseract tesseract-lang
```

**Ubuntu / Debian:**
```bash
sudo apt update && sudo apt install -y ffmpeg tesseract-ocr tesseract-ocr-rus tesseract-ocr-eng
```

## Quick start (local stdio)

You can run SocialMCP directly with `uvx` without cloning the repository. Add this to your MCP client configuration (`claude_desktop_config.json`, Cursor, Windsurf, or Antigravity `mcp_config.json`):

```json
{
  "mcpServers": {
    "social": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/bledny1099/social-mcp.git",
        "social-mcp"
      ]
    }
  }
}
```

Or install it into a local environment with `pip` / `uv`:

```bash
git clone https://github.com/bledny1099/social-mcp.git
cd social-mcp
pip install -e .
social-mcp --transport stdio
```

## Self-hosting via Docker (remote HTTP MCP)

SocialMCP also runs as a remote Streamable HTTP server with Bearer token authentication and OAuth 2.1 support (compatible with Claude web/mobile connectors).

```bash
git clone https://github.com/bledny1099/social-mcp.git
cd social-mcp
cp .env.example .env
# Edit .env and set MCP_BEARER_TOKEN (e.g. openssl rand -hex 32)
docker compose up -d --build
```

The server listens on `127.0.0.1:8087` (mapped to container port `8080`, path `/mcp`). Put Nginx or Caddy in front of it with TLS (see `Caddyfile.snippet`).

## Available MCP tools (18)

### TikTok
| Tool | Description |
| --- | --- |
| `tiktok_get_video` | Metadata, stats (views, likes, comments, shares, saves), music info, direct stream URLs, and carousel slides |
| `tiktok_get_transcript` | Transcript from subtitles, Whisper audio, and keyframe/carousel OCR for screenshot and slide videos |
| `tiktok_get_user_videos` | Recent videos from a TikTok creator handle or profile URL |

### Instagram
| Tool | Description |
| --- | --- |
| `instagram_get_media` | Metadata, caption, stats, direct video/audio URLs, and carousel items for Reels and posts |
| `instagram_get_transcript` | Whisper audio transcription + keyframe and carousel slide OCR for Reels and image posts |
| `instagram_get_user_posts` | Recent posts from an Instagram profile |

### Threads
| Tool | Description |
| --- | --- |
| `threads_get_post` | Full post text, stats, media URLs, optional media OCR/transcription, and top replies |
| `threads_search_posts` | Public search by keyword, phrase, or hashtag |
| `threads_get_user_posts` | Profile details and recent posts from a Threads user |

### YouTube
| Tool | Description | Quota cost |
| --- | --- | --- |
| `youtube_get_transcript` | Captions + Whisper/OCR fallback, paged by character `offset` | 0 |
| `youtube_list_transcript_languages` | List manual and auto-generated caption tracks | 0 |
| `youtube_search` | Search videos with channel, date, duration, and sort filters | 100 (+1) |
| `youtube_get_videos` | Batch metadata for up to 50 video IDs or URLs | 1 |
| `youtube_list_playlists` | List your playlists | 1 |
| `youtube_list_playlist_items` | Read items in a playlist | 1 (+1) |
| `youtube_add_to_playlist` | Append videos to a playlist | 50 per video |
| `youtube_remove_from_playlist` | Remove items by `playlist_item_id` | 50 per video |
| `youtube_quota_status` | Estimated daily YouTube Data API quota usage | 0 |

## How screenshot and slide video transcription works

Short-form videos on TikTok, Reels, and Shorts often have background music instead of speech while showing screenshots, chat logs, or text slides on screen.

When `tiktok_get_transcript`, `instagram_get_transcript`, or `youtube_get_transcript` runs:
1. If the post is an image carousel or photo-mode slideshow, SocialMCP downloads the slides concurrently and runs OCR on each image.
2. If the post is a video, `ffmpeg` extracts scene changes and time-spaced keyframes.
3. Consecutive frames are compared using a 256-bit perceptual difference hash on the central 80% region (ignoring animated corner watermarks) so identical frames are skipped before OCR.
4. `Pillow` checks frame luminance, inverts dark-mode screenshots so white-on-dark text becomes sharp black-on-white, applies contrast normalization, and passes the result to `tesseract` (`rus+eng`) in TSV mode.
5. Low-confidence noise tokens are filtered out, scrolling screenshots are deduplicated line-by-line across frames, and the extracted on-screen text is merged with the Whisper speech transcript.

## Optional YouTube Data API setup

You only need Google OAuth credentials if you want to use `youtube_search`, `youtube_get_videos`, or the playlist tools (`youtube_get_transcript` works without them).

1. Create a project in Google Cloud Console and enable **YouTube Data API v3**.
2. Create OAuth 2.0 Web Application credentials and add `http://localhost:8765/` to Authorized redirect URIs.
3. Run the local helper script to get your refresh token:
   ```bash
   python scripts/get_refresh_token.py --client-id YOUR_ID --client-secret YOUR_SECRET
   ```
4. Paste `YT_CLIENT_ID`, `YT_CLIENT_SECRET`, and `YT_REFRESH_TOKEN` into `.env`.

## Environment variables

| Variable | Default | Description |
| --- | --- | --- |
| `MCP_HOST` | `127.0.0.1` | Set to `0.0.0.0` for HTTP server mode; omit for `stdio` |
| `MCP_PORT` | `8080` | HTTP server port |
| `MCP_PATH` | `/mcp` | HTTP endpoint path |
| `MCP_BEARER_TOKEN` | unset | Static bearer token for HTTP mode |
| `WHISPER_MODEL` | `base` | Faster-Whisper model (`tiny`, `base`, `small`, `medium`, `large-v3-turbo`) |
| `WHISPER_DEVICE` | `cpu` | `cpu` or `cuda` |
| `WHISPER_COMPUTE_TYPE` | `int8` | Quantization type (`int8`, `float16`, `float32`) |
| `OCR_LANGUAGES` | `rus+eng` | Tesseract language packs for screenshot/frame OCR |
| `SOCIAL_MCP_PROXY` | unset | Optional HTTP/SOCKS5 proxy URL for scraping from datacenter IPs |
| `YT_CLIENT_ID` / `YT_CLIENT_SECRET` / `YT_REFRESH_TOKEN` | unset | Google OAuth credentials for YouTube Data API tools |
| `YT_WATCH_LATER_PLAYLIST_ID` | unset | Default playlist ID for playlist tools |

## License

MIT