Provides capabilities to scrape Facebook page posts by URL or search for social media research and data analysis.
Enables scraping of Instagram posts via URL, hashtag, or search, allowing for profile analysis and content theme identification.
Uses OpenAI API for generating embeddings to enable semantic search and AI-powered timeline analysis of scraped social media data.
Facilitates scraping of Reddit posts by subreddit, search terms, or specific URLs to track discussions and sentiment.
Supports scraping TikTok videos by profile, hashtag, or search for monitoring trends and engagement on the platform.
Provides tools to scrape YouTube video data by search or channel URL for social media research and cross-platform comparison.
social-research-mcp
MCP server for social media research — scrape, search, and analyze across 7 platforms with an AI-powered timeline.
Works with Claude Code, Claude Desktop, Cursor, Windsurf, VS Code Copilot, and any MCP-compatible client.
Features
7 Platform Scrapers — Twitter/X, Instagram, TikTok, YouTube, LinkedIn, Facebook, Reddit (powered by Apify)
AI Timeline — All scraped data is stored in a local LanceDB database with OpenAI embeddings for semantic search
Trend Detection — Identify trending topics, sentiment shifts, and engagement spikes across platforms
Profile Analysis — Engagement rates, posting frequency, content themes, peak hours
Sentiment Analysis — Rule-based sentiment scoring with emoji support
Cross-Platform Comparison — Compare profiles, hashtags, or topics side-by-side
Quick Start
# Run with npx (no install needed)
APIFY_TOKEN=your-token npx social-research-mcp
# With timeline/embeddings support
APIFY_TOKEN=your-token OPENAI_API_KEY=your-key npx social-research-mcpInstallation
npm install -g social-research-mcpConfiguration
Environment Variable | Required | Description |
| Yes | Apify API token for scraping |
| No | OpenAI API key for timeline embeddings (semantic search) |
| No | Data directory (default: |
| No | HTTP port (default: |
Get your Apify token here and OpenAI key here.
Usage with MCP Clients
Claude Code
Add to your project's .mcp.json:
{
"mcpServers": {
"social-research": {
"command": "npx",
"args": ["social-research-mcp"],
"env": {
"APIFY_TOKEN": "your-token",
"OPENAI_API_KEY": "your-key"
}
}
}
}Claude Desktop
Add to your claude_desktop_config.json:
{
"mcpServers": {
"social-research": {
"command": "npx",
"args": ["social-research-mcp"],
"env": {
"APIFY_TOKEN": "your-token",
"OPENAI_API_KEY": "your-key"
}
}
}
}HTTP Mode
For clients that support HTTP/SSE transport:
social-research-mcp --http # Port 3847
social-research-mcp --http 8080 # Custom portTools
Scrape Tools (7)
Each scraper pulls data from its platform via Apify and automatically saves results to the local timeline database.
Tool | Description |
| Search tweets by query, hashtag, or user timeline |
| Scrape posts by URL, hashtag, or search |
| Scrape videos by profile, hashtag, or search |
| Scrape videos by search or channel URL |
| Scrape posts by profile, company, or search |
| Scrape page posts by URL or search |
| Scrape posts by subreddit, search, or URL |
Timeline Tools (4)
Query and analyze your locally stored timeline data.
Tool | Description |
| Semantic search using AI embeddings (requires |
| Structured filtering by platform, date, engagement, author, hashtags |
| Detect trending topics with growth rates and sentiment |
| Aggregate stats: post counts, top authors, top hashtags |
Analysis Tools (3)
Tool | Description |
| Full profile analysis with engagement benchmarks |
| Sentiment scoring on timeline data or direct text |
| Side-by-side comparison of profiles, hashtags, or topics |
Architecture
src/
├── index.ts # CLI entry point
├── server.ts # MCP server (registers all tools)
├── config.ts # Environment configuration
├── types.ts # Shared types (UnifiedPost, Platform, etc.)
├── transport/
│ ├── stdio.ts # MCP stdio transport (default)
│ └── http.ts # HTTP/SSE transport
├── platforms/
│ ├── base.ts # Abstract platform class
│ ├── index.ts # Platform registry
│ └── [7 platforms] # Platform-specific scrapers
├── tools/
│ ├── scrape.ts # Scrape tool handlers
│ ├── timeline.ts # Timeline tool handlers
│ └── analysis.ts # Analysis tool handlers
├── db/
│ ├── lance.ts # LanceDB connection and queries
│ ├── schema.ts # Post row schema and converters
│ └── embeddings.ts # OpenAI embedding wrapper
└── analysis/
├── sentiment.ts # Rule-based sentiment analysis
├── trends.ts # Trend detection and topic clustering
└── engagement.ts # Platform-specific engagement formulasAll scraped data is normalized to a UnifiedPost schema, embedded via OpenAI, and stored in a local LanceDB database. This enables cross-platform semantic search and trend analysis over time.
Apify Costs
Scraping costs are billed through your Apify account. Approximate costs:
Platform | Approximate Cost |
~$0.40 / 1k tweets | |
TikTok | ~$5 / 1k results |
Others | Compute-based (varies) |
See Apify pricing for details.
Development
git clone https://github.com/TerminalGravity/social-research-mcp.git
cd social-research-mcp
npm install
npm run build
npm run typecheckLicense
MIT