# social-research-mcp
MCP server for social media research — scrape, search, and analyze across 7 platforms with an AI-powered timeline.
Works with Claude Code, Claude Desktop, Cursor, Windsurf, VS Code Copilot, and any MCP-compatible client.
## Features
- **7 Platform Scrapers** — Twitter/X, Instagram, TikTok, YouTube, LinkedIn, Facebook, Reddit (powered by [Apify](https://apify.com))
- **AI Timeline** — All scraped data is stored in a local LanceDB database with OpenAI embeddings for semantic search
- **Trend Detection** — Identify trending topics, sentiment shifts, and engagement spikes across platforms
- **Profile Analysis** — Engagement rates, posting frequency, content themes, peak hours
- **Sentiment Analysis** — Rule-based sentiment scoring with emoji support
- **Cross-Platform Comparison** — Compare profiles, hashtags, or topics side-by-side
## Quick Start
```bash
# Run with npx (no install needed)
APIFY_TOKEN=your-token npx social-research-mcp
# With timeline/embeddings support
APIFY_TOKEN=your-token OPENAI_API_KEY=your-key npx social-research-mcp
```
## Installation
```bash
npm install -g social-research-mcp
```
## Configuration
| Environment Variable | Required | Description |
|---|---|---|
| `APIFY_TOKEN` | Yes | Apify API token for scraping |
| `OPENAI_API_KEY` | No | OpenAI API key for timeline embeddings (semantic search) |
| `SOCIAL_RESEARCH_DATA_DIR` | No | Data directory (default: `~/.social-research-mcp/data/`) |
| `SOCIAL_RESEARCH_PORT` | No | HTTP port (default: `3847`) |
Get your [Apify token here](https://console.apify.com/account/integrations) and [OpenAI key here](https://platform.openai.com/api-keys).
## Usage with MCP Clients
### Claude Code
Add to your project's `.mcp.json`:
```json
{
"mcpServers": {
"social-research": {
"command": "npx",
"args": ["social-research-mcp"],
"env": {
"APIFY_TOKEN": "your-token",
"OPENAI_API_KEY": "your-key"
}
}
}
}
```
### Claude Desktop
Add to your `claude_desktop_config.json`:
```json
{
"mcpServers": {
"social-research": {
"command": "npx",
"args": ["social-research-mcp"],
"env": {
"APIFY_TOKEN": "your-token",
"OPENAI_API_KEY": "your-key"
}
}
}
}
```
### HTTP Mode
For clients that support HTTP/SSE transport:
```bash
social-research-mcp --http # Port 3847
social-research-mcp --http 8080 # Custom port
```
## Tools
### Scrape Tools (7)
Each scraper pulls data from its platform via Apify and automatically saves results to the local timeline database.
| Tool | Description |
|---|---|
| `scrape_twitter` | Search tweets by query, hashtag, or user timeline |
| `scrape_instagram` | Scrape posts by URL, hashtag, or search |
| `scrape_tiktok` | Scrape videos by profile, hashtag, or search |
| `scrape_youtube` | Scrape videos by search or channel URL |
| `scrape_linkedin` | Scrape posts by profile, company, or search |
| `scrape_facebook` | Scrape page posts by URL or search |
| `scrape_reddit` | Scrape posts by subreddit, search, or URL |
### Timeline Tools (4)
Query and analyze your locally stored timeline data.
| Tool | Description |
|---|---|
| `timeline_search` | Semantic search using AI embeddings (requires `OPENAI_API_KEY`) |
| `timeline_query` | Structured filtering by platform, date, engagement, author, hashtags |
| `timeline_trends` | Detect trending topics with growth rates and sentiment |
| `timeline_stats` | Aggregate stats: post counts, top authors, top hashtags |
### Analysis Tools (3)
| Tool | Description |
|---|---|
| `analyze_profile` | Full profile analysis with engagement benchmarks |
| `analyze_sentiment` | Sentiment scoring on timeline data or direct text |
| `compare` | Side-by-side comparison of profiles, hashtags, or topics |
## Architecture
```
src/
├── index.ts # CLI entry point
├── server.ts # MCP server (registers all tools)
├── config.ts # Environment configuration
├── types.ts # Shared types (UnifiedPost, Platform, etc.)
├── transport/
│ ├── stdio.ts # MCP stdio transport (default)
│ └── http.ts # HTTP/SSE transport
├── platforms/
│ ├── base.ts # Abstract platform class
│ ├── index.ts # Platform registry
│ └── [7 platforms] # Platform-specific scrapers
├── tools/
│ ├── scrape.ts # Scrape tool handlers
│ ├── timeline.ts # Timeline tool handlers
│ └── analysis.ts # Analysis tool handlers
├── db/
│ ├── lance.ts # LanceDB connection and queries
│ ├── schema.ts # Post row schema and converters
│ └── embeddings.ts # OpenAI embedding wrapper
└── analysis/
├── sentiment.ts # Rule-based sentiment analysis
├── trends.ts # Trend detection and topic clustering
└── engagement.ts # Platform-specific engagement formulas
```
All scraped data is normalized to a `UnifiedPost` schema, embedded via OpenAI, and stored in a local LanceDB database. This enables cross-platform semantic search and trend analysis over time.
## Apify Costs
Scraping costs are billed through your Apify account. Approximate costs:
| Platform | Approximate Cost |
|---|---|
| Twitter | ~$0.40 / 1k tweets |
| TikTok | ~$5 / 1k results |
| Others | Compute-based (varies) |
See [Apify pricing](https://apify.com/pricing) for details.
## Development
```bash
git clone https://github.com/TerminalGravity/social-research-mcp.git
cd social-research-mcp
npm install
npm run build
npm run typecheck
```
## License
MIT
