# social-research-mcp

MCP server for social media research — scrape, search, and analyze across 7 platforms with an AI-powered timeline.

Works with Claude Code, Claude Desktop, Cursor, Windsurf, VS Code Copilot, and any MCP-compatible client.

## Features

- **7 Platform Scrapers** — Twitter/X, Instagram, TikTok, YouTube, LinkedIn, Facebook, Reddit (powered by [Apify](https://apify.com))
- **AI Timeline** — All scraped data is stored in a local LanceDB database with OpenAI embeddings for semantic search
- **Trend Detection** — Identify trending topics, sentiment shifts, and engagement spikes across platforms
- **Profile Analysis** — Engagement rates, posting frequency, content themes, peak hours
- **Sentiment Analysis** — Rule-based sentiment scoring with emoji support
- **Cross-Platform Comparison** — Compare profiles, hashtags, or topics side-by-side

## Quick Start

```bash
# Run with npx (no install needed)
APIFY_TOKEN=your-token npx social-research-mcp

# With timeline/embeddings support
APIFY_TOKEN=your-token OPENAI_API_KEY=your-key npx social-research-mcp
```

## Installation

```bash
npm install -g social-research-mcp
```

## Configuration

| Environment Variable | Required | Description |
|---|---|---|
| `APIFY_TOKEN` | Yes | Apify API token for scraping |
| `OPENAI_API_KEY` | No | OpenAI API key for timeline embeddings (semantic search) |
| `SOCIAL_RESEARCH_DATA_DIR` | No | Data directory (default: `~/.social-research-mcp/data/`) |
| `SOCIAL_RESEARCH_PORT` | No | HTTP port (default: `3847`) |

Get your [Apify token here](https://console.apify.com/account/integrations) and [OpenAI key here](https://platform.openai.com/api-keys).

## Usage with MCP Clients

### Claude Code

Add to your project's `.mcp.json`:

```json
{
  "mcpServers": {
    "social-research": {
      "command": "npx",
      "args": ["social-research-mcp"],
      "env": {
        "APIFY_TOKEN": "your-token",
        "OPENAI_API_KEY": "your-key"
      }
    }
  }
}
```

### Claude Desktop

Add to your `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "social-research": {
      "command": "npx",
      "args": ["social-research-mcp"],
      "env": {
        "APIFY_TOKEN": "your-token",
        "OPENAI_API_KEY": "your-key"
      }
    }
  }
}
```

### HTTP Mode

For clients that support HTTP/SSE transport:

```bash
social-research-mcp --http          # Port 3847
social-research-mcp --http 8080     # Custom port
```

## Tools

### Scrape Tools (7)

Each scraper pulls data from its platform via Apify and automatically saves results to the local timeline database.

| Tool | Description |
|---|---|
| `scrape_twitter` | Search tweets by query, hashtag, or user timeline |
| `scrape_instagram` | Scrape posts by URL, hashtag, or search |
| `scrape_tiktok` | Scrape videos by profile, hashtag, or search |
| `scrape_youtube` | Scrape videos by search or channel URL |
| `scrape_linkedin` | Scrape posts by profile, company, or search |
| `scrape_facebook` | Scrape page posts by URL or search |
| `scrape_reddit` | Scrape posts by subreddit, search, or URL |

### Timeline Tools (4)

Query and analyze your locally stored timeline data.

| Tool | Description |
|---|---|
| `timeline_search` | Semantic search using AI embeddings (requires `OPENAI_API_KEY`) |
| `timeline_query` | Structured filtering by platform, date, engagement, author, hashtags |
| `timeline_trends` | Detect trending topics with growth rates and sentiment |
| `timeline_stats` | Aggregate stats: post counts, top authors, top hashtags |

### Analysis Tools (3)

| Tool | Description |
|---|---|
| `analyze_profile` | Full profile analysis with engagement benchmarks |
| `analyze_sentiment` | Sentiment scoring on timeline data or direct text |
| `compare` | Side-by-side comparison of profiles, hashtags, or topics |

## Architecture

```
src/
├── index.ts            # CLI entry point
├── server.ts           # MCP server (registers all tools)
├── config.ts           # Environment configuration
├── types.ts            # Shared types (UnifiedPost, Platform, etc.)
├── transport/
│   ├── stdio.ts        # MCP stdio transport (default)
│   └── http.ts         # HTTP/SSE transport
├── platforms/
│   ├── base.ts         # Abstract platform class
│   ├── index.ts        # Platform registry
│   └── [7 platforms]   # Platform-specific scrapers
├── tools/
│   ├── scrape.ts       # Scrape tool handlers
│   ├── timeline.ts     # Timeline tool handlers
│   └── analysis.ts     # Analysis tool handlers
├── db/
│   ├── lance.ts        # LanceDB connection and queries
│   ├── schema.ts       # Post row schema and converters
│   └── embeddings.ts   # OpenAI embedding wrapper
└── analysis/
    ├── sentiment.ts    # Rule-based sentiment analysis
    ├── trends.ts       # Trend detection and topic clustering
    └── engagement.ts   # Platform-specific engagement formulas
```

All scraped data is normalized to a `UnifiedPost` schema, embedded via OpenAI, and stored in a local LanceDB database. This enables cross-platform semantic search and trend analysis over time.

## Apify Costs

Scraping costs are billed through your Apify account. Approximate costs:

| Platform | Approximate Cost |
|---|---|
| Twitter | ~$0.40 / 1k tweets |
| TikTok | ~$5 / 1k results |
| Others | Compute-based (varies) |

See [Apify pricing](https://apify.com/pricing) for details.

## Development

```bash
git clone https://github.com/TerminalGravity/social-research-mcp.git
cd social-research-mcp
npm install
npm run build
npm run typecheck
```

## License

MIT