mcp-content-pipeline

A content analysis and digest pipeline for YouTube videos and X (Twitter) feeds, exposed as MCP tools: extract English transcripts, fetch posts from curated accounts, generate TL;DRs and key points, sync to GitHub.

flowchart LR
    A[YouTube URL<br/>or X feed] --> B[Extract content<br/>YouTube captions / X API]
    B --> C[LLM analysis<br/>TL;DR, key points]
    C --> D[Sync to GitHub<br/>markdown]

Quick Start

git clone https://github.com/berkayildi/mcp-content-pipeline.git
cd mcp-content-pipeline
cp .env.example .env   # fill in the keys for the tools you use
uv sync

Configuration

All config lives in a local .env (gitignored). On startup the server loads the nearest .env from its working directory upwards; set MCP_CP_ENV_FILE to load one from an explicit path instead. Variables already present in the process environment take precedence over .env.

MCP client

Launch the server from the clone so it picks up .env — the same block works for Claude Desktop, Claude Code (.mcp.json), and other MCP clients:

{
  "mcpServers": {
    "content-pipeline": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/mcp-content-pipeline", "mcp-content-pipeline"]
    }
  }
}

To run the published package without a clone, use uvx mcp-content-pipeline and point it at your .env:

{
  "mcpServers": {
    "content-pipeline": {
      "command": "uvx",
      "args": ["mcp-content-pipeline"],
      "env": { "MCP_CP_ENV_FILE": "/path/to/.env" }
    }
  }
}

Related MCP server: YouTube Transcripts MCP Server

Usage

YouTube Analysis

"Use content-pipeline to analyse this video: https://www.youtube.com/watch?v=..." "Sync the analysis to GitHub"

Or all in one prompt:

"Use content-pipeline to analyse this video and sync to GitHub: https://www.youtube.com/watch?v=..."

X Feed Digest

"Analyse the X feed" "Analyse the X feed for karpathy, bcherny, atmoio, and steipete about AI today" "Analyse the X feed from the last 7 days"

Or with the full pipeline:

"Analyse the X feed and sync to GitHub"

Tools

Environment Variables

All prefixed with MCP_CP_ and set in .env — see .env.example for a ready-to-copy template.

Pipeline driver — required for analyse_video, analyse_x_feed

YouTube — transcripts come from public English captions, no key needed

GitHub — required for sync_to_github

X/Twitter — required for analyse_x_feed

Cost Projections

Estimated monthly costs for two usage patterns:

LLM costs are estimated for the default claude-sonnet-4-6 driver and vary with PIPELINE_PROVIDER / PIPELINE_MODEL; they are billed per token by the provider's API (a chat subscription such as Claude Pro does not cover API usage) and are not included in the totals. The X API spending cap can be configured in the developer console.

What this replaces

Eval Gate

PRs touching services/, tools/, config.py, server.py, eval/, .eval-gate.yml or pyproject.toml run .github/workflows/eval-gate.yml: mcp-llm-eval scores faithfulness/relevance of 4 Anthropic/OpenAI models against eval/dataset.json and blocks the PR below the thresholds in .eval-gate.yml. Needs ANTHROPIC_API_KEY and OPENAI_API_KEY repo secrets.

Run it locally (keys exported, unprefixed):

uvx --with "mcp<2" --with anthropic --with openai mcp-llm-eval run --config .eval-gate.yml --dataset eval/dataset.json --output-dir eval/results
uvx --with "mcp<2" mcp-llm-eval check --results eval/results/latest_summary.json --config .eval-gate.yml