# Link Scan MCP Server π
λ§ν¬λ₯Ό μ€μΊνκ³ μμ½μ μ 곡νλ ν¬κ΄μ μΈ Model Context Protocol (MCP) μλ²μ
λλ€. YouTube, Instagram Reels λ± λΉλμ€ λ§ν¬μ λΈλ‘κ·Έ, κΈ°μ¬ λ± ν
μ€νΈ λ§ν¬λ₯Ό μλμΌλ‘ κ°μ§νκ³ λΆμνμ¬ 3λ¬Έμ₯ μ΄λ΄μ κ°κ²°ν μμ½μ μ 곡ν©λλ€. API ν€ μμ΄ λͺ¨λ κΈ°λ₯μ μ¬μ©ν μ μμ΅λλ€!
**Link Scan MCP Server** - A comprehensive Model Context Protocol (MCP) server for scanning and summarizing links. Automatically detects and analyzes video links (YouTube, Instagram Reels) and text links (blogs, articles) to provide concise 3-sentence summaries. All features work without requiring API keys!
**Python 3.11+** | **MCP Compatible** | **License: MIT**
## β¨ Features
### π₯ Video Link Analysis
- **YouTube Support**
- Comprehensive metadata extraction (title, description)
- Subtitle extraction for first 7 seconds (yt-dlp)
- Audio transcription using OpenAI Whisper
- Integrated summarization combining all text sources
- **Instagram Reels Support**
- Audio download and transcription (first 7 seconds)
- Automatic content summarization
- **Smart Link Detection**
- Automatic video/text link type detection
- Error handling for unsupported URLs
### π Text Link Analysis
- **Web Content Extraction**
- BeautifulSoup-based HTML parsing
- Main content area detection
- Automatic navigation/ad removal
- **Intelligent Summarization**
- Llama3-powered text summarization
- 3-sentence limit enforcement
- Natural Korean output
### π€ AI-Powered Summarization
- **Llama3 Integration**
- Local LLM via Ollama (no API keys required)
- Separate prompts for video and text content
- Fallback to original text on errors
- **Whisper Transcription**
- High-quality speech-to-text conversion
- Optimized for speed and accuracy
- Supports multiple languages
### π³ Docker Support
- **One-Command Setup**
- Docker Compose configuration
- Automatic Ollama service setup
- Llama3 model auto-download
- Development mode with hot reload
### π§ Developer-Friendly
- **Type-safe with Pydantic models**
- **Async/await support** for better performance
- **Comprehensive error handling**
- **Extensible architecture**
- **Hot reload** in development mode
## π Quick Start
### Installation
```bash
# Clone the repository
git clone https://github.com/your-username/mcp-link-scan.git
cd mcp-link-scan
# Install dependencies
pip install -r requirements.txt
```
### System Dependencies
**ffmpeg** (required for audio processing):
- macOS: `brew install ffmpeg`
- Ubuntu/Debian: `sudo apt-get install ffmpeg`
- Windows: Download from https://ffmpeg.org/download.html
**Ollama** (required for summarization):
- macOS: `brew install ollama` or download from https://ollama.com/download
- Linux: `curl -fsSL https://ollama.com/install.sh | sh`
- Windows: Download from https://ollama.com/download
- After installation: `ollama pull llama3:latest`
### Configuration
Create a `.env` file:
```bash
# μλ² μ€μ
PORT=8000 # μλ² ν¬νΈ (κΈ°λ³Έκ°: 8000)
HOST=0.0.0.0 # μλ² νΈμ€νΈ (κΈ°λ³Έκ°: 0.0.0.0)
DEBUG=False # λλ²κ·Έ λͺ¨λ (κΈ°λ³Έκ°: False)
# API κ²½λ‘ prefix (μ ν)
# κ°μ μλ²μ μ¬λ¬ MCP μλ²λ₯Ό νΈμ€ν
ν λ μ¬μ©
# κΈ°λ³Έκ°: /link-scan
API_PREFIX=/link-scan
# Ollama μ€μ (μ ν)
# Docker Composeλ₯Ό μ¬μ©νλ κ²½μ° μλμΌλ‘ μ€μ λ¨
OLLAMA_API_URL=http://localhost:11434 # Ollama API URL (κΈ°λ³Έκ°: http://localhost:11434)
OLLAMA_MODEL=llama3:latest # μ¬μ©ν Ollama λͺ¨λΈ (κΈ°λ³Έκ°: llama3)
```
#### νκ²½ λ³μ μ€λͺ
| λ³μλͺ
| νμ | κΈ°λ³Έκ° | μ€λͺ
|
|--------|------|--------|------|
| `PORT` | β | `8000` | μλ²κ° μ¬μ©ν ν¬νΈ λ²νΈ |
| `HOST` | β | `0.0.0.0` | μλ²κ° λ°μΈλ©ν νΈμ€νΈ μ£Όμ |
| `DEBUG` | β | `False` | λλ²κ·Έ λͺ¨λ νμ±ν (`True`/`False`) |
| `API_PREFIX` | β | `/link-scan` | API μλν¬μΈνΈ κ²½λ‘ prefix |
| `OLLAMA_API_URL` | β | `http://localhost:11434` | Ollama API μλ² URL |
| `OLLAMA_MODEL` | β | `llama3` | μ¬μ©ν Ollama λͺ¨λΈ μ΄λ¦ |
### Running as MCP Server
**Local Mode (stdio):**
```bash
python -m src.server
```
**Remote Mode (HTTP):**
```bash
python run_server.py
```
Or with uvicorn directly:
```bash
uvicorn src.server_http:app --host 0.0.0.0 --port 8000
```
### Docker Setup (Recommended)
**Using Docker Compose:**
```bash
# Start all services (link-scan + Ollama)
docker-compose up -d
# Check logs
docker-compose logs -f
# Stop services
docker-compose down
```
Docker Compose automatically:
- Sets up Ollama service with 8GB memory
- Downloads Llama3 model
- Configures link-scan service
- Enables development mode with hot reload
**Development Mode:**
The `docker-compose.yml` is configured for development with:
- Source code volume mounting
- Hot reload enabled (`DEBUG=True`)
- Automatic code changes detection
### Testing with MCP Inspector
You can test the server using the MCP Inspector tool:
```bash
# Test with Python
npx @modelcontextprotocol/inspector python run_server.py
# Or test stdio mode
npx @modelcontextprotocol/inspector python -m src.server
```
The MCP Inspector provides a web interface to:
- View available tools and their schemas
- Test tool execution with sample inputs
- Debug server responses and error handling
- Validate MCP protocol compliance
## π οΈ Available Tools
### 1. `scan_video_link`
Scan and summarize video links (YouTube, Instagram Reels, etc.).
**Parameters:**
- `url` (string, required): Video URL to scan
**Example:**
```json
{
"name": "scan_video_link",
"arguments": {
"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
}
}
```
**Process:**
1. Detects link type (YouTube, Instagram, etc.)
2. For YouTube: Extracts title, description, subtitles (first 7s)
3. Downloads audio (first 7 seconds)
4. Transcribes audio with Whisper
5. Combines all text sources
6. Summarizes with Llama3 (3 sentences max)
### 2. `scan_text_link`
Scan and summarize text links (blogs, articles, etc.).
**Parameters:**
- `url` (string, required): Text URL to scan
**Example:**
```json
{
"name": "scan_text_link",
"arguments": {
"url": "https://example.com/blog/article"
}
}
```
**Process:**
1. Fetches HTML content
2. Extracts main text content
3. Removes navigation, ads, and noise
4. Summarizes with Llama3 (3 sentences max)
## π Example Outputs
### Video Link Summary
**Input:** YouTube video URL
**Output:**
```
μ΄ μμμ Python νλ‘κ·Έλλ° μΈμ΄μ κΈ°λ³Έ κ°λ
μ μκ°ν©λλ€.
λ³μ, ν¨μ, ν΄λμ€ λ± ν΅μ¬ λ¬Έλ²μ μ€μ΅ μμ μ ν¨κ» μ€λͺ
ν©λλ€.
μ΄λ³΄μλ μ½κ² λ°λΌν μ μλλ‘ λ¨κ³λ³λ‘ ꡬμ±λμ΄ μμ΅λλ€.
```
### Text Link Summary
**Input:** Blog article URL
**Output:**
```
μ΄ κΈμ Docker 컨ν
μ΄λ κΈ°μ μ μ₯λ¨μ μ λΆμν©λλ€.
κ°μν κΈ°μ κ³Ό λΉκ΅νμ¬ λ¦¬μμ€ ν¨μ¨μ±κ³Ό λ°°ν¬ νΈμμ±μ κ°μ μΌλ‘ μ μν©λλ€.
λ€λ§ 보μκ³Ό 볡μ‘μ± μΈ‘λ©΄μμ μ£Όμκ° νμνλ€κ³ μ‘°μΈν©λλ€.
```
## ποΈ Architecture
```
mcp-link-scan/
βββ src/
β βββ server.py # Local server (stdio)
β βββ server_http.py # Remote server (HTTP)
β βββ tools/ # MCP tools
β β βββ link_scanner.py # Main tool definitions
β β βββ media_handler.py # Video processing (Whisper)
β β βββ text_handler.py # Text extraction
β βββ utils/ # Utilities
β β βββ link_detector.py # Link type detection
β β βββ youtube_extractor.py # YouTube metadata/subtitles
β β βββ llm_summarizer.py # Llama3 integration
β βββ prompts/ # LLM prompts
β βββ __init__.py # Video/text prompt templates
βββ docker/
β βββ init-ollama.sh # Ollama initialization script
βββ docker-compose.yml # Docker services
βββ Dockerfile # Container build config
βββ requirements.txt # Python dependencies
βββ run_server.py # Server entry point
```
## π§ Development
### Setting up Development Environment
```bash
# Clone and install
git clone https://github.com/your-username/mcp-link-scan.git
cd mcp-link-scan
pip install -r requirements.txt
# Set up environment variables
cp .env.example .env
# Edit .env with your settings
# Start Ollama (if not using Docker)
ollama serve
ollama pull llama3:latest
```
### Development Mode with Docker
```bash
# Start in development mode (hot reload enabled)
docker-compose up -d
# View logs
docker-compose logs -f link-scan
# Code changes are automatically reloaded
```
### Running Tests
```bash
# Run all tests
pytest
# Run with coverage
pytest --cov=src
# Run specific test file
pytest tests/test_link_scanner.py
```
### Customizing Prompts
Edit `src/prompts/__init__.py` to customize LLM prompts:
```python
# Video summarization prompt
VIDEO_SUMMARIZE_SYSTEM = """
Your custom system prompt here...
"""
# Text summarization prompt
TEXT_SUMMARIZE_SYSTEM = """
Your custom system prompt here...
"""
```
### Configuring Whisper Model
Edit `src/tools/media_handler.py`:
```python
# Change model size (tiny, base, small, medium, large)
_whisper_model = whisper.load_model("base") # Default: "base"
```
## π Requirements
- **Python 3.11+**
- **ffmpeg** - Audio processing
- **Ollama** - LLM runtime (for summarization)
- **yt-dlp** - Video/audio download
- **openai-whisper** - Speech-to-text
- **torch** - PyTorch (for Whisper)
- **aiohttp** - Async HTTP client
- **beautifulsoup4** - HTML parsing
- **fastapi** - HTTP server framework
- **uvicorn** - ASGI server
- **mcp** - Model Context Protocol SDK
## π Deployment
### PlayMCP Registration
1. **Deploy Server**: Deploy to cloud hosting (Render, Railway, Fly.io, AWS, GCP, etc.)
2. **Get Server URL**: Example: `https://your-server.railway.app`
3. **Register in PlayMCP**: Use URL `https://your-server.railway.app/messages`
**Important:** Server URL must be publicly accessible and support HTTPS for production use.
### Using with MCP Clients
**Amazon Q CLI:**
```json
{
"mcpServers": {
"link-scan": {
"command": "python",
"args": ["run_server.py"],
"cwd": "/path/to/mcp-link-scan"
}
}
}
```
**Other MCP Clients:**
```json
{
"mcpServers": {
"link-scan": {
"url": "https://your-server.com/messages"
}
}
}
```
## π€ Contributing
We welcome contributions! Please follow these steps:
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass (`pytest`)
6. Commit your changes (`git commit -m 'Add amazing feature'`)
7. Push to the branch (`git push origin feature/amazing-feature`)
8. Open a Pull Request
### Development Workflow
```bash
# Install in development mode
pip install -e .
# Run tests
pytest
# Format code (if using formatters)
black src/ tests/
isort src/ tests/
```
## π License
This project is licensed under the MIT License - see the LICENSE file for details.
## π Acknowledgments
- **yt-dlp** team for the excellent YouTube extraction library
- **OpenAI Whisper** team for the speech-to-text model
- **Ollama** team for the local LLM runtime
- **MCP** team for the Model Context Protocol specification
- **Pydantic** team for the data validation library
## π Support
- **Issues**: [GitHub Issues](https://github.com/your-username/mcp-link-scan/issues)
- **Discussions**: [GitHub Discussions](https://github.com/your-username/mcp-link-scan/discussions)
## πΊοΈ Roadmap
- [ ] Batch processing for multiple links
- [ ] Caching layer for improved performance
- [ ] Export functionality (JSON, CSV, etc.)
- [ ] Advanced analytics (sentiment analysis, topic extraction)
- [ ] Support for more video platforms (TikTok, Vimeo, etc.)
- [ ] WebSocket support for real-time updates
- [ ] Integration examples with popular MCP clients
- [ ] Custom prompt templates via API
- [ ] Multi-language support for summaries
- [ ] Video thumbnail extraction
## π Notes
- Audio downloads are temporarily stored and automatically cleaned up
- Whisper model is loaded once and reused for better performance
- Processing time depends on video length and Whisper model size
- **YouTube videos are processed for first 7 seconds only** to reduce processing time
- **All text sources (title, description, subtitles, transcription) are combined for YouTube videos**
- **Summaries are limited to 3 sentences maximum**
- For production, consider using GPU for faster Whisper conversion
- Ollama timeout is set to 5 minute for tool calls
