Enables integration with n8n workflow automation platform, allowing real-time monitoring of YOLO operations through Server-Sent Events (SSE) and providing workflow automation for computer vision tasks like object detection, tracking, and model training.
Transforms Ultralytics' YOLO operations into a RESTful API service, providing programmatic access to training, validation, prediction, export, tracking, and benchmarking of YOLO models for computer vision tasks.
Exposes YOLO's object detection capabilities through a RESTful API, enabling training custom models, performing predictions, tracking objects in videos, and exporting models to different formats.
Ultralytics MCP Server 🚀
A powerful Model Context Protocol (MCP) compliant server that provides RESTful API access to Ultralytics YOLO operations for computer vision tasks including training, validation, prediction, export, tracking, and benchmarking.
🎯 What is this?
The Ultralytics MCP Server transforms Ultralytics' command-line YOLO operations into a production-ready REST API service. Whether you're building computer vision applications, training custom models, or integrating YOLO into your workflow automation tools like n8n, this server provides a seamless bridge between Ultralytics' powerful capabilities and modern application architectures.
✨ Key Features
- 🌐 RESTful API: HTTP endpoints for all YOLO operations with comprehensive request/response validation
- 📡 Real-time Updates: Server-Sent Events (SSE) for monitoring long-running operations like training
- 🤝 MCP Compliance: Full Model Context Protocol support with handshake endpoint and tool discovery for workflow automation
- 🐳 Production Ready: Docker containerization with multi-stage builds and security scanning
- 🧪 Battle Tested: Comprehensive test suite with CI/CD pipeline and 90%+ code coverage
- 📊 Observability: Built-in metrics parsing, health checks, and monitoring endpoints
- 🔒 Enterprise Security: API key authentication, input validation, and vulnerability scanning
- ⚡ CPU & GPU Support: Automatic device detection with graceful fallbacks
- 📚 Self-Documenting: Auto-generated OpenAPI/Swagger documentation
🏗️ Architecture Overview
Core Components:
app/main.py
: FastAPI application with route definitions and middlewareapp/schemas.py
: Pydantic models for comprehensive request/response validationapp/ultra.py
: Ultralytics CLI integration with metrics parsing and device managementtools/UltralyticsMCPTool
: TypeScript MCP client library for workflow automation
🚀 Quick Start
📋 Prerequisites
- Python 3.11+ (required for compatibility)
- Conda/Miniconda (recommended for environment management)
- Git (for cloning the repository)
- 4GB+ RAM (for model operations)
- Optional: NVIDIA GPU with CUDA support for faster training
⚡ One-Minute Setup
🔍 Verify Installation
After setup, verify everything works:
📖 What Just Happened?
- Environment Setup: Created isolated conda environment with PyTorch CPU support
- Dependency Installation: Installed Ultralytics, FastAPI, and all required packages
- Server Start: Launched FastAPI server with auto-reload for development
- API Test: Made a prediction request using a pre-trained YOLOv8 nano model
🟢 Real-time SSE with n8n
Live streaming updates for YOLO operations with Server-Sent Events (SSE)
Using SSE in n8n
- Drag MCP Client Tool ➜ set SSE Endpoint
http://host.docker.internal:8092/sse/train
(or compose DNSultra-api:8000/sse/train
). - In Settings ➜ OpenAPI URL use
http://host.docker.internal:8092/openapi.json
. - Set Timeout
0
to keep stream open. - Run workflow → live epoch/loss lines appear in the node's execution log.
🎯 Available SSE Endpoints:
/sse
- MCP handshake endpoint with tool discovery and keep-alive/sse/train
- Real-time training progress with epoch updates/sse/predict
- Live prediction results/sse/val
- Validation metrics streaming/sse/export
- Export progress updates/sse/track
- Object tracking stream/sse/benchmark
- Performance testing results/sse/solution
- Solution execution logs
📊 SSE Examples:
MCP Handshake:
Training with Live Progress:
🔗 OpenAPI Documentation: http://localhost:8092/docs#/default/sse_endpoint_sse__op__get
🧪 Running Tests
Our comprehensive test suite ensures reliability across all operations.
🏃♂️ Quick Test
🔬 Test Categories
Test Type | Command | Duration | What it Tests |
---|---|---|---|
Unit Tests | pytest tests/test_unit.py | ~10s | Individual functions |
Integration | pytest tests/test_flow.py | ~5min | Complete workflows |
Quick Check | python run_tests.py quick | ~30s | Endpoints only |
Full Suite | python run_tests.py | ~5min | Everything including training |
📊 Understanding Test Output
The integration test performs a complete YOLO workflow:
- 🏥 Health Check - Verify API is responsive
- 🏋️ Model Training - Train YOLOv8n for 1 epoch on COCO128
- 🔍 Model Validation - Validate the trained model
- 🎯 Prediction - Run inference on a test image
- � File Verification - Check all expected outputs were created
CI/CD Workflow
The project uses GitHub Actions for continuous integration and deployment. See .github/workflows/ci.yml
for the complete configuration.
Workflow Jobs
- 🧪 Test Job
- Sets up Conda environment with caching
- Runs pytest with coverage reporting
- Uploads coverage to Codecov
- 🐳 Build Job (on success)
- Builds Docker image with multi-stage optimization
- Pushes to GitHub Container Registry
- Supports multi-platform builds (amd64, arm64)
- 🔒 Security Job
- Runs Trivy vulnerability scanner
- Uploads SARIF results to GitHub Security
- 🔗 Integration Job
- Tests complete API workflow
- Validates endpoint responses
- Checks health and documentation endpoints
Workflow Triggers
- Push to
main
ordevelop
branches - Pull Requests to
main
branch - Manual workflow dispatch
Caching Strategy
Docker Deployment
Quick Deploy
Production Deployment
Environment Configuration
Key Variables:
ULTRA_API_KEY
: API authentication keyCUDA_VISIBLE_DEVICES
: GPU selectionMEMORY_LIMIT
: Container memory limit
Service Access
Once deployed, access the service at:
- API: http://localhost:8000
- Docs: http://localhost:8000/docs
- Prometheus (if enabled): http://localhost:9090
- Grafana (if enabled): http://localhost:3000
For detailed Docker configuration, see DOCKER.md.
📚 API Reference & Examples
🎯 Core Operations
Operation | Endpoint | Purpose | Example Use Case |
---|---|---|---|
Train | POST /train | Train custom models | Training on your dataset |
Validate | POST /val | Model performance testing | Check accuracy metrics |
Predict | POST /predict | Object detection/classification | Real-time inference |
Export | POST /export | Format conversion | Deploy to mobile/edge |
Track | POST /track | Object tracking in videos | Surveillance, sports analysis |
Benchmark | POST /benchmark | Performance testing | Hardware optimization |
📝 Request/Response Format
All endpoints return a standardized response structure:
🚀 Example Operations
1. Training a Custom Model
2. Real-time Prediction
3. Model Export for Deployment
4. Video Object Tracking
📊 Common Parameters Reference
Parameter | Type | Default | Description | Example |
---|---|---|---|---|
model | string | required | Model path or name | "yolov8n.pt" |
data | string | - | Dataset YAML path | "coco128.yaml" |
source | string | - | Input source | "image.jpg" , "video.mp4" , "0" (webcam) |
epochs | integer | 100 | Training epochs | 50 |
imgsz | integer | 640 | Image size | 320 , 640 , 1280 |
device | string | "cpu" | Compute device | "cpu" , "0" , "0,1" |
conf | float | 0.25 | Confidence threshold | 0.1 to 1.0 |
iou | float | 0.7 | IoU threshold for NMS | 0.1 to 1.0 |
batch | integer | 16 | Batch size | 1 , 8 , 32 |
save | boolean | false | Save results | true , false |
extra_args | object | {} | Additional YOLO args | {"patience": 10} |
🧪 Testing & Quality Assurance
🔬 Comprehensive Test Suite
Our testing infrastructure ensures reliability across all YOLO operations:
📊 Test Coverage
Component | Tests | Coverage | Description |
---|---|---|---|
Core Flow | 9 tests | 95%+ | Complete train→validate→predict workflow |
Training | 5 tests | 98% | Model training with various configurations |
Prediction | 4 tests | 97% | Inference on images, videos, webcam |
Export | 3 tests | 95% | Model format conversion (ONNX, TensorRT) |
Tracking | 3 tests | 92% | Object tracking in video streams |
Benchmark | 2 tests | 90% | Performance testing and profiling |
🚦 CI/CD Pipeline
🔍 Example Test Run
See tests/README.md
for detailed test documentation.
🚀 n8n Integration
n8n MCP Client Setup
🤝 MCP Handshake Protocol
The /sse
endpoint now serves as a Model Context Protocol (MCP) handshake endpoint:
- Initial Connection: When you connect to
/sse
, it immediately sends a tool discovery message: - Keep-Alive: After the handshake, it sends ping comments every 15 seconds to maintain the connection:
- Tool Discovery: MCP clients can discover available tools via:
- Manifest Endpoint:
GET /mcp/manifest.json
- Static tool definitions - SSE Handshake:
GET /sse
- Dynamic tool discovery with live connection
- Manifest Endpoint:
Streaming in n8n
- Drag MCP Client Tool → SSE Endpoint
http://host.docker.internal:8092/sse/train
(or/sse/predict
). - OpenAPI URL
http://host.docker.internal:8092/openapi.json
. - Timeout 0, Auth None.
- Run workflow → live epoch/loss lines appear in execution log (see GIF).
Available SSE Endpoints:
/sse
- MCP handshake endpoint with tool discovery and keep-alive/sse/train
- Real-time training progress with epoch updates/sse/predict
- Live prediction results/sse/val
- Validation metrics streaming/sse/export
- Export progress updates/sse/track
- Object tracking stream/sse/benchmark
- Performance testing results/sse/solution
- Solution execution logs
Example SSE Training Stream:
For detailed integration examples, see tools/UltralyticsMCPTool/README.md
.
1. Environment Setup
Add the Ultralytics API URL to your n8n environment:
2. Install UltralyticsMCPTool
3. n8n Node Configuration
Create a custom n8n node or use the HTTP Request node:
4. Workflow Examples
Image Classification Workflow:
- Trigger: Webhook receives image
- Ultralytics: Predict objects
- Logic: Process results
- Output: Send notifications
Training Pipeline:
- Schedule: Daily trigger
- Ultralytics: Train model
- Validate: Check performance
- Deploy: Update production model
5. MCP Integration
For detailed integration examples, see tools/UltralyticsMCPTool/README.md
.
🐳 Docker Deployment
🚀 Quick Docker Setup
📁 Docker Configuration
Production-ready setup:
Docker Compose services:
🔧 Environment Variables
Variable | Default | Description |
---|---|---|
YOLO_CACHE_DIR | /tmp/yolo | Model cache directory |
YOLO_SETTINGS_DIR | /tmp/settings | Settings directory |
API_HOST | 0.0.0.0 | API host binding |
API_PORT | 8000 | API port |
LOG_LEVEL | INFO | Logging level |
MAX_WORKERS | 4 | Uvicorn workers |
MODEL_DIR | /app/models | Model storage path |
🌐 Production Deployment
🔧 API Documentation
Response Format
All endpoints return a standardized response:
Error Handling
🛡️ Security & Authentication
📊 Health & Monitoring
🤝 Contributing Guidelines
We welcome contributions! Please follow these guidelines:
Development Setup
- Fork and Clone
- Create Environment
- Install Development Tools
Code Standards
- Python: Follow PEP 8, use Black for formatting
- TypeScript: Use ESLint and Prettier
- Documentation: Update README.md and docstrings
- Tests: Maintain 80%+ test coverage
Pre-commit Checks
Pull Request Process
- Create Feature Branch
- Make Changes
- Write code following standards
- Add/update tests
- Update documentation
- Test Changes
- Submit PR
- Clear description of changes
- Reference related issues
- Ensure CI passes
Issue Reporting
When reporting issues, include:
- Environment: OS, Python version, dependencies
- Steps: Minimal reproduction steps
- Expected: What should happen
- Actual: What actually happens
- Logs: Error messages and stack traces
Feature Requests
For new features:
- Use Case: Why is this needed?
- Proposal: How should it work?
- Impact: Who benefits from this?
- Implementation: Any technical considerations?
📄 License & Support
📝 License
This project is licensed under the MIT License - see the LICENSE file for details.
Key permissions:
- ✅ Commercial use
- ✅ Modification
- ✅ Distribution
- ✅ Private use
🆘 Getting Help
Resource | Link | Purpose |
---|---|---|
📚 API Docs | http://localhost:8000/docs | Interactive API documentation |
🐛 Issues | GitHub Issues | Bug reports & feature requests |
💬 Discussions | GitHub Discussions | Questions & community chat |
📖 Ultralytics | Official Docs | YOLO model documentation |
🔧 MCP Protocol | Specification | MCP standard reference |
🎯 Quick Support Checklist
Before asking for help:
- Check the FAQ for common issues
- Search existing GitHub Issues
- Test with the latest version
- Include environment details in your issue
When reporting bugs:
🙏 Acknowledgments
Component | Thanks To | For |
---|---|---|
🎯 YOLO Models | Ultralytics | Revolutionary object detection |
🚀 FastAPI | Sebastian Ramirez | Lightning-fast API framework |
🔧 Pydantic | Samuel Colvin | Data validation & settings |
🐳 Docker | Docker Inc | Containerization platform |
🧪 pytest | pytest-dev | Testing framework |
🌐 Conda | Anaconda | Package management |
🌟 Built with ❤️ for the Computer Vision Community 🌟
⭐ Star this repo | 🍴 Fork & contribute | 📢 Share with friends
Empowering developers to build intelligent computer vision applications with ease 🚀
This server cannot be installed
A Model Context Protocol compliant server that provides RESTful API access to Ultralytics YOLO operations for computer vision tasks including training, validation, prediction, export, tracking, and benchmarking.
Related MCP Servers
- -securityAlicense-qualityA Model Context Protocol server that provides access to Unity Catalog Functions, allowing AI assistants to list, get, create, and delete functions within Unity Catalog directly through a standardized interface.Last updated -14PythonMIT License
- -securityFlicense-qualityA Model Context Protocol server that enables natural language interactive control of Universal Robots collaborative robots, allowing users to control robot motion, monitor status, and execute programs through direct commands to large language models.Last updated -3Python
- -securityFlicense-qualityA Model Context Protocol server that provides a standardized interface for AI models and applications to interact with the Luno cryptocurrency exchange API for trading operations.Last updated -2Python
- AsecurityFlicenseAqualityA Model Context Protocol server that enables LLMs to explore and interact with API specifications by providing tools for loading, browsing, and getting detailed information about API endpoints.Last updated -4713TypeScript