OpenEye
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
OpenEye
Open-source eyes for the agent era.
Website · Docs · Models · Blog · PyPI
OpenEye is a CLI-first perception engine that turns raw video into structured world models for robots and autonomous agents. Pull, run, and serve computer-vision models from your terminal — like Ollama, but for vision AI.
Features
Object Detection — YOLOv8, YOLO26 (currently ships YOLO11n weights), Grounding DINO, RF-DETR, and custom adapters
Depth Estimation — Monocular depth via Depth Anything V2
Segmentation — SAM2 segment anything
Scene Understanding — Spatial relationships, scene graphs, natural language queries
VLM Reasoning — Vision-language model integration (Nebius, OpenRouter)
Agentic Loop — Continuous perception → reasoning → planning pipeline
Safety Monitoring — Zone-based awareness with real-time human detection and robot halt
CLI-Native — Every capability accessible from the terminal
REST + WebSocket API — Serve models over HTTP with a single command
Fleet Management — Register, deploy, and monitor edge devices at scale
MLOps — Model registry, versioning, A/B testing, batch inference
Governance — Policy enforcement, audit logging, safety rule management
Hosted API —
https://openeye.sh/v1for object detection without running anything locally (pay per call in credits)Web Dashboard — openeye.sh/dashboard: API keys, credits, history, fleet management, and — connected to your own
openeye serve— inference, live streams, and more
Related MCP server: vision-eyes
Quick Install
pipx install openeye-shWith YOLO support:
pipx install "openeye-sh[yolo]"Quick Start
# Pull a model
openeye pull yolov8
# Run inference on an image
openeye run yolov8 photo.jpg
# Start the server with REST API, WebSocket, and browser dashboard
openeye serve yolov8
# Live camera feed with real-time detections
openeye watch --models yolov8
# Live camera feed with safety monitoring
openeye watch --models yolov8 --safetyGive Your AI Agent Eyes (MCP)
OpenEye ships an MCP server, so Claude, Cursor, or any MCP client can pull models and run detection, depth, and segmentation on your machine:
pipx install "openeye-sh[mcp]"
claude mcp add openeye -- openeye-mcpThen ask: "Run yolov8 on ~/Pictures/desk.jpg and tell me what's on my desk." Setup for Claude Desktop and other clients: docs/claude-skill.
Hosted API Quick Start
No GPU or local install needed for detection:
Sign in at openeye.sh (email/password or 1Profile).
Create a key in Dashboard → Settings → API Keys (keys start with
oe_and are shown once).Buy credits in Dashboard → Settings → Credits.
Call the API:
curl -X POST https://openeye.sh/v1/detect \
-H "Authorization: Bearer oe_your_key_here" \
-F "file=@photo.jpg"
# or with the CLI
export OPENEYE_API_KEY=oe_your_key_here
openeye api detect photo.jpg
openeye api usageEndpoint | Status | Credits |
| Live (YOLOv8n, JPEG/PNG under 4 MB) | 1 |
| Returns | 3 |
| Returns | 2 |
| Live | free |
The hosted API has no WebSocket streaming; for streaming, run openeye serve. See the docs for details.
Use the Dashboard with a Local Server
openeye serve allows requests from https://openeye.sh by default, so the hosted dashboard can drive a server on your machine:
openeye serve yolov8 # http://localhost:8000Open openeye.sh/dashboard, click Server on the Overview page (the "OpenEye Server URL" dialog), and enter http://localhost:8000. Inference, Live, Scene Graph, Metrics, Benchmark, Config, Governance, MLOps, and the live agent loop then run against your server. If your browser blocks the connection from an HTTPS page to localhost, serve OpenEye over HTTPS instead.
Interactive CLI Demo
An interactive walkthrough that runs real OpenEye commands against sample images. Great for evaluating the tool, onboarding, or live presentations.
Prerequisites
Python 3.12+
jq (optional, for JSON pipeline steps —
brew install jq)curl (for the API server step)
Setup
# Install the CLI with YOLO and camera support
pipx install -e "./cli[yolo,camera]"
# Pull the YOLOv8 model (6 MB)
openeye pull yolov8Run the Demo
./demo.shdemo.sh is a bash script: run it on macOS, Linux, WSL, or Git Bash on Windows. It uses openeye from your PATH (pipx), falling back to ./.venv or cli/.venv. In plain PowerShell or cmd, run the same steps yourself:
openeye list
openeye pull yolov8
openeye run yolov8 src/assets/demo/scene-warehouse.jpg --pretty
openeye run yolov8 src/assets/demo/scene-warehouse.jpg --visualize
openeye bench yolov8 --warmup 2 --runs 5
openeye serve yolov8 --port 8111The demo is interactive — press Enter to advance between steps. Each step shows the command being run and its live output.
What the Demo Covers
Step | Command | What it demonstrates |
1 |
| Browse the model registry (7 vision models) |
2 |
| Download a model — like |
3 |
| Object detection with structured JSON output |
4 |
| Detection on a different scene (apple, cup, donut) |
5 |
| Save an annotated image with bounding boxes |
6 |
| Unix pipeline composability — pipe JSON to jq |
7 |
| Benchmark inference speed (latency, FPS, P95) |
8 |
| Launch a REST API server and query it with curl |
9 | Batch loop | Process all 4 demo images and summarize results |
Demo Images
Four sample images are included in src/assets/demo/:
Image | Scene | Typical detections |
| Warehouse with people and vehicles | person, truck, tv |
| Kitchen countertop | apple, cup, donut, bottle |
| Workshop environment | person, dining table, remote, knife |
| Safety-critical workspace | (used for safety guardian demos) |
Output
Annotated images are saved to demo_output/. The API server step temporarily runs on port 8111 and shuts down automatically.
Beyond the Demo
Commands not covered in the scripted demo that you can try interactively:
# Live webcam detection with terminal overlay
openeye watch --models yolov8
# Safety Guardian — zone-based human proximity alerts
openeye watch --models yolov8 --safety --danger-m 0.5 --caution-m 1.5
# Unitree G1 robot safety demo (dry-run, no robot needed)
openeye g1-demo --control-mode dry_run --demo
# Agentic perception loop — perceive, reason, act
openeye agent run
# Edge device fleet management
openeye fleet ls
# Open-vocabulary detection with text prompts (requires grounding-dino)
openeye pull grounding-dino
openeye run grounding-dino photo.jpg --prompt "person. hard hat. forklift."Project Structure
openeye.sh/
├── cli/ # openeye-sh CLI & inference server (Python)
├── backend/ # Perception engine runtime (Python)
├── src/ # React web frontend (TypeScript)
├── api/ # Vercel functions (auth, data, credits, /v1 API)
├── db/ # Postgres schema
└── docs/ # MkDocs documentation sitePackage | Description | Language |
| CLI tool & FastAPI inference server (port 8000), MCP server ( | Python |
| Perception pipeline, self-hosted fleet control plane (port 8001), governance engine | Python |
| Website and dashboard at openeye.sh | TypeScript/React |
| Vercel functions: Better Auth ( | TypeScript |
| Postgres schema ( | SQL |
Tech Stack
CLI & Backend: Python, Typer, Rich, FastAPI, Uvicorn, Pydantic
Frontend: React, TypeScript, Vite, Tailwind CSS, shadcn/ui, TanStack Query
Website & API: Vercel (static SPA + functions in api/), Neon Postgres, Vercel Blob (avatars)
Auth: Better Auth — email/password and "Continue with 1Profile"; Google/Apple sign-in when their OAuth credentials are configured
Credits: Cred, proxied through /api/cred
AI/ML: Ultralytics (YOLO), Transformers, ONNX Runtime, TensorRT, LangChain
Development
Frontend
npm install
npm run dev # Vite only, http://localhost:8080 (no api/ functions)To run the api/ functions too (sign-in, dashboard data, credits, /v1), use the Vercel CLI:
vercel env pull .env.local # DATABASE_URL, BETTER_AUTH_SECRET, ... (see .env.example)
npm run db:migrate # apply db/schema.sql to that database
vercel devWithout the server functions you can still use Continue as Demo User on the login page.
CLI
cd cli
pip install -e ".[all]" --group devBackend
cd backend
pip install -e . --group devThe fleet control plane in backend/src/fleet is self-hosted: point DATABASE_URL at a Postgres database (Neon works) with db/schema.sql applied. See backend/.env.example.
Documentation
openeye.sh/docs — the main docs (source:
src/data/docs/)docs/— MkDocs Material site with longer guides and tutorials
License
See LICENSE for details.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Roboflow computer vision for AI agents: datasets, annotation, versioning, workflows, inference.
Memory, reflection and safety for AI agents. Hosted or local, no vector database.
Image & PDF tools for AI agents: compress, convert, resize, PDF, AI vision, pipeline.
PDF and image tools for AI agents: compress, merge, split, OCR, sign. Runs locally, never uploads.
Related MCP Servers
- AlicenseAqualityDmaintenanceProvides vision understanding capabilities such as image analysis, OCR, object localization, and video frame analysis, plus optional image generation and editing, to coding agents via OpenAI-compatible multimodal models. Runs as a local MCP server with HTTP and stdio transports, configurable for clients like Codex, Claude Code, Kimi, and Cursor.317 npmMIT
- AlicenseAqualityCmaintenanceEnables AI agents to perform detailed local image analysis, including pixel-level queries, cropping, grid coordinates, comparisons, and zero-shot classification, all without cloud APIs, keys, or payments.1MIT
- AlicenseNot gradedqualityAmaintenanceEnables AI coding models that lack native vision to analyze screenshots, error messages, UI layouts, documents, and OCR-heavy images by routing each request to one of several major vision LLM providers through a single unified tool. Handles local files, remote URLs, and Data URIs with automatic compression, multi-cropping for large images, retries, and optional HTTP/Docker deployment for shared access.512 npmMIT
- AlicenseNot gradedqualityAmaintenanceEnables AI coding assistants without native vision to analyze screenshots, errors, UI designs, and documents through a unified image understanding tool backed by multiple vision model providers. Supports local files, remote URLs, Data URIs, OCR, multi-crop processing, and stdio or HTTP deployment for shared access.512 npmMIT