How do I use FiftyOne MCP Server?

1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@FiftyOne MCP Server Load the quickstart dataset and show me a summary of its contents" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

FiftyOne MCP Server

fo_agent

Control FiftyOne datasets through AI assistants using the Model Context Protocol

PyPI Python

Overview

Enable ChatGPT and Claude to explore datasets, execute operators, and build computer vision workflows through natural language. This server exposes FiftyOne's operator framework (80+ built-in operators) through 16 MCP tools.

Features

Dataset Management (3 tools) - List, load, and summarize datasets
Operator System (5 tools) - Execute any FiftyOne operator dynamically
Plugin Management (5 tools) - Discover and install FiftyOne plugins
Session Management (3 tools) - Control FiftyOne App for delegated execution
Natural Language Workflows - Multi-step operations through conversation
ChatGPT & Claude Compatible - Works with desktop apps

Quick Start

Option 1: pip (Simplest)

pip install fiftyone-mcp-server

Then add to your AI tool config and restart:

Edit ~/Library/Application Support/Claude/claude_desktop_config.json:

{ "mcpServers": { "fiftyone": { "command": "fiftyone-mcp" } } }

claude mcp add fiftyone -- fiftyone-mcp

Add to Cursor MCP settings:

{ "fiftyone": { "command": "fiftyone-mcp" } }

Add to .vscode/mcp.json:

{ "servers": { "fiftyone": { "command": "fiftyone-mcp" } } }

Edit ~/Library/Application Support/ChatGPT/config.json:

{ "mcpServers": { "fiftyone": { "command": "fiftyone-mcp" } } }

Option 2: uvx (No Install Needed)

If you have uv installed:

{ "mcpServers": { "fiftyone": { "command": "uvx", "args": ["fiftyone-mcp-server"] } } }

This downloads and runs the latest version automatically.

Usage

After configuration, restart your AI assistant and try:

"List all my datasets" "Load quickstart dataset and show summary" "What operators are available for managing samples?" "Set context to my dataset, then tag high-confidence samples" "What plugins are available? Install the brain plugin" "Find similar images in my dataset"

The server starts with 50 built-in operators. Install plugins to expand functionality - the AI can discover and install plugins automatically when needed (brain, zoo, annotation, evaluation, and more).

Architecture

Component	Description
Operator System	80+ FiftyOne operators through unified interface
Plugin System	AI discovers and installs plugins on demand
Session System	Launch FiftyOne App for delegated operators
Context Management	Dataset, view, and selection state

Design Philosophy: Minimal tool count (16 tools), maximum flexibility (full operator & plugin ecosystem).

Contributing

We welcome contributions! Here's how to set up a local development environment.

Local Development Setup

# Clone the repository git clone https://github.com/AdonaiVera/fiftyone-mcp-server.git cd fiftyone-mcp-server # Install Poetry (if not installed) curl -sSL https://install.python-poetry.org | python3 - # Install dependencies poetry install # Run the server locally poetry run fiftyone-mcp

Testing Your Changes

# Run tests poetry run pytest # Code formatting poetry run black -l 79 src/ # Linting poetry run pylint --errors-only src/ # Test with MCP Inspector npx @modelcontextprotocol/inspector poetry run fiftyone-mcp