# mcp-drone-agent

Give Claude actual flight control over a drone via the Model Context
Protocol. This is the demo project for the "drones" video: an LLM agent
reads a plain-English mission, plans a search pattern itself, and calls real
MCP tools (`takeoff`, `move_to`, `scan`, `land`, ...) to fly it.

## Why a simulator instead of a real flight stack

The paper this is inspired by ("Taking Flight with Dialogue," PX4-based
Drone Agent, [arXiv 2506.07509](https://arxiv.org/abs/2506.07509)) uses
PX4 + ROS 2 + NVIDIA Isaac Sim, which needs a GPU Linux box and a fair
amount of setup. This project keeps the same idea -- natural language in,
tool calls out, flight path as the payoff -- but swaps the flight stack for
a small deterministic Python simulator (`drone_sim.py`) so it runs anywhere
in about 10 seconds with no GPU, no ROS2, no simulator install. The MCP
server (`mcp_server.py`) is real, not a mock -- swap `drone_sim.World` for a
PX4/MAVSDK backend later and the tool layer above it doesn't change.

## How it's wired together

```
mission text  -->  agent.py  --(MCP stdio)-->  mcp_server.py  -->  drone_sim.py
                      |                                                |
                 LLM (Ollama / Claude)                          physics / world
                 decides which tool                             state / flight log
                 to call next
```

- `drone_sim.py` -- a 60x60m field, three static obstacles, three hidden
  targets (one is the mission objective, two are decoys), simple kinematics,
  battery drain, and a flight event log. Pure Python, no dependencies.
- `mcp_server.py` -- wraps the simulator in five MCP tools using the
  official MCP Python SDK (`FastMCP`), served over stdio.
- `agent.py` -- the MCP client. `--mode ollama` (default) uses a local LLM
  via Ollama. `--mode mock` is a scripted lawnmower search (no LLM).
  `--mode claude` uses Anthropic's API.
- `visualize.py` -- renders the flight log into a top-down PNG: field
  boundary, obstacles, the flown path, detections, obstacle
  detours, home/landing markers.
- `demo_mission.py` -- runs the whole pipeline in one command.

## Setup

```bash
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
```

### Local LLM (recommended)

Install [Ollama](https://ollama.com), pull a tool-capable model, and keep
`ollama serve` running:

```bash
ollama pull qwen2.5:7b
```

Default model is `qwen2.5:7b` (strong tool calling). Also works with
`llama3.1:8b` / `llama3.2` if already pulled. Prefer models whose Ollama
card lists **tools** support.

### Cloud Claude (optional)

For `--mode claude`, copy `.env.example` to `.env` and set
`ANTHROPIC_API_KEY`. `agent.py` loads `.env` via `python-dotenv`.

## Live web console (recommended)

Same teal theme / sidebar layout as `medical-assistant`. Backend streams
telemetry over WebSocket; the map animates the drone in real time.

```bash
# Terminal 1 — API
cd backend
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000

# Terminal 2 — UI
cd frontend
npm install
npm run dev
```

Open [http://localhost:3000/mission](http://localhost:3000/mission), pick
**Mock** or **Ollama**, hit **Start mission**.

## CLI demo

```bash
# Local LLM via Ollama (default) -- real agent, no Anthropic bill
python demo_mission.py --mode ollama

# Scripted search, no LLM -- proves MCP + sim only
python demo_mission.py --mode mock

# Cloud Claude (needs credits / API key)
python demo_mission.py --mode claude

# Pick another local model
python demo_mission.py --mode ollama --model llama3.1:8b
```

All modes write `flight_logs/latest.jsonl` (raw event log),
`flight_logs/agent_transcript.json` (every tool call + LLM reasoning
text, if any).

Same seed (`--seed 42` by default) = same field layout every run, so you can
re-record a take without the obstacle/target positions moving around on you.

## Point Claude Desktop at it directly

Since `mcp_server.py` is a real MCP server, you can also add it straight to
Claude Desktop's config and fly missions from the chat window instead of the
script:

```json
{
  "mcpServers": {
    "drone-agent": {
      "command": "python3",
      "args": ["/absolute/path/to/mcp-drone-agent/mcp_server.py"]
    }
  }
}
```

Restart Claude Desktop, then just type a mission in chat, e.g. "take off,
search the field, and land next to the survivor marker."