# delivery-vision-mcp

An MCP (Model Context Protocol) server that exposes trained computer vision
models as tools an AI agent can call directly, built for logistics and
delivery use cases like Pathao, Redx, Carrybee, Steadfast etc.

Instead of connecting to someone else's vision service, this server wraps
your own trained YOLO models so any MCP client (Claude Desktop, Claude Code,
Cursor) can call them mid-conversation.

## Tools

| Tool | What it does |
|---|---|
| `detect_package_damage` | Detects damage on a parcel, returns type, affected area %, and severity |
| `detect_damage_batch` | Runs damage detection across up to 20 images in one call, summarized |
| `count_parcels` | Counts boxes/parcels visible in an image |
| `verify_helmet` | Checks whether a rider is wearing a helmet |
| `classify_vehicle_type` | Classifies vehicle type (bike, CNG, car, rickshaw, etc) |
| `list_available_tools` | Lists tools and shows which models have weights loaded |

## What makes this more than a demo

- **Severity is more than confidence.** `detect_package_damage` blends
  detection confidence with how much of the frame the damage covers, so a
  small high-confidence scuff doesn't register the same as a crushed side.
- **Batch support.** `detect_damage_batch` checks a whole set of images
  (e.g. one hub's daily intake) in a single tool call instead of one at a time.
- **No raw crashes.** Every tool validates the image path and model weights
  first and returns a clean, readable message on failure, missing file,
  corrupt image, missing weights, instead of a stack trace reaching the agent.
- **Self-reporting.** `list_available_tools` shows which models are actually
  loaded, so an agent (or you) can check readiness before calling a tool
  that will fail.

## ⚠️ Attention: This Won't Work Out of the Box

Before you get started, it's worth being upfront about one thing: this repo
gives you the *plumbing*, not the *intelligence*. The server, the tool
definitions, the error handling, the severity scoring, all of that is done.
What it doesn't come with are the actual computer vision models.

That part is on you. You'll need to train your own YOLO model for each
capability you want to use, whether that's spotting a dented parcel,
counting boxes in a hub photo, checking if a rider has their helmet on, or
telling a CNG apart from a rickshaw. Once you've trained a model and have a
`.pt` weights file in hand, you just drop it into the `models/` folder with
the right name, and that tool switches on automatically. No code to touch,
no config to rewrite.

Until you do that, the server will still run and Claude will still see all
six tools, but calling one without weights behind it just gets you a polite
"not ready yet" message instead of a real answer. So think of this less as
a finished product and more as a well-built frame waiting for you to fill
in the glass. The actual computer vision work, collecting images, labeling
them, training the model, is the part that turns this from a nice piece of
scaffolding into something that genuinely sees.

## Setup

1. Install dependencies:
   ```bash
   pip install -r requirements.txt
   ```

2. Add trained weights to `models/`, one file per tool you want live:
   - `models/parcel-damage.pt` — damage type detection
   - `models/parcel-count.pt` — box/parcel counting
   - `models/helmet-detection.pt` — helmet vs no-helmet detection
   - `models/vehicle-classify.pt` — vehicle type classification

   You don't need all four at once. Any tool whose weights are missing
   returns a clear setup message instead of crashing the server, so you
   can build this out incrementally.

3. Run the server locally to confirm it starts:
   ```bash
   python server.py
   ```

## Connect to Claude Desktop

Add this to your `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "delivery-vision": {
      "command": "python",
      "args": ["/full/path/to/delivery-vision-mcp/server.py"]
    }
  }
}
```

Restart Claude Desktop. It will list all six tools automatically and can
call them when you share an image path (or a list of paths, for the batch
tool) in conversation.

## Why build this instead of just using an existing MCP server

Anyone can connect to Roboflow's or Groundlight's MCP server. This project
demonstrates the other half of the skill: training your own CV models and
exposing them as agent-callable tools, the part that actually requires
computer vision expertise.