# PPE Compliance Vision Pipeline

Industrial **PPE (Personal Protective Equipment) compliance** demo for video:

1. Fine-tune **RF-DETR** on the [SH17](https://github.com/mugheesahmad/SH17) safety dataset  
2. Run video inference with **ByteTrack** + temporal compliance logic  
3. Query results through an **MCP server** (no re-inference in the agent loop)

```
Video → RF-DETR → ByteTrack (track_id) → PPE association → state machine
                                                      ↓
                              annotated MP4 + analytics JSON → MCP tools → Claude/Cursor
```

**Repo:** [github.com/Pratham-22/ppe-compliance](https://github.com/Pratham-22/ppe-compliance)  
**Weights:** [Release v0.1](https://github.com/Pratham-22/ppe-compliance/releases/tag/v0.1) (`checkpoint_epoch44.pth`)

## Demo results

Annotated factory-floor run: **ByteTrack IDs** + temporal PPE compliance overlays (`missing: helmet, safety-vest`).

![Annotated multi-frame collage](docs/images/demo_collage.jpg)

| Frame ~2.7s | Frame ~5.7s |
|:---:|:---:|
| ![Annotated frame 16](docs/images/annotated_frame_0016.jpg) | ![Annotated frame 34](docs/images/annotated_frame_0034.jpg) |

Sample clip: [`outputs/inference/factory_floor_annotated.mp4`](outputs/inference/factory_floor_annotated.mp4) · analytics: [`factory_floor_analytics.json`](outputs/inference/factory_floor_analytics.json)

> Low-resolution YouTube warehouse footage often has **no real helmets/vests**, so this demo mostly shows person tracks flagged as non-compliant (domain shift vs SH17). Prefer clear construction video for positive PPE detections.

## Features

| Layer | What it does |
|-------|----------------|
| **Detection** | RF-DETR Large fine-tuned for person / helmet / safety-vest (17 SH17 classes) |
| **Tracking** | ByteTrack assigns stable worker `track_id`s across frames |
| **Compliance** | Helmet/vest associated via geometry; 15-frame window, 60% ratio threshold |
| **MCP** | Read-only tools over precomputed JSON (summary, violations, worst offenders, trends, …) |

### MCP tools

| Tool | Purpose |
|------|---------|
| `get_site_summary` | Site-wide compliance rollup |
| `get_worker` / `explain_worker` | Per-track status + narrative |
| `list_violations` / `get_worst_offenders` | Who failed PPE |
| `get_violations_in_window` | Violations between two timestamps |
| `get_compliance_trend` | Compliance rate over time buckets |
| `get_flickering_tracks` | Short-lived / noisy track IDs |
| `get_worker_summary_ranked` | Rank all workers by ratio / duration |
| `get_timeline` | Frame-level events |
| `reload_analytics` | Re-read JSON after a new inference run |

## Repo layout

```
ppe-compliance/
├── scripts/
│   ├── train_smoke.py / train_full.py   # RF-DETR fine-tune
│   ├── video_infer.py                   # end-to-end video pipeline
│   ├── ppe_compliance.py                # association + state machine
│   ├── export_checkpoint.py             # Lightning ckpt → RF-DETR .pth
│   ├── sbatch_*.sh                      # OSC Slurm helpers
│   └── upload_release_weights.sh        # optional: upload .pth to GitHub Releases
├── ppe_mcp/
│   ├── store.py                         # JSON query layer
│   ├── server.py                        # FastMCP tools
│   └── *_config.example.json            # Claude / Cursor wiring
├── outputs/inference/                   # sample annotated video + analytics JSON
├── docs/images/                         # README stills from the annotated demo
├── setup_env.sh
└── requirements.txt
```

## Quick start

### 1. Environment (GPU recommended)

```bash
git clone https://github.com/Pratham-22/ppe-compliance.git
cd ppe-compliance

module load python/3.12   # on OSC Ascend, if available
bash setup_env.sh
source .venv/bin/activate
```

### 2. Download model weights

Weights are **not** in git (GitHub file limit is 100 MB). Download the fine-tuned checkpoint from [Release v0.1](https://github.com/Pratham-22/ppe-compliance/releases/tag/v0.1):

```bash
mkdir -p outputs/rfdetr_large
curl -L -o outputs/rfdetr_large/checkpoint_epoch44.pth \
  "https://github.com/Pratham-22/ppe-compliance/releases/download/v0.1/checkpoint_epoch44.pth"
```

### 3. Run inference on a video

A sample demo clip and prior analytics are already in the repo under `data/videos/` and `outputs/inference/`.

```bash
# GPU job on OSC
sbatch scripts/sbatch_video_infer.sh

# or locally
python scripts/video_infer.py \
  --video data/videos/factory_floor_demo.mp4 \
  --checkpoint outputs/rfdetr_large/checkpoint_epoch44.pth \
  --device cuda
```

Produces:

- `outputs/inference/*_annotated.mp4`
- `outputs/inference/*_analytics.json`

### 4. Query results with MCP

```bash
export PPE_ANALYTICS_JSON=outputs/inference/factory_floor_analytics.json
python -m ppe_mcp.server
```

Wire into Cursor / Claude Desktop using the examples in `ppe_mcp/`.

**Example agent prompts**

- “Give me a site-wide PPE summary.”
- “Who are the 3 worst helmet offenders?”
- “Explain worker #2.”
- “What happened between 5s and 10s?”

## Optional: train from scratch

### Data

Download SH17 (Kaggle / upstream repo), then convert:

```bash
python scripts/convert_sh17_to_rfdetr.py
```

### Train

```bash
# smoke test
python scripts/train_smoke.py

# full fine-tune (Slurm on OSC)
sbatch scripts/sbatch_train_rfdetr.sh
```

Export a usable checkpoint:

```bash
python scripts/export_checkpoint.py
```

## Sample outputs

This repo includes a demo run under `outputs/inference/` (stills above in `docs/images/`):

- Annotated video with `#track_id` labels and compliance colors  
- Analytics JSON consumed by the MCP server  

## Notes

- Worker IDs are **ByteTrack track IDs**, not employee names.
- MCP is intentionally **read-only** over JSON — inference stays on GPU.
- Default compliance settings: detection threshold `0.35`, window `15` frames, min ratio `0.6`.

## License / data

SH17 and third-party video remain under their original licenses. This code is provided for portfolio / interview demonstration.