\# Construction Site Safety Inspector



An LLM-powered system for automated construction site hazard detection and incident analysis.



\## Overview



This system analyzes construction site photos and incident PDF reports to detect safety hazards, retrieve similar past accidents from a KOSHA database, and generate professional bilingual safety reports with citations.



Built as a final project for LLM-AE-AI course at Kyung Hee University.



\## Features



\- Vision Hazard Detection: Claude analyzes site photos and identifies safety violations with severity levels

\- PDF Incident Analysis: Extracts and analyzes construction accident report PDFs

\- RAG Pipeline: Searches 37 real KOSHA accident cases using hybrid BM25 and VoyageAI search

\- Tool Use: classify\_hazard() function assigns hazard type and KOSHA regulation codes

\- Bilingual Reports: Professional safety reports in Korean and English with citations

\- Urgent Prevention Alerts: Automatically fires alerts when HIGH severity hazards are detected

\- 2D Hazard Visualization: Draws colored bounding boxes on site photos

\- YOLO vs Claude Comparison: Side by side comparison showing why Claude beats YOLO

\- Weekly Safety Summary: Management level weekly report of all inspections and alerts

\- MCP Server: Exposes all tools via FastMCP for Claude Desktop integration

\- PDF Report Generation: Professional PDF reports with metrics, images, and hazard cards



\## Technology Stack



W1 - Prompt Engineering: Domain safety inspection system prompt

W2 - Claude API: Core backend for all AI operations

W3 - LLM-as-Judge: Evaluates report quality automatically

W4 - Tool Use: classify\_hazard() function

W5 - RAG Pipeline: VoyageAI + BM25 + RRF on KOSHA PDFs

W6 - Vision, PDF, Citations, Caching: Photo analysis, document reading, cited output

W7 - MCP Server via FastMCP: Claude Desktop integration



\## Project Structure



safety-inspector/

    app.py              Streamlit web interface

    inspector.py        Core AI pipeline

    rag\_builder.py      Builds RAG index from KOSHA PDFs

    yolo\_compare.py     YOLO vs Claude comparison

    mcp\_server.py       MCP server

    pdf\_generator.py    PDF report generation

    data/pdfs/          KOSHA accident PDFs

    outputs/            Generated reports and alerts



\## Setup



1\. Clone the repository

2\. Create virtual environment: python -m venv venv

3\. Activate: venv\\Scripts\\activate

4\. Install dependencies: pip install -r requirements.txt

5\. Create .env file with your API keys:

   ANTHROPIC\_API\_KEY=your\_key\_here

   VOYAGE\_API\_KEY=your\_key\_here

6\. Download KOSHA PDFs into data/pdfs/

7\. Build RAG index: python rag\_builder.py

8\. Run the app: streamlit run app.py



\## Demo



Streamlit Web App:

streamlit run app.py



MCP Server:

npx @modelcontextprotocol/inspector python mcp\_server.py



\## Data Source



KOSHA construction accident case reports:

https://portal.kosha.or.kr



\## Developer



Muhammad Ali

Student ID: 2026311007

Course: LLM-AE-AI

Professor: 백장운

Kyung Hee University

Graduate School of Architecture Engineering