Skip to main content
Glama
PS-minalprasad

BugPilot MCP Server

README.md
# ๐Ÿž BugPilot

### AI-Powered Engineering Bug Intelligence Agent

BugPilot is a **production-oriented, multi-agent AI system** that analyzes engineering bug data through an **agentic ReAct workflow** and a **Model Context Protocol (MCP) tool layer**.

Instead of allowing agents to access the database directly, BugPilot enforces a strict architecture:

**React / Vite โ†’ FastAPI โ†’ ReAct Orchestrator โ†’ Specialist Agents โ†’ MCP Client โ†’ MCP Server โ†’ Analytics โ†’ Data Provider โ†’ SQLite**

The current system uses **synthetic Jira-compatible data persisted in SQLite**. The data layer is abstracted behind a provider interface so that the agent and MCP layers remain decoupled from the underlying data source.

---

## ๐ŸŽฏ Project Objective

Engineering teams often have large numbers of bugs distributed across projects, components, sprints, releases, and environments.

Finding meaningful answers such as:

- Which bugs are highest risk?
- Which components are becoming unstable?
- Which defects have been reopened?
- Which bugs are aging beyond acceptable limits?
- Is a release ready from a defect-risk perspective?
- Which issues are related to a particular bug?
- What are the historical trends?

usually requires manually querying and correlating multiple sources of information.

**BugPilot automates this analysis using AI agents and deterministic analytics exposed through MCP tools.**

---

# ๐Ÿ—๏ธ System Architecture

```text
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                    React + Vite Frontend                            โ”‚
โ”‚                       TypeScript UI                                 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                โ”‚
                                โ”‚ HTTP REST API
                                โ”‚ JWT + RBAC + Tenant Isolation
                                โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                     FastAPI Backend                                 โ”‚
โ”‚                         Port 8000                                   โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                โ”‚
                                โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                    ReAct Orchestrator                               โ”‚
โ”‚                                                                     โ”‚
โ”‚       Goal โ†’ LLM Decision โ†’ Tool/Agent Call โ†’ Observation           โ”‚
โ”‚                         โ†‘                    โ”‚                       โ”‚
โ”‚                         โ””โ”€โ”€โ”€โ”€ Next Decision โ”˜                       โ”‚
โ”‚                                                                     โ”‚
โ”‚              Groq Primary + Ollama Fallback                        โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                โ”‚                    โ”‚                โ”‚
                โ–ผ                    โ–ผ                โ–ผ
        โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
        โ”‚ Bug Analyst  โ”‚    โ”‚ Trend Analystโ”‚  โ”‚ Risk Analyst โ”‚
        โ”‚    Agent     โ”‚    โ”‚    Agent     โ”‚  โ”‚    Agent     โ”‚
        โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
               โ”‚                   โ”‚                 โ”‚
               โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                   โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                         MCP Client                                  โ”‚
โ”‚                                                                     โ”‚
โ”‚              Dynamic Tool Discovery                                 โ”‚
โ”‚              Tool Validation                                        โ”‚
โ”‚              Timeout / Execution Controls                           โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                โ”‚
                                โ”‚ MCP / JSON-RPC over stdio
                                โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                         MCP Server                                  โ”‚
โ”‚                                                                     โ”‚
โ”‚                   10 Read-Only Tools                                โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                โ”‚
                                โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                      AnalyticsService                               โ”‚
โ”‚                                                                     โ”‚
โ”‚        Deterministic Metrics + Trend + Risk Calculations            โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                โ”‚
                                โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                       DataProvider                                  โ”‚
โ”‚                                                                     โ”‚
โ”‚              SQLDataProvider / Provider Contract                    โ”‚
โ”‚                     organization_id scoping                         โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                                โ”‚
                                โ–ผ
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                         SQLite                                     โ”‚
โ”‚                                                                     โ”‚
โ”‚              Synthetic Jira-Compatible Bug Data                    โ”‚
โ”‚              Issues โ€ข Sprints โ€ข Releases โ€ข Users                    โ”‚
โ”‚              History โ€ข Comments โ€ข Audit Information                 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
```

---

# ๐Ÿ” Strict Data Access Architecture

One of the core design principles of BugPilot is **separation between reasoning and data access**.

### Allowed

```text
Agent
  โ†“
MCP Client
  โ†“
MCP Server
  โ†“
AnalyticsService
  โ†“
DataProvider
  โ†“
SQLite
```

### Forbidden

```text
Agent โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ†’ Database โŒ

Agent โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ†’ SQLite file โŒ

Agent โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ†’ Data files โŒ
```

Agents do not receive direct database access.

Instead, they interact with structured MCP tools. This provides:

- Controlled data access
- Tool-level validation
- Centralized authorization boundaries
- Easier testing
- Easier provider replacement
- Separation of AI reasoning from deterministic data operations

---

# ๐Ÿค– Agentic Architecture

BugPilot is designed around a **ReAct-style agent loop**.

ReAct means **Reasoning + Acting**.

The orchestrator repeatedly decides what action should happen next based on the user goal and previous observations.

```text
User Query
    โ†“
Understand Goal
    โ†“
Check Guardrails
    โ†“
Inspect Available Tools
    โ†“
LLM Decision
    โ†“
CALL_TOOL / DELEGATE / FINISH
    โ†“
Execute Action
    โ†“
Receive Observation
    โ†“
Evaluate Observation
    โ†“
Next Decision
    โ†“
FINISH
```

The system does not simply execute one hardcoded tool for every query.

The orchestrator determines which available tool or specialist agent is appropriate based on the task.

---

# ๐Ÿง  ReAct Decision Types

The orchestrator can make three major types of decisions.

## `CALL_TOOL`

Used when an MCP tool can directly provide the required evidence.

Example:

```text
User:
"Show me the oldest open authentication bugs."

Possible execution:

search_bugs
      โ†“
get_aging_bugs
      โ†“
FINISH
```

## `DELEGATE`

Used when a specialist agent is better suited to the requested analysis.

Available specialists include:

- Bug Analyst
- Trend Analyst
- Risk Analyst

## `FINISH`

Used when sufficient evidence has been collected and the final response can be generated.

---

# ๐Ÿ‘ฅ Specialist Agents

BugPilot separates specialized analytical responsibilities.

## Bug Analyst

Focuses on individual defects and their technical details.

Typical analysis:

- Severity
- Priority
- Status
- Root cause
- Reproduction information
- Environment
- Business impact
- Related defects

---

## Trend Analyst

Focuses on historical patterns.

Typical analysis:

- Bug creation trends
- Resolution trends
- Sprint trends
- Component patterns
- Release trends
- Reopen patterns

---

## Risk Analyst

Focuses on engineering risk.

Typical analysis:

- Severity
- Priority
- Production impact
- Security impact
- Aging
- Reopen frequency
- Component blast radius
- Release risk

---

# ๐Ÿ”Œ MCP Architecture

BugPilot uses the **Model Context Protocol (MCP)** as the controlled tool interface between agents and the data/analytics layer.

The MCP layer provides a standardized boundary:

```text
AI Agent
   โ†“
MCP Client
   โ†“
MCP Server
   โ†“
MCP Tool
   โ†“
AnalyticsService
   โ†“
DataProvider
```

The client dynamically discovers the tools exposed by the MCP server instead of requiring the agent layer to directly access implementation-specific database functions.

---

# ๐Ÿ› ๏ธ MCP Tools

BugPilot currently exposes **10 read-only analytical tools**.

| # | Tool | Parameters | Purpose |
|---|------|------------|---------|
| 1 | `search_bugs` | `query`, `limit` | Search defects using keywords |
| 2 | `get_bug` | `bug_id` | Retrieve complete details for a specific bug |
| 3 | `get_bug_metrics` | `sprint_id`, `component`, `project` | Calculate bug counts and severity/status distributions |
| 4 | `get_bug_trends` | `sprint_id`, `component`, `project` | Analyze creation, resolution, and sprint trends |
| 5 | `get_aging_bugs` | `min_age_days`, `limit` | Identify old unresolved defects |
| 6 | `get_reopened_bugs` | `component`, `limit` | Identify defects that were reopened |
| 7 | `get_component_risk` | `component`, `project` | Calculate component-level risk |
| 8 | `get_release_risk` | `release` | Evaluate release readiness from defect risk |
| 9 | `get_bug_history` | `bug_id` | Retrieve status history and discussion information |
| 10 | `get_related_bugs` | `bug_id`, `limit` | Find related defects |

All MCP tools are designed to be:

- Read-only
- Tenant-scoped
- Schema validated
- Dynamically discoverable
- Separated from direct database access

---

# ๐Ÿ“Š Deterministic Analytics

A key architectural decision in BugPilot is that **deterministic calculations are not delegated entirely to the LLM**.

Examples include:

```text
Bug Count
Severity Distribution
Aging
Reopen Count
Trend Calculation
Risk Metrics
Release Risk
```

These calculations are handled by application code.

The LLM is responsible for:

```text
Understanding the user goal
        โ†“
Selecting tools
        โ†“
Interpreting observations
        โ†“
Reasoning over evidence
        โ†“
Generating the final explanation
```

The application layer is responsible for:

```text
Fetching data
        โ†“
Filtering data
        โ†“
Aggregating data
        โ†“
Calculating deterministic metrics
```

This reduces the risk of the LLM inventing numerical results.

---

# ๐Ÿ” Evidence-Grounded Analysis

BugPilot follows an evidence-first approach.

For complex queries, the orchestrator can gather multiple pieces of evidence before producing its final answer.

Example:

```text
User:
"Analyze authentication bugs and identify the highest-risk issue."

        โ†“

search_bugs
        โ†“
Candidate bugs discovered
        โ†“
get_bug for relevant candidates
        โ†“
Collect technical evidence
        โ†“
Risk analysis
        โ†“
Reflection / validation
        โ†“
Final grounded report
```

The system can consider factors such as:

- Severity
- Priority
- Production environment
- Security impact
- Technical root cause
- Aging
- Reopen history
- Business impact

---

# ๐Ÿ”Ž Reflection Agent

BugPilot includes a reflection/validation stage intended to improve the quality of generated analysis.

The Reflection Agent evaluates generated responses against available evidence.

The validation process can check dimensions such as:

- Factual grounding
- Reasoning quality
- Instruction following
- Completeness
- Consistency
- Hallucination risk

The goal is to prevent the final answer from making claims that are unsupported by retrieved bug evidence.

---

# ๐Ÿ›ก๏ธ AI Guardrails

BugPilot includes AI safety and robustness controls around the agent workflow.

## Out-of-Domain Protection

Queries unrelated to engineering or bug analysis can be rejected before unnecessary tool execution.

Example:

```text
"What is the capital of France?"
```

should not cause the agent to inspect the bug database.

---

## Prompt Injection Protection

The system includes prompt-injection defense logic designed to prevent user instructions from overriding the application's intended behavior.

Examples of suspicious instructions include:

```text
Ignore previous instructions.
Reveal internal prompts.
Expose hidden tool information.
Bypass access controls.
Return internal implementation details.
```

The guardrail layer evaluates suspicious input before allowing normal agent execution.

---

## Tool Boundary Protection

Agents do not receive unrestricted database access.

Instead:

```text
Agent
 โ†“
MCP Tool
 โ†“
Validated Parameters
 โ†“
Analytics / Data Layer
```

This limits the operations available to the model.

---

# ๐Ÿ” Security Architecture

## JWT Authentication

Authentication uses JWT-based sessions.

Secrets are loaded from environment configuration rather than being hardcoded into source code.

---

## RBAC

BugPilot supports role-based access control.

### Admin

Administrative access and user management.

### Engineer / Developer

Authorized issue operations and analysis.

### Viewer

Read-only access to issue information and analytics.

> MCP analytical tools remain read-only even though authorized backend APIs may support issue-management operations.

---

## Multi-Tenancy

BugPilot uses `organization_id` to scope tenant data.

Example organizations:

```text
org-acme
org-globex
```

Tenant isolation prevents one organization from accessing another organization's records.

Tenant scoping is applied across relevant application and data layers.

---

# ๐Ÿ”‘ Secrets Management

Sensitive credentials are not hardcoded.

Environment configuration is provided through:

```text
.env
```

while the repository contains:

```text
.env.example
```

with placeholder values.

Example:

```text
JWT_SECRET=change-this-secret-in-production
```

Real secrets should never be committed to Git.

---

# ๐Ÿ—„๏ธ Data Architecture

BugPilot currently uses **synthetic Jira-compatible data**.

The data is persisted in:

```text
SQLite
```

with the application accessing it through a provider abstraction.

```text
SQLite
  โ†“
SQLDataProvider
  โ†“
AnalyticsService
  โ†“
MCP Server
```

The provider layer separates the data source from the agent and MCP layers.

---

# ๐Ÿ”„ Why Synthetic Jira-Compatible Data?

The project is designed so that the agent architecture does not depend directly on an external Jira instance.

The current implementation uses synthetic Jira-compatible data to provide:

- Deterministic development
- Reproducible tests
- Predictable evaluation
- No dependency on external authentication
- No dependency on Jira API availability
- Safe demonstration data

The provider abstraction creates a clean boundary for future external Jira integration.

### Current

```text
Synthetic Jira-compatible Data
        โ†“
SQLite
```

### Future

```text
Jira Cloud Provider
        โ†“
Same Provider Contract
```

The goal is to change the data provider without redesigning the agent/MCP architecture.

---

# ๐Ÿงฉ Provider Abstraction

The provider layer separates the data source from the rest of the application.

Conceptually:

```text
             โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
             โ”‚   Provider Contractโ”‚
             โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
              โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
              โ”‚                  โ”‚
              โ–ผ                  โ–ผ
       SQLDataProvider     Future Jira Provider
              โ”‚
              โ–ผ
           SQLite
```

This makes the system easier to test, maintain, and extend.

---

# ๐Ÿงช Testing Strategy

BugPilot includes unit and integration tests covering important application behavior.

Testing areas include:

- Configuration
- Authentication
- RBAC
- Multi-tenancy
- MCP tools
- Agent behavior
- AI guardrails
- Prompt injection protection
- Evaluation framework
- Production observability
- API routes
- Data access
- Analytics

Run the test suite:

```bash
pytest tests/unit tests/integration -q
```

> The test count in project documentation should always reflect the latest actual test run.

---

# ๐Ÿ“ˆ Agent Evaluation Framework

BugPilot includes an evaluation framework for measuring agent quality rather than relying only on whether the application starts successfully.

Run:

```bash
python -m evaluation.run_eval
```

The evaluation framework measures dimensions such as:

### 1. Task / Goal Success

Did the agent correctly fulfill the requested objective?

### 2. Tool Selection

Did the agent select appropriate MCP tools?

### 3. Tool Execution Success

Did selected tools execute successfully?

### 4. Tool Usage Efficiency

Did the agent avoid unnecessary tool calls?

### 5. Reasoning Quality

How well did the agent reason over the available evidence?

### 6. Groundedness

Are generated claims supported by retrieved data?

### 7. Hallucination Detection

Does the response contain unsupported claims?

### 8. Reliability

How does the system behave with missing or invalid data?

### 9. Latency

Measures:

- Mean
- P50
- P95
- P99
- Maximum latency

### 10. Instruction Following

Does the response follow required output structure and constraints?

### 11. Safety & Robustness

Measures behavior against:

- Prompt injection attempts
- Out-of-domain requests
- Invalid inputs
- Tool misuse scenarios

---

# โšก Scalability Testing

The evaluation framework includes concurrency testing.

Configured concurrency levels include:

```text
1
5
10
25
50
```

The purpose is to identify:

- Latency degradation
- Error rates
- Resource saturation
- Concurrency limitations
- Capacity bottlenecks

The evaluation output can highlight warning conditions when configured thresholds are exceeded.

---

# ๐Ÿ’ฐ LLM Architecture

BugPilot uses an LLM Gateway abstraction.

```text
                 LLM Gateway
                     โ”‚
          โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
          โ–ผ                     โ–ผ
     Groq Provider        Ollama Provider
       Primary               Fallback
          โ”‚                     โ”‚
    llama-3.3-70b          llama3.1:8b
```

This abstraction prevents the orchestration layer from being tightly coupled to a single LLM provider.

---

# ๐ŸŒ API Architecture

The backend is implemented using:

```text
FastAPI
```

Core API areas include:

- Health
- Authentication
- Agents
- MCP tools
- Chat
- Metrics
- Application operations

The frontend communicates with the backend through HTTP APIs.

The frontend does not directly access the database.

---

# ๐Ÿ’ป Frontend

The frontend uses:

```text
React
TypeScript
Vite
```

The UI provides an engineering workspace for interacting with BugPilot's analysis capabilities.

All application data flows through the backend API.

---

# ๐Ÿงฑ Technology Stack

| Layer | Technology |
|---|---|
| Frontend | React 18 + TypeScript + Vite |
| Backend | FastAPI + Uvicorn |
| Validation | Pydantic v2 |
| Agent Architecture | ReAct-style Orchestrator |
| Specialist Agents | Bug / Trend / Risk Analysts |
| LLM Gateway | Groq + Ollama |
| Primary LLM | Llama 3.3 70B |
| Fallback LLM | Llama 3.1 8B |
| Tool Protocol | MCP Python SDK |
| Transport | stdio / JSON-RPC |
| ORM | SQLAlchemy 2.0 |
| Database | SQLite |
| Authentication | JWT |
| Password Hashing | bcrypt / Passlib |
| Authorization | RBAC |
| Testing | Pytest |
| Frontend Build | Vite |

---

# ๐Ÿš€ Setup

## Prerequisites

Install:

- Python 3.12+
- Node.js 18+
- Git

---

## LLM Provider Setup (Required for Chat/Analysis)

BugPilot needs **at least one** of the following before chat or analysis
features will work. The backend and frontend will still start and the
health/data endpoints will work without this โ€” but chat requests will fail
until one provider is configured.

**Option A โ€” Groq (recommended, free, no local install)**

1. Get a free API key at [https://console.groq.com](https://console.groq.com)
2. Set it in your `.env` file:

   ```text
   GROQ_API_KEY=your-key-here
   ```

**Option B โ€” Ollama (local, no API key needed)**

1. Install Ollama: [https://ollama.com](https://ollama.com)
2. Pull the fallback model:

   ```bash
   ollama pull llama3.1:8b
   ```

3. Ollama runs automatically as the fallback if Groq is unavailable, using
   `OLLAMA_BASE_URL` (default `http://localhost:11434`) from your `.env`.

You can configure both โ€” Groq is tried first, and BugPilot automatically
fails over to Ollama if the Groq request fails.

---

## 1. Clone Repository

```bash
git clone https://github.com/PS-minalprasad/Bugpilot.git
cd Bugpilot
```

---

## 2. Create Python Environment

### Windows

```powershell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
```

### macOS / Linux

```bash
python -m venv .venv
source .venv/bin/activate
```

---

## 3. Install Backend Dependencies

```bash
pip install -r requirements.txt
```

---

## 4. Configure Environment

### Windows

```powershell
Copy-Item .env.example .env
```

### macOS / Linux

```bash
cp .env.example .env
```

Update environment values as required.

**Never commit the real `.env` file.**

---

# โ–ถ๏ธ Running BugPilot

## Start MCP Server

### Windows

```powershell
.\.venv\Scripts\python -m mcp_server.server
```

### macOS / Linux

```bash
.venv/bin/python -m mcp_server.server
```

---

## Start FastAPI Backend

```bash
uvicorn backend.main:app --host 127.0.0.1 --port 8000 --reload
```

Backend:

```text
http://127.0.0.1:8000
```

---

## Start Frontend

```bash
cd frontend
npm install
npm run dev
```

Build the frontend:

```bash
npm run build
```

---

# ๐Ÿงช Running Tests

Run all unit and integration tests:

```bash
pytest tests/unit tests/integration -q
```

Run AI guardrail tests:

```bash
pytest tests/unit/test_ai_guardrails.py -v
```

Run configuration tests:

```bash
pytest tests/unit/test_config.py -v
```

---

# ๐Ÿ“Š Running Evaluation

Run the agent evaluation framework:

```bash
python -m evaluation.run_eval
```

The evaluation framework generates metrics for agent quality, reliability, latency, safety, and tool usage.

---

# ๐Ÿ”Ž Example Queries

Once BugPilot is running, examples of supported engineering questions include:

```text
Tell me about the authentication bugs.
```

```text
Which bugs are currently the highest risk?
```

```text
Show me the oldest unresolved bugs.
```

```text
Which components have the highest risk?
```

```text
Which bugs have been reopened?
```

```text
Is the current release ready from a bug-risk perspective?
```

```text
Find bugs related to BP-157.
```

```text
Compare authentication bugs and identify the highest-risk issue.
```

---

# ๐Ÿ”„ Example Agent Execution

For a query such as:

```text
"Analyze authentication bugs and identify the highest-risk issue."
```

a possible execution flow is:

```text
User Query
    โ†“
Intent / Safety Check
    โ†“
ReAct Orchestrator
    โ†“
Discover MCP Tools
    โ†“
search_bugs
    โ†“
Candidate Issues
    โ†“
get_bug
    โ†“
Detailed Evidence
    โ†“
Risk Analysis
    โ†“
Reflection / Validation
    โ†“
Final Grounded Report
```

The exact trajectory depends on the task and the decisions made by the orchestrator.

---
# ๐Ÿ“ Project Structure

The core agentic, analytics, provider, and MCP components are organized at the repository root, while the FastAPI application logic is contained under `backend/`.

```text
bugpilot/
โ”‚
โ”œโ”€โ”€ agents/
โ”‚   โ”œโ”€โ”€ orchestrator.py          # ReAct orchestrator (LLM decision loop)
โ”‚   โ”œโ”€โ”€ specialists.py           # Bug Analyst, Trend Analyst, Risk Analyst
โ”‚   โ”œโ”€โ”€ reporting.py             # Report Agent + Reflection Agent
โ”‚   โ”œโ”€โ”€ base.py                  # BaseAgent, AgentResult
โ”‚   โ””โ”€โ”€ orchestration_models.py  # OrchestrationResult, StepMetadata
โ”‚
โ”œโ”€โ”€ analytics/
โ”‚   โ””โ”€โ”€ service.py                # Deterministic metrics/trend/risk calculations
โ”‚
โ”œโ”€โ”€ models/
โ”‚   โ”œโ”€โ”€ bug.py
โ”‚   โ”œโ”€โ”€ report.py
โ”‚   โ”œโ”€โ”€ sprint.py
โ”‚   โ”œโ”€โ”€ analysis.py
โ”‚   โ””โ”€โ”€ analytics.py
โ”‚
โ”œโ”€โ”€ providers/
โ”‚   โ”œโ”€โ”€ base.py                  # Provider contract
โ”‚   โ”œโ”€โ”€ data_provider.py         # SQL/SQLite data provider
โ”‚   โ”œโ”€โ”€ synthetic_provider.py    # Synthetic demo data generator
โ”‚   โ””โ”€โ”€ postgres_provider.py     # PostgreSQL provider stub
โ”‚
โ”œโ”€โ”€ mcp_client/
โ”‚   โ””โ”€โ”€ client.py                 # MCP client (stdio, discovery, allowlist)
โ”‚
โ”œโ”€โ”€ mcp_server/
โ”‚   โ””โ”€โ”€ server.py                 # MCP server + all 10 tools (defined inline)
โ”‚
โ”œโ”€โ”€ backend/
โ”‚   โ”œโ”€โ”€ api/
โ”‚   โ”‚   โ””โ”€โ”€ routes/
โ”‚   โ”‚       โ”œโ”€โ”€ auth.py
โ”‚   โ”‚       โ”œโ”€โ”€ health.py
โ”‚   โ”‚       โ”œโ”€โ”€ issues.py
โ”‚   โ”‚       โ””โ”€โ”€ v1.py             # Chat / orchestrator route
โ”‚   โ”œโ”€โ”€ core/
โ”‚   โ”‚   โ”œโ”€โ”€ exceptions.py
โ”‚   โ”‚   โ”œโ”€โ”€ logging.py
โ”‚   โ”‚   โ””โ”€โ”€ observability.py
โ”‚   โ”œโ”€โ”€ database/
โ”‚   โ”‚   โ”œโ”€โ”€ models.py
โ”‚   โ”‚   โ”œโ”€โ”€ repository.py
โ”‚   โ”‚   โ””โ”€โ”€ session.py
โ”‚   โ”œโ”€โ”€ llm/
โ”‚   โ”‚   โ”œโ”€โ”€ gateway.py            # Groq -> Ollama failover gateway
โ”‚   โ”‚   โ”œโ”€โ”€ base.py
โ”‚   โ”‚   โ”œโ”€โ”€ schemas.py            # ReActDecision Pydantic schema
โ”‚   โ”‚   โ”œโ”€โ”€ prompts.py
โ”‚   โ”‚   โ””โ”€โ”€ providers/
โ”‚   โ”‚       โ”œโ”€โ”€ groq.py
โ”‚   โ”‚       โ””โ”€โ”€ ollama.py
โ”‚   โ”œโ”€โ”€ security/
โ”‚   โ”‚   โ”œโ”€โ”€ auth.py               # JWT
โ”‚   โ”‚   โ”œโ”€โ”€ middleware.py
โ”‚   โ”‚   โ”œโ”€โ”€ dependencies.py
โ”‚   โ”‚   โ””โ”€โ”€ prompt_injection.py
โ”‚   โ”œโ”€โ”€ config.py
โ”‚   โ””โ”€โ”€ main.py
โ”‚
โ”œโ”€โ”€ evaluation/
โ”‚   โ”œโ”€โ”€ evaluator.py
โ”‚   โ”œโ”€โ”€ golden_dataset.py
โ”‚   โ”œโ”€โ”€ load_tester.py
โ”‚   โ””โ”€โ”€ run_eval.py
โ”‚
โ”œโ”€โ”€ tests/
โ”‚   โ”œโ”€โ”€ unit/
โ”‚   โ””โ”€โ”€ integration/
โ”‚
โ”œโ”€โ”€ frontend/
โ”‚   โ””โ”€โ”€ src/                      # React + TypeScript + Vite
โ”‚
โ”œโ”€โ”€ alembic/                       # Database migrations
โ”œโ”€โ”€ data/                          # Synthetic data generator
โ”œโ”€โ”€ generate_pdf.py
โ”œโ”€โ”€ requirements.txt
โ”œโ”€โ”€ .env.example
โ”œโ”€โ”€ .gitignore
โ””โ”€โ”€ README.md
```

### Key Architectural Modules

| Module | Responsibility |
|---|---|
| `agents/` | ReAct orchestrator (`orchestrator.py`), specialist agents (`specialists.py`), and Report/Reflection agents (`reporting.py`) |
| `analytics/` | Deterministic bug metrics, trends, and risk calculations |
| `models/` | Domain models such as `Bug`, `Report`, `Sprint`, and analysis/analytics schemas |
| `providers/` | Data-provider abstraction (`base.py`) with SQL (`data_provider.py`), synthetic, and PostgreSQL implementations |
| `mcp_client/` | MCP client โ€” dynamic tool discovery, allowlist enforcement, timeout guards |
| `mcp_server/` | MCP server exposing 10 read-only analytical tools |
| `backend/` | FastAPI API, authentication, security, LLM gateway, and database layer |
| `evaluation/` | Agent evaluation, quality metrics, and scalability/load testing |
| `tests/` | Unit and integration test suites |
| `frontend/` | React + TypeScript + Vite user interface |



---

# ๐ŸŽฏ Design Principles

BugPilot follows several core engineering principles.

### 1. Separation of Concerns

AI reasoning, tool execution, analytics, and data access are separate layers.

### 2. No Direct Agent Database Access

Agents access data only through MCP tools.

### 3. Deterministic Analytics

Numerical calculations are handled by application code rather than relying entirely on LLM reasoning.

### 4. Provider Abstraction

The data source is separated from the agent and MCP layers.

### 5. Evidence-Grounded Generation

The final response should be based on retrieved system evidence.

### 6. Defense in Depth

Security is implemented through multiple layers:

```text
Authentication
      โ†“
RBAC
      โ†“
Tenant Isolation
      โ†“
Tool Validation
      โ†“
AI Guardrails
      โ†“
MCP Boundary
```

### 7. Testability

Synthetic deterministic data and provider abstraction make the system reproducible and easier to test.

---

# โš ๏ธ Current Limitations

BugPilot currently uses **synthetic Jira-compatible data** rather than depending on a live Jira Cloud instance for its core demonstration workflow.

Therefore:

- Jira Cloud is not required to run the demonstration.
- Data is not automatically synchronized from a live Jira project.
- Real-time Jira webhook synchronization is not currently the primary data flow.
- The current architecture is designed to allow a future external Jira provider.

This limitation is intentional for reproducible development and evaluation.

---

# ๐Ÿ”ฎ Future Improvements

## Live Jira Integration

Connect the provider layer to Jira Cloud APIs.

```text
Jira Cloud
    โ†“
Jira Provider
    โ†“
Provider Contract
    โ†“
AnalyticsService
    โ†“
MCP Server
```

## Real-Time Synchronization

Add Jira webhooks/event processing for near-real-time updates.

## Background Processing

Introduce background workers for:

- Synchronization
- Large evaluations
- Report generation
- Scheduled analytics

## Production Observability

Expand monitoring with:

- OpenTelemetry
- Prometheus
- Grafana
- Distributed tracing
- Structured audit events

## Enterprise Authentication

Potential integrations:

- SSO
- OAuth
- Enterprise identity providers

---

# ๐Ÿ† What Makes BugPilot Different?

BugPilot is not simply an LLM chatbot connected to a database.

It combines:

```text
                    BugPilot
                       โ”‚
       โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
       โ”‚               โ”‚                โ”‚
       โ–ผ               โ–ผ                โ–ผ
   Multi-Agent       ReAct            MCP
   Architecture     Reasoning        Tool Layer
       โ”‚               โ”‚                โ”‚
       โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
                       โ–ผ
               Deterministic
                  Analytics
                       โ”‚
                       โ–ผ
               Evidence-Grounded
                    Analysis
                       โ”‚
                       โ–ผ
              Security + RBAC
                       โ”‚
                       โ–ผ
                 Evaluation
```

The architecture is designed to keep **LLM reasoning, deterministic computation, tool execution, and data access separated**.

---

# ๐Ÿ“Œ Project Status

### Current Implementation

- โœ… React + Vite frontend
- โœ… FastAPI backend
- โœ… ReAct-style orchestration
- โœ… Specialist agents
- โœ… MCP client
- โœ… MCP server
- โœ… Dynamic MCP tool discovery
- โœ… 10 read-only analytical MCP tools
- โœ… SQLite persistence
- โœ… Synthetic Jira-compatible dataset
- โœ… Data provider abstraction
- โœ… JWT authentication
- โœ… RBAC
- โœ… Multi-tenancy
- โœ… AI guardrails
- โœ… Prompt-injection protection
- โœ… Reflection / validation
- โœ… Automated evaluation framework
- โœ… Unit and integration testing

### Data Source

**Synthetic Jira-compatible data**

### External Jira

Designed for future provider integration; not required for the current demonstration workflow.

---

Maintenance

ActivitySlowing
ResponsivenessNo issues