BugPilot MCP Server
README.md
# ๐ BugPilot
### AI-Powered Engineering Bug Intelligence Agent
BugPilot is a **production-oriented, multi-agent AI system** that analyzes engineering bug data through an **agentic ReAct workflow** and a **Model Context Protocol (MCP) tool layer**.
Instead of allowing agents to access the database directly, BugPilot enforces a strict architecture:
**React / Vite โ FastAPI โ ReAct Orchestrator โ Specialist Agents โ MCP Client โ MCP Server โ Analytics โ Data Provider โ SQLite**
The current system uses **synthetic Jira-compatible data persisted in SQLite**. The data layer is abstracted behind a provider interface so that the agent and MCP layers remain decoupled from the underlying data source.
---
## ๐ฏ Project Objective
Engineering teams often have large numbers of bugs distributed across projects, components, sprints, releases, and environments.
Finding meaningful answers such as:
- Which bugs are highest risk?
- Which components are becoming unstable?
- Which defects have been reopened?
- Which bugs are aging beyond acceptable limits?
- Is a release ready from a defect-risk perspective?
- Which issues are related to a particular bug?
- What are the historical trends?
usually requires manually querying and correlating multiple sources of information.
**BugPilot automates this analysis using AI agents and deterministic analytics exposed through MCP tools.**
---
# ๐๏ธ System Architecture
```text
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ React + Vite Frontend โ
โ TypeScript UI โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โ HTTP REST API
โ JWT + RBAC + Tenant Isolation
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ FastAPI Backend โ
โ Port 8000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ ReAct Orchestrator โ
โ โ
โ Goal โ LLM Decision โ Tool/Agent Call โ Observation โ
โ โ โ โ
โ โโโโโ Next Decision โ โ
โ โ
โ Groq Primary + Ollama Fallback โ
โโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโ
โ โ โ
โผ โผ โผ
โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโ
โ Bug Analyst โ โ Trend Analystโ โ Risk Analyst โ
โ Agent โ โ Agent โ โ Agent โ
โโโโโโโโฌโโโโโโโโ โโโโโโโโฌโโโโโโโโ โโโโโโโโฌโโโโโโโโ
โ โ โ
โโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ MCP Client โ
โ โ
โ Dynamic Tool Discovery โ
โ Tool Validation โ
โ Timeout / Execution Controls โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โ MCP / JSON-RPC over stdio
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ MCP Server โ
โ โ
โ 10 Read-Only Tools โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ AnalyticsService โ
โ โ
โ Deterministic Metrics + Trend + Risk Calculations โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ DataProvider โ
โ โ
โ SQLDataProvider / Provider Contract โ
โ organization_id scoping โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ SQLite โ
โ โ
โ Synthetic Jira-Compatible Bug Data โ
โ Issues โข Sprints โข Releases โข Users โ
โ History โข Comments โข Audit Information โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
```
---
# ๐ Strict Data Access Architecture
One of the core design principles of BugPilot is **separation between reasoning and data access**.
### Allowed
```text
Agent
โ
MCP Client
โ
MCP Server
โ
AnalyticsService
โ
DataProvider
โ
SQLite
```
### Forbidden
```text
Agent โโโโโโโโโโโโโโโโโ Database โ
Agent โโโโโโโโโโโโโโโโโ SQLite file โ
Agent โโโโโโโโโโโโโโโโโ Data files โ
```
Agents do not receive direct database access.
Instead, they interact with structured MCP tools. This provides:
- Controlled data access
- Tool-level validation
- Centralized authorization boundaries
- Easier testing
- Easier provider replacement
- Separation of AI reasoning from deterministic data operations
---
# ๐ค Agentic Architecture
BugPilot is designed around a **ReAct-style agent loop**.
ReAct means **Reasoning + Acting**.
The orchestrator repeatedly decides what action should happen next based on the user goal and previous observations.
```text
User Query
โ
Understand Goal
โ
Check Guardrails
โ
Inspect Available Tools
โ
LLM Decision
โ
CALL_TOOL / DELEGATE / FINISH
โ
Execute Action
โ
Receive Observation
โ
Evaluate Observation
โ
Next Decision
โ
FINISH
```
The system does not simply execute one hardcoded tool for every query.
The orchestrator determines which available tool or specialist agent is appropriate based on the task.
---
# ๐ง ReAct Decision Types
The orchestrator can make three major types of decisions.
## `CALL_TOOL`
Used when an MCP tool can directly provide the required evidence.
Example:
```text
User:
"Show me the oldest open authentication bugs."
Possible execution:
search_bugs
โ
get_aging_bugs
โ
FINISH
```
## `DELEGATE`
Used when a specialist agent is better suited to the requested analysis.
Available specialists include:
- Bug Analyst
- Trend Analyst
- Risk Analyst
## `FINISH`
Used when sufficient evidence has been collected and the final response can be generated.
---
# ๐ฅ Specialist Agents
BugPilot separates specialized analytical responsibilities.
## Bug Analyst
Focuses on individual defects and their technical details.
Typical analysis:
- Severity
- Priority
- Status
- Root cause
- Reproduction information
- Environment
- Business impact
- Related defects
---
## Trend Analyst
Focuses on historical patterns.
Typical analysis:
- Bug creation trends
- Resolution trends
- Sprint trends
- Component patterns
- Release trends
- Reopen patterns
---
## Risk Analyst
Focuses on engineering risk.
Typical analysis:
- Severity
- Priority
- Production impact
- Security impact
- Aging
- Reopen frequency
- Component blast radius
- Release risk
---
# ๐ MCP Architecture
BugPilot uses the **Model Context Protocol (MCP)** as the controlled tool interface between agents and the data/analytics layer.
The MCP layer provides a standardized boundary:
```text
AI Agent
โ
MCP Client
โ
MCP Server
โ
MCP Tool
โ
AnalyticsService
โ
DataProvider
```
The client dynamically discovers the tools exposed by the MCP server instead of requiring the agent layer to directly access implementation-specific database functions.
---
# ๐ ๏ธ MCP Tools
BugPilot currently exposes **10 read-only analytical tools**.
| # | Tool | Parameters | Purpose |
|---|------|------------|---------|
| 1 | `search_bugs` | `query`, `limit` | Search defects using keywords |
| 2 | `get_bug` | `bug_id` | Retrieve complete details for a specific bug |
| 3 | `get_bug_metrics` | `sprint_id`, `component`, `project` | Calculate bug counts and severity/status distributions |
| 4 | `get_bug_trends` | `sprint_id`, `component`, `project` | Analyze creation, resolution, and sprint trends |
| 5 | `get_aging_bugs` | `min_age_days`, `limit` | Identify old unresolved defects |
| 6 | `get_reopened_bugs` | `component`, `limit` | Identify defects that were reopened |
| 7 | `get_component_risk` | `component`, `project` | Calculate component-level risk |
| 8 | `get_release_risk` | `release` | Evaluate release readiness from defect risk |
| 9 | `get_bug_history` | `bug_id` | Retrieve status history and discussion information |
| 10 | `get_related_bugs` | `bug_id`, `limit` | Find related defects |
All MCP tools are designed to be:
- Read-only
- Tenant-scoped
- Schema validated
- Dynamically discoverable
- Separated from direct database access
---
# ๐ Deterministic Analytics
A key architectural decision in BugPilot is that **deterministic calculations are not delegated entirely to the LLM**.
Examples include:
```text
Bug Count
Severity Distribution
Aging
Reopen Count
Trend Calculation
Risk Metrics
Release Risk
```
These calculations are handled by application code.
The LLM is responsible for:
```text
Understanding the user goal
โ
Selecting tools
โ
Interpreting observations
โ
Reasoning over evidence
โ
Generating the final explanation
```
The application layer is responsible for:
```text
Fetching data
โ
Filtering data
โ
Aggregating data
โ
Calculating deterministic metrics
```
This reduces the risk of the LLM inventing numerical results.
---
# ๐ Evidence-Grounded Analysis
BugPilot follows an evidence-first approach.
For complex queries, the orchestrator can gather multiple pieces of evidence before producing its final answer.
Example:
```text
User:
"Analyze authentication bugs and identify the highest-risk issue."
โ
search_bugs
โ
Candidate bugs discovered
โ
get_bug for relevant candidates
โ
Collect technical evidence
โ
Risk analysis
โ
Reflection / validation
โ
Final grounded report
```
The system can consider factors such as:
- Severity
- Priority
- Production environment
- Security impact
- Technical root cause
- Aging
- Reopen history
- Business impact
---
# ๐ Reflection Agent
BugPilot includes a reflection/validation stage intended to improve the quality of generated analysis.
The Reflection Agent evaluates generated responses against available evidence.
The validation process can check dimensions such as:
- Factual grounding
- Reasoning quality
- Instruction following
- Completeness
- Consistency
- Hallucination risk
The goal is to prevent the final answer from making claims that are unsupported by retrieved bug evidence.
---
# ๐ก๏ธ AI Guardrails
BugPilot includes AI safety and robustness controls around the agent workflow.
## Out-of-Domain Protection
Queries unrelated to engineering or bug analysis can be rejected before unnecessary tool execution.
Example:
```text
"What is the capital of France?"
```
should not cause the agent to inspect the bug database.
---
## Prompt Injection Protection
The system includes prompt-injection defense logic designed to prevent user instructions from overriding the application's intended behavior.
Examples of suspicious instructions include:
```text
Ignore previous instructions.
Reveal internal prompts.
Expose hidden tool information.
Bypass access controls.
Return internal implementation details.
```
The guardrail layer evaluates suspicious input before allowing normal agent execution.
---
## Tool Boundary Protection
Agents do not receive unrestricted database access.
Instead:
```text
Agent
โ
MCP Tool
โ
Validated Parameters
โ
Analytics / Data Layer
```
This limits the operations available to the model.
---
# ๐ Security Architecture
## JWT Authentication
Authentication uses JWT-based sessions.
Secrets are loaded from environment configuration rather than being hardcoded into source code.
---
## RBAC
BugPilot supports role-based access control.
### Admin
Administrative access and user management.
### Engineer / Developer
Authorized issue operations and analysis.
### Viewer
Read-only access to issue information and analytics.
> MCP analytical tools remain read-only even though authorized backend APIs may support issue-management operations.
---
## Multi-Tenancy
BugPilot uses `organization_id` to scope tenant data.
Example organizations:
```text
org-acme
org-globex
```
Tenant isolation prevents one organization from accessing another organization's records.
Tenant scoping is applied across relevant application and data layers.
---
# ๐ Secrets Management
Sensitive credentials are not hardcoded.
Environment configuration is provided through:
```text
.env
```
while the repository contains:
```text
.env.example
```
with placeholder values.
Example:
```text
JWT_SECRET=change-this-secret-in-production
```
Real secrets should never be committed to Git.
---
# ๐๏ธ Data Architecture
BugPilot currently uses **synthetic Jira-compatible data**.
The data is persisted in:
```text
SQLite
```
with the application accessing it through a provider abstraction.
```text
SQLite
โ
SQLDataProvider
โ
AnalyticsService
โ
MCP Server
```
The provider layer separates the data source from the agent and MCP layers.
---
# ๐ Why Synthetic Jira-Compatible Data?
The project is designed so that the agent architecture does not depend directly on an external Jira instance.
The current implementation uses synthetic Jira-compatible data to provide:
- Deterministic development
- Reproducible tests
- Predictable evaluation
- No dependency on external authentication
- No dependency on Jira API availability
- Safe demonstration data
The provider abstraction creates a clean boundary for future external Jira integration.
### Current
```text
Synthetic Jira-compatible Data
โ
SQLite
```
### Future
```text
Jira Cloud Provider
โ
Same Provider Contract
```
The goal is to change the data provider without redesigning the agent/MCP architecture.
---
# ๐งฉ Provider Abstraction
The provider layer separates the data source from the rest of the application.
Conceptually:
```text
โโโโโโโโโโโโโโโโโโโโโโ
โ Provider Contractโ
โโโโโโโโโโโฌโโโโโโโโโโโ
โ
โโโโโโโโโโดโโโโโโโโโโ
โ โ
โผ โผ
SQLDataProvider Future Jira Provider
โ
โผ
SQLite
```
This makes the system easier to test, maintain, and extend.
---
# ๐งช Testing Strategy
BugPilot includes unit and integration tests covering important application behavior.
Testing areas include:
- Configuration
- Authentication
- RBAC
- Multi-tenancy
- MCP tools
- Agent behavior
- AI guardrails
- Prompt injection protection
- Evaluation framework
- Production observability
- API routes
- Data access
- Analytics
Run the test suite:
```bash
pytest tests/unit tests/integration -q
```
> The test count in project documentation should always reflect the latest actual test run.
---
# ๐ Agent Evaluation Framework
BugPilot includes an evaluation framework for measuring agent quality rather than relying only on whether the application starts successfully.
Run:
```bash
python -m evaluation.run_eval
```
The evaluation framework measures dimensions such as:
### 1. Task / Goal Success
Did the agent correctly fulfill the requested objective?
### 2. Tool Selection
Did the agent select appropriate MCP tools?
### 3. Tool Execution Success
Did selected tools execute successfully?
### 4. Tool Usage Efficiency
Did the agent avoid unnecessary tool calls?
### 5. Reasoning Quality
How well did the agent reason over the available evidence?
### 6. Groundedness
Are generated claims supported by retrieved data?
### 7. Hallucination Detection
Does the response contain unsupported claims?
### 8. Reliability
How does the system behave with missing or invalid data?
### 9. Latency
Measures:
- Mean
- P50
- P95
- P99
- Maximum latency
### 10. Instruction Following
Does the response follow required output structure and constraints?
### 11. Safety & Robustness
Measures behavior against:
- Prompt injection attempts
- Out-of-domain requests
- Invalid inputs
- Tool misuse scenarios
---
# โก Scalability Testing
The evaluation framework includes concurrency testing.
Configured concurrency levels include:
```text
1
5
10
25
50
```
The purpose is to identify:
- Latency degradation
- Error rates
- Resource saturation
- Concurrency limitations
- Capacity bottlenecks
The evaluation output can highlight warning conditions when configured thresholds are exceeded.
---
# ๐ฐ LLM Architecture
BugPilot uses an LLM Gateway abstraction.
```text
LLM Gateway
โ
โโโโโโโโโโโโดโโโโโโโโโโโ
โผ โผ
Groq Provider Ollama Provider
Primary Fallback
โ โ
llama-3.3-70b llama3.1:8b
```
This abstraction prevents the orchestration layer from being tightly coupled to a single LLM provider.
---
# ๐ API Architecture
The backend is implemented using:
```text
FastAPI
```
Core API areas include:
- Health
- Authentication
- Agents
- MCP tools
- Chat
- Metrics
- Application operations
The frontend communicates with the backend through HTTP APIs.
The frontend does not directly access the database.
---
# ๐ป Frontend
The frontend uses:
```text
React
TypeScript
Vite
```
The UI provides an engineering workspace for interacting with BugPilot's analysis capabilities.
All application data flows through the backend API.
---
# ๐งฑ Technology Stack
| Layer | Technology |
|---|---|
| Frontend | React 18 + TypeScript + Vite |
| Backend | FastAPI + Uvicorn |
| Validation | Pydantic v2 |
| Agent Architecture | ReAct-style Orchestrator |
| Specialist Agents | Bug / Trend / Risk Analysts |
| LLM Gateway | Groq + Ollama |
| Primary LLM | Llama 3.3 70B |
| Fallback LLM | Llama 3.1 8B |
| Tool Protocol | MCP Python SDK |
| Transport | stdio / JSON-RPC |
| ORM | SQLAlchemy 2.0 |
| Database | SQLite |
| Authentication | JWT |
| Password Hashing | bcrypt / Passlib |
| Authorization | RBAC |
| Testing | Pytest |
| Frontend Build | Vite |
---
# ๐ Setup
## Prerequisites
Install:
- Python 3.12+
- Node.js 18+
- Git
---
## LLM Provider Setup (Required for Chat/Analysis)
BugPilot needs **at least one** of the following before chat or analysis
features will work. The backend and frontend will still start and the
health/data endpoints will work without this โ but chat requests will fail
until one provider is configured.
**Option A โ Groq (recommended, free, no local install)**
1. Get a free API key at [https://console.groq.com](https://console.groq.com)
2. Set it in your `.env` file:
```text
GROQ_API_KEY=your-key-here
```
**Option B โ Ollama (local, no API key needed)**
1. Install Ollama: [https://ollama.com](https://ollama.com)
2. Pull the fallback model:
```bash
ollama pull llama3.1:8b
```
3. Ollama runs automatically as the fallback if Groq is unavailable, using
`OLLAMA_BASE_URL` (default `http://localhost:11434`) from your `.env`.
You can configure both โ Groq is tried first, and BugPilot automatically
fails over to Ollama if the Groq request fails.
---
## 1. Clone Repository
```bash
git clone https://github.com/PS-minalprasad/Bugpilot.git
cd Bugpilot
```
---
## 2. Create Python Environment
### Windows
```powershell
python -m venv .venv
.\.venv\Scripts\Activate.ps1
```
### macOS / Linux
```bash
python -m venv .venv
source .venv/bin/activate
```
---
## 3. Install Backend Dependencies
```bash
pip install -r requirements.txt
```
---
## 4. Configure Environment
### Windows
```powershell
Copy-Item .env.example .env
```
### macOS / Linux
```bash
cp .env.example .env
```
Update environment values as required.
**Never commit the real `.env` file.**
---
# โถ๏ธ Running BugPilot
## Start MCP Server
### Windows
```powershell
.\.venv\Scripts\python -m mcp_server.server
```
### macOS / Linux
```bash
.venv/bin/python -m mcp_server.server
```
---
## Start FastAPI Backend
```bash
uvicorn backend.main:app --host 127.0.0.1 --port 8000 --reload
```
Backend:
```text
http://127.0.0.1:8000
```
---
## Start Frontend
```bash
cd frontend
npm install
npm run dev
```
Build the frontend:
```bash
npm run build
```
---
# ๐งช Running Tests
Run all unit and integration tests:
```bash
pytest tests/unit tests/integration -q
```
Run AI guardrail tests:
```bash
pytest tests/unit/test_ai_guardrails.py -v
```
Run configuration tests:
```bash
pytest tests/unit/test_config.py -v
```
---
# ๐ Running Evaluation
Run the agent evaluation framework:
```bash
python -m evaluation.run_eval
```
The evaluation framework generates metrics for agent quality, reliability, latency, safety, and tool usage.
---
# ๐ Example Queries
Once BugPilot is running, examples of supported engineering questions include:
```text
Tell me about the authentication bugs.
```
```text
Which bugs are currently the highest risk?
```
```text
Show me the oldest unresolved bugs.
```
```text
Which components have the highest risk?
```
```text
Which bugs have been reopened?
```
```text
Is the current release ready from a bug-risk perspective?
```
```text
Find bugs related to BP-157.
```
```text
Compare authentication bugs and identify the highest-risk issue.
```
---
# ๐ Example Agent Execution
For a query such as:
```text
"Analyze authentication bugs and identify the highest-risk issue."
```
a possible execution flow is:
```text
User Query
โ
Intent / Safety Check
โ
ReAct Orchestrator
โ
Discover MCP Tools
โ
search_bugs
โ
Candidate Issues
โ
get_bug
โ
Detailed Evidence
โ
Risk Analysis
โ
Reflection / Validation
โ
Final Grounded Report
```
The exact trajectory depends on the task and the decisions made by the orchestrator.
---
# ๐ Project Structure
The core agentic, analytics, provider, and MCP components are organized at the repository root, while the FastAPI application logic is contained under `backend/`.
```text
bugpilot/
โ
โโโ agents/
โ โโโ orchestrator.py # ReAct orchestrator (LLM decision loop)
โ โโโ specialists.py # Bug Analyst, Trend Analyst, Risk Analyst
โ โโโ reporting.py # Report Agent + Reflection Agent
โ โโโ base.py # BaseAgent, AgentResult
โ โโโ orchestration_models.py # OrchestrationResult, StepMetadata
โ
โโโ analytics/
โ โโโ service.py # Deterministic metrics/trend/risk calculations
โ
โโโ models/
โ โโโ bug.py
โ โโโ report.py
โ โโโ sprint.py
โ โโโ analysis.py
โ โโโ analytics.py
โ
โโโ providers/
โ โโโ base.py # Provider contract
โ โโโ data_provider.py # SQL/SQLite data provider
โ โโโ synthetic_provider.py # Synthetic demo data generator
โ โโโ postgres_provider.py # PostgreSQL provider stub
โ
โโโ mcp_client/
โ โโโ client.py # MCP client (stdio, discovery, allowlist)
โ
โโโ mcp_server/
โ โโโ server.py # MCP server + all 10 tools (defined inline)
โ
โโโ backend/
โ โโโ api/
โ โ โโโ routes/
โ โ โโโ auth.py
โ โ โโโ health.py
โ โ โโโ issues.py
โ โ โโโ v1.py # Chat / orchestrator route
โ โโโ core/
โ โ โโโ exceptions.py
โ โ โโโ logging.py
โ โ โโโ observability.py
โ โโโ database/
โ โ โโโ models.py
โ โ โโโ repository.py
โ โ โโโ session.py
โ โโโ llm/
โ โ โโโ gateway.py # Groq -> Ollama failover gateway
โ โ โโโ base.py
โ โ โโโ schemas.py # ReActDecision Pydantic schema
โ โ โโโ prompts.py
โ โ โโโ providers/
โ โ โโโ groq.py
โ โ โโโ ollama.py
โ โโโ security/
โ โ โโโ auth.py # JWT
โ โ โโโ middleware.py
โ โ โโโ dependencies.py
โ โ โโโ prompt_injection.py
โ โโโ config.py
โ โโโ main.py
โ
โโโ evaluation/
โ โโโ evaluator.py
โ โโโ golden_dataset.py
โ โโโ load_tester.py
โ โโโ run_eval.py
โ
โโโ tests/
โ โโโ unit/
โ โโโ integration/
โ
โโโ frontend/
โ โโโ src/ # React + TypeScript + Vite
โ
โโโ alembic/ # Database migrations
โโโ data/ # Synthetic data generator
โโโ generate_pdf.py
โโโ requirements.txt
โโโ .env.example
โโโ .gitignore
โโโ README.md
```
### Key Architectural Modules
| Module | Responsibility |
|---|---|
| `agents/` | ReAct orchestrator (`orchestrator.py`), specialist agents (`specialists.py`), and Report/Reflection agents (`reporting.py`) |
| `analytics/` | Deterministic bug metrics, trends, and risk calculations |
| `models/` | Domain models such as `Bug`, `Report`, `Sprint`, and analysis/analytics schemas |
| `providers/` | Data-provider abstraction (`base.py`) with SQL (`data_provider.py`), synthetic, and PostgreSQL implementations |
| `mcp_client/` | MCP client โ dynamic tool discovery, allowlist enforcement, timeout guards |
| `mcp_server/` | MCP server exposing 10 read-only analytical tools |
| `backend/` | FastAPI API, authentication, security, LLM gateway, and database layer |
| `evaluation/` | Agent evaluation, quality metrics, and scalability/load testing |
| `tests/` | Unit and integration test suites |
| `frontend/` | React + TypeScript + Vite user interface |
---
# ๐ฏ Design Principles
BugPilot follows several core engineering principles.
### 1. Separation of Concerns
AI reasoning, tool execution, analytics, and data access are separate layers.
### 2. No Direct Agent Database Access
Agents access data only through MCP tools.
### 3. Deterministic Analytics
Numerical calculations are handled by application code rather than relying entirely on LLM reasoning.
### 4. Provider Abstraction
The data source is separated from the agent and MCP layers.
### 5. Evidence-Grounded Generation
The final response should be based on retrieved system evidence.
### 6. Defense in Depth
Security is implemented through multiple layers:
```text
Authentication
โ
RBAC
โ
Tenant Isolation
โ
Tool Validation
โ
AI Guardrails
โ
MCP Boundary
```
### 7. Testability
Synthetic deterministic data and provider abstraction make the system reproducible and easier to test.
---
# โ ๏ธ Current Limitations
BugPilot currently uses **synthetic Jira-compatible data** rather than depending on a live Jira Cloud instance for its core demonstration workflow.
Therefore:
- Jira Cloud is not required to run the demonstration.
- Data is not automatically synchronized from a live Jira project.
- Real-time Jira webhook synchronization is not currently the primary data flow.
- The current architecture is designed to allow a future external Jira provider.
This limitation is intentional for reproducible development and evaluation.
---
# ๐ฎ Future Improvements
## Live Jira Integration
Connect the provider layer to Jira Cloud APIs.
```text
Jira Cloud
โ
Jira Provider
โ
Provider Contract
โ
AnalyticsService
โ
MCP Server
```
## Real-Time Synchronization
Add Jira webhooks/event processing for near-real-time updates.
## Background Processing
Introduce background workers for:
- Synchronization
- Large evaluations
- Report generation
- Scheduled analytics
## Production Observability
Expand monitoring with:
- OpenTelemetry
- Prometheus
- Grafana
- Distributed tracing
- Structured audit events
## Enterprise Authentication
Potential integrations:
- SSO
- OAuth
- Enterprise identity providers
---
# ๐ What Makes BugPilot Different?
BugPilot is not simply an LLM chatbot connected to a database.
It combines:
```text
BugPilot
โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโ
โ โ โ
โผ โผ โผ
Multi-Agent ReAct MCP
Architecture Reasoning Tool Layer
โ โ โ
โโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโ
โ
โผ
Deterministic
Analytics
โ
โผ
Evidence-Grounded
Analysis
โ
โผ
Security + RBAC
โ
โผ
Evaluation
```
The architecture is designed to keep **LLM reasoning, deterministic computation, tool execution, and data access separated**.
---
# ๐ Project Status
### Current Implementation
- โ
React + Vite frontend
- โ
FastAPI backend
- โ
ReAct-style orchestration
- โ
Specialist agents
- โ
MCP client
- โ
MCP server
- โ
Dynamic MCP tool discovery
- โ
10 read-only analytical MCP tools
- โ
SQLite persistence
- โ
Synthetic Jira-compatible dataset
- โ
Data provider abstraction
- โ
JWT authentication
- โ
RBAC
- โ
Multi-tenancy
- โ
AI guardrails
- โ
Prompt-injection protection
- โ
Reflection / validation
- โ
Automated evaluation framework
- โ
Unit and integration testing
### Data Source
**Synthetic Jira-compatible data**
### External Jira
Designed for future provider integration; not required for the current demonstration workflow.
---
This server cannot be deployed
Maintenance
ActivitySlowing
ResponsivenessNo issues