# MCP Issue Tracker

A small issue-tracking backend built with FastAPI and SQLAlchemy, exposed to AI clients through a Model Context Protocol (MCP) server and integrated with an OpenAI agent.

The project demonstrates how an MCP server can act as an integration layer on top of an existing REST API instead of accessing the database directly. It also includes a small Retrieval-Augmented Generation (RAG) pipeline using OpenAI embeddings and Chroma, allowing the agent to retrieve troubleshooting knowledge before answering technical questions.

Write operations remain protected by explicit human approval.

## Architecture

```text
                              User
                                |
                                | Natural language
                                v
                       +------------------+
                       |   OpenAI Agent   |
                       |   Agents SDK     |
                       +--------+---------+
                                |
                                | MCP tool selection
                                v
                       +------------------+
                       |    MCP Server    |
                       |      Python      |
                       +----+---------+---+
                            |         |
          issue CRUD tools  |         | search_knowledge
                            |         |
                            v         v
                  +----------------+  +-------------------+
                  | Human approval |  |  RAG Retrieval    |
                  | for write ops  |  |                   |
                  +-------+--------+  | query embedding   |
                          |           |        |          |
                          v           |        v          |
                    FastAPI REST API  |     Chroma        |
                          |           |        |          |
                          |           | relevant chunks  |
                          |           +--------+----------+
                          |                    |
                          v                    |
                  SQLAlchemy / SQLite          |
                                               |
                                               v
                                      grounded answer
```

The issue-management tools communicate with the FastAPI REST API over HTTP using HTTPX.

The `search_knowledge` tool uses OpenAI embeddings and a local persistent Chroma collection. Knowledge documents are indexed separately, then retrieved at query time using cosine-distance vector search.

A simpler MCP client is also included for direct MCP tool discovery and invocation without an AI model.

## Tech Stack

- Python 3.12+
- FastAPI
- Pydantic
- SQLAlchemy 2
- SQLite
- Model Context Protocol Python SDK
- OpenAI Agents SDK
- OpenAI Embeddings API
- Chroma
- HTTPX
- pytest
- uv

## Features

- REST API for issue management
- Persistent storage with SQLite and SQLAlchemy
- Request and response validation with Pydantic
- Automatic OpenAPI documentation with FastAPI
- MCP server exposing issue operations as tools
- Asynchronous HTTP communication between the MCP server and REST API
- HTTP and connection error handling translated into MCP tool errors
- OpenAI agent capable of selecting MCP tools from natural-language requests
- Multi-turn conversational context within an interactive session
- Human-in-the-loop approval for MCP write operations
- RAG-based troubleshooting knowledge retrieval
- OpenAI embeddings for semantic search
- Persistent vector storage with Chroma
- Document chunking before indexing
- Cosine-distance retrieval
- Relevance threshold to reject unrelated results
- Strict grounding for troubleshooting answers
- Source attribution in grounded responses
- Automated API integration tests
- Automated AI approval workflow tests
- Automated RAG retrieval tests
- Isolated in-memory SQLite database for API tests

## Issue Model

An issue contains:

```json
{
  "id": 1,
  "title": "Login fails",
  "description": "Login fails after session expiration.",
  "status": "open",
  "priority": "high"
}
```

Supported statuses:

- `open`
- `in_progress`
- `closed`

Supported priorities:

- `low`
- `medium`
- `high`

## REST API

| Method | Endpoint | Description |
|---|---|---|
| `GET` | `/health` | API health check |
| `GET` | `/issues` | List all issues |
| `GET` | `/issues/{id}` | Retrieve an issue |
| `POST` | `/issues` | Create an issue |
| `PATCH` | `/issues/{id}` | Partially update an issue |
| `DELETE` | `/issues/{id}` | Delete an issue |

FastAPI automatically provides interactive API documentation at:

```text
http://127.0.0.1:8000/docs
```

## MCP Tools

The MCP server exposes the following tools:

| Tool | Description | Approval |
|---|---|---|
| `ping` | Check whether the MCP server is running | No |
| `list_issues` | Retrieve all issues | No |
| `get_issue` | Retrieve an issue by ID | No |
| `create_issue` | Create a new issue | Required in AI client |
| `update_issue` | Partially update an issue | Required in AI client |
| `delete_issue` | Delete an issue | Required in AI client |
| `search_knowledge` | Search troubleshooting knowledge using vector similarity | No |

The MCP server communicates with the REST API using HTTPX instead of accessing SQLite directly.

This keeps the REST API reusable by clients that do not use MCP and prevents the MCP layer from depending on database sessions or persistence details.

The RAG path is separate: `search_knowledge` calls the knowledge-base layer, which embeds the query and searches the persistent Chroma collection.

## AI Agent

The project includes an interactive AI client built with the OpenAI Agents SDK.

The agent receives natural-language requests, discovers the available MCP tools, and decides which tool to invoke.

Example write operation:

```text
You: Create a high priority issue titled "Login problem"
     with the description "Users cannot log in."

--- Approval required ---
Tool: create_issue
Arguments: {"title":"Login problem","description":"Users cannot log in.","priority":"high"}
Approve? [y/N]: y

Assistant: Created issue #2: Login problem
```

Read operations can execute directly, while write operations require explicit user approval.

### Human-in-the-loop write protection

The following tools are considered write operations:

```text
create_issue
update_issue
delete_issue
```

If the agent attempts to call one of them, execution is interrupted before the MCP tool runs.

The application then asks the user to approve or reject the operation:

```text
LLM selects write tool
        |
        v
Run is interrupted
        |
        v
User sees tool + arguments
        |
   +----+----+
   |         |
 approve    reject
   |         |
   v         v
execute     do not execute
```

This allows the model to propose changes without giving it unrestricted authority to modify application data.

## RAG Knowledge Retrieval

The project includes a small local troubleshooting knowledge base stored as Markdown files.

Example sources:

```text
knowledge/
├── authentication.md
├── database.md
└── deployment.md
```

The RAG flow has two separate phases.

### Ingestion

```text
knowledge documents
        |
        v
     chunking
        |
        v
OpenAI embeddings
        |
        v
      Chroma
```

Each chunk is stored with metadata such as:

```json
{
  "source": "database.md",
  "chunk_index": 0
}
```

The Chroma collection is persisted locally under `.chroma/`, which is ignored by Git because it is generated from the source documents.

### Retrieval

At query time:

```text
user question
      |
      v
query embedding
      |
      v
cosine-distance search
      |
      v
most relevant chunk
      |
      v
MCP search_knowledge result
      |
      v
OpenAI agent
      |
      v
grounded answer + source
```

Only the query is embedded at search time. Document embeddings are generated during indexing and reused from the persistent vector store.

### Grounding and abstention

The agent is instructed to answer troubleshooting questions only using information returned by `search_knowledge`.

If no retrieved chunk passes the configured relevance threshold, the agent must abstain instead of answering from its own general knowledge.

Example:

```text
You: How do I center a div with CSS?

Assistant: The knowledge base does not contain enough relevant
information to answer how to center a div with CSS.
```

This avoids treating the nearest vector as automatically relevant. Nearest-neighbor search always returns the closest item, even when the corpus contains no genuinely useful document.

For grounded answers, the agent also identifies the source document when possible.

## Getting Started

### 1. Clone the repository

```bash
git clone https://github.com/jgonzalezar/mcp-issue-tracker.git
cd mcp-issue-tracker
```

### 2. Install dependencies

The project uses `uv` for Python and dependency management.

```bash
uv sync
```

### 3. Configure the OpenAI API key

The AI client and RAG embedding workflow require an OpenAI API key.

PowerShell:

```powershell
$env:OPENAI_API_KEY="your-api-key"
```

Do not commit API keys to the repository.

### 4. Start the REST API

```bash
uv run uvicorn mcp_issue_tracker.api:app --reload
```

The API will be available at:

```text
http://127.0.0.1:8000
```

### 5. Index the knowledge base

Before using RAG search, index the Markdown documents:

```bash
uv run python -m mcp_issue_tracker.knowledge_base index
```

This creates or updates the persistent Chroma collection.

The generated `.chroma/` directory is intentionally not committed to Git.

You can test retrieval directly:

```bash
uv run python -m mcp_issue_tracker.knowledge_base search "How should I investigate a slow SQL query?"
```

### 6. Run the basic MCP demo client

Keep the FastAPI server running and open another terminal:

```bash
uv run python -m mcp_issue_tracker.mcp_client
```

The demo client starts the MCP server as a subprocess using the `stdio` transport and discovers the available tools.

### 7. Run the AI client

```bash
uv run python -m mcp_issue_tracker.ai_client
```

Example session:

```text
Issue Tracker AI
Type 'exit' to quit.

You: How should I investigate a slow SQL query?

Assistant:
- Check database query execution times.
- Use EXPLAIN or EXPLAIN ANALYZE.
- Look for sequential scans on large tables.
- Check whether filters and joins use appropriate indexes.
- Verify database connection-pool usage.

Source: database.md

You: Create a high priority issue called "Slow database queries".

--- Approval required ---
Tool: create_issue
Arguments: {...}
Approve? [y/N]:
```

The conversation remains contextual during the current process, allowing follow-up prompts such as:

```text
What issue did you just create?
Change its status to in progress.
Based on our knowledge base, how should I investigate it?
```

## MCP Server

The MCP server can also be started directly:

```bash
uv run python -m mcp_issue_tracker.mcp_server
```

With the default `stdio` transport, the process waits for an MCP client to communicate through standard input and output.

The AI client launches the MCP server as a subprocess and passes the required environment variables explicitly, including the OpenAI API key needed by the RAG embedding flow.

## Configuration

By default, the MCP server connects to:

```text
http://127.0.0.1:8000
```

A different REST API URL can be provided through:

```text
ISSUE_TRACKER_API_URL
```

PowerShell example:

```powershell
$env:ISSUE_TRACKER_API_URL="http://localhost:9000"
```

The OpenAI integration uses:

```text
OPENAI_API_KEY
```

The API key should be provided through the environment and must not be committed to the repository.

## Testing

Run the complete automated test suite with:

```bash
uv run pytest -v
```

The current suite contains **10 tests** covering the REST API, AI approval workflow, and RAG retrieval behavior.

### API tests

The API tests use a separate in-memory SQLite database so that test execution does not modify the development database.

They cover:

- complete issue CRUD lifecycle
- request validation
- invalid priority handling
- `404 Not Found` behavior

### AI approval tests

The AI approval tests do not call the real OpenAI API.

`pytest` monkeypatching is used to replace the OpenAI `Runner` and user input with deterministic test doubles.

They cover:

- the set of write tools that require approval
- successful approval of an interrupted write operation
- rejection of a write operation
- resumption of an interrupted agent run after the approval decision

### RAG tests

The RAG tests mock both the embedding and vector-store boundaries.

They cover:

- paragraph-based chunking
- successful retrieval of a relevant result
- filtering of results above the relevance threshold
- failure behavior when the knowledge base is empty

These tests do not require a live OpenAI API call or a real Chroma database.

## Design Decisions

### Separate REST and MCP layers

The MCP server does not access the relational database directly. It consumes the FastAPI REST API over HTTP.

This introduces an additional network dependency, but keeps responsibilities separated and allows the REST API to be reused independently of MCP.

The MCP server therefore acts as an adapter between AI/MCP clients and the existing application API.

### Separate ingestion and retrieval

Knowledge documents are embedded during a dedicated indexing phase.

At query time, the application embeds only the user query and searches against already persisted vectors.

This avoids recalculating document embeddings for every request.

### Pydantic models vs. SQLAlchemy entities

Pydantic models define the external API contract and perform validation.

SQLAlchemy entities represent the persistence layer and database schema.

Keeping them separate prevents the database representation from becoming tightly coupled to the HTTP API.

### Synchronous database access

The project uses synchronous SQLAlchemy sessions.

FastAPI supports synchronous route handlers and can execute them appropriately without requiring the database layer to be converted to asynchronous code.

The MCP server uses asynchronous HTTPX calls because communication with the REST API is I/O-bound.

### Lazy initialization for RAG dependencies

Chroma and the OpenAI client are initialized lazily rather than at module import time.

This avoids delaying the MCP server startup and keeps the initial `stdio` handshake responsive.

The synchronous RAG search path is executed through `asyncio.to_thread()` so that expensive imports and blocking operations do not block the MCP event loop.

### Human approval for AI write operations

Read tools can be used directly by the agent because they do not modify application state.

Before `create_issue`, `update_issue`, or `delete_issue` is executed, the agent run is interrupted and the user must explicitly approve the proposed tool invocation.

This keeps the language model useful for deciding what action to take while keeping the final authorization decision with the user.

### Relevance threshold

Vector search always returns the closest result, even when the closest result is not actually relevant.

The retrieval layer therefore applies a cosine-distance threshold and discards results that are too distant from the query.

If no result passes the threshold, the agent is instructed to abstain.

The threshold used in this small prototype is intentionally empirical and would need calibration against representative queries in a production system.

### Grounded troubleshooting answers

For troubleshooting and technical guidance, the agent is instructed to answer only from retrieved knowledge.

This prevents the model from silently mixing its own general knowledge with the project's knowledge base and makes source attribution possible.

### SQLite and local Chroma

SQLite and a local persistent Chroma collection were chosen to keep the project lightweight and easy to run without external infrastructure.

For a production system, these could be replaced with services such as PostgreSQL and a production-oriented vector-search solution depending on scale and architecture.

## Error Handling

FastAPI returns standard HTTP responses such as:

- `201 Created`
- `204 No Content`
- `404 Not Found`
- `422 Unprocessable Entity`

The MCP layer translates REST and network failures into MCP `ToolError` responses.

For example:

```text
Issue Tracker API returned HTTP 404: Issue not found
```

or:

```text
Issue Tracker API is unavailable.
```

The RAG MCP tool also converts retrieval failures into MCP tool errors.

This keeps transport failures, REST API errors, and knowledge-retrieval failures explicit instead of silently treating them as successful results.

## Project Structure

```text
mcp-issue-tracker/
├── knowledge/
│   ├── authentication.md
│   ├── database.md
│   └── deployment.md
│
├── src/
│   └── mcp_issue_tracker/
│       ├── __init__.py
│       ├── ai_client.py
│       ├── api.py
│       ├── database.py
│       ├── db_models.py
│       ├── knowledge_base.py
│       ├── models.py
│       ├── mcp_client.py
│       └── mcp_server.py
│
├── tests/
│   ├── test_ai_client.py
│   ├── test_api.py
│   └── test_knowledge_base.py
│
├── .gitignore
├── .python-version
├── pyproject.toml
├── uv.lock
└── README.md
```

The generated `.chroma/` directory is excluded from version control.

## Possible Improvements

Possible next steps for a production-oriented version include:

- PostgreSQL instead of SQLite
- PostgreSQL with `pgvector` or a managed vector database
- Alembic database migrations
- Authentication and authorization
- API keys or OAuth for MCP access
- More granular authorization policies for AI actions
- Persistent conversation storage
- Token-based chunking and chunk overlap
- Larger evaluation dataset for tuning the relevance threshold
- Retrieval evaluation metrics
- Hybrid keyword + vector search
- Reranking
- Docker support
- CI/CD with GitHub Actions
- MCP Streamable HTTP transport
- Remote MCP deployment
- Additional MCP resources and prompts
- Pagination and filtering
- Structured logging and observability
- Tracing of AI tool calls, retrievals, and approvals
- More extensive automated testing

## Purpose

This project was built as a hands-on exercise to explore Python backend development, FastAPI, SQLAlchemy, REST APIs, asynchronous HTTP communication, automated testing, the Model Context Protocol, AI tool calling, conversational agents, human-in-the-loop approval workflows, embeddings, vector search, and Retrieval-Augmented Generation.