# DevPulse PM Agent — MCP Challenge

This repository contains the **implemented solution** for the DevPulse PM Agent challenge.

All 4 MCP tool functions are fully implemented inside the `tools/` directory, along with validation scripts and cross-validation reports.

---

## Repo structure

```
├── server.py               # MCP server entry point
├── agent_config.json       # run contract — tells the agent how to launch your server
├── requirements.txt        # pip dependencies (pinned)
├── env_vars.json           # env var declarations
├── .gitignore              # excludes .venv, __pycache__, .env, etc.
├── validate.py             # local validation script
├── cross_validate.py       # cross-validation against oracle
├── oracle_comparison.py    # oracle comparison utility
├── data/                   # sample data for local development
│   ├── product_backlog.json
│   ├── customer_feedback.json
│   ├── sprint_history.json
│   ├── sample_roster.json
│   └── sample_dependencies.json
└── tools/
    ├── __init__.py
    ├── prioritize_backlog.py   # implemented
    ├── analyze_feedback.py     # implemented
    ├── assess_capacity.py      # implemented
    └── map_dependencies.py     # implemented
```

> **Note:** `team_roster.json` and `dependency_map.json` are **not** in the repo.
> Your server fetches them at startup from the shared MCP data server (see below).

---

## The 4 tools you must implement

### `prioritize_backlog`
Rank backlog items using a scoring method (`rice`, `value_effort`, or `customer_signal`).
Accepts optional filters (squad, status, tags) and a dependency check flag.

### `analyze_feedback`
Extract and rank themes from customer feedback.
Supports filtering by time range, customer tier, and source.
Supports grouping by theme, customer, or source.

### `assess_capacity`
Calculate available sprint capacity per engineer.
Inputs include sprint ID, squad filter, carry-over flag, and skill-fit check flag.

> **Note on data sources for `assess_capacity`:** This tool uses **both** `sprint_history.json`
> (for sprint-level metadata such as velocity and planned points) **and** `team_roster.json`
> (fetched from the MCP server, for per-engineer capacity, PTO, and skill data).
> Both sources are passed directly into `assess_capacity_impl` via the `sprints` and `roster` arguments.

### `map_dependencies`
Trace dependency chains for backlog items up to a configurable depth.
Handle circular dependencies and flag external blockers.

---

## Data sources

Your tools receive data from two sources:

### 1. Local files — injected by the eval agent via `PM_AGENT_DATA`

Three files are provided at eval time by setting the `PM_AGENT_DATA` env var to a directory:

| File | Used by |
|---|---|
| `product_backlog.json` | `prioritize_backlog`, `map_dependencies` |
| `customer_feedback.json` | `analyze_feedback` |
| `sprint_history.json` | `assess_capacity` |

`server.py` already reads these with `DATA_DIR = Path(os.getenv("PM_AGENT_DATA", "./data"))`.
You do not need to change this.

### 2. Remote MCP server — team roster and dependency map (can also be called Nimbus Oracle)

`team_roster.json` and `dependency_map.json` are served from a shared MCP data server.
`server.py` fetches them at startup using the `MCP_DATA_URL` env var:

```
MCP_DATA_URL=https://co-mcp-server-dev.apps-internal.lrl.lilly.com/mcp
```

This is already wired in `server.py` and pre-set in `env_vars.json`. You do not need to change anything — `roster` and `deps` are passed directly into your `*_impl` functions just like the local data files.

> **Network access:** The MCP data server is hosted on the Lilly internal network.
> You must be on the **Lilly corporate network or connected via VPN** to reach it during local development.
> If the server is unreachable, `server.py` will log a warning and fall back to empty `roster` and `deps` lists —
> your tools must handle empty lists gracefully without crashing.

For local development with Claude Desktop, add the `mcpServers` block to `claude_desktop_config.json`.

**Mac path:** `~/Library/Application Support/Claude/claude_desktop_config.json`

**Windows path:** `%APPDATA%\Claude\claude_desktop_config.json`
_(Paste this path into File Explorer's address bar to open the folder directly.)_

Open the file and paste below `"mcpServers"` block directly above the `coworkUserFilesPath` line, replacing `YOUR_LILLY_EMAIL@lilly.com` and 'L-User ID' accordingly. Please make sure to take care of alignment and syntax properly. There will be other content as well apart from 'coworkUserFilesPath'. Keep everything else same and do not change anything (We have tried below two versions during testing and one of them worked (for different machines). In case if both does not work, you will need to troubleshoot to fix it for your system since system settings may differ from person to person. Also, you need NOT change the default path for your 'coworkUserFilesPath'):

```json
{
  "mcpServers": {
    "pm-data-agent": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://co-mcp-server-dev.apps-internal.lrl.lilly.com/mcp",
        "--header", "X-User-Email: YOUR_LILLY_EMAIL@lilly.com"
      ]
    }
  },
  "coworkUserFilesPath": "/Users/YOUR-L-USER/Claude",
```
OR

```json
{
    "coworkUserFilesPath": "/Users/YOUR-L-USER/Claude",
    "mcpServers": {
    "pm-data-agent": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://co-mcp-server-dev.apps-internal.lrl.lilly.com/mcp",
        "--header", "X-User-Email: YOUR_LILLY_EMAIL@lilly.com"
      ]
    }
  },
}
```


### Enabling the MCP connector in Claude Desktop

After saving `claude_desktop_config.json`, follow these steps to activate the connector:

1. **Fully quit Claude Desktop** — do not just close the window. On Mac, right-click the Claude icon in the Dock and select **Quit**. On Windows, right-click the Claude icon in the system tray and select **Quit**.
2. **Relaunch Claude Desktop.**
3. Open a new conversation and click the **🔌 (plug/tools) icon** in the bottom-left of the chat input box. You should see `pm-data-agent` listed as an available tool server.
4. If `pm-data-agent` does not appear, open **Claude → Settings → Developer** (Mac: `Cmd+,` → Developer tab; Windows: `Ctrl+,` → Developer tab) and check the MCP server logs for errors. Common causes:
   - `npx` is not installed — run `npm install -g npx` to fix.
   - `mcp-remote` package is not available — run `npx mcp-remote --version` to verify.
   - VPN is not connected — the MCP server URL is only reachable on the Lilly network.
5. Once the connector is active, Claude Desktop will automatically pass `roster` and `dependency_map` data to your server at startup whenever you start a session.

---

## Data schema

### `product_backlog.json`
```
id, title, description, status, priority, effort_points,
business_value_score, confidence_score, requester, requested_date,
last_updated, tags, dependencies, squad_assignment, acceptance_criteria
```

### `customer_feedback.json`
```
id, customer_id, customer_name, customer_tier, customer_status,
arr, source, date, text, sentiment_score
```

### `team_roster.json` (from MCP server)
```
name, role, squad, total_capacity_points, sprint_allocation_percent,
pto_days_this_sprint, skills, carry_over_items, current_sprint_assignments
```

### `dependency_map.json` (from MCP server)
```
source_item_id, target_item_id, dependency_type,
external_team, external_eta, notes
```

### `sprint_history.json`
```
sprint_id, sprint_number, start_date, end_date, planned_points,
completed_points, items_planned, items_completed, items_carried_over,
velocity, notes
```

**Never hardcode IDs, names, or numbers from the sample data** — your tools must work correctly on any conformant dataset with the same schema.

---

## `agent_config.json` — run contract

**Do not modify the schema of `agent_config.json`.** The agent reads this file to know how to launch your server. The fields are:

| Field | Description |
|---|---|
| `challenge_id` | Identifies the challenge — do not change |
| `type` | Always `"python"` |
| `runtime_version` | Python version to use — **must match your venv** (3.11, 3.12, or 3.13) |
| `entry` | Entry point file — always `"server.py"` |
| `run_command` | Command to start the server — always `"python server.py"` |
| `requirements` | Dependency file — always `"requirements.txt"` |
| `env_file` | Env vars file — always `"env_vars.json"` |
| `data_env_var` | Env var name for the local data directory — always `"PM_AGENT_DATA"` |
| `required_env` | List of env var names your code requires at runtime |
| `timeout_seconds` | Max seconds allowed for server startup |

**The only fields you should change are `runtime_version`** (if you use Python 3.12 or 3.13) **and `required_env`** (if your implementation needs additional env vars like LLM credentials).

`MCP_DATA_URL` is already in `required_env` — the eval runtime injects it automatically.

---

## Environment variables (`env_vars.json`)

Declare any environment variables your implementation needs in `env_vars.json`.

The default `env_vars.json` contains:
```json
{
    "PM_AGENT_DATA": "",
    "MCP_DATA_URL": "https://co-mcp-server-dev.apps-internal.lrl.lilly.com/mcp"
}
```

`PM_AGENT_DATA` is overridden at eval time with the blind dataset directory.
`MCP_DATA_URL` default points to the shared data server — leave it as-is unless told otherwise.

If your implementation needs additional variables (e.g. LLM credentials), add them here **with empty values** — the runtime will inject the actual values. Also add their names to `required_env` in `agent_config.json`.

**Never hardcode credentials or secrets in source code.**

---

## Supported Python versions

**Use Python 3.11, 3.12, or 3.13.** Declare your version in `agent_config.json`:

```json
{ "runtime_version": "3.11" }
```

---

## Dependencies (`requirements.txt`)

**Pin your dependencies to exact versions compatible with your chosen Python version.**
The `requirements.txt` in this repo uses placeholder versions (`0.0.0`) — replace them with real pinned versions before submitting.

Example for Python 3.11 (**Below are just examples for declaring versions in requirements.txt. The actual may differ depedning on the python version you choose to go ahead with**):
```
mcp==1.9.4
pydantic==2.11.5
openai==1.93.0
requests==2.32.3
python-dotenv==1.1.0
httpx==0.28.1
```

> **Important:** `httpx` is required — `server.py` uses it to connect to the MCP data server at startup.
> Unpinned or incorrect versions can cause your server to fail to start. Always test `pip install -r requirements.txt` from a fresh virtual environment with your declared Python version before submitting.

---

## Quick start (local dev)

**Mac / Linux:**
```bash
python3.11 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# Run the server locally (fetches roster+deps from MCP server at startup)
python server.py
```

**Windows (PowerShell):**
```powershell
python3.11 -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

# Run the server locally (fetches roster+deps from MCP server at startup)
python server.py
```

The server fetches `team_roster` and `dependency_map` from the MCP data server on startup.
Local files (`product_backlog.json`, `customer_feedback.json`, `sprint_history.json`) are read from `./data/`.

---

## Testing your tools locally

You do not need Claude Desktop to verify your implementations. You can call the `*_impl` functions directly from a Python script using the sample data files:

```python
# test_tools.py — run from the repo root with your venv active
import json
from pathlib import Path

from tools.prioritize_backlog import prioritize_backlog_impl
from tools.analyze_feedback import analyze_feedback_impl
from tools.assess_capacity import assess_capacity_impl
from tools.map_dependencies import map_dependencies_impl

DATA = Path("data")
backlog  = json.loads((DATA / "product_backlog.json").read_text())
feedback = json.loads((DATA / "customer_feedback.json").read_text())
sprints  = json.loads((DATA / "sprint_history.json").read_text())

# roster and deps come from the MCP server at runtime; use empty lists offline
roster = []
deps   = []

print("=== prioritize_backlog ===")
print(prioritize_backlog_impl("value_effort", {}, True, backlog, feedback, deps))

print("=== analyze_feedback ===")
print(analyze_feedback_impl(None, None, None, "theme", feedback))

print("=== assess_capacity ===")
print(assess_capacity_impl(None, "all", True, False, roster, backlog, sprints))

print("=== map_dependencies ===")
print(map_dependencies_impl([], True, 3, backlog, deps))
```

Run it with:
```bash
python test_tools.py
```

Each function should return a dict without raising a `NotImplementedError` or crashing.

---

## Critical checklist before submitting

- [x] **All 4 tools are implemented** and return valid dicts (no `NotImplementedError`)
- [x] **`requirements.txt` has real pinned versions** (no `0.0.0` placeholders), including `httpx`, tested with your Python version
- [x] **`agent_config.json` has the correct `runtime_version`** (3.11, 3.12, or 3.13)
- [x] **`env_vars.json` is present** and lists all env var names your code reads
- [x] **Local data tools read from `PM_AGENT_DATA`**, not hardcoded paths
- [x] **`MCP_DATA_URL` is in `required_env`** in `agent_config.json`
- [x] **Tools do not crash on unexpected inputs** — handle edge cases gracefully (including empty `roster` or `deps` lists)
- [x] **Same call returns the same result every time** — no randomness unless seeded; if using an LLM, sort final output deterministically by score so rankings are stable
- [x] **If using an LLM**, credentials are declared in `env_vars.json` and `required_env` in `agent_config.json`
- [x] **`.venv/`, `__pycache__/`, and `.env` are not committed** — verify your `.gitignore` covers these before pushing