# Project Sentry

An intelligent MCP (Model-Context-Protocol) server for QA automation that bridges AI agents with backend microservices for seamless end-to-end testing.

## Overview

Project Sentry transforms your Postman collections into a powerful MCP server that enables AI agents to perform complex, stateful API testing using natural language. It handles authentication, state management, and provides a clean interface for test execution and reporting.

## Features

- **Autonomous Authentication**: Automatic username/password authentication with Basic auth and retry logic
- **Stateful Testing**: Session-based state store for multi-step test scenarios  
- **Postman Integration**: Convert existing Postman collections into MCP tools
- **AI-Friendly**: Designed to reduce cognitive load on AI agents
- **Transparent Error Handling**: Structured error responses with full downstream API details (v1.3)
- **Environment Configuration**: Programmatic access to valid test data and configuration (v1.3)
- **Comprehensive Reporting**: Generate structured test reports in Markdown format

## Quick Start

### 1. Installation

```bash
# Clone the repository
git clone <repository-url>
cd postman-replacement

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install fastmcp fastapi uvicorn pydantic pydantic-settings requests typer python-dotenv
```

### 2. Configuration

Copy the example environment file and configure your settings:

```bash
cp .env.example .env
```

Edit `.env` with your actual values:

```env
# Authentication Configuration
AUTH_BASE_URL=https://steven01-auth.cp.manh.cloud
AUTH_BASIC_TOKEN=b21uaWNvbXBvbmVudC4xLjAuMDpiNHM4cmdUeWc1NVhZTnVu
USERNAME=admin@ecomorg.com
PASSWORD=Password@1

# Target API Configuration
API_BASE_URL=https://omni-steven01.cp.manh.cloud
API_VERSION=v1

# Server Configuration
HOST=localhost
PORT=8000
LOG_LEVEL=INFO
```

### 3. Ingest Postman Collection

Convert your Postman collection into MCP tools:

```bash
sentry-ingest Orders.postman_collection.json
```

This will generate Python tool files in `sentry/generated_tools/`.

### 4. Start the Server

```bash
source venv/bin/activate
python -m sentry
```

The server will start on `http://localhost:8000` and be ready for MCP client connections.

### 5. Connect MCP Clients

#### Cursor IDE Configuration

Add this to your Cursor settings (`.cursor-settings/settings.json`):

```json
{
  "mcp": {
    "servers": {
      "project-sentry": {
        "command": "python",
        "args": ["-m", "sentry"],
        "cwd": "/Users/nmorrisroe/Desktop/postman-replacement",
        "env": {
          "PATH": "/Users/nmorrisroe/Desktop/postman-replacement/venv/bin:${PATH}"
        }
      }
    }
  }
}
```

#### Windsurf IDE Configuration

Add this to your Windsurf MCP settings (`.windsurf/mcp_servers.json`):

```json
{
  "mcpServers": {
    "project-sentry": {
      "command": "python",
      "args": ["-m", "sentry"],
      "cwd": "/Users/nmorrisroe/Desktop/postman-replacement",
      "env": {
        "PATH": "/Users/nmorrisroe/Desktop/postman-replacement/venv/bin:${PATH}"
      }
    }
  }
}
```

#### Generic MCP Client Configuration

For other MCP-compatible clients, use these connection details:

```json
{
  "name": "project-sentry",
  "description": "QA automation server for API testing",
  "transport": {
    "type": "stdio",
    "command": "python",
    "args": ["-m", "sentry"],
    "cwd": "/Users/nmorrisroe/Desktop/postman-replacement",
    "env": {
      "PATH": "/Users/nmorrisroe/Desktop/postman-replacement/venv/bin:${PATH}"
    }
  }
}
```

## Usage

### Available Tools

#### Static Tools
- `state_setValue(session_id, key, value)` - Store data in session state
- `state_getValue(session_id, key)` - Retrieve data from session state  
- `state_clear(session_id)` - Clear all session data
- `generate_test_report(log, test_name, session_id)` - Generate test reports
- `search_api_documentation(query)` - Search available API tools
- `context_get_environment_config()` - Get environment configuration (v1.3)
- `context_get_valid_skus(category)` - Get valid test SKUs (v1.3)
- `context_get_default_customer()` - Get default customer data (v1.3)

#### Generated Tools (37 Total)
Tools automatically generated from your Orders.postman_collection.json:

**Order Management:**
- `create_order_1_item` - Create single item order
- `create_order_2_item` - Create multi-item order  
- `confirm_order` - Confirm order details
- `allocate_order` - Allocate inventory
- `ship_order` - Ship completed order
- `cancel_order_line` - Cancel specific order lines

**Payment Processing:**
- `create_paymentheader1_valid_cc` - Create valid credit card payment
- `create_paymentheader1_fraud_cc` - Test fraud detection
- `trigger_payment_processing` - Process payments
- `fetch_paymentheader` - Retrieve payment details

**Fulfillment & Config:**
- `fetch_allocation_config` - Get allocation settings
- `release_order` - Release order for fulfillment
- `short_order` - Handle short shipments
- `fetch_release_config` - Get release configuration

And 27 more tools covering your complete API surface.

## Unified Tool Registry and Discovery

Project Sentry now centralizes endpoint definitions in `sentry/presets/endpoints.json` and dynamically registers both:
- Preset tools: one per entry (backward compatible names)
- Canonical tools: one per unique `(method, path)` named like `invoke_{method}_{path}`

Key components:
- `sentry/core/tool_registry.py`: Loads rich metadata (description, tags, optional schemas/examples) and exposes helpers:
  - `list_presets()` and `list_canonicals()`
  - `describe(name)` for presets or canonical tools
- Static discovery tools:
  - `list_endpoints(kind="all|presets|canonicals")`
  - `describe_endpoint(name)`
  - `search_api_documentation(query)`

Authoring presets:
- Minimal fields supported: `method`, `path`, `default_payload`, `description`, `tags`
- Optional rich context fields (all optional): `params_schema`, `body_schema`, `headers`, `examples`, `usage`, `notes`

Runtime:
- `sentry/__main__.py` loads presets, registers tools, and records registrations in the registry for discoverability.
- Use `describe_endpoint` to see how to use a tool correctly, including presets that share the same canonical endpoint.

## What's New in v2.0

### 🚀 Dynamic Configuration Building
Project Sentry now builds `config.json` from real tenant API data instead of using static test data:

**New CLI Command:**
```bash
# Build config from tenant APIs
python -m sentry build-config

# Preview what would be built (dry run)
python -m sentry build-config --dry-run

# Save to custom location
python -m sentry build-config --output /path/to/config.json
```

**Automatic Config Refresh:**
- Server automatically rebuilds config on startup
- Fails fast if tenant APIs are unreachable
- Ensures configuration is always current with your tenant

**What Gets Fetched from APIs:**
- Enterprise codes from Organization API
- Supported currencies from Organization configuration
- Real SKUs from Item Search API
- Order types and document types from Order Config API

**What Remains Static:**
- Test credit card numbers for payment testing
- Default customer information (customizable)
- Environment metadata

### 🔥 Transparent Error Propagation (v1.3)
API errors are no longer hidden! When a tool makes an API call that results in a non-2xx status code, you now receive:
```json
{
  "success": false,
  "error": {
    "status_code": 400,
    "body": {
      "errors": [{
        "field": "currency",
        "message": "Invalid currency code 'EUR'. Must be 'USD' for this enterprise."
      }]
    },
    "headers": {...}
  }
}
```

### 🎯 Environment Configuration (v1.3)
Static tools provide programmatic access to configuration data:
- `context_get_environment_config()` - Get all environment configuration
- `context_get_valid_skus()` - Get valid test product SKUs
- `context_get_default_customer()` - Get default customer information

### Example Test Flow

Here's how an AI agent would use Project Sentry for end-to-end testing:

```
1. Start with session: "test-session-123"
2. Call: create_order_1_item() -> Store OrderId in state
3. Call: state_setValue(session_id="test-session-123", key="order_id", value="ORD-456")
4. Call: confirm_order() using stored OrderId
5. Call: allocate_order() -> Update order status
6. Call: ship_order() -> Complete fulfillment
7. Call: generate_test_report() -> Create markdown report
```

The AI agent can chain these API calls naturally using stored state, without needing to manage authentication or remember IDs between calls.

## Architecture

### Core Components

- **AuthManager**: Thread-safe singleton for username/password authentication with Basic auth
- **StateStore**: Session-aware in-memory state management
- **RequestWrapper**: HTTP client with automatic retry on 401 errors
- **Tool Registry**: Dynamic discovery and registration of API tools

### Project Structure

```
project-sentry/
├── .env.example              # Environment configuration template
├── pyproject.toml            # Project dependencies and metadata
├── README.md                 # This file
└── sentry/
    ├── __main__.py           # Server entry point
    ├── core/                 # Core framework modules
    │   ├── auth.py           # Authentication management
    │   ├── config.py         # Configuration loading
    │   ├── state.py          # State management
    │   └── request_wrapper.py # HTTP client with retry logic
    ├── generated_tools/      # Auto-generated API tools
    ├── static_tools/         # Hand-crafted utility tools
    │   ├── state_tools.py    # State management tools
    │   ├── reporting.py      # Test reporting tools
    │   └── search_tools.py   # API discovery tools
    └── ingest.py             # Postman collection ingestion CLI
```

## Development

### Adding Custom Tools

Create new tools in `sentry/static_tools/` following this pattern:

```python
from pydantic import BaseModel, Field

class MyToolParams(BaseModel):
    param1: str = Field(..., description="Description of parameter")

def my_custom_tool(params: MyToolParams) -> dict:
    """Tool description for AI agents."""
    # Implementation here
    return {"success": True, "result": "data"}
```

### Testing

```bash
# Install development dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Format code
black sentry/
isort sentry/
```

## Configuration Files

### Environment Variables (.env)

| Variable | Description | Example |
|----------|-------------|---------|
| `AUTH_BASE_URL` | Authentication server base URL | `https://auth.example.com` |
| `CLIENT_ID` | OAuth2 client identifier | `your-client-id` |
| `CLIENT_SECRET` | OAuth2 client secret | `your-secret` |
| `TENANT_ID` | Tenant identifier for multi-tenant auth | `tenant-123` |
| `API_BASE_URL` | Target API server base URL | `https://api.example.com` |
| `API_VERSION` | API version to use | `v1` |
| `HOST` | Server bind address | `localhost` |
| `PORT` | Server port | `8000` |
| `LOG_LEVEL` | Logging verbosity | `INFO` |

### Environment Configuration (config.json)

The `config.json` file is now **automatically generated** from your tenant's live API data:

```json
{
  "environment": {
    "name": "Live Tenant Environment",
    "description": "Configuration built from real tenant API data",
    "version": "2.0",
    "built_at": "2025-09-09T20:43:10.285Z"
  },
  "enterprise_codes": {
    "default": "YOUR-TENANT-CODE",
    "alternatives": ["ALT-CODE-1", "ALT-CODE-2"]
  },
  "currencies": {
    "default": "USD",
    "supported": ["USD", "EUR", "GBP", "CAD"]
  },
  "skus": {
    "general": [
      {
        "id": "REAL-SKU-001",
        "name": "Real Product from Your Catalog",
        "category": "general",
        "price": 99.99,
        "description": "Actual product from your tenant"
      }
    ]
  }
}
```

**Configuration Sources:**
- **Enterprise codes**: Fetched from `/cfg/api/configDirector/export`
- **Currencies**: Retrieved from `/api/organization/organization/search`
- **SKUs**: Pulled from `/api/search/item` with real product data
- **Order types**: Loaded from `/api/order/orderConfig`

This configuration is accessible via the `context_get_environment_config()` tool and is automatically refreshed on server startup.

## Troubleshooting

### Authentication Issues
- Verify your credentials in `.env`
- Check that the auth server is accessible
- Review server logs for detailed error messages

### Tool Generation Issues
- Ensure your Postman collection is valid JSON
- Check that request bodies contain valid JSON examples
- Review generated files in `sentry/generated_tools/`

### Server Startup Issues
- Verify all dependencies are installed
- Check that the configured port is available
- Review the startup logs for specific errors

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

## License

[Add your license information here]