# Armin Tavassoli - SE333 Final Project

## Testing Agent with Decision Table Test Generation and Security Scanning

For my SE333 final project, I built an MCP (Model Context Protocol) server that helps automate testing workflows for Java Maven projects. The main features are decision table-based test generation and security vulnerability scanning, along with coverage analysis and Git automation tools.

## What This Does

The agent provides a few key capabilities:

1. **Decision Table-Based Test Generation**: Generates JUnit test cases by analyzing method logic and creating test matrices that cover different decision paths
2. **Security Vulnerability Scanning**: Scans Java code for common security issues like SQL injection, XSS, command injection, etc.
3. **Coverage Analysis**: Finds code that's missing test coverage and suggests what to test
4. **Git Automation**: Handles common Git workflows like staging, committing, and creating PRs

## Features

### Coverage Tools

- **`find_jacoco_path`**: Finds where the JaCoCo coverage reports are located (HTML, XML, or exec files)
- **`missing_coverage`**: Looks through JaCoCo XML reports to find uncovered code and shows coverage stats

### Test Generation

- **`generate_decision_table_tests`**: Creates JUnit tests using decision tables. It:
  - Looks at method signatures and logic
  - Finds decision points (if/else, switch statements, ternary operators)
  - Builds a test matrix that covers:
    - Null inputs
    - Empty inputs
    - Valid inputs
    - Boundary conditions
    - Edge cases
    - Exception scenarios

### Security Scanning

- **`scan_security_vulnerabilities`**: Scans Java source code for:
  - SQL Injection risks
  - Command Injection vulnerabilities
  - Path Traversal issues
  - Hardcoded secrets/passwords
  - Insecure random number generation
  - Deserialization risks
  - XSS vulnerabilities
  - Each finding is classified by severity (high, medium, low)

### Git Tools

- **`git_status`**: Shows what's staged, unstaged, and untracked
- **`git_add_all`**: Stages everything (skips build artifacts)
- **`git_commit`**: Creates commits with messages that include coverage stats
- **`git_push`**: Pushes to the remote repo
- **`git_pull_request`**: Creates PRs (needs GitHub CLI or you can do it manually)

## Setup

### What You Need

- Python 3.12 or newer
- Node.js 18+ (LTS version works best)
- VS Code with Chat view
- Java 11+ and Maven 3.6+
- Git and a GitHub account
- uv package manager ([install here](https://docs.astral.sh/uv/))

### Installation Steps

1. **Install uv**
   ```bash
   # macOS/Linux
   curl -LsSf https://astral.sh/uv/install.sh | sh
   
   # Windows
   powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
   ```

2. **Set up Python environment**
   ```bash
   cd Armin_Tavassoli_SE333_Final_project
   uv init
   uv venv
   source .venv/bin/activate  # On Windows: .venv\Scripts\activate
   ```

3. **Install dependencies**
   ```bash
   uv sync
   # Or if that doesn't work:
   uv add mcp fastmcp httpx pypdf python-dotenv
   ```

4. **Connect VS Code to the MCP server**
   - Start the server in HTTP mode:
     ```bash
     python server.py
     ```
     The server will start on `http://localhost:8001/sse` (or port 8000 if MCP_PORT is not set)
   - **Note**: VS Code's HTTP MCP client has known compatibility issues with FastMCP's SSE transport. For VS Code, you can either:
     - **Option A (Recommended for VS Code)**: Use stdio mode by setting `MCP_USE_STDIO=true` and configuring VS Code to use local process (see `.vscode/settings.json`)
     - **Option B**: Use HTTP mode - the server runs on HTTP as required, but VS Code may have connection issues. HTTP mode works well with other MCP clients or for testing with curl.
   - In VS Code, press `CTRL+SHIFT+P` (or `CMD+SHIFT+P` on Mac) and search for "MCP: Add Server"
   - Enter the server URL: `http://localhost:8001` (or `http://localhost:8000` if using default port)
   - Name it something like "SE333 Testing Agent"
   - Make sure the tools show up in the Chat view

5. **Enable Auto-Approve**
   - Press `CTRL+SHIFT+P` and search for "Chat: Settings"
   - Turn on Auto-Approve
   - Check that all tools are highlighted

6. **Set up the Maven project**
   ```bash
   cd codebase
   mvn clean install
   mvn test jacoco:report
   ```

## How to Use

### Basic Workflow

1. **Check coverage**
   - Use `find_jacoco_path` to find the reports
   - Use `missing_coverage` to see what's not covered

2. **Generate tests**
   - Use `generate_decision_table_tests` with a class and method name
   - Example: `generate_decision_table_tests(class_name="org.apache.commons.lang3.StringUtils", method_name="isEmpty")`

3. **Scan for security issues**
   - Use `scan_security_vulnerabilities` to check the codebase
   - Start with high-severity issues

4. **Git workflow**
   - `git_status` → `git_add_all` → `git_commit` → `git_push` → `git_pull_request`

### Example: Generating Tests

```python
# In VS Code Chat:
generate_decision_table_tests(
    class_name="org.apache.commons.lang3.StringUtils",
    method_name="isEmpty",
    input_parameters='{"str": "String"}'
)
```

This will:
1. Look at the method signature and code
2. Find decision points in the logic
3. Generate test cases
4. Give you JUnit test code you can save

### Example: Security Scan

```python
scan_security_vulnerabilities(
    class_name="org.apache.commons.lang3.StringUtils",
    severity="high"
)
```

You'll get JSON back with:
- What vulnerabilities were found
- Where they are (file and line)
- How severe they are
- Code snippets
- How to fix them

## Project Structure

```
Armin_Tavassoli_SE333_Final_project/
├── codebase/                     # Apache Commons Lang3 Maven project
│   ├── pom.xml                   # Maven config with JaCoCo
│   ├── src/
│   │   ├── main/java/            # Source code
│   │   └── test/java/            # Test code
│   └── target/
│       └── site/jacoco/          # Coverage reports
├── .github/
│   └── prompts/
│       └── tester.prompt.md     # Agent prompt config
├── server.py                     # MCP server code
├── pyproject.toml                # Python dependencies
├── README.md                     # This file
├── demo/                         # Demo materials
│   └── final_presentation.mp4   # Video (if applicable)
├── report/                       # Written report
│   └── reflection.pdf           # LaTeX reflection report
└── docs/                         # Extra docs
```

## Tool Reference

### find_jacoco_path

Finds where the JaCoCo coverage reports are in the Maven project.

**Inputs**: None

**Returns**: Path to the report file, or an error message if reports haven't been generated yet

**Example**:
```
find_jacoco_path()
```

---

### missing_coverage

Looks at JaCoCo XML reports to find code that isn't covered by tests.

**Inputs**:
- `class_name` (optional): Specific class to check

**Returns**: JSON with:
- Coverage summary (lines, branches, methods, classes)
- List of classes with low coverage
- Suggestions for what to test

**Example**:
```python
missing_coverage(class_name="org.apache.commons.lang3.StringUtils")
```

---

### generate_decision_table_tests

Generates JUnit test cases using decision tables.

**Inputs**:
- `class_name` (required): Full class name like `org.apache.commons.lang3.StringUtils`
- `method_name` (required): Method to test
- `input_parameters` (optional): JSON describing the parameters

**Returns**: JUnit test code as a string

**Example**:
```python
generate_decision_table_tests(
    class_name="org.apache.commons.lang3.StringUtils",
    method_name="isEmpty"
)
```

---

### scan_security_vulnerabilities

Scans Java code for security issues.

**Inputs**:
- `class_name` (optional): Specific class to scan
- `severity` (optional): Filter by "all", "high", "medium", or "low"

**Returns**: JSON with:
- Total count of vulnerabilities
- Breakdown by severity
- Details for each one:
  - File and line number
  - Type of vulnerability
  - Severity
  - Description
  - Code snippet
  - How to fix it

**Example**:
```python
scan_security_vulnerabilities(severity="high")
```

---

### Git Tools

#### git_status
Shows what files are staged, unstaged, or untracked.

#### git_add_all
Stages all changes (build artifacts are excluded).

#### git_commit(message)
Creates a commit. The message will include coverage stats if available.

#### git_push(remote, branch)
Pushes commits to the remote repo.

#### git_pull_request(base, title, body)
Creates a pull request. You'll need GitHub CLI installed, or you can create it manually.

## Troubleshooting

### Server Won't Start

If `python server.py` fails:

1. Check Python version: `python --version` (needs 3.12+)
2. Make sure dependencies are installed: `uv sync`
3. Check that the virtual environment is activated
4. See if port 8000 is already in use

### Can't Find JaCoCo Reports

If `find_jacoco_path` says reports aren't found:

1. Run the tests first: `cd codebase && mvn clean test`
2. Generate the reports: `mvn jacoco:report`
3. Check that the directory exists: `ls codebase/target/site/jacoco/`

### VS Code Won't Connect

If tools don't show up in Chat:

1. Make sure the server is actually running (check the terminal)
2. Double-check the server URL in VS Code settings
3. Try restarting VS Code
4. Make sure Auto-Approve is enabled
5. Check the server logs for any errors

### Test Generation Fails

If `generate_decision_table_tests` gives an error:

1. Make sure the class name is fully qualified (e.g., `org.apache.commons.lang3.StringUtils`)
2. Check that the method name is spelled correctly
3. Verify the source file exists in `codebase/src/main/java/`
4. Make sure the file is readable

### Security Scan Finds Nothing

If `scan_security_vulnerabilities` returns empty results:

1. This might be fine if the code is actually secure
2. Try scanning a specific class: `scan_security_vulnerabilities(class_name="...")`
3. Try `severity="all"` to see everything
4. Make sure Java files are in `codebase/src/main/java/`

### Git Commands Fail

If Git tools return errors:

1. Make sure Git is initialized: `git status`
2. Check that your Git credentials are set up
3. For `git_pull_request`: Install GitHub CLI (`gh`) or create the PR manually
4. Make sure you have write access to the repository

## Development Notes

### Adding New Tools

If you want to add a new MCP tool:

1. Create a function with the `@mcp.tool()` decorator
2. Write a good docstring
3. Add the tool name to the list in `tester.prompt.md`
4. Update this README

### Testing

To test the agent:

1. Start the server: `python server.py`
2. In VS Code Chat, try each tool one at a time
3. Check that the outputs look right
4. Try some invalid inputs to test error handling

## Tracking Results

### Coverage Metrics

To track coverage improvements, use JaCoCo reports. After running `mvn test jacoco:report`, you can:

1. View HTML report: Open `codebase/target/site/jacoco/index.html` in a browser
2. Use the MCP tools:
   - `find_jacoco_path` to locate reports
   - `missing_coverage` to get detailed statistics

The reports show:
- Line coverage percentage
- Branch coverage percentage
- Method coverage percentage
- Class coverage percentage

**For Presentation**: Compare before/after coverage by:
- Running initial coverage: `cd codebase && mvn clean test jacoco:report`
- Generating tests using the agent
- Running coverage again: `mvn test jacoco:report`
- Comparing the metrics

### Security Metrics

Track security improvements:
- Total vulnerabilities found (use `scan_security_vulnerabilities`)
- Breakdown by severity (high/medium/low)
- Remediation progress
- Files affected

### Git Commit History

If you use the Git automation tools, you can track improvements through commit history:
- Each commit includes coverage statistics
- Review commit messages to see progress over time
- Use `git log` to see the improvement timeline

## Future Ideas

Some things I'd like to add later:
- Integration with mutation testing (PIT)
- Support for other languages besides Java
- Better test case prioritization
- Automated test refactoring
- CI/CD integration
- Real-time coverage monitoring
- More advanced security pattern detection

## License

This is my SE333 coursework project at DePaul University.

The codebase (Apache Commons Lang3) uses the Apache License 2.0.

## Contact

**Student**: Armin Tavassoli  
**Course**: SE333 - Software Agents  
**Institution**: DePaul University