# Armin Tavassoli - SE333 Final Project
## Testing Agent with Decision Table Test Generation and Security Scanning
For my SE333 final project, I built an MCP (Model Context Protocol) server that helps automate testing workflows for Java Maven projects. The main features are decision table-based test generation and security vulnerability scanning, along with coverage analysis and Git automation tools.
## What This Does
The agent provides a few key capabilities:
1. **Decision Table-Based Test Generation**: Generates JUnit test cases by analyzing method logic and creating test matrices that cover different decision paths
2. **Security Vulnerability Scanning**: Scans Java code for common security issues like SQL injection, XSS, command injection, etc.
3. **Coverage Analysis**: Finds code that's missing test coverage and suggests what to test
4. **Git Automation**: Handles common Git workflows like staging, committing, and creating PRs
## Features
### Coverage Tools
- **`find_jacoco_path`**: Finds where the JaCoCo coverage reports are located (HTML, XML, or exec files)
- **`missing_coverage`**: Looks through JaCoCo XML reports to find uncovered code and shows coverage stats
### Test Generation
- **`generate_decision_table_tests`**: Creates JUnit tests using decision tables. It:
- Looks at method signatures and logic
- Finds decision points (if/else, switch statements, ternary operators)
- Builds a test matrix that covers:
- Null inputs
- Empty inputs
- Valid inputs
- Boundary conditions
- Edge cases
- Exception scenarios
### Security Scanning
- **`scan_security_vulnerabilities`**: Scans Java source code for:
- SQL Injection risks
- Command Injection vulnerabilities
- Path Traversal issues
- Hardcoded secrets/passwords
- Insecure random number generation
- Deserialization risks
- XSS vulnerabilities
- Each finding is classified by severity (high, medium, low)
### Git Tools
- **`git_status`**: Shows what's staged, unstaged, and untracked
- **`git_add_all`**: Stages everything (skips build artifacts)
- **`git_commit`**: Creates commits with messages that include coverage stats
- **`git_push`**: Pushes to the remote repo
- **`git_pull_request`**: Creates PRs (needs GitHub CLI or you can do it manually)
## Setup
### What You Need
- Python 3.12 or newer
- Node.js 18+ (LTS version works best)
- VS Code with Chat view
- Java 11+ and Maven 3.6+
- Git and a GitHub account
- uv package manager ([install here](https://docs.astral.sh/uv/))
### Installation Steps
1. **Install uv**
```bash
# macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
# Windows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
```
2. **Set up Python environment**
```bash
cd Armin_Tavassoli_SE333_Final_project
uv init
uv venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
```
3. **Install dependencies**
```bash
uv sync
# Or if that doesn't work:
uv add mcp fastmcp httpx pypdf python-dotenv
```
4. **Connect VS Code to the MCP server**
- Start the server in HTTP mode:
```bash
python server.py
```
The server will start on `http://localhost:8001/sse` (or port 8000 if MCP_PORT is not set)
- **Note**: VS Code's HTTP MCP client has known compatibility issues with FastMCP's SSE transport. For VS Code, you can either:
- **Option A (Recommended for VS Code)**: Use stdio mode by setting `MCP_USE_STDIO=true` and configuring VS Code to use local process (see `.vscode/settings.json`)
- **Option B**: Use HTTP mode - the server runs on HTTP as required, but VS Code may have connection issues. HTTP mode works well with other MCP clients or for testing with curl.
- In VS Code, press `CTRL+SHIFT+P` (or `CMD+SHIFT+P` on Mac) and search for "MCP: Add Server"
- Enter the server URL: `http://localhost:8001` (or `http://localhost:8000` if using default port)
- Name it something like "SE333 Testing Agent"
- Make sure the tools show up in the Chat view
5. **Enable Auto-Approve**
- Press `CTRL+SHIFT+P` and search for "Chat: Settings"
- Turn on Auto-Approve
- Check that all tools are highlighted
6. **Set up the Maven project**
```bash
cd codebase
mvn clean install
mvn test jacoco:report
```
## How to Use
### Basic Workflow
1. **Check coverage**
- Use `find_jacoco_path` to find the reports
- Use `missing_coverage` to see what's not covered
2. **Generate tests**
- Use `generate_decision_table_tests` with a class and method name
- Example: `generate_decision_table_tests(class_name="org.apache.commons.lang3.StringUtils", method_name="isEmpty")`
3. **Scan for security issues**
- Use `scan_security_vulnerabilities` to check the codebase
- Start with high-severity issues
4. **Git workflow**
- `git_status` → `git_add_all` → `git_commit` → `git_push` → `git_pull_request`
### Example: Generating Tests
```python
# In VS Code Chat:
generate_decision_table_tests(
class_name="org.apache.commons.lang3.StringUtils",
method_name="isEmpty",
input_parameters='{"str": "String"}'
)
```
This will:
1. Look at the method signature and code
2. Find decision points in the logic
3. Generate test cases
4. Give you JUnit test code you can save
### Example: Security Scan
```python
scan_security_vulnerabilities(
class_name="org.apache.commons.lang3.StringUtils",
severity="high"
)
```
You'll get JSON back with:
- What vulnerabilities were found
- Where they are (file and line)
- How severe they are
- Code snippets
- How to fix them
## Project Structure
```
Armin_Tavassoli_SE333_Final_project/
├── codebase/ # Apache Commons Lang3 Maven project
│ ├── pom.xml # Maven config with JaCoCo
│ ├── src/
│ │ ├── main/java/ # Source code
│ │ └── test/java/ # Test code
│ └── target/
│ └── site/jacoco/ # Coverage reports
├── .github/
│ └── prompts/
│ └── tester.prompt.md # Agent prompt config
├── server.py # MCP server code
├── pyproject.toml # Python dependencies
├── README.md # This file
├── demo/ # Demo materials
│ └── final_presentation.mp4 # Video (if applicable)
├── report/ # Written report
│ └── reflection.pdf # LaTeX reflection report
└── docs/ # Extra docs
```
## Tool Reference
### find_jacoco_path
Finds where the JaCoCo coverage reports are in the Maven project.
**Inputs**: None
**Returns**: Path to the report file, or an error message if reports haven't been generated yet
**Example**:
```
find_jacoco_path()
```
---
### missing_coverage
Looks at JaCoCo XML reports to find code that isn't covered by tests.
**Inputs**:
- `class_name` (optional): Specific class to check
**Returns**: JSON with:
- Coverage summary (lines, branches, methods, classes)
- List of classes with low coverage
- Suggestions for what to test
**Example**:
```python
missing_coverage(class_name="org.apache.commons.lang3.StringUtils")
```
---
### generate_decision_table_tests
Generates JUnit test cases using decision tables.
**Inputs**:
- `class_name` (required): Full class name like `org.apache.commons.lang3.StringUtils`
- `method_name` (required): Method to test
- `input_parameters` (optional): JSON describing the parameters
**Returns**: JUnit test code as a string
**Example**:
```python
generate_decision_table_tests(
class_name="org.apache.commons.lang3.StringUtils",
method_name="isEmpty"
)
```
---
### scan_security_vulnerabilities
Scans Java code for security issues.
**Inputs**:
- `class_name` (optional): Specific class to scan
- `severity` (optional): Filter by "all", "high", "medium", or "low"
**Returns**: JSON with:
- Total count of vulnerabilities
- Breakdown by severity
- Details for each one:
- File and line number
- Type of vulnerability
- Severity
- Description
- Code snippet
- How to fix it
**Example**:
```python
scan_security_vulnerabilities(severity="high")
```
---
### Git Tools
#### git_status
Shows what files are staged, unstaged, or untracked.
#### git_add_all
Stages all changes (build artifacts are excluded).
#### git_commit(message)
Creates a commit. The message will include coverage stats if available.
#### git_push(remote, branch)
Pushes commits to the remote repo.
#### git_pull_request(base, title, body)
Creates a pull request. You'll need GitHub CLI installed, or you can create it manually.
## Troubleshooting
### Server Won't Start
If `python server.py` fails:
1. Check Python version: `python --version` (needs 3.12+)
2. Make sure dependencies are installed: `uv sync`
3. Check that the virtual environment is activated
4. See if port 8000 is already in use
### Can't Find JaCoCo Reports
If `find_jacoco_path` says reports aren't found:
1. Run the tests first: `cd codebase && mvn clean test`
2. Generate the reports: `mvn jacoco:report`
3. Check that the directory exists: `ls codebase/target/site/jacoco/`
### VS Code Won't Connect
If tools don't show up in Chat:
1. Make sure the server is actually running (check the terminal)
2. Double-check the server URL in VS Code settings
3. Try restarting VS Code
4. Make sure Auto-Approve is enabled
5. Check the server logs for any errors
### Test Generation Fails
If `generate_decision_table_tests` gives an error:
1. Make sure the class name is fully qualified (e.g., `org.apache.commons.lang3.StringUtils`)
2. Check that the method name is spelled correctly
3. Verify the source file exists in `codebase/src/main/java/`
4. Make sure the file is readable
### Security Scan Finds Nothing
If `scan_security_vulnerabilities` returns empty results:
1. This might be fine if the code is actually secure
2. Try scanning a specific class: `scan_security_vulnerabilities(class_name="...")`
3. Try `severity="all"` to see everything
4. Make sure Java files are in `codebase/src/main/java/`
### Git Commands Fail
If Git tools return errors:
1. Make sure Git is initialized: `git status`
2. Check that your Git credentials are set up
3. For `git_pull_request`: Install GitHub CLI (`gh`) or create the PR manually
4. Make sure you have write access to the repository
## Development Notes
### Adding New Tools
If you want to add a new MCP tool:
1. Create a function with the `@mcp.tool()` decorator
2. Write a good docstring
3. Add the tool name to the list in `tester.prompt.md`
4. Update this README
### Testing
To test the agent:
1. Start the server: `python server.py`
2. In VS Code Chat, try each tool one at a time
3. Check that the outputs look right
4. Try some invalid inputs to test error handling
## Tracking Results
### Coverage Metrics
To track coverage improvements, use JaCoCo reports. After running `mvn test jacoco:report`, you can:
1. View HTML report: Open `codebase/target/site/jacoco/index.html` in a browser
2. Use the MCP tools:
- `find_jacoco_path` to locate reports
- `missing_coverage` to get detailed statistics
The reports show:
- Line coverage percentage
- Branch coverage percentage
- Method coverage percentage
- Class coverage percentage
**For Presentation**: Compare before/after coverage by:
- Running initial coverage: `cd codebase && mvn clean test jacoco:report`
- Generating tests using the agent
- Running coverage again: `mvn test jacoco:report`
- Comparing the metrics
### Security Metrics
Track security improvements:
- Total vulnerabilities found (use `scan_security_vulnerabilities`)
- Breakdown by severity (high/medium/low)
- Remediation progress
- Files affected
### Git Commit History
If you use the Git automation tools, you can track improvements through commit history:
- Each commit includes coverage statistics
- Review commit messages to see progress over time
- Use `git log` to see the improvement timeline
## Future Ideas
Some things I'd like to add later:
- Integration with mutation testing (PIT)
- Support for other languages besides Java
- Better test case prioritization
- Automated test refactoring
- CI/CD integration
- Real-time coverage monitoring
- More advanced security pattern detection
## License
This is my SE333 coursework project at DePaul University.
The codebase (Apache Commons Lang3) uses the Apache License 2.0.
## Contact
**Student**: Armin Tavassoli
**Course**: SE333 - Software Agents
**Institution**: DePaul University