# Development Workflow
## π¨ **IMPORTANT: All Changes Must Use Pull Requests**
**Direct commits to `main` are not allowed.** All changes must go through the Pull Request (PR) process to ensure:
- Code review and quality control
- Proper version management
- CI/CD validation
- Documentation updates
## π **Standard Development Workflow**
### 1. **Create Feature Branch**
```powershell
# Create and switch to feature branch
git checkout -b feature/your-feature-name
# Or for bug fixes
git checkout -b fix/issue-description
# Or for documentation updates
git checkout -b docs/update-description
```
### 2. **Make Your Changes**
```powershell
# Make code changes, add tests, update docs
# Commit frequently with clear messages
git add .
git commit -m "Add feature X with tests and documentation
- Implement core functionality
- Add comprehensive unit tests
- Update command documentation
- Include usage examples"
```
### 3. **Push Feature Branch**
```powershell
# Push your feature branch to GitHub
git push origin feature/your-feature-name
```
### 4. **Create Pull Request**
1. Go to [GitHub Repository](https://github.com/sbroenne/mcp-server-excel)
2. Click **"New Pull Request"**
3. Select your feature branch
4. Fill out the PR template:
- **Clear title** describing the change
- **Detailed description** of what was changed and why
- **Testing information** - what tests were added/run
- **Breaking changes** - if any
- **Documentation updates** - what docs were updated
### 5. **PR Review Process**
- **Automated checks** will run (build, tests, linting)
- **Code review** by maintainers
- **Address feedback** if requested
- **Merge** once approved and all checks pass
### 6. **After Merge**
```powershell
# Switch back to main and pull latest
git checkout main
git pull origin main
# Delete the feature branch (cleanup)
git branch -d feature/your-feature-name
git push origin --delete feature/your-feature-name
```
## π·οΈ **Release Process**
### Creating a New Release
**Only maintainers** can create releases. The process is:
1. **Ensure all changes are merged** to `main` via PRs
2. **Create and push a version tag**:
```powershell
# Create version tag (semantic versioning)
git tag v1.1.0
# Push the tag (triggers release workflow)
git push origin v1.1.0
```
1. **Automated Release Workflow**:
- β
Updates version numbers in project files
- β
Builds the release binaries
- β
Creates GitHub release with ZIP file
- β
Updates release notes
### Version Numbering
We follow [Semantic Versioning](https://semver.org/):
- **Major** (v2.0.0): Breaking changes
- **Minor** (v1.1.0): New features, backward compatible
- **Patch** (v1.0.1): Bug fixes, backward compatible
## π **Branch Protection Rules**
The `main` branch is protected with:
- **Require pull request reviews** - Changes must be reviewed
- **Require status checks** - CI/CD must pass
- **Require up-to-date branches** - Must be current with main
- **No direct pushes** - All changes via PR only
## π§ͺ **Testing Requirements & Organization**
### **Three-Tier Test Architecture**
ExcelMcp uses a **production-ready three-tier testing approach** with organized directory structure:
```
tests/
βββ ExcelMcp.Core.Tests/
β βββ Unit/ # Fast tests, no Excel required (~2-5 sec)
β βββ Integration/ # Medium speed, requires Excel (~1-15 min)
β βββ RoundTrip/ # Slow, comprehensive workflows (~3-10 min each)
βββ ExcelMcp.McpServer.Tests/
β βββ Unit/ # Fast tests, no server required
β βββ Integration/ # Medium speed, requires MCP server
β βββ RoundTrip/ # Slow, end-to-end protocol testing
βββ ExcelMcp.CLI.Tests/
βββ Unit/ # Fast tests, no Excel required
βββ Integration/ # Medium speed, requires Excel & CLI
```
### **Development Workflow Commands**
**During Development (Fast Feedback):**
```powershell
# Quick validation - run tests for specific feature
dotnet test --filter "Feature=PowerQuery&RunType!=OnDemand"
dotnet test --filter "Feature=DataModel&RunType!=OnDemand"
```
**Before Commit (Comprehensive):**
```powershell
# Full local validation - runs in 10-20 minutes (excludes VBA)
dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA&Feature!=VBATrust"
```
**Session/Batch Code Changes (MANDATORY):**
```powershell
# When modifying ExcelSession.cs or ExcelBatch.cs
dotnet test --filter "RunType=OnDemand"
```
### **Test Categories & Guidelines**
**β οΈ No Unit Tests** - See `docs/ADR-001-NO-UNIT-TESTS.md` for architectural rationale
**Integration Tests (`Category=Integration`)**
- β
Test business logic with real Excel COM interaction
- β
Medium speed (10-20 minutes for full suite)
- β
Requires Excel installation
- β
These ARE our unit tests (Excel COM cannot be mocked)
- β
Run specific features during development
- β
Slow execution (3-10 minutes each)
- β
Verifies actual Excel state changes
- β
Comprehensive scenario coverage
### **Adding New Tests**
When creating tests, follow these placement guidelines:
```csharp
// Unit Test Example
[Trait("Category", "Unit")]
[Trait("Speed", "Fast")]
[Trait("Layer", "Core")]
public class CommandLogicTests
{
// Tests business logic without Excel
}
// Integration Test Example
[Trait("Category", "Integration")]
[Trait("Speed", "Medium")]
[Trait("Feature", "PowerQuery")]
[Trait("RequiresExcel", "true")]
public class PowerQueryCommandsTests
{
// Tests single Excel operations
}
// Round Trip Test Example
[Trait("Category", "RoundTrip")]
[Trait("Speed", "Slow")]
[Trait("Feature", "EndToEnd")]
[Trait("RequiresExcel", "true")]
public class VbaWorkflowTests
{
// Tests complete workflows: import β run β verify β export
}
```
### **PR Testing Requirements**
Before creating a PR, ensure:
```powershell
# Required - Integration tests pass (excludes VBA)
dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA&Feature!=VBATrust"
# Code builds without warnings
dotnet build -c Release
# Code follows style guidelines (automatic via EditorConfig)
```
**For Complex Features:**
- β
Add integration tests for all Excel operations
- β
Test round-trip persistence (create β save β reload β verify)
- β
Update documentation
- β
No unit tests needed (see ADR-001-NO-UNIT-TESTS.md)
## π **MCP Server Configuration Management**
### **CRITICAL: Keep server.json in Sync**
When modifying MCP Server functionality, **you must update** `src/ExcelMcp.McpServer/.mcp/server.json`:
#### **When to Update server.json:**
- β
**Adding new MCP tools** - Add tool definition to `"tools"` array
- β
**Modifying tool parameters** - Update `inputSchema` and `properties`
- β
**Changing tool descriptions** - Update `description` fields
- β
**Adding new capabilities** - Update `"capabilities"` section
- β
**Changing requirements** - Update `"environment"."requirements"`
#### **server.json Synchronization Checklist:**
```powershell
# After making MCP Server code changes, verify:
# 1. Tool definitions match actual implementations
Compare-Object (Get-Content "src/ExcelMcp.McpServer/.mcp/server.json" | ConvertFrom-Json).tools (Get-ChildItem "src/ExcelMcp.McpServer/Tools/*.cs")
# 2. Build succeeds with updated configuration
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
# 3. Test MCP server starts without errors
dnx Sbroenne.ExcelMcp.McpServer --yes
```
#### **server.json Structure:**
```json
{
"version": "2.0.0", // β Updated by release workflow
"tools": [ // β Must match Tools/*.cs implementations
{
"name": "excel_file", // β Must match [McpServerTool] attribute
"description": "...", // β Keep description accurate
"inputSchema": { // β Must match method parameters
"properties": {
"action": { ... }, // β Must match actual actions supported
"filePath": { ... } // β Must match parameter types
}
}
}
]
}
```
#### **Common server.json Update Scenarios:**
1. **Adding New Tool:**
```csharp
// In Tools/NewTool.cs
[McpServerTool]
public async Task<string> NewTool(string action, string parameter)
```
```json
// Add to server.json tools array
{
"name": "excel_newtool",
"description": "New functionality description",
"inputSchema": { ... }
}
```
2. **Adding Action to Existing Tool:**
```csharp
// In existing tool method
case "new-action":
return HandleNewAction(parameter);
```
```json
// Update inputSchema properties.action enum
"action": {
"enum": ["list", "create", "new-action"] // β Add new action
}
```
## οΏ½π **PR Template Checklist**
When creating a PR, verify:
- [ ] **Code builds** with zero warnings
- [ ] **All tests pass** (unit tests minimum)
- [ ] **New features have tests**
- [ ] **Documentation updated** (README, etc.)
- [ ] **MCP server.json updated** (if MCP Server changes) β **NEW**
- [ ] **Breaking changes documented**
- [ ] **Follows existing code patterns**
- [ ] **Commit messages are clear**
## π« **What NOT to Do**
- β **Don't commit directly to `main`**
- β **Don't create releases without PRs**
- β **Don't skip tests**
- β **Don't ignore build warnings**
- β **Don't update version numbers manually** (release workflow handles this)
## π‘ **Tips for Good PRs**
### Commit Messages
```text
β
Good: "Add PowerQuery batch refresh command with error handling"
β Bad: "fix stuff"
```
### PR Titles
```text
β
Good: "Add batch operations for Power Query refresh"
β Bad: "Update code"
```
### PR Size
- **Keep PRs focused** - One feature/fix per PR
- **Break large changes** into smaller, reviewable chunks
- **Include tests and docs** in the same PR as the feature
## π§ **Local Development Setup**
```powershell
# Clone the repository
git clone https://github.com/sbroenne/mcp-server-excel.git
cd ExcelMcp
# Install dependencies
dotnet restore
# Run all tests
dotnet test
# Build release version
dotnet build -c Release
# Test the built executable
.\src\ExcelMcp.CLI\bin\Release\net10.0\excelcli.exe --version
```
## π **Application Insights / Telemetry Setup**
ExcelMcp uses Azure Application Insights (Classic SDK with WorkerService integration) for anonymous usage telemetry and crash reporting. Telemetry is **opt-out** (enabled by default in release builds).
### **How It Works**
The Application Insights connection string is **embedded at build time** via MSBuild - there is no runtime environment variable lookup.
**Build-time flow:**
1. MSBuild reads `AppInsightsConnectionString` property (from `Directory.Build.props.user` or env var)
2. Generates `TelemetryConfig.g.cs` with the connection string as a `const string`
3. Compiled assembly contains the embedded connection string
### **What is Tracked**
- **Tool invocations**: Tool name, action, duration (ms), success/failure
- **Unhandled exceptions**: Exception type and redacted stack trace
- **User ID**: SHA256 hash of machine identity (anonymous, 16 chars)
- **Session ID**: Random GUID per process (8 chars)
### **What is NOT Tracked**
- File paths, file names, or file contents
- User identity, machine name, or IP address
- Excel data, formulas, or cell values
- Connection strings, credentials, or passwords
### **Sensitive Data Redaction**
All telemetry passes through `SensitiveDataRedactingProcessor` which removes:
- Windows file paths (`C:\Users\...` β `[REDACTED_PATH]`)
- UNC paths (`\\server\share\...` β `[REDACTED_PATH]`)
- Connection string secrets (`Password=...` β `[REDACTED_CREDENTIAL]`)
- Email addresses β `[REDACTED_EMAIL]`
### **Local Development with Telemetry**
To enable telemetry in local builds:
```powershell
# 1. Copy the template file
Copy-Item "Directory.Build.props.user.template" "Directory.Build.props.user"
# 2. Edit Directory.Build.props.user and add your connection string
# <AppInsightsConnectionString>InstrumentationKey=xxx;IngestionEndpoint=...</AppInsightsConnectionString>
# 3. Build - connection string is embedded at compile time
dotnet build src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
# 4. Run - telemetry is automatically sent to Azure
dotnet run --project src/ExcelMcp.McpServer/ExcelMcp.McpServer.csproj
```
**Note:** `Directory.Build.props.user` is gitignored - your connection string won't be committed.
### **Local Development without Telemetry**
If you don't create `Directory.Build.props.user`, builds will have an empty connection string and telemetry will be disabled. This is the default for local development.
### **Azure Resources Setup (Maintainers Only)**
To deploy the Application Insights infrastructure:
```powershell
# 1. Login to Azure
az login
# 2. Deploy resources (creates RG, Log Analytics, App Insights)
.\infrastructure\azure\deploy-appinsights.ps1 -SubscriptionId "<your-subscription-id>"
# 3. Copy the connection string from output
# Output: "Connection String: InstrumentationKey=xxx;IngestionEndpoint=..."
```
### **GitHub Secret Configuration (Maintainers Only)**
After deploying Azure resources:
1. Go to GitHub repo β **Settings** β **Secrets and variables** β **Actions**
2. Add new secret: `APPINSIGHTS_CONNECTION_STRING`
3. Paste the connection string from deployment output
The release workflow sets this as an environment variable, and MSBuild embeds it at build time.
### **Telemetry Architecture**
```text
Build Time:
MSBuild β reads AppInsightsConnectionString β generates TelemetryConfig.g.cs
Runtime:
MCP Tool Invocation
β
βΌ
ExcelMcpTelemetry.TrackToolInvocation()
β (tracks: tool, action, duration, success)
βΌ
SensitiveDataRedactingProcessor
β (removes: paths, credentials, emails)
βΌ
TelemetryClient β Application Insights
```
### **Files Overview**
| File | Purpose |
|------|---------|
| `Telemetry/ExcelMcpTelemetry.cs` | Static helper for tracking events |
| `Telemetry/ExcelMcpTelemetryInitializer.cs` | Sets User.Id and Session.Id on telemetry |
| `Telemetry/SensitiveDataRedactingProcessor.cs` | Redacts PII before transmission |
| `Program.cs` | Application Insights WorkerService configuration |
| `ExcelMcp.McpServer.csproj` | MSBuild target that generates TelemetryConfig.g.cs |
| `Directory.Build.props.user.template` | Template for local dev connection string |
| `infrastructure/azure/appinsights.bicep` | Azure resource definitions |
| `infrastructure/azure/deploy-appinsights.ps1` | Deployment script |
## π **Need Help?**
- **Read the docs**: [Contributing Guide](CONTRIBUTING.md)
- **Ask questions**: Create a GitHub Issue with the `question` label
- **Report bugs**: Use the bug report template
---
**Remember: Every change, no matter how small, must go through a Pull Request!**
This ensures code quality, proper testing, and maintains the project's reliability for all users.
