OpenShift SRE Copilot
README.md
# MCP OpenShift Enterprise Agent
Enterprise-grade AI-powered OpenShift SRE Copilot platform using the Model Context Protocol (MCP).
## Overview
This platform provides intelligent OpenShift/Kubernetes cluster management through:
- **MCP Server** - Exposes 9 diagnostic tools for LLM integration
- **Multi-Cluster Support** - ARO, ROSA HCP, OSD-GCP, and generic OpenShift/Kubernetes
- **RAG Knowledge Base** - Runbooks, SOPs, and troubleshooting guides
- **AI SRE Analysis** - Intelligent cluster diagnostics with severity classification
- **Read-Only Security** - Safe cluster inspection without modification risk
- **Autonomous Remediation** - AI-powered recommendation engine
---
## Architecture
```text
User / LLM (Claude, GPT-4, etc.)
↓
MCP Server (stdio)
↓
9 Diagnostic Tools
↓
Kubernetes API / OpenShift API
↓
Multi-Cluster (ARO, ROSA HCP, etc.)
```
**Data Flow:**
```text
Cluster → MCP Tools → AI Analysis → RAG Context → Recommendations
```
---
## Quick Start
### Prerequisites
- **Node.js** 18+ ([Download](https://nodejs.org/))
- **OpenShift CLI** (`oc`) ([Installation Guide](https://docs.openshift.com/container-platform/latest/cli_reference/openshift_cli/getting-started-cli.html))
- Access to an OpenShift/Kubernetes cluster
### 1. Install Dependencies
```bash
npm install
```
Or use the bootstrap script:
```bash
bash scripts/bootstrap-enterprise.sh
```
### 2. Configure Cluster Access
Copy the example environment file:
```bash
cp .env.example .env
```
Edit `.env` with your cluster details:
```bash
# ARO Cluster Configuration
ARO_CLUSTER_NAME=my-aro-cluster
ARO_API_URL=https://api.aro-cluster.location.aroapp.io:6443
ARO_USERNAME=kubeadmin
ARO_PASSWORD=your-password
# ROSA HCP Cluster Configuration
HCP_CLUSTER_NAME=my-rosa-cluster
HCP_API_URL=https://api.cluster-name.region.openshiftapps.com:443
HCP_USERNAME=admin
HCP_PASSWORD=your-password
```
**Finding your API URL:**
For ARO:
```bash
az aro show --name <cluster> --resource-group <rg> --query apiserverProfile.url -o tsv
```
For ROSA HCP:
```bash
rosa describe cluster -c <cluster-name> | grep "API URL"
```
### 3. Authenticate to Your Cluster
**For username/password auth:**
```bash
oc login <API_URL> -u <username> -p <password> --insecure-skip-tls-verify=true
```
**For token auth:**
```bash
oc login --token=<token> --server=<API_URL>
```
### 4. Test Connectivity
```bash
npm test
```
Expected output:
- ✅ Cluster connection successful
- ✅ Nodes and namespaces listed
- ✅ RAG system loaded
- ✅ SRE analysis working
### 5. Start MCP Server
```bash
npm start
```
The server runs in stdio mode and waits for MCP requests.
---
## Available MCP Tools
| Tool | Description |
|------|-------------|
| `list_clusters` | List all configured clusters |
| `get_cluster_health` | Overall cluster health assessment with severity |
| `get_nodes` | List nodes with status and resource info |
| `get_pods` | List pods in a namespace |
| `get_failing_pods` | Find pods not in Running/Succeeded state |
| `get_events` | Get recent Kubernetes events |
| `diagnose_crashloop` | Detailed CrashLoopBackOff diagnostics |
| `get_storage_info` | PVC and storage status |
| `get_cluster_operators` | OpenShift cluster operator status |
---
## Integration with Claude Desktop
Add this to your Claude Desktop config at:
`~/Library/Application Support/Claude/claude_desktop_config.json` (Mac)
`%APPDATA%\Claude\claude_desktop_config.json` (Windows)
```json
{
"mcpServers": {
"openshift-sre": {
"command": "node",
"args": [
"/absolute/path/to/openshift-mcp-sre-tools/src/index.js"
]
}
}
}
```
**Restart Claude Desktop**, then ask:
- "What clusters do I have available?"
- "Check the health of my cluster"
- "Are there any failing pods?"
- "Show me recent events in the openshift-monitoring namespace"
---
## Project Structure
```
.
├── src/
│ ├── index.js # MCP Server entry point
│ ├── mcp/tools.js # MCP tool definitions
│ ├── openshift/client.js # OpenShift/K8s client wrapper
│ ├── agents/sre-copilot.js # AI SRE analysis engine
│ ├── rag/retriever.js # RAG knowledge retrieval
│ ├── utils/
│ │ ├── logger.js # Winston logging
│ │ └── cluster-config.js # Cluster configuration loader
│ └── test-client.js # Test suite
├── rag/
│ ├── runbooks/ # Operational runbooks
│ ├── sop/ # Standard operating procedures
│ ├── incidents/ # Past incident reports (examples)
│ └── architecture/ # Architecture docs (examples)
├── config/
│ └── clusters.json # Multi-cluster configuration
├── docs/ # Comprehensive documentation
├── scripts/
│ └── bootstrap-enterprise.sh # Setup automation
├── .env.example # Environment template
└── package.json # Dependencies
```
---
## Enterprise Features
### AI SRE Capabilities
- Cluster diagnostics with severity classification (healthy/medium/high/critical)
- Node health analysis
- Storage analysis
- Event correlation
- Autonomous remediation suggestions
- Incident summarization
### RAG Knowledge Base
- OpenShift runbooks
- Standard Operating Procedures (SOPs)
- Incident reports
- Troubleshooting guides
- Expandable with custom documentation
### Security
- Read-only mode by default
- No cluster modifications without explicit approval
- Audit logging
- Rate limiting
- Credential isolation via .env
---
## Configuration
### Multi-Cluster Setup
Edit `config/clusters.json` to add/modify clusters:
```json
{
"clusters": [
{
"name": "production-aro",
"type": "ARO",
"apiUrl": "${ARO_API_URL}",
"auth": {
"type": "basic",
"username": "${ARO_USERNAME}",
"password": "${ARO_PASSWORD}"
},
"enabled": true,
"readOnly": true
}
],
"defaultCluster": "production-aro"
}
```
### Environment Variables
See `.env.example` for all available configuration options.
---
## Troubleshooting
### "Cannot connect to cluster"
- Verify API URL is correct (`oc cluster-info`)
- Check credentials in `.env`
- Ensure you've run `oc login` for basic auth clusters
- Test manually: `oc get nodes`
### "HTTP request failed" or "Unauthorized"
- Token may have expired - re-login with `oc login`
- Check username/password are correct
- Verify RBAC permissions (need at least cluster-reader)
### "Permission denied"
- User needs read access to cluster resources
- Grant cluster-reader role: `oc adm policy add-cluster-role-to-user cluster-reader <user>`
### "MCP server not showing in Claude Desktop"
- Verify absolute path in config (no `~` or relative paths)
- Restart Claude Desktop completely
- Check Claude Desktop logs for errors
---
## Authentication Methods
### Username/Password (Basic Auth)
1. Configure credentials in `.env`
2. Run `oc login` before starting the MCP server
3. The client loads credentials from your `~/.kube/config`
### Token-Based (Bearer Token)
1. Get token from OpenShift Console
2. Add `HCP_TOKEN=sha256~...` to `.env`
3. Update cluster config to use token auth
**Note:** The Kubernetes client library doesn't support direct username/password auth. For basic auth, you must run `oc login` first to create a valid kubeconfig.
---
## Documentation
- **[SETUP-GUIDE.md](SETUP-GUIDE.md)** - Detailed setup instructions
- **[docs/architecture/](docs/architecture/)** - System architecture
- **[docs/setup/](docs/setup/)** - Getting started guides
- **[docs/troubleshooting/](docs/troubleshooting/)** - Common issues
- **[rag/runbooks/](rag/runbooks/)** - Operational runbooks
---
## Development
### Run Tests
```bash
npm test
```
### Watch Mode
```bash
npm run dev
```
### Bootstrap Fresh Install
```bash
npm run bootstrap
```
---
## Supported Platforms
- ✅ **ROSA HCP** - Red Hat OpenShift Service on AWS (Hosted Control Plane)
- ✅ **ARO** - Azure Red Hat OpenShift
- ✅ **OSD-GCP** - OpenShift Dedicated on Google Cloud
- ✅ **Generic OpenShift** - Self-managed OpenShift
- ✅ **Kubernetes** - Generic Kubernetes clusters
---
## Security Notes
🔒 **READ_ONLY_MODE** is enabled by default - no modifications to cluster state
**Never commit:**
- `.env` file (contains credentials)
- `kubeconfig` files
- API keys or tokens
The `.gitignore` is configured to protect sensitive files.
---
## Future Enhancements
- [ ] Loki integration for log analysis
- [ ] Prometheus/Grafana dashboards
- [ ] Slack/Teams bot integration
- [ ] Fine-tuned SRE LLM model
- [ ] n8n workflow automation
- [ ] Multi-cluster federation support
---
## License
MIT
---
## Support
- Check logs in `logs/combined.log` and `logs/error.log`
- Review cluster configuration in `config/clusters.json`
- See runbooks in `rag/runbooks/` for common issues
---
**Built with:**
- [@modelcontextprotocol/sdk](https://github.com/modelcontextprotocol/sdk) - MCP protocol
- [@kubernetes/client-node](https://github.com/kubernetes-client/javascript) - Kubernetes API
- [Winston](https://github.com/winstonjs/winston) - Logging
- [ChromaDB](https://www.trychroma.com/) - RAG vector storage (optional)
TDQS
A3.6/5.0
Scored across 9 tools
Disambiguation4/5
Tools have distinct purposes, with slight overlap between get_pods and get_failing_pods, but descriptions clearly differentiate them. Others like diagnose_crashloop and get_cluster_health are distinct.
Naming Consistency5/5
All tools follow a consistent verb_noun pattern (diagnose_, get_, list_), with clear and predictable naming.
Tool Count5/5
With 9 tools, the set is well-scoped for an SRE diagnosis and monitoring tool, covering essential operations without bloat.
Completeness4/5
Covers key diagnosis tasks (health, pods, nodes, events, storage, crash loops). Missing log retrieval or resource updates, but fits a focused diagnostic scope.
Maintenance
ActivityInactive
ResponsivenessNo issues