# DSR Processor Agentic Agent

Complete implementation of DSR (Daily Status Report) processing using IBM watsonx Orchestrate ADK Agent with Agentic Workflow tools, plus MCP (Model Context Protocol) server for Claude Desktop integration.

## Overview

This agent provides conversational, human-in-the-loop DSR processing capabilities for the C4I SOT system. It handles bulk file processing from Cloud Object Storage (COS), data extraction from multiple formats, schema validation, and automated storage of processed results.

**Two deployment options:**
1. **WxO ADK Agent** - Deploy to IBM watsonx Orchestrate for enterprise workflows
2. **MCP Server** - Use with Claude Desktop or other MCP clients for local AI assistance

**Uses WxO Knowledge Base for schema management** - The DSR schema is stored in WxO Knowledge Base as a `.json.txt` file, making it easy to update without redeploying tools.

### Key Features

- **Conversational Interface**: Natural language interaction for DSR processing tasks
- **Multi-Format Support**: Processes JSON, DOCX, and XLSX DSR files
- **Batch Processing**: Handle multiple files with single commands
- **Schema Validation**: Validates against C4I SOT DSR unified schema v1.2 from Knowledge Base
- **Knowledge Base Integration**: Schema stored in WxO Knowledge Base for easy updates
- **Human Review**: Optional review steps for quality control
- **Error Handling**: Graceful error handling with helpful suggestions
- **COS Integration**: Direct integration with IBM Cloud Object Storage

## Architecture

### Agentic Workflow Tools

The agent uses 5 custom tools that work together:

1. **list_cos_files** - List and filter files in COS bucket
2. **download_cos_file** - Download files from COS to processing area
3. **extract_dsr_data** - Extract structured data from DSR files
4. **validate_schema** - Validate data against DSR schema
5. **save_to_cos** - Save processed data back to COS

### Agent Flow

```
User Request
    ↓
Agent (LLM) interprets intent
    ↓
Agent selects appropriate tool(s)
    ↓
Tool executes and returns result
    ↓
Agent processes result and responds
    ↓
[Repeat for multi-step workflows]
```

## Project Structure

```
dsr-agent-agentic/
├── README.md                           # This file
├── requirements.txt                    # Python dependencies (ADK)
├── requirements-mcp.txt                # Python dependencies (MCP)
├── dsr-processor-agentic.yaml         # Agent specification (ADK)
├── mcp_server.py                       # MCP server implementation
├── tools/                              # Agentic Workflow tools
│   ├── list_cos_files.py              # COS file listing
│   ├── download_cos_file.py           # COS file download
│   ├── extract_dsr_data.py            # Data extraction
│   ├── validate_schema.py             # Schema validation
│   └── save_to_cos.py                 # COS file upload
├── docs/                               # Documentation
│   ├── DEPLOYMENT-GUIDE.md            # ADK deployment guide
│   ├── MCP-SERVER-GUIDE.md            # MCP server guide
│   └── TESTING-GUIDE.md               # Comprehensive testing
└── examples/                           # Usage examples
    └── EXAMPLE-CONVERSATIONS.md       # Conversation examples
```

## Quick Start

### Option 1: MCP Server (Claude Desktop)

**Prerequisites:**
- Python 3.8+
- Claude Desktop or other MCP client
- IBM Cloud Object Storage credentials

**Setup:**
```bash
# Install dependencies
cd dsr-agent-agentic
pip install -r requirements-mcp.txt

# Configure environment variables
cp .env.example .env
# Edit .env with your COS credentials

# Add to Claude Desktop config
# See docs/MCP-SERVER-GUIDE.md for details
```

**Usage:**
Open Claude Desktop and use natural language:
- "List all DSR files in Cloud Object Storage"
- "Download and process the USS VALOR DSR file"
- "Validate the latest DSR and save it"

See [MCP-SERVER-GUIDE.md](docs/MCP-SERVER-GUIDE.md) for complete setup instructions.

### Option 2: WxO ADK Agent

**Prerequisites:**
- IBM Cloud account with watsonx Orchestrate TZ Essentials (trial)
- Cloud Object Storage instance with bucket created
- COS credentials (API key, instance CRN, endpoint, bucket name)

**Note:** This implementation uses the `gpt-oss-120b-groq` model (GPT-OSS 120B - OpenAI via Groq) which is available by default in WxO TZ Essentials. No additional watsonx.ai project setup is required.

**Deployment:**

1. **Deploy Tools to WxO:**
   - Follow [DEPLOYMENT-GUIDE.md](docs/DEPLOYMENT-GUIDE.md) for detailed steps
   - Deploy each of the 5 tools to WxO UI
   - Configure environment variables for COS access

2. **Deploy Agent:**
   - Create agent in WxO AI assistant builder
   - Use configuration from `dsr-processor-agentic.yaml`
   - Connect all 5 tools to the agent
   - Test and publish

3. **Test:**
   - Follow [TESTING-GUIDE.md](docs/TESTING-GUIDE.md)
   - Try example conversations from [EXAMPLE-CONVERSATIONS.md](examples/EXAMPLE-CONVERSATIONS.md)

### Basic Usage

```
User: "List all DSR files in COS"
Agent: [Lists files with details]

User: "Process the USS VALOR file from August 14"
Agent: [Downloads → Extracts → Validates → Saves]

User: "Process all JSON files from last week"
Agent: [Batch processes multiple files]
```

## Features in Detail

### Multi-Format Processing

- **JSON**: Direct parsing of structured DSR data
- **DOCX**: Heuristic extraction from Word documents
- **XLSX**: Cyber findings extraction from Excel spreadsheets

### Schema Validation

Validates against C4I SOT DSR unified schema v1.2 from Knowledge Base:
- **Knowledge Base Integration**: Schema retrieved automatically from WxO Knowledge Base
- **Easy Updates**: Update schema without redeploying tools
- Required fields verification
- Data type checking
- Pattern matching (dates, GUIDs, hull numbers)
- Enum validation (status, enclave, issue types)
- Detailed error messages with fix suggestions
- Fallback to minimal schema if Knowledge Base unavailable

### Batch Processing

Process multiple files with single commands:
- Filter by date range
- Filter by ship name
- Filter by file format
- Automatic error recovery
- Progress reporting

### Human Review

Optional review workflows:
- Review before saving
- Review only on warnings
- Approve/reject/modify options
- Summary views for quick decisions

## Environment Variables

Required for all tools:

```bash
COS_API_KEY_ID=<your-cos-api-key>
COS_INSTANCE_CRN=<your-cos-instance-crn>
COS_ENDPOINT=https://s3.us-south.cloud-object-storage.appdomain.cloud
COS_BUCKET_NAME=dsr-files-in-cloud-object-storage-cos-standard-7q2
DOWNLOAD_DIR=/tmp/dsr-downloads
DSR_SCHEMA_KB_NAME=C4I_SOT_DSR_unified.schema.v1_2.iso.json.txt
```

**Note:** The schema is retrieved from WxO Knowledge Base using `DSR_SCHEMA_KB_NAME`. The old `DSR_SCHEMA_PATH` variable is deprecated.

## Dependencies

See [requirements.txt](requirements.txt) for complete list:

- `ibm-cos-sdk` - IBM Cloud Object Storage SDK
- `jsonschema` - JSON schema validation
- `python-docx` - DOCX file processing
- `openpyxl` - XLSX file processing

## Documentation

- **[MCP-SERVER-GUIDE.md](docs/MCP-SERVER-GUIDE.md)** - MCP server setup and usage
- **[DEPLOYMENT-GUIDE.md](docs/DEPLOYMENT-GUIDE.md)** - WxO ADK agent deployment
- **[TESTING-GUIDE.md](docs/TESTING-GUIDE.md)** - Comprehensive testing guide
- **[EXAMPLE-CONVERSATIONS.md](examples/EXAMPLE-CONVERSATIONS.md)** - Usage examples

## Comparison with Skill Flows

This Agentic Workflow implementation differs from Skill Flows:

| Feature | Agentic Workflow | Skill Flows |
|---------|------------------|-------------|
| Interaction | Conversational | Structured forms |
| Flexibility | High - natural language | Low - predefined paths |
| Human Review | Built-in, conversational | Requires explicit steps |
| Error Handling | Contextual suggestions | Fixed error messages |
| Batch Processing | Natural language commands | Requires loops/iteration |
| Learning Curve | Lower (natural language) | Higher (flow design) |

## Use Cases

### Daily Operations
- Process new DSR files as they arrive
- Validate and archive processed data
- Generate daily summaries

### Batch Processing
- Process historical data
- Reprocess files after schema updates
- Bulk validation of existing files

### Quality Control
- Review files before saving
- Identify validation issues
- Suggest corrections

### Reporting
- List processed files
- Show processing statistics
- Identify problematic files

## Troubleshooting

### Common Issues

**COS Connection Errors:**
- Verify environment variables
- Check API key permissions
- Confirm endpoint URL

**Validation Failures:**
- Review error messages
- Check schema requirements
- Verify data formats

**Knowledge Base Access:**
- Verify schema file uploaded to Knowledge Base
- Check file name: `C4I_SOT_DSR_unified.schema.v1_2.iso.json.txt`
- Ensure Knowledge Base connected to agent
- Verify `DSR_SCHEMA_KB_NAME` environment variable
- Test with `action: schema_info` parameter

**Tool Not Found:**
- Ensure tools are published
- Check agent configuration
- Refresh agent

See [DEPLOYMENT-GUIDE.md](docs/DEPLOYMENT-GUIDE.md#troubleshooting) for detailed troubleshooting.

## Support

For issues or questions:
1. Check documentation in `docs/` folder
2. Review example conversations
3. Consult IBM watsonx Orchestrate documentation
4. Contact C4I SOT team

## Version History

- **v1.1.0** (2026-03-10) - MCP Server Implementation
  - Added MCP (Model Context Protocol) server for Claude Desktop integration
  - New `mcp_server.py` for standalone MCP deployment
  - New `requirements-mcp.txt` for MCP dependencies
  - Comprehensive MCP-SERVER-GUIDE.md documentation
  - Updated README with dual deployment options (MCP + WxO)
  - Updated DEPLOYMENT-GUIDE.md to reference MCP option

- **v1.0.1** (2026-03-10) - Knowledge Base Integration
  - Updated to use WxO Knowledge Base for schema storage
  - Schema file in `.json.txt` format for Knowledge Base compatibility
  - Updated validate_schema tool to use `get_knowledge()` function
  - Added Knowledge Base upload instructions
  - Enhanced troubleshooting for Knowledge Base access
  - Deprecated `DSR_SCHEMA_PATH` in favor of `DSR_SCHEMA_KB_NAME`

- **v1.0.0** (2026-03-10) - Initial release
  - 5 Agentic Workflow tools
  - Complete agent specification
  - Comprehensive documentation
  - Example conversations

## License

MIT License - See LICENSE file for details

## Authors

C4I SOT Team

## Acknowledgments

- IBM watsonx Orchestrate team
- ADK Agent framework
- C4I SOT DSR schema contributors

---

**For detailed deployment instructions, see [DEPLOYMENT-GUIDE.md](docs/DEPLOYMENT-GUIDE.md)**