IRIS Legacy

# Session Persistence & Auto-Injection **Status**: ✅ FULLY WORKING (2025-11-25) **Commits**: `4266df1`, `b14d922`, `1066609`, `78ece27`, `24a8200`, `0a772ed` ## Overview The MCP server now implements **session-based credential persistence** and **file reference management** to improve user experience and reduce redundant authentication. ## 🎯 The Complete Solution ### Problem (Before) Agent repeatedly asked for credentials even after user provided them: ``` User: "read my PEC emails" Agent: "What are your PEC credentials?" User: "filippo@legalmail.it password123" Agent: [reads emails successfully] User: "download attachment 6" Agent: "What are your PEC credentials?" ← ASKED AGAIN! ``` ### Root Causes Identified **Issue #1**: Tool definitions had credentials in `required` arrays - Agent sees `pec_email` is REQUIRED → refuses to call without it - When agent loses credentials from LLM context → asks USER instead of calling tool - MCP auto-inject code never executed because agent always provided credentials OR asked user **Issue #2**: Session migration didn't work properly - Started as guest session, migrated to user session - But migration happened AFTER credential check → credentials in guest session, not user session **Issue #3**: PEC attachment file_path not saved to session - Microsoft attachments worked, PEC didn't - Code tried to access `result["content"]` but result was JSON string - Never parsed → never saved → agent forgot file paths ### Solution (After) **Fix #1**: Made ALL credentials optional in tool definitions (commits `78ece27`, `24a8200`) ```python # Before "required": ["pec_email", "pec_password", "message_id"] # After "required": ["message_id"] # pec_email and pec_password marked as OPTIONAL with description: # "(OPTIONAL if already provided in this conversation - will be auto-filled from session)" ``` **Fix #2**: Proper session migration + credential re-injection (commits `4266df1`, `b14d922`) - Migrate guest → user session BEFORE credential check - Re-inject credentials from migrated session if needed - Only migrate once (check if target session exists) **Fix #3**: Fixed PEC attachment file_path persistence (commit `0a772ed`) - Parse JSON string result first: `json.loads(result)` - Navigate nested structure: `result → data → data → file_path` (PEC has 2 levels!) - Save to `session['last_attachment']` with metadata ### Result (Now) ✅ ``` User: "read my PEC emails" Agent: "What are your PEC credentials?" User: "filippo@legalmail.it password123" Agent: [reads emails] ✅ Credentials saved to session User: "download attachment 6" Agent: [downloads without asking] ✅ Auto-injected from session User: "read my emails" (switch to Microsoft) Agent: "What's your email?" User: "yyi9910@infocert.it" Agent: [reads emails] ✅ user_email saved to session User: "search for emails with attachments" Agent: [searches without asking] ✅ Auto-injected from session User: "go back to PEC, read again" Agent: [reads PEC without asking] ✅ PEC credentials still in session! ``` **Agent never asks for credentials again in the same conversation!** 🎉 ## Features ### 1. Credential Auto-Injection **Problem Solved**: Users had to provide PEC credentials and user_email on every tool call. **Solution**: After first use, credentials are saved in session and auto-injected into subsequent tool calls. #### How It Works ```python # First PEC call - user provides credentials pec_list_messages( pec_email="filippo@legalmail.it", pec_password="password123" ) # → Credentials saved to session # Second PEC call - credentials auto-injected pec_get_attachment( message_id="6", attachment_index=0 # pec_email and pec_password auto-injected from session! ) ``` #### Supported Tools **PEC Tools** (auto-inject `pec_email`, `pec_password`): - `pec_list_messages` - `pec_get_message` - `pec_get_attachment` - `pec_send_message` **Microsoft Tools** (auto-inject `user_email`): - `email_*` (all email operations) - `calendar_*` (all calendar operations) - `teams_*` (all Teams operations) - `users_*` (all user operations) ### 2. Session Migration **Problem Solved**: Sessions started as "guest" and needed to migrate to user-specific sessions. **Solution**: One-time migration from guest session to user session when `user_id` (user_email) is detected. #### Flow ``` Request 1: guest session (anonymous) ↓ User provides email/credentials ↓ Request 2: Migrate guest → user session (one-time) Copy all data (credentials, files, etc.) Update active_connections mapping ↓ Request 3+: Use existing user session (no re-migration) Auto-inject credentials from user session ``` #### Implementation **File**: `src/mcp_servers/http_server.py` **Lines**: 1066-1110 ```python # Check if migration needed if "guest@trustysign.local" in current_session_id: user_id = state.get("user_id") if user_id: target_session_id = f"session_user_{user_id}" target_session = SessionManager(target_session_id) # Only migrate if target doesn't exist if not target_session.get("user_id"): # Copy data from guest to user session old_data = session.load() old_data["user_id"] = user_id target_session.save(old_data) # Update mapping active_connections[f"{connection_id}_session"] = target_session_id session = target_session ``` ### 3. Post-Migration Credential Re-injection **Problem Solved**: After migration, guest session was empty so auto-injection failed. **Solution**: Re-inject credentials from migrated user session. **File**: `src/mcp_servers/http_server.py` **Lines**: 1100-1110 ```python # After migration, re-inject from migrated session if still missing if tool_name.startswith("pec_"): if not tool_params.get("pec_email") or not tool_params.get("pec_password"): saved_email = target_session.get("pec_email") saved_password = target_session.get("pec_password") if saved_email and not tool_params.get("pec_email"): tool_params["pec_email"] = saved_email if saved_password and not tool_params.get("pec_password"): tool_params["pec_password"] = saved_password ``` ### 4. File Reference Management **Status**: 🔄 In Progress (commit `1066609`) **Goal**: Remember `file_path` from downloaded attachments so agent can reuse without re-downloading. #### Expected Flow ```python # Download attachment result = pec_get_attachment(message_id="6", attachment_index=0) # Returns: { "file_path": "/app/attachments/...", "filename": "doc.pdf" } # Save to session session.update("last_attachment", { "file_path": result["file_path"], "filename": result["filename"], "download_url": result["download_url"], "timestamp": "2025-11-25T12:00:00" }) # Reuse in email email_send_message( to=["user@example.com"], subject="Document", body="See attached", attachments=[{ "file_path": session.get("last_attachment")["file_path"] }] ) ``` #### Current Issue The file is saved to disk but NOT to session. Debugging in progress with detailed logging (commit `1066609`). **Suspected Cause**: Response structure from `pec_get_attachment` may have nested `data` key that we're not navigating to. ## Logging ### Credential Management ```bash # Credential saves 💾 PEC credentials saved for filippo@legalmail.it # Auto-injection 🔑 Re-injected pec_email after migration 🔑 Re-injected pec_password after migration ``` ### Session Migration ```bash # Migration flow 🔄 Migrating session from guest to user_id: filippo@legalmail.it ✅ Session created: session_user_filippo@legalmail.it 📂 Using user session: session_user_filippo@legalmail.it ``` ### File Management ```bash # File saves to disk (working) 💾 Saved attachment: /app/attachments/20251125_122201_xxx_doc.pdf (138717 bytes) # File reference saves to session (NOT working yet) 💾 Saved attachment reference: doc.pdf ``` ### Debug Logs (commit 1066609) ```bash Checking if we should save attachment reference for pec_get_attachment Got file_data type: <class 'str'> Parsed file_info keys: dict_keys(['success', 'data', ...]) Using nested data, keys: dict_keys(['file_path', 'filename', ...]) 💾 Saved attachment reference: doc.pdf ``` ## Agent Integration **Agent Prompt**: `/workspace/shared/iris_agent_prompt_updated.md` Agents using the MCP server should: 1. **Provide credentials once** in conversation context 2. **Know that MCP will auto-inject** if missing from tool calls 3. **Remember file_path** from attachment responses 4. **Reuse file_path** instead of re-downloading See agent prompt for detailed instructions and examples. ## Testing ### Test Credential Persistence ```python # 1. First PEC call with credentials pec_list_messages( pec_email="test@legalmail.it", pec_password="test123" ) # 2. Second PEC call WITHOUT credentials pec_get_message( message_id="1" # Should auto-inject pec_email and pec_password ) # Expected: Both calls succeed without asking credentials again ``` ### Test Session Migration ```python # 1. Start with guest session # 2. First Microsoft call with user_email email_list_messages(user_email="user@infocert.it") # → Triggers migration to user session # 3. Check logs for migration message # "🔄 Migrating session from guest to user_id: user@infocert.it" # 4. Subsequent calls should use migrated session # "📂 Using user session: session_user_user@infocert.it" ``` ### Test File Reuse ```python # 1. Download attachment result = pec_get_attachment(message_id="6", attachment_index=0) file_path = result["file_path"] # 2. Use in Teams teams_send_message( chat_id="...", message="Document", attachments=[{"file_path": file_path}] ) # 3. Use in Email email_send_message( to=["user@example.com"], subject="Document", attachments=[{"file_path": file_path}] ) # Expected: Same file reused, not re-downloaded ``` ## Troubleshooting ### Credentials Not Persisting **Check**: 1. Session migration completed? Look for `🔄 Migrating session` log 2. Credentials saved? Look for `💾 PEC credentials saved` log 3. Re-injection working? Look for `🔑 Re-injected` log **Solution**: Ensure user session exists and credentials were saved to it. ### File Path Not Found **Status**: ✅ FIXED (commit `0a772ed`) **Problem**: PEC attachments file_path not saved to session (Microsoft worked, PEC didn't). **Solution**: Fixed JSON parsing in attachment saving code. ### Session Lost on Container Restart **Cause**: Sessions stored in memory, not persisted to Redis/DB. **Solution**: This is expected behavior. Sessions last for container lifetime (~24h). ## Architecture ### Session Types 1. **Guest Session**: `session_guest_{connection_id}` - Initial anonymous session - No user_id set - Short-lived 2. **User Session**: `session_user_{user_email}` - User-specific session - Contains credentials, file refs, etc. - Persistent across requests ### Session Storage **Class**: `SessionManager` (`src/mcp_servers/session_manager.py`) **Methods**: - `get(key)` - Retrieve value from session - `update(key, value)` - Store value in session - `load()` - Load entire session data - `save(data)` - Save entire session data **Storage**: In-memory dict (could be extended to Redis) ### Active Connections Mapping ```python active_connections = { "{connection_id}_session": "session_user_filippo@legalmail.it", "{connection_id}_state": {...} } ``` Maps SSE connection IDs to session IDs and conversation state. ## Future Enhancements 1. **Redis Session Storage**: Persist sessions across container restarts 2. **Session Expiry**: Auto-delete sessions after 24h inactivity 3. **Multi-Account Support**: Allow switching between multiple PEC accounts 4. **File Path Auto-Inject**: Auto-inject last downloaded file in attachment params 5. **Session Export/Import**: Save/restore session state ## Related Files - `src/mcp_servers/http_server.py` - Main session logic - `src/mcp_servers/session_manager.py` - Session storage - `src/mcp_servers/attachment_helper.py` - File management - `/workspace/shared/iris_agent_prompt_updated.md` - Agent instructions ## Change Log ### Phase 3: Tool Definition Fix (2025-11-25) - **Commit `0a772ed`**: Fixed PEC attachment file_path persistence (JSON parsing) - **Commit `24a8200`**: Made user_email optional in ALL Microsoft tools - **Commit `78ece27`**: Made pec_email/pec_password optional in PEC tools **ROOT CAUSE IDENTIFIED**: Tool definitions had credentials in `required` arrays! - Agent sees required → won't call without them → asks user instead - When agent loses credentials from context → can't call tool → repeatedly asks user - Solution: Remove credentials from required, add "(OPTIONAL if already provided)" to descriptions ### Phase 2: Session Management (2025-11-25) - **Commit `1066609`**: Added debug logging for file reference saving - **Commit `b14d922`**: Implemented credential re-injection after migration - **Commit `4266df1`**: Fixed persistent session migration logic ### Phase 1: Initial Implementation (2025-11-24) - Initial session persistence implementation