Igloo MCP

# evolve_report - LLM-Agnostic Report Evolution Framework The `evolve_report` tool provides a framework for LLMs to evolve living reports. Unlike traditional tools that call external LLMs, this tool is **LLM-agnostic** - it expects the calling LLM to analyze the report structure and generate structured changes. ## Overview Living reports are JSON-backed, auditable business reports that evolve safely over time. The `evolve_report` tool enables LLMs to propose and apply structured changes to these reports while maintaining audit trails and data integrity. ## Architecture ### Two-Phase Workflow 1. **Discovery Phase**: LLM calls `evolve_report` with `dry_run=True` to understand current report structure 2. **Evolution Phase**: LLM generates structured changes and calls `evolve_report` with `dry_run=False` to apply them ### Automatic Index Synchronization The `evolve_report` tool automatically refreshes the report index before and after operations: - **Before selector resolution**: Ensures CLI-created reports are immediately visible to MCP tools - **After successful changes**: Maintains index consistency after report evolution This eliminates the need for manual `refresh_reports` calls and ensures seamless synchronization between CLI and MCP workflows. ### LLM Responsibility The calling LLM is responsible for: - Analyzing the current report outline structure - Understanding sections, insights, and their relationships - Generating valid `ProposedChanges` objects - Ensuring semantic consistency of changes ## Parameters ### Required Parameters | Parameter | Type | Description | |-----------|------|-------------| | `report_selector` | string | Report ID (e.g., `rpt_550e8400e29b11d4a716446655440000`) or title | | `instruction` | string | Natural language instruction for audit trail (e.g., "Add revenue insights") | | `proposed_changes` | object | Structured changes generated by LLM (see ProposedChanges schema) | ### Optional Parameters | Parameter | Type | Description | |-----------|------|-------------| | `constraints` | object | Optional constraints on evolution (max_importance_delta, sections) | | `dry_run` | boolean | Validate changes without applying (default: false) | | `response_detail` | string | Response verbosity: `minimal`, `standard` (default), or `full` | ## ProposedChanges Schema The `proposed_changes` object must conform to this structure: ```json { "insights_to_add": [ { "insight_id": "uuid-string (optional - auto-generated if omitted)", "summary": "Brief insight summary", "importance": 1-10, "supporting_queries": [ { "execution_id": "optional-execution-id", "sql_sha256": "optional-sql-hash", "cache_manifest": "optional-cache-path" } ] } ], "sections_to_add": [ { "section_id": "uuid-string (optional - auto-generated if omitted)", "title": "Section Title", "order": 1, "notes": "Optional section notes", "insights": [ { "summary": "Inline insight summary", "importance": 7 } ] } ], "insights_to_modify": [ { "insight_id": "existing-uuid (required)", "summary": "Updated summary (optional - partial update)", "importance": 8 } ], "sections_to_modify": [ { "section_id": "existing-uuid (required)", "title": "Updated Title", "insight_ids_to_add": ["insight-uuid"], "insight_ids_to_remove": ["old-insight-uuid"] } ], "insights_to_remove": ["uuid1", "uuid2"], "sections_to_remove": ["uuid1", "uuid2"] } ``` ## New Features ### Auto-generated UUIDs For additions (`insights_to_add`, `sections_to_add`), UUIDs are now optional and will be auto-generated if not provided. This simplifies the API and reduces boilerplate. **Example: Adding insight without UUID** ```json { "insights_to_add": [ { "summary": "Revenue increased 20% YoY", "importance": 9 } ] } ``` The tool will automatically generate a UUID for the insight. The generated UUID is returned in the response summary under `insight_ids_added`. **Note**: UUIDs are still **required** for modifications (`insights_to_modify`, `sections_to_modify`) to ensure you're modifying the correct item. ### Partial Updates When modifying insights or sections, you can now update only specific fields. Fields not provided (or set to `None`) will remain unchanged. **Example: Updating only insight summary** ```json { "insights_to_modify": [ { "insight_id": "existing-insight-uuid", "summary": "Updated summary text" // importance, status, etc. remain unchanged } ] } ``` **Example: Updating only importance** ```json { "insights_to_modify": [ { "insight_id": "existing-insight-uuid", "importance": 10 // summary, status, etc. remain unchanged } ] } ``` **Important**: At least one field besides the ID must be provided for modifications. ### Atomic Add-and-Link You can now create insights inline within section additions, automatically linking them to the section. This enables atomic operations where insights and sections are created together. **Example: Adding section with inline insights** ```json { "sections_to_add": [ { "title": "Customer Retention Analysis", "order": 3, "insights": [ { "summary": "30-day retention improved 15%", "importance": 8 }, { "summary": "Churn rate decreased 20%", "importance": 7 } ] } ] } ``` The insights are created atomically with the section and automatically linked. UUIDs for inline insights are auto-generated if not provided. **Note**: The `insights` field is mutually exclusive with `insight_ids_to_add`. Use `insights` for inline creation or `insight_ids_to_add` for referencing existing insights. ### Section Templates Use pre-built markdown templates to generate structured section content. Templates produce clean, consistently formatted markdown. > **💡 Tip**: Use `get_report_schema(schema_type="section_templates")` to discover all available templates, their required fields, and example usage. #### Available Templates | Template | Aliases | Required Fields | Description | |----------|---------|-----------------|-------------| | `findings` | `findings_list` | `findings` | Key findings with metrics and action items | | `metrics` | `metrics_snapshot` | `metrics` | Metrics table with optional callouts | | `bullet_list` | `bullet`, `summary_bullets` | `items` | Simple bullet point summary | | `executive_summary` | `exec_summary` | _(none)_ | Executive summary with takeaways and recommendations | | `action_items` | `next_steps`, `actions` | `actions` | Action items with optional owners and due dates | | `methodology` | _(none)_ | _(none)_ | Methodology section with data sources and approach | #### Discovering Templates Programmatically ```python # Get full template documentation result = get_report_schema(schema_type="section_templates") # Returns: { "status": "success", "schema_type": "section_templates", "description": "Section content templates generate formatted markdown...", "available_names": ["action_items", "actions", "bullet", "bullet_list", ...], "templates": { "findings": { "description": "Key findings with optional metrics and action items", "aliases": ["findings_list"], "required_fields": ["findings"], "optional_fields": ["heading"], "example_template_data": {...} }, ... } } ``` #### Example: Findings List Template ```json { "sections_to_add": [ { "title": "North Star Metrics", "order": 1, "template": "findings_list", "template_data": { "heading": "North Star Metrics - Q4 2024", "findings": [ { "title": "Activation Rate Improvement", "metric": {"name": "Activation", "value": "62%", "trend": "+4 pp"}, "description": "Week-over-week activation improved on the new onboarding path.", "actions": ["Continue A/B testing", "Expand to mobile"] }, { "title": "Revenue Growth", "metric": {"name": "MRR", "value": "$1.2M", "trend": "+15%"}, "description": "Monthly recurring revenue exceeded targets." } ] } } ] } ``` **Generated Markdown**: ```markdown ## North Star Metrics - Q4 2024 ### 1. Activation Rate Improvement | Metric | Value | Trend | | --- | --- | --- | | Activation | 62% | +4 pp | Week-over-week activation improved on the new onboarding path. **Next Steps** - Continue A/B testing - Expand to mobile ### 2. Revenue Growth | Metric | Value | Trend | | --- | --- | --- | | MRR | $1.2M | +15% | Monthly recurring revenue exceeded targets. ``` #### Example: Executive Summary Template ```json { "sections_to_add": [ { "title": "Executive Summary", "order": 0, "template": "executive_summary", "template_data": { "headline": "Q4 Performance Summary", "context": "This quarter saw significant growth across all business units.", "key_points": [ {"title": "Revenue Growth", "detail": "Up 25% YoY"}, {"title": "Customer Retention", "detail": "Improved to 95%"}, "Expanded into 3 new markets" ], "recommendation": "Continue investment in customer success initiatives.", "conclusion": "Overall, Q4 exceeded expectations." } } ] } ``` #### Example: Action Items Template ```json { "sections_to_add": [ { "title": "Next Steps", "order": 5, "template": "action_items", "template_data": { "heading": "Follow-up Actions", "actions": [ { "description": "Review Q4 metrics", "owner": "Alice", "due": "2024-01-20", "priority": "High" }, { "description": "Schedule team retrospective", "owner": "Bob", "due": "2024-01-25", "priority": "Medium" } ] } } ] } ``` **Note**: If action items include `owner`, `due`, or `priority` fields, they render as a table. Otherwise, they render as a numbered list. #### Example: Methodology Template ```json { "sections_to_add": [ { "title": "Methodology", "order": 1, "template": "methodology", "template_data": { "heading": "Analysis Methodology", "data_sources": [ "Snowflake analytics warehouse (DEX_TRADES_V2)", "On-chain transaction data via Allium", "CoinGecko price feeds" ], "time_period": "Q4 2024 (October 1 - December 31)", "approach": "We aggregated daily trading volumes and computed 7-day moving averages to smooth volatility. Price impacts were calculated using VWAP methodology across all trades above $1,000." } } ] } ``` **Generated Markdown**: ```markdown ## Analysis Methodology **Data Sources:** - Snowflake analytics warehouse (DEX_TRADES_V2) - On-chain transaction data via Allium - CoinGecko price feeds **Time Period:** Q4 2024 (October 1 - December 31) **Analysis Approach:** We aggregated daily trading volumes and computed 7-day moving averages to smooth volatility. Price impacts were calculated using VWAP methodology across all trades above $1,000. ``` ### Response Detail Control The `response_detail` parameter controls response verbosity for significant token reduction. #### Options | Level | Token Count | Description | Use When | |-------|-------------|-------------|----------| | `minimal` | ~200 tokens | Status, ID, version, counts only | Batch operations, automated workflows | | `standard` | ~400 tokens (default) | + Created IDs and warnings | Interactive workflows, balanced detail | | `full` | ~1000+ tokens | + Complete applied changes echo | Debugging, detailed audit requirements | #### Minimal Response Example ```json { "report_selector": "Q1 Analysis", "instruction": "Add revenue insight", "proposed_changes": {...}, "response_detail": "minimal" } ``` **Response**: ```json { "status": "success", "report_id": "rpt_550e8400...", "outline_version": 6, "summary": { "insights_added": 1, "sections_modified": 1 } } ``` **Token Savings**: Substantial reduction vs. standard response #### Standard Response Example (Default) ```json { "report_selector": "Q1 Analysis", "instruction": "Add revenue insight", "proposed_changes": {...} // response_detail omitted = "standard" } ``` **Response**: ```json { "status": "success", "report_id": "rpt_550e8400...", "outline_version": 6, "summary": { "insights_added": 1, "insight_ids_added": ["ins_abc123..."], "sections_modified": 1, "section_ids_modified": ["sec_def456..."], "sections_removed": 0, "section_ids_removed": [], "insights_removed": 0, "insight_ids_removed": [] }, "warnings": [ "Section 'Revenue Analysis' has only 1 insight (recommended: 3+)" ] } ``` **Token Cost**: Balanced (default behavior) #### Full Response Example ```json { "report_selector": "Q1 Analysis", "instruction": "Add revenue insight", "proposed_changes": {...}, "response_detail": "full" } ``` **Response**: ```json { "status": "success", "report_id": "rpt_550e8400...", "outline_version": 6, "summary": { "insights_added": 1, "insight_ids_added": ["ins_abc123..."], "sections_modified": 1, "section_ids_modified": ["sec_def456..."], "sections_removed": 0, "section_ids_removed": [], "insights_removed": 0, "insight_ids_removed": [] }, "warnings": [...], "changes_applied": { "insights_to_add": [{ "insight_id": "ins_abc123...", "summary": "Revenue grew 40% in Q1", "importance": 9 }], "sections_to_modify": [{ "section_id": "sec_def456...", "insight_ids_to_add": ["ins_abc123..."] }] }, "timing": { "total_duration_ms": 145.2 } } ``` **Token Cost**: Highest detail for debugging #### Token Efficiency Comparison | Operation Type | Standard Response | Minimal Response | Savings | |---------------|------------------|------------------|---------| | Add 1 insight | 400 tokens | 150 tokens | Significant reduction | | Modify 3 sections | 550 tokens | 180 tokens | Substantial reduction | | Complex operation (5+ changes) | 800 tokens | 200 tokens | Maximum reduction | **Recommendation**: Use `minimal` for multi-turn workflows, `standard` for interactive work, `full` only for debugging. --- ### Enhanced Error Messages Validation errors now include structured information with field paths, values, and available IDs to help debug issues more easily. **Error Response Format** ```json { "status": "validation_failed", "error_type": "semantic_validation", "validation_errors": [ "insights_to_modify[0].insight_id: insight_id not found (value: 550e8400-e29b-41d4-a716-446655440999) (available: 550e8400-e29b-41d4-a716-446655440001, 550e8400-e29b-41d4-a716-446655440002, ...)" ], "context": { "structured_errors": [ { "field": "insights_to_modify[0].insight_id", "value": "550e8400-e29b-41d4-a716-446655440999", "error": "insight_id not found", "available_ids": ["550e8400-e29b-41d4-a716-446655440001", "550e8400-e29b-41d4-a716-446655440002"] } ] } } ``` The structured format includes: - **field**: Full path to the field (e.g., `insights_to_modify[0].insight_id`) - **value**: The actual value that failed validation - **error**: Human-readable error message - **available_ids**: List of valid IDs (for "not found" errors) String format is maintained for backward compatibility. ## Validation Rules The `proposed_changes` object must conform to strict validation rules: ### UUID Requirements - All `insight_id` and `section_id` values must be valid UUIDs - Use `uuid.uuid4()` to generate new IDs - Example: `"550e8400-e29b-41d4-a716-446655440000"` ### Reference Integrity - `insight_ids_to_add` must reference existing insight UUIDs - `section_ids_to_modify` must reference existing section UUIDs - Circular references are not allowed ### Content Constraints - `importance` must be integer 1-10 - `summary` cannot be empty (minimum 10 characters recommended) - `title` cannot be empty for new sections ### Semantic Validation - Modified insights must exist in the report - Removed items must exist in the report - Section orders must be unique positive integers ## Usage Examples ### Example 1: Adding Revenue Insights **Step 1: Discovery** ```python result = evolve_report( report_selector="Q1 Revenue Report", instruction="Add insights about top revenue drivers", proposed_changes={}, # Empty for discovery dry_run=True ) # Returns current outline structure for analysis ``` **Step 2: Evolution** ```python result = evolve_report( report_selector="Q1 Revenue Report", instruction="Add insights about top revenue drivers", proposed_changes={ "insights_to_add": [{ "insight_id": "550e8400-e29b-41d4-a716-446655440000", "summary": "Enterprise segment drove 45% YoY growth", "importance": 9, "supporting_queries": [{ "execution_id": "exec_123" }] }], "sections_to_modify": [{ "section_id": "revenue_overview", "insight_ids_to_add": ["550e8400-e29b-41d4-a716-446655440000"] }] }, dry_run=False ) ``` ### Example 2: Removing Outdated Insights ```python result = evolve_report( report_selector="Monthly Metrics", instruction="Remove Q3 insights that are no longer relevant", proposed_changes={ "insights_to_remove": ["q3-insight-uuid"], "sections_to_modify": [{ "section_id": "quarterly_breakdown", "insight_ids_to_remove": ["q3-insight-uuid"] }] } ) ``` ## Error Handling ### Validation Errors The tool validates changes at multiple levels: - **Schema Validation**: Ensures `proposed_changes` conforms to expected structure - **Semantic Validation**: Checks for logical consistency (e.g., referenced insights exist) - **Business Logic**: Prevents unsafe operations ### Error Response Format ```json { "status": "validation_failed", "error_type": "semantic_validation", "validation_errors": [ "Insight ID 'nonexistent-uuid' not found in report" ], "proposed_changes": {...} } ``` ### Common Errors | Error Type | Cause | Resolution | |------------|-------|------------| | `selector_error` | Report not found | Check report ID/title spelling | | `schema_validation` | Invalid JSON structure | Verify ProposedChanges schema | | `semantic_validation` | Logical inconsistencies | Ensure referenced IDs exist | | `error` | Unexpected error | Check tool logs | ## Practical Examples #### Example 3: Building a Complete Report Section (Atomic Add-and-Link) ```python # Modern approach: Create section with inline insights atomically result = evolve_report( report_selector="Q4 Analytics Report", instruction="Add customer behavior analysis section with insights", proposed_changes={ "sections_to_add": [{ "title": "Customer Behavior Analysis", "order": 3, "notes": "Analysis of user engagement patterns and retention metrics", "insights": [ { "summary": "Average session duration increased 15% to 8.5 minutes", "importance": 7, "supporting_queries": [{ "execution_id": "sess_analysis_exec_2025q4", "sql_sha256": "abc123..." }] }, { "summary": "User retention improved 12% month-over-month", "importance": 8 } ] }] } ) # UUIDs are auto-generated for both section and insights # Insights are automatically linked to the section ``` #### Example 3b: Traditional Two-Step Approach (Still Supported) ```python import uuid # Step 1: Add a new section section_id = str(uuid.uuid4()) result = evolve_report( report_selector="Q4 Analytics Report", instruction="Add customer behavior analysis section", proposed_changes={ "sections_to_add": [{ "section_id": section_id, "title": "Customer Behavior Analysis", "order": 3, "notes": "Analysis of user engagement patterns and retention metrics" }] }, dry_run=True # Validate first ) # Step 2: Add insights to the new section insight_id = str(uuid.uuid4()) result = evolve_report( report_selector="Q4 Analytics Report", instruction="Add session duration analysis insight", proposed_changes={ "insights_to_add": [{ "insight_id": insight_id, "summary": "Average session duration increased 15% to 8.5 minutes", "importance": 7, "supporting_queries": [{ "execution_id": "sess_analysis_exec_2025q4", "sql_sha256": "abc123..." }] }], "sections_to_modify": [{ "section_id": section_id, "insight_ids_to_add": [insight_id] }] } ) ``` #### Example 3c: Adding Insights Without UUIDs ```python # UUIDs are auto-generated automatically result = evolve_report( report_selector="Q4 Analytics Report", instruction="Add revenue insights", proposed_changes={ "insights_to_add": [ { "summary": "Q4 revenue exceeded target by 15%", "importance": 9 }, { "summary": "Enterprise segment grew 45% YoY", "importance": 8 } ] } ) # Generated UUIDs are returned in response.summary.insight_ids_added print(result["summary"]["insight_ids_added"]) ``` #### Example 3d: Partial Update ```python # Update only the summary, leaving importance and other fields unchanged result = evolve_report( report_selector="Q4 Analytics Report", instruction="Update insight summary", proposed_changes={ "insights_to_modify": [ { "insight_id": "existing-insight-uuid", "summary": "Updated summary text" # importance, status, etc. remain unchanged } ] } ) ``` #### Example 4: Error Handling and Validation ```python # This will fail validation (non-existent section ID) result = evolve_report( report_selector="Q4 Analytics Report", instruction="Add insight to non-existent section", proposed_changes={ "sections_to_modify": [{ "section_id": "nonexistent-section-id", "insight_ids_to_add": ["some-insight-id"] }] } ) # Response: { "status": "validation_failed", "error_type": "semantic_validation", "validation_errors": [ "Section ID 'nonexistent-section-id' not found in report" ] } ``` ## Best Practices ### Incremental Changes Make small, focused changes rather than large restructures: ✅ **Good**: Add one insight at a time ❌ **Bad**: Add 10 insights and modify 5 sections simultaneously ### Dry Run Validation Always use `dry_run=True` first to validate changes: ```python # Validate first validation = evolve_report(..., dry_run=True) if validation["status"] == "dry_run_success": # Then apply evolve_report(..., dry_run=False) ``` ### Audit Trail Clarity Use descriptive `instruction` parameters: ✅ **Good**: `"Add customer retention analysis for Q4"` ❌ **Bad**: `"update report"` ### Insight Importance Use importance scores (1-10) strategically: - 1-3: Minor observations - 4-6: Standard insights - 7-8: Key findings - 9-10: Critical business impacts ### UUID Generation Always generate proper UUIDs for new content: ```python import uuid # Correct - generates valid UUID new_insight_id = str(uuid.uuid4()) # Result: "550e8400-e29b-41d4-a716-446655440000" # Incorrect - not a valid UUID fake_id = "my-custom-id" ``` ### Query References Link insights to actual executed queries for traceability: ```python # Reference real query executions insight = { "insight_id": str(uuid.uuid4()), "summary": "Revenue grew 15% MoM", "supporting_queries": [{ "execution_id": "exec_2025_q1_rev_analysis_001", "sql_sha256": "abc123def456..." # From execute_query response }] } ``` ### Content Quality Guidelines - **Summaries**: 10-50 words, actionable insights - **Importance**: Use 1-10 scale consistently across reports - **Section ordering**: Use order field to control report flow - **Notes**: Optional but helpful for complex sections ## Administrative CLI Operations For administrative and power-user operations, the CLI provides direct access to report management. The primary interface for report evolution is through MCP tools: ```bash # Administrative report creation igloo report create "Q1 Analysis" # Administrative audit trail viewing igloo report history "rpt_123" ``` MCP tools handle the primary workflow - CLI commands are for administrative setup and management. ## Constraints Use `constraints` to limit evolution scope: ```python constraints = { "max_importance_delta": 2, # Limit importance changes "sections": ["revenue", "customers"] # Only modify these sections } ``` ## Return Values ### Success Response ```json { "status": "success", "report_id": "rpt_550e8400e29b11d4a716446655440000", "changes_applied": {...}, "outline_version": 3 } ``` ### Dry Run Success ```json { "status": "dry_run_success", "report_id": "rpt_550e8400e29b11d4a716446655440000", "proposed_changes": {...}, "validation_passed": true } ``` ## Troubleshooting ### Report Not Found - Verify report ID format: `rpt_` prefix + UUID - Use title resolution: `"Q1 Sales Report"` - Check report exists: `igloo report list` ### Schema Errors - Validate JSON against ProposedChanges schema - Ensure UUIDs are valid format - Check required fields are present ### Semantic Errors - Verify insight/section IDs exist in current outline - Ensure insight references are consistent - Check section ordering is logical