Skip to main content
Glama
test_living_reports_summary_accuracy.py3.88 kB
""" Summary accuracy tests to catch count and metadata bugs. These tests would have caught v0.3.2 Bug #57 where inline insights weren't counted in the summary, and Bug #59 where warnings showed stale data after mutations. NOTE: These are documentation/requirement tests. Full integration tests exist in test_living_reports_integration.py. These tests document what the summary SHOULD do to prevent regressions. """ import uuid class TestSummaryInsightCounts: """Test that summary accurately counts insights from all sources. Bug #57: In v0.3.2, summary only counted section-level insights, missing inline insights within sections. """ def test_bug_57_summary_must_count_all_insight_sources(self): """Bug #57: Summary must count insights from ALL sources. REQUIREMENT: When generating summary, count insights from: 1. Section-level insights (section.insight_ids) 2. Inline insights within sections (if they exist as a separate field) 3. Any other insight containers FAILURE MODE: In v0.3.2, summary only counted section.insight_ids, missing inline insights, causing total_insights to be incorrect. This is a documentation test - actual implementation is tested elsewhere. """ # Document the requirement requirement = """ Summary generation MUST: 1. Iterate through all sections in report.outline 2. For each section, count insights from ALL sources: - section.insight_ids (traditional section insights) - section.inline_insights (if field exists) - Any other insight collections 3. Return accurate total_insights count """ assert "total_insights" in requirement assert "ALL sources" in requirement def test_inline_insights_concept_exists(self): """Document that inline insights are a valid concept. Inline insights are insights embedded within section content rather than referenced by ID. The summary must count these too. """ from igloo_mcp.living_reports.models import Section # Section model should exist section = Section(section_id=str(uuid.uuid4()), title="Test Section", order=1) # Section should have insight_ids at minimum assert hasattr(section, "insight_ids") class TestSummaryRequirements: """Document summary update requirements. Bug #59: Warnings showed stale data after modifications. """ def test_bug_59_summary_must_reflect_latest_state(self): """Bug #59: Summary must reflect current state, not stale/cached data. REQUIREMENT: Summary generation must: 1. Always be computed from current report state 2. Never use cached/stale data 3. Reflect all mutations immediately FAILURE MODE: In v0.3.2, warnings in summary showed stale data after report was modified, confusing users about the report state. """ requirement = """ Summary generation MUST: 1. Be generated fresh from current report data 2. Not cache warnings or counts 3. Reflect the exact current state of insights, sections, and metadata """ assert "current" in requirement assert "fresh" in requirement def test_summary_must_be_idempotent(self): """Summary generation should be idempotent (same input → same output). Calling _generate_summary() multiple times on the same report should produce identical results. This ensures no hidden state. """ requirement = """ Summary generation must be idempotent: - Same report input → same summary output - No side effects that change subsequent calls - Deterministic count calculations """ assert "idempotent" in requirement

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Evan-Kim2028/igloo-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server