Chroma MCP Server

# Implicit Learning and Analysis in Phase 1 ## Introduction In Phase 1 of the Chroma MCP Server integration, "implicit learning" refers to the system's capability to automatically capture and structure valuable data from your daily development activities. Even without active model fine-tuning (like LoRA, planned for later phases), this captured data provides a rich foundation for understanding development patterns, AI assistant performance, and identifying effective solutions. The cornerstone of implicit learning in Phase 1 is the `chat_history_v1` collection, which stores summaries of AI interactions with enhanced contextual information. This, combined with indexed code and test results, allows for powerful insights through manual or semi-automated analysis. ## Key Data Sources for Implicit Learning ### 1. `chat_history_v1` Collection This ChromaDB collection stores summarized AI prompt/response pairs along with rich contextual metadata. Each entry can include: - **Core Metadata:** - `session_id`: Groups related interactions. - `timestamp`: When the interaction occurred. - `prompt_summary`: A concise summary of the user's request. - `response_summary`: A concise summary of the AI's response. - `involved_entities`: Key terms, file paths, function names discussed. - `raw_prompt_hash` / `raw_response_hash`: Hashes for de-duplication or integrity checks. - `status`: Lifecycle status (e.g., `captured`, `analyzed`, `promoted_to_learning`). - **Enhanced Context Fields (Crucial for Implicit Learning):** - `code_context`: Snippets of code before and after modifications made by the AI. - `diff_summary`: A summary of the key changes made to files. - `tool_sequence`: The sequence of MCP tools used by the AI (e.g., "read_file→edit_file→run_terminal_cmd"). - `modification_type`: Categorization of the interaction (e.g., `refactor`, `bugfix`, `feature`, `documentation`). - `confidence_score`: An AI-assessed score (0.0-1.0) indicating the perceived value or success of the interaction. ### 2. `codebase_v1` Collection While primarily for RAG, the `codebase_v1` collection contributes to implicit learning through: - `related_chat_ids`: A field that can link code chunks back to the `chat_history_v1` entries that resulted in their modification. This helps trace the origin of code changes. ### 3. `test_results_v1` Collection Test outcomes, especially those captured by the `--auto-capture-workflow` pytest plugin, offer strong implicit signals: - **Failure-to-Success Transitions:** When a test goes from failing to passing after code changes (often AI-assisted), it signifies effective problem-solving. These transitions are logged with context. - **Error Patterns:** Analyzing recurring test failures can highlight areas where developers or the AI struggle. ## Workflow for Implicit Learning & Analysis (Phase 1) The process involves automated data capture followed by manual or semi-automated analysis: ### 1. Automated Data Capture - **Code Indexing:** The `chroma-mcp-client index` command, typically run via Git `post-commit` hooks, keeps `codebase_v1` up-to-date. - **Rich Chat Logging:** The `auto_log_chat` IDE rule (e.g., in `.cursorrules`) automatically captures AI interactions, enriches them with context (diffs, tool sequences, etc.), and logs them to `chat_history_v1`. - **Test Result Capture:** Running tests with `hatch test --cover -v --auto-capture-workflow` (or via `scripts/test.sh -c -v --auto-capture-workflow`) logs test results, including failure-to-success transitions, to `test_results_v1` and `validation_evidence_v1`. ### 2. Manual/Semi-Automated Analysis (Using `chroma-mcp-client`) - **Analyzing Chat History (`chroma-mcp-client analyze-chat-history`):** - This command fetches entries from `chat_history_v1` (typically those with `status: captured`). - It leverages the rich context (confidence scores, diff summaries, modification types) to help developers identify potentially valuable interactions or patterns in AI assistance. - It can update the status of processed entries (e.g., to `analyzed`), making them candidates for promotion to `derived_learnings_v1`. - **Reviewing Test Results & Transitions:** - Developers can manually review the `test_results_v1` collection or use the `chroma-mcp-client check-test-transitions` command. - `check-test-transitions` specifically looks for tests that were failing and subsequently passed after code changes, logging these as valuable `ValidationEvidence`. This is a strong indicator of a successful problem-solving loop, often involving AI assistance. ### 3. Identifying Potential Learnings The human developer plays a crucial role in this phase by: - Reviewing the output of `analyze-chat-history`. - Examining successful test transitions identified by `check-test-transitions`. - Spotting recurring problems, effective solutions proposed by the AI, common pitfalls, or particularly insightful AI interactions. ## Bridging to Explicit Learning (Derived Learnings) The insights and high-value interactions identified through this implicit learning and analysis process are the primary candidates for promotion to the `derived_learnings_v1` collection. This is where knowledge becomes explicit and directly reusable by the RAG system. The `chroma-mcp-client review-and-promote` command provides an interactive workflow to facilitate this promotion, allowing developers to curate and formalize these learnings. (This will be detailed further in `docs/usage/derived_learnings.md`). ## Benefits of Phase 1 Implicit Learning - **Valuable Dataset from Day One:** Your development activities immediately start building a rich, contextualized dataset without requiring complex setup. - **Understanding AI & Developer Patterns:** Helps identify how the AI assistant is being used, where it excels, and common challenges faced during development. - **Foundation for Advanced Learning:** Provides the raw material and validated signals necessary for more advanced learning techniques in Phases 2 and 3, such as LoRA fine-tuning. - **Improved Debugging and Onboarding:** Past interactions can serve as examples or troubleshooting guides. By actively capturing and providing tools to analyze this "development exhaust," Phase 1 offers immediate value in understanding and improving the software development lifecycle.