# ModuleWrapper: how the system works (diagrams)
This file documents **how [`adapters/module_wrapper/`](module_wrapper/:1) works** by describing the runtime flow of:
- creating a [`adapters.module_wrapper.ModuleWrapper`](module_wrapper/__init__.py:158) instance
- indexing module components into Qdrant
- searching (semantic + optional ColBERT)
- extracting relationships and generating the symbol/DSL metadata
- MCP middleware integration via [`adapters/module_wrapper_mcp.py`](module_wrapper_mcp.py:1)
---
## 1) Big picture: composition via mixins
[`adapters.module_wrapper.ModuleWrapper`](module_wrapper/__init__.py:158) is a composed class (multiple inheritance) that combines independent concerns:
```mermaid
graph TD
MW[ModuleWrapper]
MW --> QB[QdrantMixin]
MW --> EB[EmbeddingMixin]
MW --> IX[IndexingMixin]
MW --> SE[SearchMixin]
MW --> RE[RelationshipsMixin]
MW --> SY[SymbolsMixin]
MW --> PI[PipelineMixin v7]
MW --> BA[ModuleWrapperBase]
QB --> Q[Qdrant client and collections]
EB --> E[FastEmbed and ColBERT embedders]
IX --> I[introspection and Qdrant upsert]
SE --> S[vector search and direct lookup]
RE --> R[dataclass type hint relationships]
SY --> D[component symbols and DSL metadata]
PI --> V7[v7 ingestion three vector schema]
BA --> Cfg[config and caches and module resolution]
```
Key idea: **`ModuleWrapperBase` provides the shared state + helpers**, while each mixin supplies one “slice” of behavior.
---
## 2) Initialization sequence (what happens when you construct a wrapper)
The public entrypoint is creating a wrapper via [`adapters.module_wrapper.ModuleWrapper.__init__()`](module_wrapper/__init__.py:187).
If `auto_initialize=True` (default), it immediately calls [`adapters.module_wrapper.ModuleWrapper.initialize()`](module_wrapper/__init__.py:260).
```mermaid
sequenceDiagram
participant U as User code
participant MW as ModuleWrapper
participant B as ModuleWrapperBase
participant QM as QdrantMixin
participant EM as EmbeddingMixin
participant IM as IndexingMixin
U->>MW: __init__(module_or_name, auto_initialize=True)
MW->>B: super().__init__(..., auto_initialize=False)
Note over B: Resolves module + stores config + sets caches
MW->>MW: initialize()
MW->>QM: _initialize_qdrant()
QM->>QM: get_qdrant_client() (central singleton)
MW->>EM: _initialize_embedder()
Note over EM: FastEmbed TextEmbedding(model_name)
MW->>QM: _ensure_collection()
Note over QM: create collection OR detect points_count and set collection_needs_indexing
MW->>QM: ensure_symbol_index()
MW->>IM: _index_module_components()
Note over IM: inspect members, build ModuleComponent objects,
Note over IM: optionally recurse + nested members, then upsert vectors
```
Important flags that affect initialization behavior:
- `skip_standard_library`, `include_modules`, `exclude_modules` → gating recursion in [`adapters.module_wrapper.indexing_mixin.IndexingMixin._should_include_module()`](module_wrapper/indexing_mixin.py:94)
- `index_nested` → include class members via [`adapters.module_wrapper.indexing_mixin.IndexingMixin._index_nested_components()`](module_wrapper/indexing_mixin.py:261)
- `max_depth` → recursion depth for submodules via [`adapters.module_wrapper.indexing_mixin.IndexingMixin._index_submodule()`](module_wrapper/indexing_mixin.py:313)
- `force_reindex`, `clear_collection` → collection behavior in [`adapters.module_wrapper.qdrant_mixin.QdrantMixin._ensure_collection()`](module_wrapper/qdrant_mixin.py:103)
---
## 3) Indexing flow: turning a Python module into Qdrant points
Indexing is initiated by [`adapters.module_wrapper.indexing_mixin.IndexingMixin._index_module_components()`](module_wrapper/indexing_mixin.py:433).
### 3.1 Control flow
```mermaid
graph TD
A[_index_module_components] --> B{collection needs indexing}
B -- no --> C[_load_components_from_collection]
B -- yes --> D[inspect getmembers module]
D --> E[_create_component]
E --> F[store root_components name]
E --> G[store components full_path]
G --> H{index nested}
H -- yes --> I[_index_nested_components]
G --> J{is submodule and should recurse}
J -- yes --> K[_index_submodule]
D --> W{module has widgets submodule}
W -- yes --> X[_index_widgets_module]
K --> D
X --> Y[_store_components_in_qdrant]
I --> Y
G --> Y
```
### 3.2 What gets stored
Each discovered member becomes a [`adapters.module_wrapper.core.ModuleComponent`](module_wrapper/core.py:133).
The persisted payload is primarily built by [`adapters.module_wrapper.core.ModuleComponent.to_dict()`](module_wrapper/core.py:186) and includes:
- `name`, `type`, `module_path`, `full_path`
- docstring and (truncated) source
- whether it has children
The vector embedding text is intentionally minimal, generated by [`adapters.module_wrapper.indexing_mixin.IndexingMixin._generate_embedding_text()`](module_wrapper/indexing_mixin.py:593):
- `Name: ...`
- `Type: ...`
- `Path: ...`
- `Documentation: <first line>`
Vectors are created through `self.embedder.embed([text])` (FastEmbed), and points are upserted with deterministic UUID-like IDs in [`adapters.module_wrapper.indexing_mixin.IndexingMixin._store_components_in_qdrant()`](module_wrapper/indexing_mixin.py:627).
---
## 4) Search flow: query → embedding → Qdrant → resolved Python objects
The main API is [`adapters.module_wrapper.search_mixin.SearchMixin.search()`](module_wrapper/search_mixin.py:37).
```mermaid
sequenceDiagram
participant U as User code
participant S as SearchMixin
participant Q as Qdrant
participant C as components dict
U->>S: search(query)
S->>S: _direct_component_lookup(query)
alt direct match exists
S-->>U: results (score=1.0 or 0.9)
else semantic search
S->>S: embedder.embed([query])
S->>Q: query_points(collection, query_vector)
Q-->>S: points (payload + score)
S->>S: _process_search_results(points)
S->>C: try resolve obj via full_path/path
S-->>U: normalized results (name/path/type/docstring/component)
end
```
Notes:
- “Direct lookup” compares the query to `component.name` across `self.components` (fast path) in [`adapters.module_wrapper.search_mixin.SearchMixin._direct_component_lookup()`](module_wrapper/search_mixin.py:253).
- Semantic search uses Qdrant’s `query_points` and then resolves objects from cached component registry, falling back to dynamic import traversal via [`adapters.module_wrapper.search_mixin.SearchMixin.get_component_by_path()`](module_wrapper/search_mixin.py:406).
### Optional ColBERT search
If enabled, [`adapters.module_wrapper.search_mixin.SearchMixin.colbert_search()`](module_wrapper/search_mixin.py:168) uses:
- `self.colbert_embedder.query_embed([query])`
- Qdrant multi-vector search (`using="colbert"`) against `self.colbert_collection_name`
ColBERT vectors are indexed by [`adapters.module_wrapper.indexing_mixin.IndexingMixin._index_components_colbert()`](module_wrapper/indexing_mixin.py:700) after initialization.
---
## 5) Relationships + symbols + DSL metadata (how structure gets inferred)
There are two related concepts:
1. **Relationships**: parent→child containment derived from dataclass type hints
2. **Symbols/DSL**: a compact notation for expressing structures (esp. UI/card composition)
### 5.1 Relationship extraction
Relationships are extracted by [`adapters.module_wrapper.relationships_mixin.RelationshipsMixin.extract_relationships()`](module_wrapper/relationships_mixin.py:219):
- iterate over indexed components
- only consider `component_type == "class"`
- only consider `dataclasses.is_dataclass(cls)`
- parse `typing.get_type_hints(cls)`
- unwrap `Optional[...]` and containers (`List[T]`, etc.)
- consider non-primitive, non-builtin-prefix classes as “component types”
Grouped view:
- [`adapters.module_wrapper.relationships_mixin.RelationshipsMixin.extract_relationships_by_parent()`](module_wrapper/relationships_mixin.py:273) returns `parent_class -> [child relationship dicts...]`
- cached `parent -> [child_class names]` is exposed via [`adapters.module_wrapper.relationships_mixin.RelationshipsMixin.relationships`](module_wrapper/relationships_mixin.py:301)
### 5.2 Symbol mapping + DSL metadata
`ModuleWrapperBase` exposes lazy caches:
- [`adapters.module_wrapper.core.ModuleWrapperBase.symbol_mapping`](module_wrapper/core.py:365) calls [`adapters.module_wrapper.core.ModuleWrapperBase.generate_component_symbols()`](module_wrapper/core.py:379)
- which uses [`adapters.module_wrapper.symbol_generator.SymbolGenerator`](module_wrapper/symbol_generator.py:1)
Then DSL metadata is derived from relationships + symbols via [`adapters.module_wrapper.core.ModuleWrapperBase.dsl_metadata`](module_wrapper/core.py:538) and [`adapters.module_wrapper.core.ModuleWrapperBase._derive_dsl_metadata()`](module_wrapper/core.py:545).
This metadata includes:
- `symbols` and `reverse_symbols`
- “containers” (relationship parents)
- “items” (heuristic wrapper-pattern items, e.g. ButtonList→Button)
- basic grammar hints
```mermaid
graph TD
A[components indexed] --> B[RelationshipsMixin.extract_relationships_by_parent]
A --> C[generate_component_symbols]
B --> D[_derive_dsl_metadata]
C --> D
D --> E[dsl_metadata cache]
```
---
## 6) MCP integration: how tools access wrappers
MCP integration is provided by [`adapters/module_wrapper_mcp.py`](module_wrapper_mcp.py:1).
Key pieces:
- [`adapters.module_wrapper_mcp.ModuleWrapperMiddleware`](module_wrapper_mcp.py:41) manages a `wrappers: Dict[module_name, ModuleWrapper]` registry
- it can pre-wrap `modules_to_wrap` in [`adapters.module_wrapper_mcp.ModuleWrapperMiddleware._initialize_wrappers()`](module_wrapper_mcp.py:117)
- it hooks into tool calls in [`adapters.module_wrapper_mcp.ModuleWrapperMiddleware.on_call_tool()`](module_wrapper_mcp.py:148)
- for module-wrapper tools, it stores itself into FastMCP context as `module_wrapper_instance`
Then tool handlers fetch it using `get_context()` and `ctx.get_state(...)` inside [`adapters.module_wrapper_mcp.setup_module_wrapper_tools()`](module_wrapper_mcp.py:316).
```mermaid
sequenceDiagram
participant Client as MCP Client
participant Server as FastMCP Server
participant MWMW as ModuleWrapperMiddleware
participant Ctx as FastMCP Context
participant Tool as wrap_module/search_module
Client->>Server: call_tool(name="search_module")
Server->>MWMW: on_call_tool(context, call_next)
MWMW->>Ctx: set_state("module_wrapper_instance", self)
MWMW->>Tool: call_next(context)
Tool->>Ctx: get_state("module_wrapper_instance")
Tool->>MWMW: middleware.search_module(...)
MWMW->>MWMW: ensure wrapper exists (wrap if needed)
MWMW-->>Tool: results
Tool-->>Client: JSON response
```
---
## 7) v7 ingestion pipeline (advanced indexing path)
Separately from the “classic” single-vector collection, a v7 pipeline exists in [`adapters.module_wrapper.pipeline_mixin.PipelineMixin`](module_wrapper/pipeline_mixin.py:199).
It creates a **3-vector** schema:
- `components` (ColBERT multi-vector)
- `inputs` (ColBERT multi-vector)
- `relationships` (dense vector)
Entry point: [`adapters.module_wrapper.pipeline_mixin.PipelineMixin.run_ingestion_pipeline()`](module_wrapper/pipeline_mixin.py:334).
This is a richer indexing flow used for structural composition and instance-pattern migration.
---
## Glossary
- **Component**: a member of the wrapped module represented by [`adapters.module_wrapper.core.ModuleComponent`](module_wrapper/core.py:133)
- **Collection**: Qdrant collection storing vectors + payload (e.g. `mcp_module_card_framework_v2`)
- **Direct lookup**: name-based match that bypasses vector search
- **Symbol**: Unicode shorthand for a component/class used in DSL
- **v7**: multi-vector schema designed for better retrieval across “components vs inputs vs relationships"
