---
phase: 03-memory-layer
plan: 02
type: execute
wave: 1
depends_on: []
files_modified:
- src/skill_retriever/memory/vector_store.py
- src/skill_retriever/memory/__init__.py
- tests/test_vector_store.py
- pyproject.toml
autonomous: true
must_haves:
truths:
- "Vector store returns semantically similar components when queried with a text description (top-5 cosine similarity)"
- "Embeddings are persisted and reloaded without re-computation"
artifacts:
- path: "src/skill_retriever/memory/vector_store.py"
provides: "FAISSVectorStore with string ID mapping and FastEmbed embedding"
exports: ["FAISSVectorStore"]
min_lines: 100
- path: "tests/test_vector_store.py"
provides: "Vector store unit tests"
min_lines: 60
- path: "pyproject.toml"
provides: "faiss-cpu dependency added"
contains: "faiss-cpu"
key_links:
- from: "src/skill_retriever/memory/vector_store.py"
to: "src/skill_retriever/config.py"
via: "imports EMBEDDING_CONFIG for dimensions"
pattern: "from skill_retriever\\.config import"
- from: "src/skill_retriever/memory/vector_store.py"
to: "faiss"
via: "IndexFlatIP wrapped in IndexIDMap"
pattern: "faiss\\.IndexFlatIP|faiss\\.IndexIDMap"
---
<objective>
Build the vector store subsystem: a FAISS-backed store with string-to-int ID mapping, L2 normalization for cosine similarity, batch add support, and JSON+binary persistence.
Purpose: The vector store enables semantic search over component descriptions. Given a natural language query, it returns the top-k most similar components by cosine similarity. This is one of two retrieval signals (the other being graph PPR) that Phase 4 will fuse.
Output: `vector_store.py` with FAISSVectorStore class, tests, faiss-cpu dependency added
</objective>
<execution_context>
@C:\Users\33641\.claude/get-shit-done/workflows/execute-plan.md
@C:\Users\33641\.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/03-memory-layer/03-RESEARCH.md
@src/skill_retriever/config.py
</context>
<tasks>
<task type="auto">
<name>Task 1: Add faiss-cpu dependency and create FAISSVectorStore</name>
<files>
pyproject.toml
src/skill_retriever/memory/vector_store.py
src/skill_retriever/memory/__init__.py
</files>
<action>
1. Add `faiss-cpu>=1.13.2` to pyproject.toml dependencies array. Run `uv sync` to install.
2. Create `src/skill_retriever/memory/vector_store.py` with `FAISSVectorStore` class:
a) `__init__(self, dimensions: int | None = None)`:
- Default dimensions from `EMBEDDING_CONFIG.dimensions` (384)
- Create `faiss.IndexFlatIP(dim)` wrapped in `faiss.IndexIDMap(base)`
- Initialize `_id_to_int: dict[str, int]`, `_int_to_id: dict[int, str]`, `_next_id: int = 0`
- Store `_dimensions: int` for validation
b) `add(self, component_id: str, embedding: np.ndarray) -> None`:
- Assign next int ID, update both mapping dicts
- Reshape to (1, dim), cast to float32, call `faiss.normalize_L2(vec)` BEFORE adding
- `self._index.add_with_ids(vec, np.array([int_id], dtype=np.int64))`
c) `add_batch(self, ids: list[str], embeddings: np.ndarray) -> None`:
- Bulk version: assign int IDs for all, update mappings
- Cast to float32, call `faiss.normalize_L2(vecs)` on the batch
- Single `add_with_ids` call
d) `search(self, query_embedding: np.ndarray, top_k: int = 5) -> list[tuple[str, float]]`:
- Reshape, cast, normalize query vector
- `distances, indices = self._index.search(vec, top_k)`
- Filter out idx == -1 results (FAISS pads with -1 when fewer than k results exist)
- Return list of (component_id_string, similarity_score) tuples
e) `remove(self, component_id: str) -> None`:
- Look up int ID from mapping
- Call `self._index.remove_ids(np.array([int_id], dtype=np.int64))`
- Remove from both mapping dicts
- Raise KeyError if component_id not found
f) `save(self, directory: str) -> None`:
- Create directory with `Path(directory).mkdir(parents=True, exist_ok=True)`
- `faiss.write_index(self._index, str(path / "vectors.faiss"))`
- Write JSON mapping file: `{"id_to_int": ..., "int_to_id": {str(k): v}, "next_id": ..., "dimensions": ...}`
g) `load(self, directory: str) -> None`:
- `self._index = faiss.read_index(str(path / "vectors.faiss"))`
- Load JSON mapping, reconstruct `_id_to_int`, `_int_to_id` (convert keys back to int), `_next_id`, `_dimensions`
h) `count` property returning `self._index.ntotal`
i) `contains(self, component_id: str) -> bool` returning `component_id in self._id_to_int`
Use `from __future__ import annotations`. Import config with standard pattern.
AVOID:
- Do NOT skip `faiss.normalize_L2()` -- raw inner product without normalization gives meaningless similarity scores
- Do NOT use IVF or HNSW indexes for this scale (<50k vectors)
- Do NOT forget to save the ID mapping alongside the FAISS index (pitfall 5 from research)
- Do NOT create FastEmbed model instances in this class -- embedding generation belongs to the caller/ingestion pipeline, this class only stores/searches pre-computed vectors
</action>
<verify>
Run `uv run python -c "from skill_retriever.memory.vector_store import FAISSVectorStore; print('import OK')"` succeeds.
Run `uv run python -c "import faiss; print(faiss.__version__)"` prints version.
Run `uv run pyright src/skill_retriever/memory/vector_store.py` passes.
Run `uv run ruff check src/skill_retriever/memory/vector_store.py` passes.
</verify>
<done>
FAISSVectorStore class exists with add, add_batch, search, remove, save, load, count, contains methods. FAISS is installed. Imports work. Type checks and linting pass.
</done>
</task>
<task type="auto">
<name>Task 2: Write vector store tests covering CRUD, search, and persistence</name>
<files>
tests/test_vector_store.py
</files>
<action>
Create `tests/test_vector_store.py` with pytest tests. Use `numpy` to generate synthetic embeddings (random vectors of dimension 384).
1. **test_add_and_count** -- Add 3 vectors with different IDs, verify count == 3.
2. **test_add_and_search** -- Add 5 vectors. Create a query vector identical to one of them. Search top_k=3. Verify the identical vector is returned first with similarity close to 1.0 (>0.99 after normalization).
3. **test_search_returns_cosine_similarity** -- Add 2 known orthogonal vectors (e.g., [1,0,0,...] and [0,1,0,...]). Query with vector close to first. Verify first result has high similarity, second has low similarity.
4. **test_search_filters_negative_one** -- Add only 2 vectors, search with top_k=5. Verify result list has exactly 2 entries (no -1 padding).
5. **test_add_batch** -- Create 10 vectors as a numpy array, add via add_batch. Verify count == 10. Search and verify results come back correctly.
6. **test_remove** -- Add 3 vectors, remove one. Verify count == 2. Verify contains() returns False for removed ID. Verify search does not return removed ID.
7. **test_remove_nonexistent_raises** -- Verify `remove("nonexistent")` raises KeyError.
8. **test_save_and_load** -- Add vectors, save to tmp_path. Create new FAISSVectorStore, load. Verify count matches. Search and verify same results as before save.
9. **test_contains** -- Add a vector, verify contains() returns True for that ID and False for unknown ID.
Helper fixture: Create a `make_random_embedding` fixture that returns `np.random.default_rng(42).random(384).astype(np.float32)` for reproducibility.
</action>
<verify>
Run `uv run pytest tests/test_vector_store.py -v` -- all tests pass.
Run `uv run ruff check tests/test_vector_store.py` passes.
</verify>
<done>
9 tests covering add, search, cosine similarity correctness, -1 filtering, batch add, remove, save/load persistence, and contains all pass.
</done>
</task>
</tasks>
<verification>
- `uv run pytest tests/test_vector_store.py -v` -- all tests green
- `uv run pyright src/skill_retriever/memory/vector_store.py` -- zero errors
- `uv run ruff check src/skill_retriever/memory/` -- zero warnings
- Search with identical vector returns similarity > 0.99
- Save/load round-trip preserves all vectors and ID mappings
</verification>
<success_criteria>
1. FAISSVectorStore uses IndexFlatIP wrapped in IndexIDMap with L2 normalization
2. String-to-int ID mapping is maintained bidirectionally
3. Search returns correct cosine similarity scores (normalized inner product)
4. FAISS -1 padding is filtered out when fewer than k results exist
5. Save persists both FAISS index and JSON ID mapping; load restores both
6. All linting and type checks pass
</success_criteria>
<output>
After completion, create `.planning/phases/03-memory-layer/03-02-SUMMARY.md`
</output>
