Monitor embedding drift
monitor_embedding_driftCompare two time windows of pgvector embeddings and flag distributional drift by measuring cosine distance between centroids and L2-norm changes.
Instructions
Compare two time windows of a pgvector column and flag distributional drift. Samples up to sample_size (default 5000) non-NULL embeddings from each window (filtered by timestamp_column), computes the centroid (per-dimension mean vector) and L2-norm distribution of each, then reports the cosine distance between the two centroids (the main drift signal), the relative change in mean / std of the L2-norm distribution, and a boolean drift_detected that flips when cosine distance exceeds drift_threshold (default 0.05). Each window is treated as a half-open [start, end) interval. Useful for ops monitoring of embedding pipelines — an upstream model swap typically shows up as a large centroid cosine distance even if the norm distribution looks stable. insufficient_data is returned distinctly from drift_detected=false when either window is empty. Reports available=false if pgvector is not installed.
Example: monitor_embedding_drift(schema='public', table='docs', embedding_column='embedding', timestamp_column='created_at', baseline_start='2026-01-01', baseline_end='2026-02-01', current_start='2026-02-01', current_end='2026-03-01')
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| table | Yes | ||
| schema | Yes | ||
| database | No | Optional: target a configured secondary (read-only) database by name; omit for the primary. Call list_databases to see the configured ids. | |
| current_end | Yes | ||
| sample_size | No | ||
| baseline_end | Yes | ||
| current_start | Yes | ||
| baseline_start | Yes | ||
| drift_threshold | No | ||
| embedding_column | Yes | ||
| timestamp_column | Yes |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| notes | Yes | ||
| current | Yes | ||
| baseline | Yes | ||
| available | Yes | ||
| dimension | Yes | ||
| drift_detected | Yes | ||
| drift_threshold | Yes | ||
| insufficient_data | Yes | ||
| centroid_cosine_distance | Yes | ||
| norm_std_relative_change | Yes | ||
| norm_mean_relative_change | Yes |