find_similar_columns
Detect duplicate columns, naming drift (email vs email_address), or matching concepts across datasets by clustering columns with composite similarity scores from name, type, and value signals.
Instructions
Multi-signal cross-dataset column consolidation. Fuses name (token Jaccard), type, top-value overlap, cardinality similarity, and (when present) embedding cosine into a composite score. Clusters via union-find and classifies each cluster: near_duplicate, naming_drift, parallel_definition, or overlapping_topic. Use to find duplicate columns across datasets, surface naming drift (email vs email_address), or detect the same conceptual column spread across multiple datasets. Mirrors jcm's find_similar_symbols. Every signal is heuristic, so a high score means investigate, not merge.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| top_n | No | Max clusters returned. Default 50, capped at 200. | |
| datasets | No | Datasets to scan. Omit to scan every indexed dataset. | |
| min_score | No | Composite-score floor for surfacing pairs. | |
| same_type_only | No | Drop pairs where types don't match. |