# carsearch — Canadian used-car search, monitoring & market-analysis platform

`carsearch` turns Canadian marketplace listings into a **persistent, searchable,
historical database** and exposes it to Claude Code (or any agent) over **MCP**,
plus a full **CLI**. It collects real listings from AutoTrader.ca (more
providers pluggable), keeps every price/status change as history, deduplicates
physical vehicles across listings, geocodes properly (real distances), extracts
risk signals from descriptions, computes comparable-price statistics, scores
project cars under configurable profiles, detects deals, reruns saved searches
on a schedule and raises alerts.

Ask Claude things like:

* *Find me every manual BMW E90 328i under $8,000 within 800 km of Toronto, prefer private sellers, reject rust buckets, rank by project-car value.*
* *What interesting project cars under $7k appeared in Ontario in the last 48 hours?*
* *Is this $6,200 Audi TT actually cheap relative to similar listings we've observed?*
* *Show me every price drop > 10% this week.* / *Cars sitting 30+ days where the seller already cut the price twice.*

Status per phase, tested commands and known issues: **`PROJECT_STATUS.md`**.

---

## Architecture (short)

```
providers/ (autotrader ✓, cargurus/kijiji/facebook interface+probe)  →  VehicleListing (canonical)
   → collectors/ (checkpointed ProviderRun, ingest w/ snapshots+events, raw payloads)
   → dedupe/ (VIN → weighted soft signals → vehicles + candidates)  → geo/ (geocode, haversine)
   → SQLite/PostgreSQL (SQLAlchemy)  → search/  analysis/ (description, comps, deals, market)
   → scoring/ (profiles + model knowledge JSON)  → monitoring/ (saved searches, scheduler) → notifications/
   → cli/ (carsearch …)  and  mcp/ (23 tools)
```
Details: `docs/architecture.md`, `docs/providers.md`, `docs/schema.md`.

## Installation

```bash
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"            # add ".[postgres]" for PostgreSQL
cp .env.example .env               # optional; SQLite in ./data works out of the box
carsearch db-init                  # creates/upgrades data/carsearch.db
```
Python ≥ 3.11 (developed on 3.14). No browser, no API keys required for AutoTrader.

## Database setup

* Default: `sqlite:///./data/carsearch.db` (WAL). Nothing to do.
* PostgreSQL: `CARSEARCH_DATABASE_URL=postgresql+psycopg://user:pass@host/carsearch` in `.env`,
  `pip install -e ".[postgres]"`, `carsearch db-init`. Schema/migrations in
  `src/carsearch/database/` (versioned, recorded in `schema_migrations`).

## Crawling (collecting real data)

```bash
# targeted, with detail pages for new listings (VIN, exact coordinates, drivetrain, created date)
carsearch crawl autotrader --make BMW --model "3 Series" --transmission manual --max-price 8000 --min-year 2006 --max-year 2013 --details new
# regional bulk (20 listings/page, ~1.5 s/page, no detail pages)
carsearch crawl autotrader --max-price 7000 --location Toronto --radius-km 500 --max-pages 200 --details none
# province scope (centroid + covering radius server-side, exact province locally)
carsearch crawl autotrader --province ON --transmission manual --max-price 6000 --seller-type private
# incremental refresh: stop after 3 consecutive pages with nothing new/changed
carsearch crawl autotrader --transmission manual --max-price 8000 --stop-when-seen 3
# resume an interrupted run
carsearch crawl autotrader --resume 12
# re-check listings not seen for 3 days (marks removed on 404); fetch missing detail pages; geocode backfill
carsearch refresh autotrader --older-than-days 3 --limit 200
carsearch details autotrader --limit 200
carsearch geocode --limit 60
scripts/bootstrap_crawl.sh   # example multi-band bootstrap used to seed the DB
```
AutoTrader caps a query at 200 pages (4000 listings) → split by price band/region for full coverage (`truncated=True` in the run summary tells you when).

## Searching

```bash
carsearch search --make BMW --model 328i --max-price 7000 --transmission manual --location Toronto --radius-km 800 --seller-type private
carsearch search --generation E90 --keywords '"one owner" -salvage' --province ON,QC --sort price_asc
carsearch new --since 48h --province ON --max-price 7000
carsearch price-drops --min 10 --since 7d
carsearch deals --province ON --max-price 7000 --profile project_car_enthusiast
carsearch deals --location Toronto --radius-km 500 --transmission manual --max-price 7000 --since 7d \
    --profile project_car_enthusiast --sort-by project --exclude-flags salvage_title,frame_rust,doesnt_run   # "best project cars" ranking
carsearch listing 42            # full record + price history + description flags
carsearch history 42            # snapshots + change events
carsearch comps 42              # comparable-price analysis with methodology
carsearch score 42 --profile project_car_enthusiast
carsearch analyze 42            # everything above + deal signals in one JSON
carsearch market --metric manual_premium --make Subaru --model WRX
carsearch market --metric price_by_mileage --generation E90 --province ON
carsearch market --metric cheapening --days 90
carsearch status [--check-providers]  ·  carsearch runs  ·  carsearch alerts
```
Every command accepts `--json` for machine-readable output.

## Monitoring / saved searches / alerts

```bash
carsearch saved seed                       # the four example searches from the brief
carsearch saved create "manual E90 328i under $8k" '{"make":"BMW","model":"3 Series","min_year":2006,"max_year":2011,"max_price":8000,"transmission":"manual"}' \
    --interval-minutes 240 --alert-rules '{"price_drop_pct":10,"target_price":6500,"min_deal_score":65}' --profile project_car_enthusiast
carsearch saved run "manual E90 328i under $8k"   # incremental crawl → new listings + alerts
carsearch schedule                          # long-running loop: runs due searches, maintenance
carsearch schedule --once                   # one pass (cron-friendly)
```
Alerts go to console, `data/alerts.jsonl` and an optional webhook (`CARSEARCH_WEBHOOK_URL`);
new sinks = subclass `notifications.alerts.AlertSink`. Every alert carries `reasons[]` (and risks).

## MCP server

```bash
carsearch serve-mcp                # stdio (default)  |  --transport streamable-http --port 8765
```
Tools: `search_cars`, `search_live_marketplace`, `get_listing`, `get_vehicle`,
`get_new_listings`, `get_price_history`, `get_listing_changes`, `get_comparables`,
`compare_cars`, `find_deals`, `find_price_drops`, `find_long_sitting_listings`,
`analyze_listing`, `score_listing`, `analyze_market`, `list_saved_searches`,
`create_saved_search`, `run_saved_search`, `delete_saved_search`, `get_alerts`,
`get_status`, `get_generation_codes`, `analyze_listing_images`. All return
structured JSON with pagination (`limit/offset/next_offset`); heavy fields
(description, photos, raw) are opt-in.

### Connect Claude Code

```bash
claude mcp add carsearch -- /ABSOLUTE/PATH/autotrader/.venv/bin/carsearch serve-mcp
# or in .mcp.json:
{ "mcpServers": { "carsearch": { "command": "/ABSOLUTE/PATH/autotrader/.venv/bin/carsearch", "args": ["serve-mcp"] } } }
```
Then: *"Find me the best manual project cars under $7,000 CAD within 500 km of
Toronto that appeared in the last week. Compare them against historical prices,
flag major mechanical/rust risks from the listings, and rank the top 10."* —
Claude will typically call `search_cars`/`get_new_listings` (or
`search_live_marketplace` for fresh data), then `analyze_listing`/`compare_cars`.

## How checkpoints work

Each crawl is a `provider_runs` row. After **every page** the listings are
committed and `last_checkpoint.next_page` is advanced; per-listing failures are
logged in `errors` and skipped. Detail enrichment commits per listing
(`detail_fetched_at`). A run that dies is `failed/interrupted` and
`carsearch crawl --resume <id>` continues from `next_page` (already fetched
pages come from the local `http_cache`). Saved-search state lives in the DB.
See `docs/architecture.md`.

## How to add a provider

Implement `BaseProvider` in `src/carsearch/providers/<name>/`, map to
`VehicleListing` in a `parser.py` (keep `raw`), register it, add fixtures +
tests, document it in `docs/providers.md`. Nothing else changes.

## Tests

```bash
pytest -q          # 42 tests: normalization, parser fixtures, ingest/price history, dedupe,
                   # geo/search, analysis (flags/comps/scoring/deals/market), crawler checkpoints, MCP tools
```
No test touches the network (real payload fixtures + a fake provider).

## Known limitations

* **Providers**: only AutoTrader.ca collects. CarGurus (DataDome), Kijiji (edge 403 / kijijiautos.ca gone) and Facebook (login) are interfaces with live health probes; no anti-bot or login circumvention is attempted (see `docs/providers.md`).
* **Asking prices, not sale prices.** Disappearance ≠ sold; we record `not_observed`/`removed`.
* **Coverage** depends on what you crawl (200-page cap per query; split by bands). "Appeared in the last N days" uses the marketplace creation date when known (detail page) else our first observation.
* **Coordinates**: exact for listings with detail pages; otherwise city-level (seed table / learned centroids / marketplace resolver). Distance filters skip listings without coordinates.
* **Scoring, flags and deal scores are heuristics** with stated evidence and configurable rules (`src/carsearch/config/*.json`); they are inputs to reasoning, not verdicts. Regex flags can misfire on unusual phrasing (evidence is always returned).
* **Comparables** use progressive relaxation; small samples are flagged (`warnings`, `confidence`).
* **Image analysis** ships hashing/caching and a pluggable vision hook (Anthropic backend included, off by default); no vision model runs unless configured.
* SQLite is fine for hundreds of thousands of listings; switch to PostgreSQL for concurrent writers.