# Signal Prospector

**A signal-based B2B prospecting engine that runs on free infrastructure.** It finds companies that just entered "buying mode" — funding, new leadership, hiring, expansion, competitor/topic engagement — scores them against your Ideal Customer Profile, enriches the decision-maker with a free MX-validated email guess, and drafts a personalized, signal-aware outreach message. Inbound replies get a ready-to-send AI draft.

The whole thing is exposed three ways: a CLI, a self-serve Flask dashboard, **and an MCP server** so you can prospect from Claude in natural language.

![Prospector dashboard: pipeline KPIs, a lead funnel, and the hottest scored leads with their triggering signals.](docs/dashboard.png)

*The Flask dashboard. Every score and the funnel above are computed by the engine — the companies, contacts and signals shown here are illustrative demo data, not real records.*

The leads table makes the scoring transparent: each row carries its triggering signal, the enriched contact and email, a lifecycle status, and a plain-English **"why"** the lead scored where it did.

![Leads table: score, company, signal, contact, email, status and an explainable 'why' column.](docs/leads.png)

---

## Why it's interesting

Most "prospecting tools" are either a static lead list or a black box. This one is the opposite: a transparent detect → score → enrich → draft → reply → optimize loop where every number is explainable and every external call is free by default.

- **Eight signal families behind one contract.** Funding, job-change, leadership-change, hiring, expansion, competitor-engagement, topic-engagement and news are each a small detector returning a typed `Signal`, registered in a registry and fed *injected* free `search_fn` / `fetch_fn` functions — so adding a detector, or swapping the search backend, is a drop-in.
- **An explainable ICP scorer, not a ranking black box.** The 0–100 score is a precise weighted blend of *right buyer × right moment* (formula below), and every lead ships a plain-English "why this lead" reason string.
- **A free email-enrichment waterfall.** Decision-maker discovery parses public search snippets, strips RTL/zero-width Unicode, then infers the most likely corporate email from name+domain patterns and validates the domain over DNS — no paid enrichment vendor, and it never invents an address it can't form.
- **Free-first by construction.** Discovery is self-hosted SearXNG; fetching is an `httpx → headless-browser → PDF` cascade with `trafilatura` extraction; generation is DeepSeek (any OpenAI-compatible endpoint). Paid fallbacks stay off unless you key them.
- **561 tests** across detectors, scorer, enrichment waterfall, reply classifier and the web layer. Unknown data always stays `n/a` rather than being fabricated.

## The scoring, precisely

No hand-waving — this is exactly how a lead's score is computed (`prospector/icp.py`):

| Quantity | Definition | Notes |
|---|---|---|
| **Final score (0–100)** | `100 × (0.55 · signal_score + 0.45 · icp_match)` | "right moment" weighted slightly over "right buyer" |
| **icp_match (0–1)** | `0.45 · company_fit + 0.55 · contact_fit` | the *person* matters more than firmographics |
| **company_fit** | mean of the configured industry / geo / size / keyword checks | missing evidence is **neutral** (0.5 baseline); only positive matches lift, and an `exclude_keyword` hit is a hard **0** |
| **signal_score (0–1)** | `clamp(best + 0.35·second + 0.12·third, 0..1)` where each term is `strength × recency` | the strongest recent signal dominates; extra strong, fresh signals add a decayed bonus |
| **strength** | per-kind base weight (funding `0.95`, leadership `0.9`, …) | a detector may override per-signal |
| **recency** | step decay: `≤7d → 1.0`, `≤30d → 0.85`, `≤90d → 0.55`, `≤180d → 0.3`, older/future → `0.15` | a future dateline is treated as stale, never as "freshest" |

Because missing evidence is neutral rather than punishing, a strong buying signal can still surface a lead whose firmographics are only partly known — which is the whole point of *signal-based* prospecting.

## The email waterfall, precisely

Enrichment never fabricates an address (`prospector/enrich.py`):

1. Resolve a mail-capable domain from the company site, rejecting aggregator/social hosts and website-builder domains (`*.wixsite.com`, `*.github.io`, …) that are never the company's own mail domain.
2. Generate candidates from the person's name, most-likely-first: `{first}.{last}` · `{first}` · `{f}{last}` · `{first}{l}` · `{first}_{last}` · `{last}` — dropping `al-`/`el-` prefixes and rejecting RFC-implausible locals (a bare initial like `bisrat.d.@` is never emitted).
3. Classify the domain over DNS: **verified** (publishes MX), **risky** (resolves but no MX), **invalid** (NXDOMAIN or RFC 7505 null-MX). Outbound-25 SMTP probing is deliberately not done.

## How it works

```
 ICP (a preset, or auto-drafted from a website URL)
        │
        ▼
  1. DETECT   eight signal detectors ──► Signal[]          prospector/signals/*
        │      (SearXNG discovery + free httpx→browser→PDF scrape cascade,
        │       trafilatura extraction, optional LLM entity refinement)
        ▼
  2. SCORE    fit × signal-strength × recency ──► 0..100   prospector/icp.py
        │      + explainable "why this lead" reason
        ▼
  3. ENRICH   decision-maker + email waterfall (MX-validated)   prospector/enrich.py
        │
        ▼
  4. DRAFT    DeepSeek signal-aware outreach (+ A/B variants)    prospector/outreach.py
        │      multilingual (en/ar/fr), draft-first (never auto-sent)
        ▼
  5. REPLY    classify inbound + draft a response      prospector/reply.py
        │
        ▼
  6. OPTIMIZE weekly "what converts" feeds back into step 4     prospector/campaign.py

  Surfaces:  CLI (cli.py) · Flask dashboard (prospector/web) · MCP server (prospector/mcp_server.py)
  Storage:   SQLite (prospector/db.py)  ·  Export: CSV / XLSX / HubSpot / Pipedrive
```

- **`prospector/signals/`** — one detector per signal family behind a shared `Signal` dataclass + `persist_signals`; a `registry` runs them all with injected free search/fetch functions, so detectors are unit-tested offline.
- **`prospector/icp.py`** — the ICP model and the transparent composite scorer above.
- **`prospector/enrich.py`** — contact discovery + the email-pattern waterfall + free MX validation.
- **`prospector/outreach.py` / `reply.py` / `campaign.py`** — signal-aware draft generation, reply classification, and the weekly optimization loop.
- **`prospector/web/`** — the self-serve Flask dashboard (onboarding, run progress, draft review, replies, exports).
- **`cli.py` / `prospector/mcp_server.py`** — the same engine over a CLI and an MCP stdio server.

## Tech stack

Python · Flask + waitress · SQLite · httpx · BeautifulSoup / lxml / trafilatura · feedparser · rapidfuzz · dnspython · openpyxl · DeepSeek (any OpenAI-compatible endpoint) · self-hosted SearXNG · MCP · pytest

## Running it locally

```bash
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
pip install -e .              # exposes the `prospector` command (same as `python cli.py`)

cp .env.example .env          # fill in DEEPSEEK_API_KEY etc. (all optional; it degrades)
prospector selfcheck          # verify config + external deps -> READY

# Fastest path: seed an ICP from a preset, run the pipeline, show leads
prospector quickstart

# ...or everything in the browser
prospector serve              # http://127.0.0.1:8101

# ...or drive it from Claude
prospector mcp                # stdio MCP server (find_leads / draft_outreach / query_analytics / ...)
```

Manual pipeline:

```bash
prospector init-icp "KSA RE developers" \
    --titles "CEO,Managing Director,Head of Development" \
    --industries "real estate,proptech" --geos "Saudi Arabia,Riyadh,GCC" \
    --keywords "off-plan,residential,giga-project" --competitors "ROSHN,Dar Al Arkan"

prospector run --icp "KSA RE developers" --limit 30    # detect -> score -> enrich -> draft
prospector leads --min-score 70
prospector export --xlsx data/leads.xlsx
```

Full command list: `quickstart`, `presets`, `init-icp` / `list-icp`, `detect`, `score`, `enrich`, `draft`, `run`, `replies ingest|list`, `export`, `notify`, `report` / `leads`, `serve`, `mcp`, `selfcheck`.

## Safety defaults

- **Draft-first** — nothing is sent automatically. Email sends only with `EMAIL_AUTOSEND=1` **and** SMTP configured; LinkedIn is never auto-sent.
- **Free-first** — paid fallbacks (Serper, Firecrawl, CRM) stay off unless keyed.
- **Never fabricates** — unknown data stays `n/a`; detectors skip results they can't ground to a real company.

## Data & privacy

This repository contains **code only** — no leads, no database, no scraped output. The `data/` directory (SQLite DB, search caches, session key) and `.env` are gitignored and never published. The example ICP presets name public real-estate developers, and all test fixtures use synthetic people with reserved example.com domains.

## Project layout

```
prospector/      the engine: signals/ · icp · enrich · outreach · reply · campaign · db · web/ · mcp_server
cli.py           the `prospector` command surface
tests/           561 tests (detectors, scorer, enrichment, replies, web)
deploy/          systemd unit + Caddy reverse-proxy snippet
scripts/         nightly cron pipeline
```

## License

MIT — see [LICENSE](LICENSE).