HN "Who's Hiring" Job Scanner

An automated job discovery and tracking pipeline that scrapes multiple sources, applies a multi-stage filter cascade, and exposes a 24-tool MCP server so Claude Desktop can analyze job fit against a resume — all without requiring an API key.

What It Does

Aggregates job postings from two sources (Hacker News "Who is hiring?" threads via Algolia API, and YC's Work at a Startup board via Playwright headless browser), runs them through a shared multi-stage filter pipeline, maintains a bounded tracking system with backlog promotion, and provides a full MCP (Model Context Protocol) tool interface for an LLM to analyze, rank, and manage a job search pipeline end-to-end.

Related MCP server: JobSpy MCP Server

Architecture

  Algolia API ──→ hn_jobs.py ──┐
                               ├──→ filters.py ──→ mcp_server.py ──→ Claude Desktop
  Playwright  ──→ waas.py   ──┘         │              │
                                    Shared filter    24 MCP tools
                                    pipeline         8 JSON stores
                                                     Description cache

Multi-Source Ingestion

• HN threads: Algolia search API → parallelized comment fetch (20-worker ThreadPoolExecutor) → HTML comment parsing to extract company, location, remote status, emails, job board URLs

• WAAS: Playwright headless browser with authenticated session → infinite-scroll scraping → structured field extraction (title, salary, batch, company size, seniority)

• Both sources feed into the same filters.py pipeline — keyword scoring, negative filters, seniority estimation, job type classification, and location detection share a single implementation

Filter Pipeline (filters.py)

Five-stage filter cascade, each with configurable behavior:

1. Weighted keyword scoring — Three categories scored by relevance (AI tooling: 3, Systems: 2, General AI+SWE: 1). Scoring is per-category, not per-keyword — multiple hits in one category don't stack. All matching uses compiled \b word-boundary regex, case-insensitive.

2. Negative keyword filter — Detects senior/management titles (staff, principal, director, VP) and high experience thresholds (10+, 15+ years). Matched posts aren't silently dropped — they go to a "Filtered Out" section so nothing is lost.

3. Seniority estimation — Infers seniority from job title keywords and description experience-year requirements. Maps to a level scale (intern → junior → mid → senior → staff+). Configurable max level — jobs above the threshold are filtered. Unknown seniority is never filtered (benefit of the doubt).

4. Job type classification — Classifies roles as coding (engineer, developer, SRE, etc.) vs non-coding (PM, designer, sales, recruiter). Engineering management is classified as non-coding. Unknown types are kept.

5. Location filter — Regex matching against known US and non-US cities/countries. Non-US, non-remote jobs are filtered. No detected location = kept (benefit of the doubt).

Bounded Job Tracking System

Eight JSON files managed atomically (read-modify-write) by the MCP server:

scan_waas ──→ [all new jobs] ──→ backlog_jobs.json (overflow, ranked by score)
                                        │
                                        ▼ (top N promoted)
                                 tracked_jobs.json (max N active, default 20)
                                        │
                          ┌──────────────┼──────────────┐
                          ▼              ▼              ▼
                   applied_jobs    dismissed_jobs  longshot_jobs
                   (permanent)     (validated)     (validated)
                          │
                    ┌─────┴─────┐
                    ▼           ▼
             rejected_jobs  accepted_jobs
             (outcome)      (outcome)

• Tracked — Bounded active pipeline (max N, configurable). Only the highest-scoring jobs from backlog fill these slots. Every mark_applied, mark_dismissed, or mark_longshot call frees a slot and auto-promotes the top backlog entry, returning the promoted job in the response to avoid redundant get_tracked_jobs calls.

• Backlog — Unbounded overflow, sorted by keyword score. Jobs that pass all filters but don't make the top N cut. Promotion happens automatically when tracked slots open.

• Applied — Permanent record. Not validated against WAAS (job might be filled but the application record matters).

• Dismissed / Longshot — Validated periodically against WAAS — dead listings are pruned automatically.

• Rejected / Accepted — Outcome tracking for applied jobs. mark_rejected and mark_accepted move jobs from applied to their respective stores. Both can be reverted to tracked via mark_open.

Description Caching

Full job descriptions are cached to disk during scanning (job_descriptions.json). get_job_details reads from cache first, falls back to live HTTP + BeautifulSoup parsing. Cache is auto-pruned — descriptions are removed when jobs leave all active stores (tracked, backlog, applied, longshot, rejected, accepted). Applied/rejected/accepted jobs retain their descriptions permanently.

MCP Server (24 Tools)

Stdio-transport MCP server for Claude Desktop. Claude Desktop acts as the LLM ranker, eliminating the need for an API key. The server owns all state — Claude never writes to JSON files directly, only through tool calls.

Scanning: scan_jobs, scan_waas, scan_all, get_job_details

• scan_waas returns only run metadata (counts, timing, active filters) — not job data. Job data flows through the tracking system.

• scan_all combines HN + WAAS with cross-source dedup by company name (case-insensitive, whitespace-stripped). HN takes priority.

• get_job_details serves from disk cache, falls back to live fetch with structured extraction (JSON-LD → Open Graph → meta tags → title tag).

Tracking: get_tracked_jobs, get_applied_jobs, get_dismissed_jobs, get_longshot_jobs, get_rejected_jobs, get_accepted_jobs, update_job_analysis, mark_applied, mark_dismissed, mark_longshot, mark_rejected, mark_accepted, mark_open, swap_role, validate_tracked_jobs, reset_tracking

• swap_role replaces a tracked job with an alternate role URL from the same company (e.g., a better-fit position discovered through other_roles), clearing stale analysis.

• validate_tracked_jobs checks open/dismissed/longshot jobs against WAAS, removes dead listings, and backfills from backlog.

• mark_applied/mark_dismissed/mark_longshot free a tracked slot and return the newly promoted backlog job inline, enabling a dismiss-and-analyze loop without re-fetching the full tracked list.

• mark_rejected/mark_accepted move applied jobs to outcome tracking. Both can be reverted via mark_open.

Config: get_resume, get_preferences, get_config, update_config, get_latest_results

• get_resume extracts text from a configured PDF via PyMuPDF.

• update_config writes to config.yaml for runtime filter/preference changes.

Daily Workflow (Automated via MCP Prompt)

1. validate_tracked_jobs — prune dead listings from tracked/dismissed/longshot, backfill from backlog

2. scan_waas — discover new jobs, auto-track top N by score, overflow to backlog

3. get_tracked_jobs → for each unanalyzed job: get_job_details → update_job_analysis (or mark_dismissed/mark_longshot with inline backfill loop)

4. Render tracked/applied/dismissed/longshot/rejected/accepted into a React artifact

Deduplication

• Within-source: seen_posts.json (HN comment IDs), seen_waas.json (WAAS job URLs). Auto-pruned after 6 months.

• Cross-source: scan_all deduplicates by company name (case-insensitive, stripped). HN takes priority since it has richer context.

• First-run backfill: Empty seen files trigger a 3-month HN backfill scan.

Tech Stack

Setup

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp config.yaml.example config.yaml  # edit with your preferences

Environment Variables (.env, gitignored)

Claude Desktop MCP Config

{
  "mcpServers": {
    "hn-jobs": {
      "type": "stdio",
      "command": "wsl",
      "args": [
        "bash", "-c",
        "set -a; source /path/to/.env; set +a; /path/to/.venv/bin/python3 /path/to/mcp_server.py"
      ]
    }
  }
}

CLI Usage

python3 hn_jobs.py --dry-run --no-rank   # HTML preview, no ranking
python3 hn_jobs.py --no-email             # Terminal output
python3 hn_jobs.py --dry-run --resume resume.pdf  # Rank against resume
python3 hn_jobs.py                        # Full scan + email delivery