# job-agent-mcp

Async job aggregator that fetches listings from multiple public job sources, dedupes and caches them in Redis, ranks them against a resume, and exposes the whole thing as an MCP server — usable from Claude Desktop, or from [`bot/`](bot/README.md), a custom floating desktop assistant with its own MCP client and Gemini-driven tool-calling loop.

## Stack

- Python 3.11+, FastAPI, `httpx` (async fetch)
- Redis via [Upstash](https://upstash.com) (free tier) — shared cache, TTL-based
- [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk) (`app/mcp_server.py`) — exposes `search_jobs`, `get_job_details`, `refresh_source`, `explain_fit`, `track_application`, `list_applications` as MCP tools over stdio
- Google Gemini (`gemini-flash-lite-latest`, free tier) — resume skill extraction + job scoring/reasoning, 2 batched calls per search
- SQLite — local application-status tracking

## Sources

| Source | Auth |
|---|---|
| RemoteOK | none (public JSON) |
| WeWorkRemotely | none (RSS) |
| HN "Who's Hiring" | none (Algolia HN API) |
| Adzuna | free API key, paginated (2×50/page = up to 100 results per search) |
| Jooble | free API key (instant, [jooble.org/api/about](https://jooble.org/api/about)) |

LinkedIn, Wellfound, and Naukri are intentionally excluded — none have a public self-serve API, and scraping them is a real ToS/legal risk (LinkedIn specifically has an active history of suing scrapers), not just a technicality. Jooble and Careerjet were added/considered instead as legitimate aggregator APIs with the same free, ToS-compliant access pattern as Adzuna.

**Resilience**: if any one source fails (bad/missing key, rate limit, outage), `app/search.py` logs a warning and returns results from the sources that worked rather than failing the whole search. Verified live — Jooble (missing key) and Adzuna (rate-limited from heavy testing) both failed simultaneously and the search still returned real results from RemoteOK + HN.

## Latency benchmark

The core engineering story of this project: same job search, three execution strategies.

| Strategy | Latency |
|---|---|
| Sequential fetch | **7.78s** (94 jobs, query="python", 2026-07-24) |
| Parallel (`asyncio.gather`) + dedupe | **3.09s** (94 → 89 jobs after dedupe) — **2.5x faster** |
| Parallel + Redis cache hit | **0.14s** — **~22x faster than cache miss, ~57x faster than sequential** |

The first two numbers use identical fetcher code (`app/fetchers/`) — the only variable is `await`-in-a-loop vs `asyncio.gather`, so the 2.5x is purely the effect of concurrency. The cache-hit number replaces all 4 network calls with 4 parallel Redis `GET`s (Upstash), keyed `jobs:{source}:{query}:v1`, TTL 30 min, with a `refresh=True` bypass for manual invalidation.

Reproduce: `python scripts/bench_sequential.py python` / `python scripts/bench_parallel.py python` / `python scripts/bench_cached.py python` — raw numbers saved to `benchmarks/*.json`.

## Resume ranking

Your resume PDF stays on your machine — `RESUME_PATH` in `.env` points to it, it is never copied into the repo. Before any text reaches Gemini, `app/resume.py` strips the name line, email, phone number, and LinkedIn/GitHub URLs (`redact_contact_info`). This matters specifically because Gemini's *free* API tier allows Google to use submitted content for training/human review — the paid tier doesn't, but redaction removes the actual PII either way at zero cost.

Pipeline: one Gemini call extracts a skills + **years_experience** + summary profile from the redacted resume (cached in Redis, 30-day TTL so an onboarded/edited profile doesn't silently get overwritten by a fresh unaudited re-extraction); one batched Gemini call scores every fetched job 0-100, given `{id, title, company, tags, description_snippet}` per job.

For a specific job, `explain_fit(job, profile)` (`app/rank.py`) goes deeper — using the job's full description (now captured by all 4 fetchers, `Job.description`) plus the resume profile, it returns a verdict, concrete strengths, concrete gaps, and actionable recommendations for closing them, grounded in the actual posting text rather than generic advice.

**Deterministic experience guard, tallied against the JD's actual requirement**: LLM scoring alone doesn't reliably enforce "needs more experience than the candidate has = low score" — the same "Lead AI Engineer" posting scored 100 in one run and 75 in another with identical inputs (Gemini isn't fully deterministic even at temperature=0, see below). The batched scoring call now also extracts `required_years` per job — its best estimate of the *minimum years that specific posting* expects, grounded in the actual title/description snippet, not a generic label. `app/rank.py`'s `_apply_seniority_guard` then compares this number against the candidate's resume-derived years (1-year margin for reasonable stretch roles) and force-caps the score at 35 if it's a clear mismatch, regardless of what the LLM's own score said. A title-keyword fallback (`senior/staff/lead/director/...` → assume ~5 years) only kicks in on the rare posting where the LLM couldn't extract any signal at all.

This is a real improvement over an earlier title-only version: verified live that `required_years` genuinely varies with the posting's actual content, not just its title — "Senior Staff Software Engineer" extracted 8, plain "Senior Software Engineer" postings mostly 5, a non-senior "Frontend Product Software Engineer" got 3 — and the final ranking correctly put senior/staff/CTO-level postings at the bottom (25-30) while non-senior roles took the top spots, for a profile with 2 years of experience.

Try it: `python scripts/rank_demo.py python`

**Known data-quality fixes**:
- HN "Who's Hiring" threads occasionally contain meta-comments (e.g. someone replying "can you change this to the following:" to edit an earlier post) that aren't real job postings. `app/fetchers/hn.py` now requires a `|`-delimited header (the convention nearly every real posting follows) before treating a comment as a job — filters this noise out before it ever reaches scoring.
- The batch scoring pass originally sent only `{title, company, tags}` per job — enough for topical overlap, but blind to seniority requirements (a "Lead, 6+ years" role could score 100 purely on keyword match). Fixed by including a `description_snippet` and an explicit `years_experience` field in the prompt, so a topically-perfect but experience-mismatched role now correctly scores lower with reasoning that says why.

**Editing your profile**: the CLI/`rank_demo.py` path reads `RESUME_PATH` from `.env` and auto-extracts on first use. The floating bot (`bot/`) has a proper onboarding flow instead — pick a PDF, review the extracted skills/experience/summary, edit anything that's off, then confirm — see [bot/README.md](bot/README.md#onboarding-flow).

## MCP server

Nine tools, all backed by the same `app/search.py` + `app/rank.py` + `app/resume.py` + `app/tracking.py` code the benchmarks and demos above already exercise — the MCP layer is a thin wrapper, not a separate implementation:

| Tool | Does |
|---|---|
| `search_jobs(query, location?, min_score?, country?)` | Deduped, cached, resume-ranked results across all 4 sources. `country` is an Adzuna region code ('us'/'in'/'gb'/...) the LLM infers from phrasing like "jobs in India" |
| `get_job_details(job_id)` | Full listing incl. description, for a job id from `search_jobs` |
| `refresh_source(source_name, query, country?)` | Force a live re-fetch for one source, bypassing the cache |
| `explain_fit(job_id)` | Verdict + specific strengths/gaps/recommendations for one job |
| `extract_resume_profile(path)` | Draft skills/years_experience/summary from a resume PDF, unsaved |
| `save_resume_profile(profile)` | Persist a (possibly user-edited) profile as the one used going forward |
| `get_saved_resume_profile()` | The currently saved profile, or null if never onboarded |
| `track_application(job_id, status, title?, company?, url?, location?)` | Upsert status: saved / applied / interviewing / rejected / offer. Snapshots the optional job fields permanently in SQLite (see below) |
| `list_applications(status?)` | Tracked applications, most recent first; optionally filtered to one status |

Every job returned by `search_jobs` is also cached individually (`job:{id}:v1`), so `get_job_details`/`explain_fit`/`track_application` can reference it afterward regardless of which query originally surfaced it.

**Applications outlive the cache**: `track_application` originally stored only `{job_id, status, timestamp}`, relying on the 30-min job cache for title/company/url — fine for a same-session lookup, broken for tracking meant to last weeks. `app/tracking.py` now snapshots title/company/url/location into SQLite at the moment you track a job; a later status-only update (e.g. applied → interviewing) preserves that snapshot via `COALESCE` rather than blanking it out.

**Verify without Claude Desktop**: `python scripts/test_mcp_client.py python` spawns the server over stdio (the same transport Claude Desktop uses) and drives the full search → details → explain_fit → track_application → list_applications loop with a real MCP client.

**Connect to Claude Desktop**: add to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "job-agent-mcp": {
      "command": "C:\\path\\to\\job-agent-mcp\\.venv\\Scripts\\job-agent-mcp.exe"
    }
  }
}
```

(Installing the project with `pip install -e .` registers `job-agent-mcp` as a console script — `app/mcp_server.py:main`. Alternatively use `command: python`, `args: ["-m", "app.mcp_server"]`, `cwd: <project path>`.)

## Floating desktop bot

[`bot/`](bot/README.md) is a second MCP client for this same server — an always-on-top Electron + React chat widget with its own Gemini-driven tool-calling loop, for daily use without opening Claude Desktop. See its README for setup and how the client-side tool-calling loop works.

## Setup

```bash
pip install -e ".[dev]"
cp .env.example .env   # fill in ADZUNA_*, JOOBLE_API_KEY, UPSTASH_REDIS_URL, GEMINI_API_KEY, RESUME_PATH
uvicorn app.main:app --reload
```

Check `http://localhost:8000/health` — `redis_connected: true` confirms your Upstash credentials are wired up correctly.

## Status

- [x] Phase 0 — project scaffold
- [x] Phase 1 — sequential fetchers per source
- [x] Phase 2 — parallel fetch + fuzzy dedupe
- [x] Phase 3 — Redis caching layer
- [x] Phase 4 — resume-ranking
- [x] Phase 5 — MCP server
- [x] Bonus — floating desktop bot (`bot/`), a second MCP client with its own Gemini tool-calling loop
- [ ] Phase 6 — deploy + write-up