Skip to main content
Glama
marat-kabirov

mcp_server

AI Job Application Tracker Agent

45+ applications a week, and for every posting you need to decide in a couple of minutes: is this even worth applying to? This agent reads a job posting, extracts its requirements, compares them against a resume profile, produces a fit-score with an explanation (go/maybe/no_go), and logs the decision to a tracker — automating the exact manual work you'd otherwise do by hand for every posting.

Full technical scope — MCP tool contracts, data schemas, the reasoning behind architectural decisions — lives in SPEC.md.

Architecture

                 python cli.py score --file posting.txt
                 python cli.py stats / update <id> <outcome>
                              │
                              ▼
              ┌─────────────────────────────────────┐
              │   LangGraph agent (agent/graph.py)    │
              │                                        │
              │  ingest → extract ─┬→ retrieve_profile │
              │                    │        │          │
              │                    │        ▼          │
              │              (low confidence) score     │
              │                    │        │          │
              │                    ▼        ▼          │
              │                clarify    decide → log │
              └───────────────────┬────────────────────┘
                                  │ calls MCP tools along the way
                                  ▼
        ┌───────────────────────────────────────────────────────┐
        │              MCP Server (mcp_server/)                  │
        │                                                         │
        │  extract_job_requirements   Groq, structured output      │
        │  load_resume_profile        reads resume_profile.json    │
        │  compute_fit_score          hard-filters + skill match    │
        │                             + final LLM scoring           │
        │  log_application / update_application_outcome /           │
        │  query_tracker_stats        SQLite (SQLAlchemy)            │
        └───────────────────────────┬───────────────────────────┘
                                     ▼
              data/tracker.db (SQLite) + data/resume_profile.json

The clarify node is the branch for vague postings: if extract_job_requirements comes back with an empty required_stack, the graph doesn't try to guess a score — it asks for clarification and writes nothing to the tracker.

Related MCP server: job-agent-mcp

MCP tools

Tool

What it does

extract_job_requirements

Groq (ChatGroq.with_structured_output) parses raw posting text into JobRequirements — no manual regex/JSON parsing. Doesn't crash on vague text: honestly returns whatever it could extract.

load_resume_profile

Reads and validates data/resume_profile.json via ResumeProfile.

compute_fit_score

Two-tier scoring: deterministic hard-filters + two-phase skill matching + final LLM scoring. Details below.

log_application

Writes an application (posting + fit-score + decision) to SQLite, returns an application_id (uuid4).

update_application_outcome

Records the real-world outcome (rejected / interview / ghosted / offer) once a response comes in, via cli.py update.

query_tracker_stats

Tracker-wide aggregates: number of applications, average score by outcome, average time-to-rejection, share of go-decisions that still got rejected.

How the score is computed

Scoring isn't a single "gut feeling" LLM call — it's three layers, each with its own area of responsibility.

1. Hard-filters (deterministic, no LLM). These check facts, not judgment calls: a seniority gap (a rough read of candidate level from years_experience_total vs. the posting's stated seniority_level), remote mismatch, location mismatch, language mismatch, a conflict on work_authorization (a heuristic over phrases like "no visa sponsorship" in the posting's raw text). If even one filter fires, the final score is forced below the maybe threshold — the LLM cannot override a fact: if the resume plainly states a student visa with no sponsorship, and the posting plainly states "no sponsorship," the verdict physically cannot become go, no matter what the model decides in the next step.

2. Two-phase skill matching. Phase 1 is an exact-string, case-insensitive match between the posting's required_stack and the resume's skill names — cheap and reliable wherever the wording literally lines up. Phase 2 handles what didn't match (e.g. the posting asks for "Next.js", and the resume has "React" and "TypeScript" but not the literal string "Next.js"): one extra Groq call with structured output decides whether the candidate's resume covers that need semantically, with a short justification. This isn't a hand-maintained alias table (which doesn't generalize to new posting vocabulary) — it's an LLM decision for exactly the case plain string matching can't resolve. Every semantic match is flagged separately and stated explicitly in the final explanation, so it never reads as if it were a direct match: "covered via adjacent experience, not direct use of the tool."

3. Final LLM scoring on top of the facts. The model receives the already-computed matched_skills / missing_skills / hard_fail_reasons / semantic_match_notes plus the full posting text, evaluates nice-to-have stack and implicit signals (red flags like "rockstar ninja," unrealistic requirements), and produces score (0–100), explanation, and confidence. verdict is derived from score by a threshold in code (GO_THRESHOLD=70, MAYBE_THRESHOLD=50), never asked from the LLM directly — the threshold stays the single place where go/maybe/no_go gets decided.

Eval

Fit-scoring is validated against a golden dataset of 4 real job postings from this job search (GeneralMind, Manex AI, Lucid Labs, WaveSix), each with a verdict the author assigned manually before any agent existed. The agent currently reaches 3/4 (75%) verdict accuracy against that dataset, with hard-filters enforced deterministically so the LLM can't override an explicit disqualifying fact.

Groq isn't strictly bit-for-bit deterministic even at temperature=0, so scores on borderline cases can shift slightly between runs.

23 pytest tests cover the pipeline: deterministic hard-filters and exact/semantic skill matching (mocked, no network), the SQLite tracker (temp DB, not data/tracker.db), LangGraph routing (mocked tools), and one live end-to-end pass against Groq (network-gated behind the requires_groq pytest marker).

pytest              # full suite
python evals/run_eval.py   # run the golden dataset with a report

What I'd do next

  • Email parser for automatic outcome updates — right now update_application_outcome is called manually via the CLI when a response arrives; an inbox parser would close that gap automatically.

  • Browser extension for one-click logging — straight from a job posting's page, without copy-pasting text into a file.

  • URL ingestion — fetch and clean a posting directly from a link instead of pasting text in.

  • Grow the golden dataset — more real postings, covering a wider range of stacks and seniority levels.

Setup

python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
cp .env.example .env   # fill in GROQ_API_KEY (console.groq.com/keys, free tier)

Fill data/resume_profile.json with your real skills (instructions in data/RESUME_PROFILE_INSTRUCTIONS.md) — the years/proficiency fields directly determine how sensible the fit-score is: empty/zero values mean the seniority hard-filter will fire almost every time.

Default model is openai/gpt-oss-120b (see mcp_server/tools/_llm.py); override via GROQ_MODEL in .env if Groq's model lineup changes.

Usage

python -m mcp_server.server                 # start the MCP server (stdio)

python cli.py score --file posting.txt      # run a posting through the agent
python cli.py stats [--since YYYY-MM-DD]    # tracker-wide aggregates
python cli.py update <application_id> <outcome>   # rejected/interview/ghosted/offer

Repository structure

job-tracker-agent/
├── SPEC.md                        # architecture, tool contracts, day-by-day plan
├── README.md                      # this file
├── cli.py                         # Typer CLI: score / stats / update
├── agent/
│   ├── state.py                   # AgentState (TypedDict)
│   └── graph.py                   # graph nodes: ingest/extract/retrieve_profile/score/decide/log/clarify
├── mcp_server/
│   ├── server.py                  # registers the MCP tools
│   ├── schemas.py                 # Pydantic models (JobRequirements, ResumeProfile, FitScoreResult, ...)
│   └── tools/
│       ├── _llm.py                # shared helper — ChatGroq client
│       ├── extraction.py          # posting parsing via Groq (structured output)
│       ├── scoring.py             # load_resume_profile, compute_fit_score (hard-filters + skill match + LLM)
│       └── tracker.py             # log_application, update_application_outcome, query_tracker_stats
├── data/
│   ├── resume_profile.json        # fill in with your own data
│   ├── RESUME_PROFILE_INSTRUCTIONS.md
│   └── tracker.db                 # created automatically, not in git
├── evals/
│   ├── golden_cases.yaml          # 4 real job postings with manual verdict/score
│   └── run_eval.py                # run + report
└── tests/
    ├── conftest.py                # requires_groq marker, load_dotenv
    ├── test_scoring_deterministic.py   # hard-filters, exact/semantic skill match
    ├── test_pipeline_llm.py            # end-to-end raw_text -> JobRequirements -> FitScoreResult (real Groq)
    ├── test_tracker.py                 # SQLite tracker against a temp database
    └── test_graph_smoke.py             # LangGraph routing (tools mocked)

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables to interact with job application workflows through MCP, allowing users to find jobs, generate non-trivial applications with proof-maps, and build offline dashboards, all without auto-submitting.
    Apache 2.0
  • A
    license
    Not graded
    quality
    C
    maintenance
    MCP server that aggregates and deduplicates job listings from multiple public sources, ranks them against a user's resume, and exposes tools for searching, viewing details, explaining fit, and tracking applications.
    MIT
  • F
    license
    A
    quality
    C
    maintenance
    Enables searching job listings, tracking applications, managing resumes, and tailoring resumes to job posts, all locally via MCP.
    6
    20
    -
  • A
    license
    A
    quality
    C
    maintenance
    A privacy-first MCP server for locally managing job, fellowship, and graduate-school applications. It offers tools for tracking application status, analyzing role fit, generating LaTeX CV/cover letters, interview prep, and discovering public jobs from ATS APIs.
    8
    1
    MIT