Helios HR Assistant

An agentic HR assistant for the fictional Helios Dynamics: policy retrieval, synthetic employee records, deterministic compliance checks, and reversible mock actions, all accessed through a real Model Context Protocol (MCP) server.

Application: https://helios-hr-assistant-wz3c.onrender.com

Readiness: https://helios-hr-assistant-wz3c.onrender.com/health

Design and evaluation: design-and-evaluation.md

Deployment: deployed.md

AI tooling disclosure: ai-tooling.md

/health.build_sha reports the commit currently deployed.

The free instance sleeps when idle, so the first visit can take about a minute while it wakes. See Cold starts and expected waits.

At a glance

• 16 policy documents in Markdown, HTML, PDF and TXT, indexed into 221 chunks.

• 12 MCP tools covering policy search, employee records, compliance rules and mock actions.

• 30 evaluation cases and two canonical demo workflows.

Live evaluation scores recorded in September are historical: the older citation metric measured retrieved-label presence rather than factual support. See the design report for the current scoring method and results.

Related MCP server: HR Assist MCP Server

Setup

Use Python 3.12+. From the repository directory:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -e ".[dev]"
Copy-Item .env.example .env

On macOS/Linux, activate with source .venv/bin/activate and copy the template using cp .env.example .env.

Set GROQ_API_KEY in .env or the host environment. A Groq tool-capable model is required for multi-step workflows. Gemini's configured compatibility endpoint is only available for non-tool calls; it is not a replacement for Groq here. Never commit API keys, demo codes, or approval secrets.

Direct dependencies are pinned in pyproject.toml; transitive dependencies are not locked.

Run

uvicorn app.main:app --reload --port 8000

Open http://127.0.0.1:8000. The app starts and discovers the MCP subprocess. All dates are evaluated against the clearly labelled 2026-09-12 synthetic snapshot, not today's date. This prevents the example answers drifting.

/health checks configuration, not whether a provider key is valid or has quota. Only a live chat can establish that. The UI shows cached results explicitly; their trace/timings belong to the original run.

API quota protection

• Unit/integration tests use scripted models and prohibit real chat-completion calls. Indexing and retrieval evaluation use the local embedding model, not an LLM API.

• Live evaluation and deployed-task scripts require explicit --allow-live.

• The app admits one chat at a time and rejects duplicate/concurrent submissions. The default hourly chat allowance is 12 requests.

• Provider requests are capped at 8 per turn, 60 per hour and 100 per day by default. The deployed service allows 16 per turn and 16 agent steps, because demo runs on a fallback model can need more steps. A request the provider refuses before doing any work (rate limit, oversized request, retired model) costs no tokens and is refunded; anything it may have processed counts.

• A rate-limited model is passed over for the next model in the chain, which has its own per-minute budget. A call waits only when every model is rate-limited, for the soonest to recover (honouring Retry-After, at most 60 s per call). Longer limits are reported, not waited out. Retired models are skipped.

• Read-only identical requests within the same session/context may reuse a labelled 10-minute cache. Mutating workflows and ticket listings are not cached. Run live (skip cache) bypasses the cache and uses quota.

• Set DEMO_ACCESS_CODE to protect a public deployment from casual quota consumption. Share it privately; users enter it in the Demo access code field, not the chat.

• Set LLM_ENABLED=false to disable all real model calls. Chat explicitly reports that calls are disabled; it does not fake an answer.

These are process-local application limits, not provider quota readings. They reset on restart and cannot protect against other applications using the same API key. Provider daily limits, and per-minute limits that outlast the wait, can still interrupt a task.

The two demo workflows

The canonical prompts live in agent/demo_tasks.py and are reused by the UI and deployed verifier:

1. International remote work: Maya's 42-day Portugal request; read her rolling usage, check compliance, read POL-INTL-001 §2 The 30-Day Rule and POL-REMOTE-001 §6 Temporary Work Outside the Home Country, and explain, citing both, that 12 + 42 exceeds 30 by 24 days.

2. PTO and ticket preview: Jonas's three-day request; check 13 available hours against 24 requested and 6 days' notice against 10, read POL-PTO-001 §3.1 Notice Requirements and §3.3 Insufficient Balance, show a mock ticket preview, and explain the manager/skip-level exception routes. Select Create mock ticket on the preview to create the in-memory ticket and show its ID.

Every policy claim, including alternatives, needs evidence. A preview is not a completed write, and a tool call alone is not a correct answer.

Offline checks

pytest -q
ruff check .
python scripts/validate_mock_data.py
python scripts/check_rules.py
python scripts/healthcheck.py
python -m rag.ingest.build_index --check
python -m evaluation.run_retrieval_eval

The committed index avoids embedding downloads at application startup. When ingestion code or policies change, rebuild locally:

python -m rag.ingest.build_index

The embedding weights must already be cached for fully offline execution.

Live evaluation (opt-in)

These commands make real LLM calls and consume quota:

python -m evaluation.run_eval --allow-live --case C01
python -m evaluation.run_eval --allow-live --resume
python scripts/verify_deployed_tasks.py --allow-live --task pto --confirm-mock-actions

The last command performs a real agent turn and then explicitly authorizes the mock ticket. It saves the complete response and confirmation result in evidence/ with each run's wall time, rate-limit waits, model calls and tokens. --task and --case can be repeated, and the run stops at the first provider or HTTP error rather than spending quota on requests that would fail the same way. Current evaluation records contain scorer version, configuration/code fingerprint and full traces. A checkpoint from another revision is rejected.

Architecture

Browser -> FastAPI -> agent orchestrator -> LLM provider
                         |
                      MCP client
                         | stdio JSON-RPC
                      MCP server
                      /        \
             local RAG index   synthetic HR records + rule engine

The folder is called mcp_server/, not mcp/, to avoid shadowing the SDK package. All employee data is synthetic; tickets and email drafts disappear on restart.

Deployment

The live service runs on Render Free as one Docker service with one worker. The web app, agent, local vector index, synthetic data and stdio MCP server share the container; only the LLM is external. GitHub Actions checks the code, the real MCP transport, app startup and the container, then deploys that exact commit. render.yaml disables Render's own auto-deploy, so only revisions that pass CI are deployed.

To deploy your own copy:

1. Fork the repository and create a Render Blueprint from render.yaml. Render builds the Dockerfile, which bakes the embedding model into the image, and the app listens on Render's $PORT.

2. In the Render dashboard, set GROQ_API_KEY. GEMINI_API_KEY and DEMO_ACCESS_CODE are optional; render.yaml supplies the other settings.

3. In GitHub, add the secret RENDER_API_KEY and the repository variables RENDER_SERVICE_ID (the srv-… ID) and APP_URL (the service URL).

4. Push to main. CI runs lint, tests, the MCP smoke test and the container check, triggers the Render deploy for that commit, and waits until /health reports it.

To run the production image locally with the settings from .env:

docker build -t helios-hr .
docker run --rm -p 8000:8000 --env-file .env helios-hr

deployed.md lists every environment variable and the demo steps.

Cold starts and expected waits

Render's free plan puts the service to sleep after about 15 minutes without traffic, and the next request has to wake it. The measured cold start was 52.5 seconds, so allow about a minute. Before a demo, open /healthz and wait for it to respond; neither health endpoint calls an LLM.

A multi-step agent task is slower than a single answer even when the service is warm, because each model step waits for the provider's per-minute token budget. On 3 October 2026, eight back-to-back requests to the deployed service took a median of 15 seconds and at most 47 seconds. Sent about a minute apart, the two demo tasks on build c88530a took 7 and 5 seconds; design-and-evaluation.md has the details.