Agent Lab

Educational application for inspecting the technical flow of AI agents over enterprise data. The project shows contracts, protocols, tool calls, structured results, and sanitized traces.

Status

Stages 1 through 11 — Foundation, MCP, Agent Runtime, Auth/RBAC, A2A, evaluations, cloud, CI/CD, deterministic contracts, resilience, and semantic robustness:

• React + Vite frontend;

• Node.js + Express backend;

• PostgreSQL schema and migrations;

• admin and viewer application users;

• deterministic calendar periods;

• TraceEvent technical contract;

• unit tests and visual shell of the Agent Lab;

• official MCP server over stdio transport;

• seven read-only MCP tools with structured responses;

• parameterized PostgreSQL queries using a least-privilege role.

• Ollama with qwen3:8b for local inference and tool calling with no token cost;

• OpenAI Responses API kept as an optional provider;

• local MCP Client with tool discovery and execution;

• grounded orchestrator and POST /api/agent/query endpoint;

• query interface with response and real technical trace.

• authentication with opaque sessions persisted in PostgreSQL;

• HttpOnly, SameSite=Strict cookie and configurable expiration;

• RBAC authorization with admin and viewer profiles;

• audited user creation and transactional deletion of HR data.

• two agents publishing A2A 1.0 Agent Cards;

• HR → Finance delegation via JSON-RPC SendMessage;

• financial task with lifecycle and structured Artifact;

• absence loss report queried via MCP.

• behavior evaluation suite with deterministic assertions;

• reference cases, empty result, and PostgreSQL freshness;

• isolated dynamic fixture with guaranteed cleanup and residual verification.

• budgeted timeout, bounded transient retry, and circuit breaker per hot instance;

• safe degradation that preserves the grounded answerPayload if only the narrative fails;

• resilience evaluation with controlled fault injection.

• interpretation of neutral, informal, and Rioplatense Spanish via LLM semantic proposal;

• backend validation of capability, schema, period, polarity, and limits before MCP;

• typed clarification and unsupported-query decisions without PostgreSQL access;

• versioned linguistic benchmark with before/after baseline and stability across repeated runs.

The application is deployed with a static frontend on Render, a serverless backend on Vercel, and PostgreSQL on Neon. GitHub Actions applies quality gates and smoke tests against production.

Related MCP server: Employee Management MCP Server

Structure

frontend/   React, inspector técnico y system index
backend/    API, dominio, migraciones, acceso PostgreSQL y trazas

Requirements

• Node.js 22 or higher.

• npm 10 or higher.

• Ollama 0.32 or higher and the local model qwen3:8b.

• Cloud PostgreSQL with three separate credentials when the provider allows it.

Installation

npm --prefix backend install
npm --prefix frontend install

Copy backend/.env.example to backend/.env and fill in the PostgreSQL provider URLs. Never use the owner credential in DATABASE_READONLY_URL or DATABASE_ADMIN_URL.

The default provider is local Ollama:

LLM_PROVIDER=ollama
OLLAMA_HOST=http://127.0.0.1:11434
OLLAMA_MODEL=qwen3:8b

Install Ollama and download the model once with ollama pull qwen3:8b. Inference uses local CPU/GPU and storage, with no paid API consumption.

OpenAI remains available as an alternative by setting LLM_PROVIDER=openai, OPENAI_API_KEY, and OPENAI_MODEL. The key belongs only in backend/.env, which is ignored by Git. It must never be sent to the frontend or included in traces.

Database

1. Create the PostgreSQL database in the cloud provider.

2. Create or configure the roles following backend/ops/database-roles.example.sql.

3. Configure the environment variables.

4. Run:

npm --prefix backend run db:migrate
npm --prefix backend run db:seed
npm --prefix backend run db:smoke
npm --prefix backend run db:verify-permissions

db:smoke uses exclusively DATABASE_READONLY_URL and queries the hr_late_arrivals view with date parameters.

The seed requires SEED_ADMIN_PASSWORD and SEED_VIEWER_PASSWORD, both at least 12 characters long. There are no default passwords in the repository.

Development

In two terminals:

npm run dev:backend
npm run dev:frontend

• Frontend: http://localhost:5173

• Backend: http://localhost:3000

Verification without a cloud database

npm test
npm run typecheck
npm run build

The calendar tests verify:

• current month from day 1 through today inclusive;

• full previous calendar month;

• leap-year February;

• last 30 calendar days.

Time intervals

The interface speaks of inclusive dates, but internally half-open intervals are used:

startInclusive <= timestamp < endExclusive

This avoids relying on 23:59:59 and correctly preserves PostgreSQL precision.

Traceability

The frontend shows technical events with:

• event name;

• technology;

• component;

• category;

• concepts;

• sanitized input and output;

• duration and status.

Credentials, session tokens, and internal model reasoning will not be shown.

MCP Server

The server uses the official Model Context Protocol SDK to expose:

• count_employees: counts total, active, and inactive employees;

• list_employees: lists the full directory with employee ID, name, department, and status;

• find_employee: searches by name or employee number;

• summarize_employee_delays: aggregates a person's historical lateness by name or employee ID;

• list_late_arrivals: lists late arrivals by period and optional employee;

• list_employees_without_late_arrivals: computes in PostgreSQL which active employees had no late arrivals during the period;

• list_absences: lists absences by period and optional employee.

The tools declare readOnlyHint, validate input and output with Zod, and return both textual content and structuredContent. Results include source, query date, applied period, total count, and truncation flag. Each call queries PostgreSQL again; there is no caching at this stage.

To start the local MCP server:

npm run mcp:server

To verify discovery, real calls, seeded data, and empty results:

npm run mcp:smoke

stdout is reserved for the MCP protocol; operational errors go to stderr and the client response is sanitized.

Agent Runtime

Implemented flow:

React → POST /api/agent/query → HrAgentOrchestrator
      → Ollama local + qwen3:8b (tool calling)
      → MCP Client → MCP Server → PostgreSQL
      → tool result → Ollama → respuesta + TraceEvent[]

The MCP Client discovers the available tools, but the router delivers to the model a single definition of the execution-controlled allowlist. Function calling schemas are strict and parallel calls are disabled so each execution is simple to inspect.

grounded: true means the orchestrator verified a call to an approved tool and received structuredContent before requesting the final response. It does not mean there is a mathematical guarantee about every token produced by the model; that quality must be measured with evaluations.

The system prompt requires that enterprise data come exclusively from the tools, that empty results be reported explicitly, and that received content be treated as data, not as instructions.

Deterministic integration test, without consuming API:

npm run agent:smoke

Real test with local Ollama, MCP, and Neon:

npm run agent:smoke:ollama

Real test with Groq, MCP, and Neon:

npm run agent:smoke:groq

Optional test with OpenAI, MCP, and Neon:

npm run agent:smoke:openai

Endpoint:

POST /api/agent/query
Content-Type: application/json

{"question":"¿Qué empleados llegaron tarde durante el último mes?"}

The response contains answer, model, grounded, toolsUsed, and a sequence of sanitized technical events. It does not include tokens, credentials, or internal reasoning.

Semantic robustness, validated routing, and deterministic presentation

The LLM receives the seven controlled capabilities and proposes exactly one decision. The backend does not trust that proposal: it validates the allowlist, Zod schema, user-expressed period, polarity, and business limits before allowing an MCP call:

LLM propone → backend valida → MCP ejecuta → PostgreSQL → payload determinista

In addition to the seven MCP tools, planning has two internal decisions that never reach MCP: request_clarification and reject_unsupported_query. The first returns agent_clarification_required when a period is missing or ambiguity exists; the second returns unsupported_agent_query when the request requires a nonexistent capability, ranking, frequency, or filter. The public catalog is available at GET /api/agent/capabilities and also appears in the System index.

Informal expressions are interpreted by meaning. For example, arriving, coming in, showing up, clocking in, or punching in late can refer to late_arrivals; "no lateness" and "always on time" are interpreted as zero events only within an explicit period. Expressions such as "a bunch", "a ton", "always", or "constantly" are never turned into invented quantities.

After MCP execution, AnswerPresentation validates the structuredContent using a discriminated Zod union. The API returns two separate surfaces:

• presentation: typed, deterministic answerPayload rendered by a specific React component;

• answer: grounded narrative generated by the LLM, visible in a secondary panel identified as non-deterministic.

Quantities, tables, dates, empty states, and source metadata are displayed from presentation; they are not extracted from the model's text. Stage 11 does not modify this Stage 9 contract. The trace incorporates llm.semantic_proposal.completed, agent.semantic_decision.validated, and presentation.payload.validated to separate proposal, validation, and deterministic representation.

The negated query is implemented as a set difference: active employees minus employees with at least one late arrival within the period. PostgreSQL executes that semantics via NOT EXISTS; the LLM does not compute the complement. If Groq returns an empty final response or attempts a second tool call during completion, the adapter performs a single textual retry with the same grounded data. The llm.grounded_response.completed event reports recovery=not_required, the type of retry applied, or the fallback to the deterministic presentation.

LLM provider resilience

Stage 10 contains external failures through four explicit mechanisms:

The public endpoint GET /api/resilience exposes the policy and the sanitized circuit state, never credentials. The agent is reused within each warm instance so that the circuit breaker keeps state between requests. On Vercel, each instance has its own circuit; coordinating it globally would require a distributed store, which is not justified for this lab.

If the initial planning fails, there is still no MCP call or grounded data, and the API returns a typed error (llm_timeout, llm_rate_limited, llm_provider_unavailable, or llm_circuit_open). If only the final drafting fails, the API responds successfully with the deterministic table and emits llm.grounded_response.degraded.

Authentication and authorization

Credentials are validated against bcrypt hashes in app_users. Upon authentication, the backend creates a random token, stores only its SHA-256 hash in app_sessions, and delivers the token via an HttpOnly cookie. The frontend never accesses the token.

Session duration is configured with SESSION_TTL_HOURS=8.

Application permissions:

• viewer: can query the agent and view the technical index;

• admin: includes query capabilities, user creation, and controlled deletion of operational data.

Administrative deletion does not run DROP DATABASE. It deletes attendance_records, employees, and departments within a single transaction; it preserves schema, users, sessions, and audit_events. It requires the literal confirmation DELETE HR DATA and records the result in the audit log.

The PostgreSQL role app_admin does not have DROP, CREATE DATABASE, superuser privileges, or neon_superuser membership. This separation demonstrates that application RBAC and database privileges are distinct layers.

Real test of both users and the full session cycle:

npm run auth:smoke

Agents and A2A

The project implements A2A Protocol 1.0 with the official SDK @a2a-js/sdk:

• HR Grounding Agent: employee and attendance queries grounded via MCP;

• Absence Finance Agent: deterministic economic analysis of absences.

Agent Cards:

/.well-known/agent-card.json
/.well-known/hr-agent-card.json
/.well-known/finance-agent-card.json

Financial flow:

Usuario → HR Agent / A2A Client
        → descubre Finance Agent Card
        → JSON-RPC SendMessage
        → Finance Agent Task: submitted → working
        → MCP list_absences → PostgreSQL
        → calculadora determinista
        → A2A Artifact application/json
        → Task completed → reporte + TraceEvent[]

A2A endpoints use an internal random bearer token. The Agent Card describes the security scheme but never contains the credential.

The base does not contain salaries. That is why the report requires explicit parameters: currency, daily cost, replacement premium, and productivity impact. The formula is:

días × costo diario × (1 + prima de reemplazo + impacto de productividad)

The LLM does not perform arithmetic. A deterministic TypeScript function calculates amounts rounded to two decimal places. If MCP indicates that the result was truncated, the agent rejects the calculation to avoid an incomplete report.

The implementation uses in-memory A2A tasks because the flow is short and synchronous. For multiple instances or long-running tasks, the TaskStore should be migrated to persistent storage.

Real test of Agent Card, A2A, MCP, and Neon:

npm run a2a:smoke

Agent evaluations

Unit tests validate functions and contracts with controlled dependencies. The evaluation suite measures the behavior of the full real agent with the configured model, MCP, and PostgreSQL.

Implemented cases:

• employee-count: verifies that a quantity question routes exclusively to count_employees;

• employee-directory: verifies that a name request routes to list_employees and retrieves the directory;

• employee-delay-summary: verifies deterministic aggregation of Bruno's delays via summarize_employee_delays;

• employees-without-late-arrivals: verifies negation routing, set difference, and the expected result EMP-003;

• known-late-arrivals: compares tool, grounding, and quantity against the seeded dataset;

• unknown-employee: requires an empty PostgreSQL result and an explicit response without invented data;

• source-of-truth-freshness: inserts a unique temporary employee and a late arrival, queries the newly created record, and verifies that the agent observes the update.

• finalization-failure-degradation: injects a controlled failure after MCP and verifies that the PostgreSQL payload remains available.

• semantic-robustness-v1: runs 80 neutral, formal, informal, Rioplatense, and edge-case phrasings; measures intent, decision, arguments, temporality, ambiguity, and stability.

The dynamic fixture uses the administrative role only during preparation and cleanup. The agent query continues using the read-only role. A finally block deletes by UUID and exact employee number; afterward, an additional query verifies that no EVAL-% employees or agent-evaluation-sourced attendance records remain.

Real execution with the configured LLM provider, MCP, and Neon:

npm run evals:run
npm run resilience:eval
npm run semantic:eval
npm run semantic:stability

semantic:eval runs the 80 cases once, and semantic:stability repeats the critical set five times. Both report validDecisionRate, intentRecognitionRate, toolSelectionRate, argumentExtractionRate, temporalInterpretationRate, exactOutcomeRate, stabilityRate, ambiguityPassRate, and unsupportedPassRate. By default they wait 30 seconds between calls to respect Groq's free token budget and separate provider rate limits. The baseline Stage 10 is kept in backend/evals/baselines/ and Stage 11 results in backend/evals/results/.

The other commands return reproducible JSON with passRate, duration, expected/actual checks, and grounded evidence per case. They exit with a non-zero code if an evaluation fails or if any temporary fixture remains. The reference case assumes the demo seed is present.

Cloud deployment

The repository keeps frontend/ and backend/ separate, with two deployment surfaces:

• agent-lab-ignac: Vite frontend as a Render Static Site;

• agent-lab-api-ignac: Express backend as a Vercel Function with Fluid Compute.

Production URLs:

• application: https://agent-lab-ignac.onrender.com;

• API: https://agent-lab-api-ignac.vercel.app;

• direct health check: https://agent-lab-api-ignac.vercel.app/api/health.

render.yaml only manages the frontend and rewrites /api/* to https://agent-lab-api-ignac.vercel.app. For the browser, authentication and cookies remain under the frontend origin; the session token stays HttpOnly and is never exposed to React.

backend/vercel.json declares Express, a maximum of 300 seconds, and the gru1 region (São Paulo), close to the Neon database. Vercel detects the lazy handler exported by src/app.ts; the application and its pools initialize upon receiving the first request of an instance. src/server.ts keeps app.listen() for local development.

The MCP transport is selected via MCP_TRANSPORT:

• stdio: local development; the client starts an independent MCP process;

• in_process: Vercel; client and MCP server connect with an in-memory transport pair, without losing the protocol, contracts, validation, or tool discovery.

In local development, LLM_PROVIDER=ollama keeps qwen3:8b. On Vercel, LLM_PROVIDER=groq uses openai/gpt-oss-20b, which supports function calling. The Groq adapter forces at least one tool and returns its result to the model to produce the grounded response.

Required production variables in the Vercel project:

NODE_ENV=production
FRONTEND_ORIGIN=https://agent-lab-ignac.onrender.com
APP_TIMEZONE=America/Argentina/Buenos_Aires
SESSION_TTL_HOURS=8
PUBLIC_BASE_URL=https://agent-lab-api-ignac.vercel.app
MCP_TRANSPORT=in_process
LLM_PROVIDER=groq
GROQ_MODEL=openai/gpt-oss-20b
GROQ_API_KEY=<secret>
LLM_TIMEOUT_MS=12000
LLM_TRANSIENT_RETRIES=1
LLM_CIRCUIT_FAILURE_THRESHOLD=3
LLM_CIRCUIT_RESET_MS=30000
DATABASE_READONLY_URL=<secret>
DATABASE_ADMIN_URL=<secret>
A2A_INTERNAL_TOKEN=<secret-aleatorio-de-32-o-mas-caracteres>

The public backend adds Helmet headers, rate limits, explicit error handling, and GET /api/health. The in-memory limits are demonstrative and operate per warm instance; a distributed production application would use a shared store. PostgreSQL keeps users, sessions, and data, so the serverless filesystem remains disposable.

CI/CD and quality gates

Each push to main and each pull request runs .github/workflows/ci.yml. Backend and frontend are validated in independent, reproducible jobs on Node.js 22:

checkout → npm ci → typecheck → build → tests → audit de dependencias productivas

npm ci installs exactly the tree locked by each package-lock.json. The jobs only have read permission on the repository, have a timeout, and cancel previous runs of the same branch. No production credential is delivered to the CI workflow.

Vercel is connected to the repository with backend/ as the Root Directory; an accepted commit on main generates the serverless deployment. Render keeps the static frontend from frontend/. This separation distinguishes two controls:

• pre-runtime quality gate: types, compilation, tests, and audit;

• post-deployment smoke test: the real deployed public HTTP contract.

.github/workflows/production-smoke.yml listens for successful deployment states and also allows manual execution. scripts/production-smoke.mjs checks:

• direct Vercel backend health;

• /api/system contract and current stage;

• public /api/resilience contract;

• /api/* proxy served under the Render origin;

• availability of the frontend HTML document.

Local execution of the same production contract:

node scripts/production-smoke.mjs