Enterprise AI Chat Agent

An internal assistant: multi-turn chat, plus filesystem tools served by a real MCP server that is connected to only when a file is actually involved.

• Architecture and reasoning → WRITEUP.md

• Conventions, guardrails, decision log → CLAUDE.md

• Verbatim prompts used to build it → PROMPTS.md

What you need to install, and in what order

The project runs on its own. Traces and behavioural evals are two separate optional layers, each with its own prerequisite — neither is needed to see the agent work. Pick a level and stop there.

Levels 2 and 3 are independent — you can do either, both, or neither. Nothing in level 1 breaks if Docker or Node are absent.

Related MCP server: Files MCP Server

Level 1 — Core (required)

Requirements

Exact versions are in requirements.txt; ranges are in pyproject.toml.

Why mcp is held below 2.0.0: MCP Python SDK v2 renamed FastMCP to MCPServer and removed the mcp.server.fastmcp module. langchain-mcp-adapters 0.3.2 declares the same upper bound, so the two cannot be installed together above it. The pin makes an implicit constraint explicit; it is not a downgrade.

Run

# 0. clone
git clone https://github.com/Semmargl/enterprise-ai-chat-agent.git
cd enterprise-ai-chat-agent

# 1. environment
python3 -m venv .venv                 # or: uv venv --python 3.14
source .venv/bin/activate             # Windows: .venv\Scripts\activate
pip install -r requirements.txt       # or: uv pip install -r requirements.txt

# 2. secrets
cp .env.example .env
# then open .env and put your OpenRouter key in OPENROUTER_API_KEY

# 3. sample files to play with (the working folder starts empty)
mkdir -p workspace && cp samples/* workspace/

# 4. check the wiring before spending a token
pytest -q                             # expect: 44 passed, <1s, no network needed

# 5. start
python -m src.main

Checkpoint: the agent greets you according to the time of day and answers a question. If step 4 printed 44 passed, the sandbox, memory window and tool visibility are all verified without a single API call.

The agent holds a normal conversation and connects to the file service the first time you ask for a file.

python -m src.main --thread report   # a separate, named conversation

Quit with exit. Start it again with the same --thread and the conversation continues: state is checkpointed to SQLite, not held in memory.

Two ways to run the MCP server

Note the two "defaults": the code falls back to 0 when the variable is unset, while .env.example ships MCP_AUTOSTART=1 so a clean clone runs without a second terminal. Copy the template and you get autostart; deploy without a .env and you get the production shape.

Either way nothing is connected until you ask for a file.

Things worth trying

What to look for in the logs

Logs are JSON, one line per event, on stderr. Keys correlation_id (one user turn) and thread_id (one conversation) tie them together.

var/audit.jsonl is the audit trail: one line per tool call with the outcome and duration. It records paths, sizes and hashes — never file contents, never secrets.

It sits in var/, not in workspace/, on purpose: the agent's own file tools reach every path under the sandbox root, so an audit trail kept there could be edited by the process it records. The checkpoint database (var/checkpoints.sqlite) is out for the same reason — it is the agent's memory, not its workspace. Startup refuses a configuration that puts either one back inside the sandbox.

Level 2 — Traces in Phoenix (optional)

Prerequisite: Docker. Skip this whole section if you do not have it — the agent does not need it. Default is OTEL_EXPORTER=none, so a clean clone runs with no collector at all.

Instrumentation is OpenTelemetry with OpenInference semantics, so spans describe LLM and tool calls rather than generic HTTP work. The exporter is an environment variable, not a code path — swapping Phoenix for any other OTLP backend is one variable, not a refactor.

# 1. start Phoenix (first run pulls the image, ~1-2 min)
docker run -d --name phoenix -p 6006:6006 -p 4317:4317 arizephoenix/phoenix

# 2. wait for it, then confirm it answers
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:6006    # expect: 200

# 3. run the agent pointed at it — two turns, one plain and one about a file
OTEL_EXPORTER=otlp OTEL_ENDPOINT=http://127.0.0.1:6006/v1/traces \
  python -m src.main --thread traces

Checkpoint: open http://localhost:6006. Two turns produce two traces. Open the file one — the tool call is nested inside the create_agent loop, under the model call. That nesting is the point: it is the agent's control flow, not a flat list of HTTP requests.

# when finished
docker stop phoenix && docker rm phoenix

Do not use OTEL_EXPORTER=console for anything but local debugging: those spans print the whole prompt and every tool result — that is, file contents — which the audit trail deliberately never stores.

Level 3 — Behavioural evaluations (optional)

Prerequisite: Node 18+. Skip if absent; pytest already covers everything deterministic.

pytest covers what can be checked without a network: path confinement, window arithmetic, tool visibility. What it cannot cover is whether the system still behaves after a real nine-turn conversation with a real model. Those three cases live in promptfooconfig.yaml and run against the actual agent — not against the bare model — through scripts/promptfoo_provider.py.

# 1. install
npm i -g promptfoo@latest

# 2. the venv must be active and .env filled in — the provider spawns the real agent
source .venv/bin/activate

# 3. run
NODE_NO_WARNINGS=1 PROMPTFOO_DISABLE_TELEMETRY=1 PROMPTFOO_DISABLE_UPDATE=1 \
  promptfoo eval -o results.json; echo "EXIT=$?"

# 4. browse the results (optional)
promptfoo view

Checkpoint: EXIT=0, and results.json contains "successes": 3, "failures": 0, "errors": 0. A full run takes 40–60 seconds — it is nine real turns plus two single-turn conversations.

Two things that look like failures and are not. The progress bar can appear stuck at 0% | 0/3: Node writes a warning into the same terminal line and overwrites it. Judge by EXIT and the JSON, not by the bar. And assertions.cached > 0 on a repeat run is the grader caching its own calls — the agent conversations themselves are never cached. Add --no-cache for a fully cold run.

This costs tokens. Three cases, eleven turns total, plus an LLM grader for the rubric assertions — all on the same OPENROUTER_API_KEY from your .env.

Configuration

Every setting is documented in .env.example. The ones that change behaviour most: MCP_AUTOSTART, SANDBOX_ROOT, MAX_FILE_BYTES, HISTORY_WINDOW_MESSAGES, SUMMARIZATION_TRIGGER_MESSAGES, OTEL_EXPORTER.

.env is in .gitignore from the first commit and has never been committed:

git log --all --full-history -- .env    # returns nothing