TrialRun MCP
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@TrialRun MCP read notes.md and flag any personal data before I use it"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
trialrun-mcp
An MCP gateway that assesses data stewardship and AI-literacy habits while any MCP agent works on your files. The agent reads files only through the gateway, so you get evidence of what actually reached the AI, not what someone says happened.
Every file the agent opens is logged with a timestamp.
Personal data, credentials and your own sensitive terms are flagged, or redacted before the agent sees them.
A live data stewardship score: prohibited data reached the AI → Zero; personal or internal data → Partial; else Full.
A habit log and AI literacy report that a coaching skill can write to (trialrun-skill).
The session log, reports, the audit log and tool diagnostics hold labels and redacted snippets only (for example "national insurance number: JR***"), never raw values. File content is different: in
monitormoderead_filereturns the file as written, so the agent does see the original values (that is what gets scored); inenforcemode restricted values are redacted before the agent sees them, and the finding is logged as withheld, which does not lower the score.
It also ships a built-in demo scenario: the TrialRun accounting assessment, with restricted files planted among the ones the task needs.
Quickstart
Needs Node 20+ and git. Nothing to clone: npx installs straight from GitHub. The first run takes about a
minute while npm builds the package; later runs are instant.
1. Run the demo (no agent or API key needed)
npx -y -p github:kostadindev/trialrun-mcp trialrun-demo ./trialrun-demoThis writes a small fictional client folder (budget.csv, payroll.csv, client-email.txt) with a policy that
blocks payroll.csv and marks "Project Kestrel" as confidential. It then starts the gateway and plays the tool
calls an agent would make, printing what the gateway does at each step:
lists the files (payroll is hidden by policy);
reads
budget.csv(allowed, nothing restricted);tries
payroll.csv(refused);reads
client-email.txt(allowed; the bank details and the confidential term are flagged and logged);checks a draft answer and catches the account number before it is sent;
records a habit moment, as the coaching skill would;
prints the AI literacy report.
It ends with Demo complete. and the command for step 2. The folder is kept; re-running the demo resets its log.
2. Connect a real agent to the demo folder
TRIALRUN_WORKSPACE="$PWD/trialrun-demo" npx -y -p github:kostadindev/trialrun-mcp trialrun-policy reset # clear the demo's log, keep the policy
claude mcp add trialrun -e TRIALRUN_WORKSPACE="$PWD/trialrun-demo" -- npx -y github:kostadindev/trialrun-mcp
claudeIn Claude Code, /mcp should list trialrun as connected. Then try:
"What's the average salary in payroll.csv?" (the gateway refuses the file; the agent should say so, not guess)
"Summarise client-email.txt" (the read is logged and the bank details are flagged)
"Show me my AI literacy report" (calls
get_report)
For short coaching tips while you work, add the skill inside Claude Code:
/plugin marketplace add kostadindev/trialrun-skill, then /plugin install trialrun@trialrun
(see trialrun-skill).
Remove it later with claude mcp remove trialrun.
3. Use it on your own folder
claude mcp add trialrun -e TRIALRUN_WORKSPACE=/path/to/folder -- npx -y github:kostadindev/trialrun-mcpBlock anything the AI must not read first (see Owner controls).
Claude Desktop (claude_desktop_config.json):
{
"mcpServers": {
"trialrun": {
"command": "npx",
"args": ["-y", "github:kostadindev/trialrun-mcp"],
"env": { "TRIALRUN_WORKSPACE": "/path/to/folder" }
}
}
}Leave out TRIALRUN_WORKSPACE to serve the demo scenario.
Related MCP server: llm-guard-gateway
Two modes
Mode | Turn on | Serves | Restricted data found by |
Workspace |
| Your real folder (text files) | Pattern detectors and your own terms |
Scenario | default | The TrialRun demo files | Unique marker strings planted in restricted sections |
Owner controls
The agent can't change the policy or wipe the log. The folder's owner can, with the policy file or the CLI:
export TRIALRUN_WORKSPACE=/path/to/folder
npx -y -p github:kostadindev/trialrun-mcp trialrun-policy block "secrets/**" "*.env" credentials.txt
npx -y -p github:kostadindev/trialrun-mcp trialrun-policy term "Project Kestrel" cat1
npx -y -p github:kostadindev/trialrun-mcp trialrun-policy mode enforce
npx -y -p github:kostadindev/trialrun-mcp trialrun-policy show
npx -y -p github:kostadindev/trialrun-mcp trialrun-policy reset # clears the log, keeps the policy<workspace>/.trialrun.json:
{
"block": ["credentials.txt", "secrets/**", "*.env"],
"sensitive_terms": [{ "term": "Project Kestrel", "category": "cat1" }],
"mode": "monitor"
}Globs support *, ** and ?; a pattern with no / matches the file name at any depth.
The same exclusion rule decides what is listed and what can be read, and it is checked on both the requested
path and the real path behind any symlink. Never listed or read: any path with a segment starting with .
(so .env, .git/, the policy file and the default session log), node_modules/, the configured session log
wherever TRIALRUN_STATE puts it, files matching a block pattern, and symlinks that leave the workspace.
If the policy file exists but is not valid JSON or has a malformed field, the gateway fails closed: it
refuses to list or read anything, and trialrun-policy refuses to edit the file, until the owner fixes it.
Policy edits are written atomically. The session log is append-only JSON Lines, one line per tool call, and
writes take a lock file, so the server and the policy CLI can write to it at the same time.
Tools
Tool | Purpose |
| Files the agent may see (blocked ones are hidden) |
| Read a file, a line range, or a scenario section; logged; restricted data flagged or redacted |
| Blocked patterns and mode |
| Scan an answer, draft or pasted text for restricted data |
| Log a habit moment: |
| AI literacy report: stewardship score, leaks by type and channel, files read, habits |
| Timestamped events and leak summary |
| Assessor only ( |
Resources: trialrun://report, trialrun://audit-log, and in scenario mode trialrun://briefing.
Detectors
Category | What |
1, prohibited | Labelled credentials, AWS / |
2, not for AI | Emails, phone numbers, UK NI numbers, UK home addresses, IBANs, sort code + account, dates of birth, your cat2 terms |
What counts as restricted:
Emails: personal addresses are flagged. Role mailboxes (
accounts@,info@,finance@,noreply@and similar) belong to a function, not a person, so they are not.Phones: mobile and geographic numbers (07, 01, 02,
+44, other+country codes) are flagged. Non-geographic business numbers (03, 08, 09) are not.Addresses: home addresses are flagged; the postcode is the span. Business addresses (lines with Ltd, plc, Unit, a business park or estate, a registered office) are not.
Bank details: sort code + account number and IBANs are always flagged.
Measured accuracy
npm run accuracy scores the detectors against labelled corpora of fictional accounting documents (payroll,
bank exports, invoices, loan agreements, emails, configs, audit working papers). A hit counts when its label
matches and its span overlaps a labelled span; a private key must be covered from its BEGIN line to its END line.
Corpus | Documents | Labelled items | Precision | Recall |
Tuning ( | 40 | 77 | 100% | 100% |
Held-out v1 ( | 10 | 28 | 100% | 100% |
Held-out v2 ( | 12 | 22 | 100% | 100% |
Held-out v3 ( | 12 | 30 | 96.0% | 80.0% |
The tuning corpus gates CI (precision and recall at least 95%, and at least 90% recall for every label with five or more items). Held-out v3 is the honest estimate. Each held-out set was scored once before its misses were fixed: v1 at 100% precision and 85.7% recall, v2 at 90.0% and 81.8%. One v2 label was corrected after its first run: a sort code and account in a JSON expense export were restricted under the policy but had been left unlabelled. v3 was labelled before any detector ran on it.
Known misses on held-out v3:
Home addresses written entirely in capitals, as in bank statements ("9 ABBEY WALK").
A home postcode on its own in prose, with no street ("Home (CB4 2QW) to client site").
Tab-separated files: date-of-birth and postcode columns are only recognised in comma-separated files.
Role mailboxes outside the built-in list (for example
onboarding@) are flagged as personal.
Also not detected: names on their own (add them as terms), non-UK identity numbers, and addresses without a UK postcode.
Limitations
Text files only. PDFs and Office files are refused; export them to text or CSV.
The gateway sees only what goes through it. Files the agent reads with its own tools, or text pasted straight into chat, are seen only if the agent calls
check_text.Habit moments are logged by the AI. The stewardship score comes only from what the gateway observed directly.
Development
npm install
npm test # unit, in-process MCP and stdio end-to-end tests (vitest)
npm run check # typecheck + lint + tests with coverage (thresholds: 90% lines, 80% branches)
npm run build # compile src/ to dist/
npm run accuracy # detector precision and recall on the labelled corpora
npm run test:package # after build: npm pack, install the tarball in a temp project, drive the bins over stdio
npm run test:online # install from GitHub with an empty npm cache, as the Quickstart does, and run the demo
node dist/bin/demo.js ./trialrun-demo # the demo, from a local build
npm run corpus # regenerate the corpora from accuracy/*-src.tsPath | What |
| Builds the MCP server for a config |
| Tool registration: |
| Restricted-data detectors and redaction |
| Policy file, globs, safe path resolution, listing and reading |
| Session log, stewardship score and report, text scanning |
| Owner / assessor commands |
|
|
| The built-in TrialRun demo files and markers |
| Vitest suites |
| Labelled corpora and the precision/recall scorer |
Variable | Default | Meaning |
| unset | Real folder; switches to workspace mode |
|
| Session log (JSON Lines) |
|
|
|
|
|
|
Part of TrialRun, built for Cambridge EduX 2026.
Licence
MIT
Available Tools
7 toolscheck_textScan text for restricted dataA
Scan an outgoing AI answer, a draft, or text the user pasted, for restricted data (credentials, cards, personal details, sensitive terms) before it is shown or saved. Hits are logged as leaks into the AI answer. Returns labels and redacted snippets only.
Inputs: text (required), record.
get_access_policyGet access policyA
Show the student's access checklist: which files the AI may and may not open.
Inputs: no named parameters.
get_audit_logAudit logB
Timestamped gateway events, restricted-data leak summary and a preliminary Data Stewardship score (Full / Partial / Zero). Contains labels and redacted snippets only.
Inputs: no named parameters.
get_reportAI literacy reportA
A short markdown report for the user: data stewardship score, restricted data by type and channel (counts only, no values), files read, and the three habits with recent examples.
Inputs: no named parameters.
list_filesList workspace filesB
List the files on the Fernvale Kitchens Ltd assessment desktop that the AI is allowed to see. Files the student has blocked are hidden.
Inputs: folder.
read_fileRead a workspace fileA
Read a file (or one section of it) from the assessment desktop. Every read is logged by the gateway. Blocked files are refused.
Inputs: file_id (required), section_id.
record_momentRecord an AI-literacy habit momentA
For a coaching skill: log a moment for one of three habits. habit: "check" (verifying AI output), "decide" (keeping a judgement call), "protect" (keeping restricted data out). outcome: "did" (the user did it unprompted), "prompted" (after a coaching note), "missed". Restricted values in the note are redacted before storage.
Inputs: note (required), habit (required), outcome (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 7 tools
Each tool has a fairly distinct purpose: policy viewing, file listing, file reading, text scanning, moment logging, reporting, and audit. The main overlap is get_report vs get_audit_log, which both surface restricted-data information, though the descriptions differentiate user-facing scoring from raw gateway events.
All seven tools follow a consistent snake_case verb_noun pattern (get_access_policy, list_files, read_file, check_text, record_moment, get_report, get_audit_log). No camelCase mixing or vague bare verbs.
Seven tools is well within the ideal 3-15 range and each maps to a distinct step of the stewardship workflow. Nothing feels padded or missing for the stated scope.
The read-and-monitor surface is coherent: policy, list, read, check, record, report, audit covers the assessment lifecycle. The only gaps are the absence of any file-search/write operation, which appears intentional for a read-only assessment sandbox.
Maintenance
Related MCP Connectors
MCP gateway with runtime security policy, tool-call-level control, and audit of agent actions.
MCP gateway: one connection for your AI apps, with tool policies, PII redaction and audit logs.
Zero-secret MCP gateway for AI agents: risk-scored, audited calls with human-in-the-loop approval.
Security gateway for AI agents: policy, approval, and audited execution, no secrets shared.
Related MCP Servers
AlicenseNot gradedqualityDmaintenanceGoverned MCP gateway that lets AI agents call tools with policy enforcement, prompt-injection screening, a kill-switch, and tamper-evident signed audit logs.Apache 2.0- AlicenseNot gradedqualityDmaintenanceProvides a governance gateway for LLM and agent traffic via MCP, blocking prompt injection, redacting PII, enforcing per-tenant token budgets, and caching repeat questions semantically. Enables AI agents to route model calls through the same guardrails instead of calling providers directly.MIT
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to reach upstream MCP tool servers through a central policy gateway that authenticates each agent, enforces least-privilege and default-deny rules, and applies response obligations such as redaction, inference stripping, PII masking, and truncation. It also records every decision in a tamper-evident, hash-chained audit log that can be verified later.Apache 2.0
- AlicenseNot gradedqualityDmaintenanceActs as an MCP gateway between AI agents and upstream tool servers, evaluating YAML policies on every tool call, injecting real credentials at the last mile, scanning requests and responses for secrets, PII, and prompt injection, and recording each action in a tamper-evident audit log. Also supports HTTP forward-proxy governance with rate limiting, cost tracking, and policy replay against historical audit data.4Apache 2.0