A
licenseNot graded
qualityD
maintenanceEnables AI assistants to query workplace incident data using RAG, providing search, analysis, and corrective action plans.
1
MIT
foreman
by YashNirwan
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@foremanpull up the verified alerts from this morning"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Foreman
Agentic vision AI for warehouse safety review, built on NVIDIA NIM.
Point it at warehouse footage and it returns a short queue of verified safety alerts, each one carrying the evidence it was confirmed on and the standard it maps to, plus a searchable index of everything else it saw. It runs on NVIDIA-hosted NIM endpoints, so it reproduces on a laptop with an API key and no GPU.
Live demo · analysis of a real run, no setup required. Evidence clips are local-only there, for the licensing reason in DATA.md.
The review console: verified alerts with the evidence each was confirmed on, natural language search across every window, and an audit view of everything suppressed.
The problem this is actually solving
Ask a vision language model "is there a safety hazard in this footage" and it will almost always say yes. On the labelled set in this repo, a single-pass VLM flags a hazard in 77% of windows that contain none. That is not a system a safety supervisor will use twice, and no amount of prompt rewriting fixes it, because the prompt is not the problem. Showing a model a safety camera and asking about hazards hands it an overwhelming prior that hazards are present.
Foreman treats that as an architecture problem rather than a wording problem. The perception pass is allowed to over-report, and a second model that can only remove things re-opens the same frames and decides whether the evidence actually meets the bar for the class that was claimed.
Measured on 49 hand-labelled windows:
arm | precision | recall | F1 | windows falsely alerting |
| 0.15 | 0.80 | 0.25 | 30/39 (77%) |
| 0.18 | 0.80 | 0.30 | 28/39 (72%) |
| 0.14 | 0.30 | 0.19 | 12/39 (31%) |
| 0.35 | 0.60 | 0.44 | 10/39 (26%) |
Over three repeats the verifier arm runs P=0.34 [0.31–0.38], R=0.50 [0.40–0.60],
F1=0.40 [0.36–0.46]. Precision roughly doubles; recall pays for it. Full numbers,
including a per-class false-positive breakdown, are in
evals/RESULTS.md. Reproduce with python evals/run_eval.py.
Three findings worth more than the headline number
1. Confidence thresholds barely work. Going from every candidate to only those the model rated 0.80+ moved precision 0.15 → 0.18. A VLM's self-reported confidence is close to useless as a filter here: it is confident about the things it is wrong about.
2. Text-only verification is worse than no verification. The llm_verifier arm
reads the perception pass's written evidence and adjudicates on that. It collapses to
F1 0.19 — below the naive baseline. The written evidence is too thin to clear the
bar the verifier is applying, so it rejects nearly everything, real hazards included.
The verifier has to look at the pixels. This one surprised me and it is the reason the
architecture has a VLM in the second position rather than a cheaper LLM.
3. VLMs read on-screen text as though it were the world. The first window of the first video is a title card: black screen, white words, "PASSING IN FRONT OF A FORKLIFT". The perception model reported a pedestrian walking in front of a moving forklift at 0.95 confidence. The verifier confirmed it as high severity. Both models had turned printed words into an observed event.
Real deployments are full of this — burned-in timestamps, camera labels, training
overlays. The fix is a scene gate: the perception pass declares whether it is
looking at real camera footage at all, in the same call, at no extra cost. It removes
an entire class of confident nonsense and is worth +0.07 precision on top of the
verifier (vlm_verifier 0.35 vs vlm_verifier_nogate 0.28).
Every suppressed detection stays visible and auditable. A safety tool that cannot show what it threw away is one nobody should trust.
Related MCP server: ppe-compliance-mcp
Architecture
flowchart LR
A[video] --> B[chunker<br/>8s windows, 4 frames]
B --> C[perception<br/>Nemotron Nano VL 12B]
C -->|is_camera_footage=false| X[gated: slides,<br/>title cards, presenters]
C -->|hazard candidates| D[verifier<br/>Nemotron Nano Omni 30B<br/>re-opens the frames]
D -->|rejected + reason| X
D -->|confirmed| E[alert queue<br/>+ evidence clip]
C --> F[embeddings<br/>Llama Nemotron Embed VL]
F --> G[semantic search<br/>over every window]
E --> H[MCP server]
G --> H
H --> I[Claude Code /<br/>any agent]stage | model | why this one |
perception |
| smallest model that reliably holds the JSON contract across every chunk; ~2s per window |
verification |
| reasoning VLM, different family and size from the proposer, so a confirmation is a genuine second look rather than a model agreeing with itself |
search |
| vision-language retriever, so queries land in the space the perception pass described |
On Cosmos. nvidia/cosmos-reason2-8b is the natural perception model here — Cosmos
Reason is post-trained for physical-world spatial and temporal reasoning rather than
generic captioning, which is exactly the axis a hazard call turns on. It is not
provisioned on the free build.nvidia.com tier (account-scoped 404 as of Aug 2026), so
this runs on Nemotron VL. VLM_PRIMARY in src/foreman/nim.py
is one line; swap it on a self-hosted NIM and the eval harness will report the
comparison honestly instead of my asserting it.
Design decisions I would defend in review
8-second windows, 4 sampled frames. Shorter and the model cannot see motion, so a pedestrian standing still and a pedestrian walking into a travel path look identical. Longer and an alert cannot be localised to a moment a reviewer can act on.
The taxonomy is upstream of the model. hazards.py
defines five classes from OSHA 29 CFR 1910.178 and the struck-by categories that
dominate warehouse injury data — chosen from what an inspector needs, not from what a
VLM happens to describe well. The perception prompt, the verifier's reject criteria,
the label schema and the UI filters all read from that one definition, so a class
cannot drift between the model proposing it and the harness grading it.
The proposer and the adjudicator get different information. The verifier sees each class's known false positive; the perception pass does not. Telling the proposer what not to say suppresses real detections along with phantom ones. Telling the adjudicator what to watch for raises precision without touching recall.
Tools are shaped like questions, not like the call graph.
mcp_server.py exposes search_timeline, list_alerts,
explain_alert, list_rejected_detections, shift_summary, hazard_taxonomy —
the questions a supervisor actually asks. Agent surfaces that mirror internal module
boundaries are how they become unusable.
Quickstart
git clone https://github.com/YashNirwan/foreman && cd foreman
python3 -m venv .venv && ./.venv/bin/pip install -e .
cp .env.example .env # add a free key from https://build.nvidia.com
python scripts/fetch_data.py # pull sample footage locally
./.venv/bin/python -m foreman.pipeline data/raw_yt/MqvOjo62BHQ.mp4
./.venv/bin/streamlit run app.py # review consoleRequires ffmpeg and yt-dlp on PATH (brew install ffmpeg yt-dlp).
A 172-second video: 21 windows, 26 candidates, 11 confirmed alerts, 98 seconds wall clock on a laptop against the free tier, 47 NIM calls.
Drive it from an agent
claude mcp add foreman -- /abs/path/to/foreman/.venv/bin/python -m foreman.mcp_serverThen ask in plain language: "what were the high-severity alerts in that shift, and show me what the verifier threw out."
Reproduce the eval
python evals/run_eval.py --perceive # perception + all 7 arms
python evals/variance.py --repeats 3 # run-to-run spread on the headline armsWhat this is not
Honest limits, because a work sample that oversells is worse than one that is small:
The eval is small. 49 windows, 10 positive events, one annotator (me). Class balance is skewed — 7 of 10 positives are
pedestrian_in_path, andblocked_egresshas no positive examples at all, so its numbers mean nothing yet. Treat the precision figures as a directional result on one labelled set, not a benchmark.Recall drops meaningfully. 0.80 → 0.50. On a real floor you would tune the verifier's bar per class, and you would almost certainly accept lower precision on
pedestrian_in_paththan onmissing_ppe, because the cost of missing them differs.Labels are judgement calls. The guideline is written at the top of
ground_truth.json's generator and ambiguous windows are labelled negative. A second annotator would move these numbers.Sampled frames are not video. Four frames across eight seconds miss fast events. The production answer is a CV pipeline (DeepStream, or the tracking stage in the Metropolis VSS blueprint) triggering VLM review on clips it has already localised, rather than a VLM scanning uniformly.
No post-training. Everything here is prompted and orchestrated, not fine-tuned. The obvious next step is distilling the verifier's confirmed/rejected decisions into a small VLM and checking whether it holds precision at a fraction of the cost.
Not a substitute for a trained safety professional, and the eval footage is training material rather than live operational CCTV.
Where this goes at scale
The shape here is deliberately the shape of NVIDIA's VSS blueprint: ingest, VLM perception, retrieval over descriptions, verified alerts. Foreman is the laptop-scale version, and each stage has a real replacement — flat numpy index → a vector DB, uniform sampling → DeepStream-triggered clips, hosted NIM → self-hosted NIM containers, single video → multi-stream. What survives the swap is the part I would argue matters most and that the blueprint leaves to you: the verification stage and the eval harness that proves it earns its place.
Data and licensing
Code is MIT. No video is redistributed in this repo. The eval ships as labels and
derived model output keyed on chunk ids; scripts/fetch_data.py rebuilds the local
footage from public sources for analysis. See DATA.md.
Built by Yash Nirwan · github.com/YashNirwan
Available Tools
7 toolsexplain_alertA
Return the full evidence chain behind one alert.
Gives the proposing model's claim, its confidence, the verifier's reasoning and the standard cited, so a supervisor can judge the alert rather than take it on trust.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| alert_id | Yes | ||
| video_id | No |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
A3.8/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden for behavioral disclosure. It transparently describes the return content (proposing model's claim, confidence, verifier's reasoning, standard cited) and the purpose, providing significant context beyond a simple 'explain.' It does not discuss error behavior or the optional video_id, but the core behavior is well disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exactly two sentences. The first sentence provides a clear, front-loaded summary, and the second adds essential detail about the evidence chain and its purpose. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema, so return values are covered, and the core purpose and content are well described. However, the optional video_id parameter is not explained, and there is no mention of error behavior or constraints. Given the simplicity of the tool, the description is adequate but has clear gaps in parameter semantics and edge-case behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description mentions neither alert_id nor video_id. The phrase 'behind one alert' implies alert_id is the identifier, but the optional video_id is completely unexplained. The description fails to add meaning beyond the bare type titles in the schema, so it does not compensate for the lack of parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Return the full evidence chain behind one alert.' It clearly distinguishes itself from sibling listing tools by focusing on a single alert and elaborates on the components it returns (claim, confidence, verifier's reasoning, standard cited). This is explicit and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states the intended use case: 'so a supervisor can judge the alert rather than take it on trust,' giving clear context for when to use this tool. It does not explicitly name alternatives or exclusions, which would warrant a 5, but the purpose is clear enough to guide tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hazard_taxonomyA
The hazard classes this system detects, with evidence bars and known failure modes.
Read this before interpreting alerts: each class has a specific evidence standard, and knowing it is the difference between reading an alert and trusting it.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
A3.9/5.0
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that each class has a specific evidence standard and known failure modes, adding meaningful context beyond the tool name. However, it does not explicitly state that the tool is read-only, what the return object looks like, or any side effects. For a simple taxonomy lookup, this is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first defines the content, the second explains why to read it. Both sentences earn their place, with no filler or redundancy. It is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with an output schema, the description covers the essential purpose and use context. It doesn't enumerate the taxonomy contents, but the output schema is responsible for that. The description is sufficiently complete for an agent to decide when and how to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and schema description coverage is 100% (vacuously). The baseline for 0 params is 4, and the description adds no parameter-specific semantics because none exist. No further explanation is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description identifies the resource as 'the hazard classes this system detects, with evidence bars and known failure modes,' establishing a clear topic. It lacks an explicit verb like 'retrieve' or 'list,' but the noun phrasing is unambiguous and distinguishes it from sibling tools that list videos or alerts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides explicit when-to-use guidance: 'Read this before interpreting alerts.' This gives clear context that the tool is a precursor to alerts, though it does not name alternatives or explicitly say when not to use it. The context is sufficient for an agent to select it appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_alertsA
List verified safety alerts for a processed video.
Args: video_id: which processed video; omit if only one has been processed. hazard_type: optional filter, one of the taxonomy keys. min_severity: low, medium or high. Defaults to low, meaning everything.
Returns confirmed alerts only. Rejected detections are available via
list_rejected_detections if you want to audit what the verifier removed.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| video_id | No | ||
| hazard_type | No | ||
| min_severity | No | low |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
A4.8/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that only confirmed alerts are returned, that there is a verification process, and that rejected items are excluded. It also explains the default behavior of `min_severity` and the optional nature of `video_id`. It does not discuss side effects, but as a list operation, the read-only nature is reasonably clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded with the core purpose, then structured into parameter explanations. Every sentence provides value: the purpose, per-arg guidance, and the pointer to an alternative tool. No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (3 optional params, output schema exists), the description is complete. It explains what is returned, how to narrow results, and how to access a related data set. The existence of an output schema means the description need not detail the return structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description compensates fully. Each parameter is given added meaning: `video_id` is explained with the 'omit if only one' rule, `hazard_type` is defined as a taxonomy key, and `min_severity` is given allowed values, a default, and what the default means. This goes well beyond the schema's type/default information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('List') and resource ('verified safety alerts for a processed video'), and it clearly differentiates from siblings by emphasizing 'confirmed alerts only' and pointing to `list_rejected_detections` for the rejected set. This is a precise, non-tautological purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'omit if only one has been processed' for `video_id`, and an alternative tool for a different need: 'Rejected detections are available via `list_rejected_detections` if you want to audit what the verifier removed.' This clearly indicates when to use this tool vs. an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_processed_videosA
List videos that have been processed and are available to query.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
A4/5.0
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds the behavioral condition 'processed and available to query', which is useful context. There are no annotations, so the description carries the full burden, but it doesn't disclose details like pagination, ordering, or access requirements. The output schema may cover return structure, so this is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence (10 words) that is front-loaded with the main action and resource. There is no unnecessary repetition or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, zero-parameter list operation with an output schema, the description fully states what the tool lists and the selection criterion. It doesn't mention edge cases or require excessive detail; the output schema handles return value descriptions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there is no parameter semantics to convey. The schema coverage is trivially 100%, and the baseline for 0 parameters is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('videos') with a specific qualifier ('processed and available to query'). This distinguishes it from sibling tools like list_rejected_detections and list_alerts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a usage context: use when you need a list of processed videos that can be queried. However, it provides no explicit guidance on when to prefer this over siblings like search_timeline or list_alerts, nor any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_rejected_detectionsA
List detections the verifier rejected, with the evidence it found missing.
Useful for auditing whether the system is suppressing real hazards. A rejection that looks wrong here is the fastest signal that the taxonomy's evidence bar for that class is set incorrectly.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| video_id | No |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
A3.5/5.0
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the burden of behavioral disclosure. It does reveal that the output includes the evidence found missing, which is a behavioral detail. However, it does not explicitly state whether the operation is read-only, whether any state is changed, or mention permissions or rate limits. This is a minimal viable level of transparency for a list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is highly concise: two short paragraphs. The first sentence immediately states the tool's primary action, and the second paragraph adds relevant audit context. No unnecessary words or repetitive details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description explains the tool's purpose and provides useful audit context, but it completely omits any explanation of the sole parameter (video_id). Since the output schema exists, return values are documented, but the input parameter is undocumented in both schema and description. This makes the tool incomplete for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has only one parameter, video_id, with 0% description coverage. The tool description does not mention video_id at all, so an agent has no idea that the parameter exists or how it filters results. This is a critical gap; the description completely fails to compensate for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb and resource: 'List detections the verifier rejected, with the evidence it found missing.' This distinguishes it from sibling tools like list_processed_videos and list_alerts, which likely list different entities. The inclusion of 'with the evidence it found missing' further clarifies the tool's unique output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear use case: 'Useful for auditing whether the system is suppressing real hazards.' It gives context on when to use the tool but does not explicitly mention alternatives or when not to use it. This is more than implied usage but lacks the explicit exclusions that would merit a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_timelineA
Search the footage in natural language.
Searches every analysed window, not just the ones that produced an alert, so it answers questions the hazard taxonomy does not cover: "someone carrying a long load past the racking", "the aisle by the loading door", "anyone on a phone".
Args: query: what to look for, in plain language. video_id: which processed video; omit if only one has been processed. k: how many windows to return.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| k | No | ||
| query | Yes | ||
| video_id | No |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
A4.4/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals a key non-obvious behavior: it searches all analyzed windows, not just those with alerts, and supports natural-language queries. This is substantial transparency, though it does not cover performance or error behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with a clear one-sentence summary, a paragraph elaborating scope with useful examples, and a structured Args list. The examples are valuable and justify the length, though the overall text is slightly longer than minimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, unique search scope, and all parameters. Since an output schema exists, it need not explain return values. A minor improvement would be referencing list_processed_videos for users unsure of video_id, but this is not a critical omission.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, but the description's Args section fully explains every parameter: query ('what to look for, in plain language'), video_id ('which processed video; omit if only one'), and k ('how many windows to return'). This adds complete semantics beyond the schema's types and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Search the footage in natural language.' It distinguishes itself from siblings by noting it searches every analyzed window, not just alert-producing ones, and explicitly contrasts with the hazard taxonomy. This makes its unique role unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context, explaining that the tool answers questions the hazard taxonomy does not cover and giving concrete examples. It also gives parameter-level guidance, such as omitting video_id when only one video is processed. However, it does not explicitly state when to prefer alternative tools like list_alerts, so it stops short of a full 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
shift_summaryB
Summarise a processed shift: volumes, alert mix, and what was filtered out.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| video_id | No |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
B3.1/5.0
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden for behavioral disclosure. It mentions what the summary includes but does not explain side effects, permissions, or behavior when the optional video_id is null. The tool appears to be a read-only summary, but this is not stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It efficiently conveys the core purpose and scope of the summary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although an output schema exists, the description fails to provide enough context about the optional parameter and the overall shift aggregation. It lacks usage guidance and behavioral caveats, making it insufficient for a tool with such sparse schema information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has zero description coverage, so the description must clarify the parameter's meaning. It does not explain that video_id identifies a processed shift, nor how the summary changes when video_id is null. The connection between 'video_id' and 'shift' is left ambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action ('Summarise') and resource ('processed shift'), and specifies the summary content (volumes, alert mix, filtered out). This distinguishes it from sibling list tools like list_processed_videos and list_alerts by presenting an aggregated view.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used when a summary of a processed shift is needed, but it does not explicitly state when to prefer this over the sibling tools or provide any exclusions. Guidance is implicit rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
7 tool updates
v0.1.0- First observed
explain_alert - First observed
hazard_taxonomy - First observed
list_alerts - First observed
list_processed_videos - First observed
list_rejected_detections - First observed
search_timeline - First observed
shift_summary
TDQS
A3.9/5.0
Scored across 7 tools
Disambiguation5/5
Each tool has a clear, non-overlapping purpose: listing videos, listing alerts, summarizing shifts, listing rejected detections, semantic search, explaining alert evidence, and exposing the taxonomy. There is no ambiguity about which tool to use for a given task.
Naming Consistency4/5
Most tools follow a verb_noun pattern (list_processed_videos, list_alerts, list_rejected_detections, search_timeline, explain_alert), but shift_summary and hazard_taxonomy break the pattern by leading with a noun. The deviation is minor and the naming remains readable.
Tool Count5/5
Seven tools is well-scoped for a video safety analysis and auditing system. Each tool covers a distinct query or audit function without redundancy, and the count is neither too thin nor too heavy.
Completeness4/5
The surface covers the core workflows: enumerating videos, querying alerts, auditing rejections, searching footage, and understanding alerts. A minor gap is the lack of a dedicated get_video detail tool, but list_processed_videos provides sufficient access for most use cases.
Maintenance
ActivitySlowing
ResponsivenessNo issues
Related MCP Connectors
A paid remote MCP for AI SDK data query MCP, built to return verdicts, receipts, usage logs, and aud
Query real user session replay data: tapes, transcripts, error/rage-click filters, alerts.
Query Churn Solution cancellation-flow metrics, revenue, and feedback analytics (read-only).
Read-only AI search visibility data: citations, AEO audits, and advisor insights from AI-Advisors.
Related MCP Servers
- FlicenseNot gradedqualityBmaintenanceEnables querying PPE compliance analytics from precomputed video inference results, providing tools for site summaries, worker status, violations, and trends without re-running inference.1-
- AlicenseNot gradedqualityBmaintenanceEnables natural language queries across AWS multi-account security scan history, including infrastructure configurations, public exposures, and organizational relationships.MIT
- AlicenseNot gradedqualityCmaintenanceEnables governed, read-only investigations of Amazon Athena access logs by searching exact user, IP, or correlation indicators and summarizing activity by correlation ID, with parameterized queries, validation, and result limits.MIT