The ui-diff-mcp server enables automated visual comparison of mobile app screenshots against mockup designs using deterministic methods and free/paid visual language models (VLMs).
Tools available:
compare_ui_images — Compare a mockup against a screenshot using only deterministic methods; returns a diff report with counts, artifacts, and coverage status.
discover_ui_diffs — Run a full UI diff analysis with visual model-based target discovery and classification; supports provider modes (free, free_gemini, free_mistral, free_opencode, free_openrouter, free_nvidia, paid, deterministic_only).
ui_diff_model_health — Check the health and availability of all visual models (auditor, reviewer, target recovery roles), returning pass/fail status per model.
read_ui_diff_report — Read and hydrate a previously generated report.json, including all referenced part files (elements, pairs, diffs, usage summary, etc.).
capture_mobile_screen — Capture a live screenshot from a connected mobile device via adb (Android) or ios-simctl (iOS simulator).
start_ui_diff_run — Start a UI diff run in the background; returns a runId for polling, suitable for large or slow audits.
get_ui_diff_run_status — Poll the status of a background diff run, including progress stage, pair index, report path, and completion timestamps.
Key capabilities:
Free-first model routing (Gemini, Mistral, OpenCode, NVIDIA, OpenRouter) with runtime probing before selection
Paid model opt-in via environment variable (UI_DIFF_ENABLE_PAID_MODE=1)
Configurable diff scopes: full screen, named regions, or specific UI targets
Quota estimation and pre-flight check before consuming free-tier API calls
Structured report.json output with usage accounting and artifact references
Background (async) run support with resume capability
Provides integration with NVIDIA's Vision Language Model (VLM) for analyzing mobile app screenshots and comparing them against mockup designs.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@ui-diff-mcp compare screenshot of login page to design mockup"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP server for comparing mobile app screenshots against mockup designs using free-first visual model diffing. Calorix is the primary integration target.
Design spec: docs/superpowers/specs/2026-06-12-ui-diff-mcp-research-design.md
The default mode (free) never calls paid OpenRouter routes. It probes direct Gemini routes first, then direct Mistral routes, OpenCode Zen's image-capable mimo-v2.5-free, native NVIDIA free VLM endpoints, and finally OpenRouter :free routes. Every route must pass the role's real image-count and JSON probe before selection. Paid OpenRouter models are disabled unless mode: "paid" is passed and UI_DIFF_ENABLE_PAID_MODE=1 is set.
OpenCode currently also lists deepseek-v4-flash-free, but its model metadata is text-only. It is intentionally excluded from auditor, reviewer, and target-recovery roles because those roles require four or five crop images. OpenCode documents its free models as limited-time routes, so catalog presence never replaces runtime probes.
Gemini model-list probing on this machine showed gemini-3.1-pro-preview, gemini-3.5-flash, and gemini-3.1-flash-lite are visible. A direct live call to gemini-3.1-pro-preview returned free-tier quota limit 0, so it remains first in the quality ranking but is expected to fail closed unless quota is available. gemini-3.5-flash is the current direct Gemini live-gate model.
Mistral model-list probing showed vision-capable routes including mistral-large-2512, mistral-medium-2604, mistral-small-2603, ministral-14b-2512, and ministral-8b-2512. Live role probing on this machine showed ministral-14b-2512 and ministral-8b-2512 correctly handle the five-image payload required by auditor/reviewer/recovery. mistral-large-2512, mistral-medium-2604, and mistral-small-2603 miscounted or misclassified simple probe images, so they are not current pipeline routes.
Before starting a free-model run, the pipeline estimates the required request count and checks available quota against the OpenRouter key info endpoint. If estimated calls exceed available free quota, the run exits immediately with status: "insufficient_free_quota" rather than consuming quota silently.
Related MCP server: imugi
mode selects the provider/model route policy. It does not select how much of the UI to inspect.
Mode | Behavior |
| Default. Gemini direct, Mistral direct, OpenCode MiMo, NVIDIA free endpoints, then OpenRouter |
| Only direct Gemini routes. Current live gate uses |
| Only direct Mistral routes. Current live gate uses |
| Only OpenCode Zen visual routes. Currently |
| Only OpenRouter |
| Only native NVIDIA free endpoint routes. |
| Explicit opt-in requiring |
| No VLM calls. Returns deterministic signal evidence only. |
Use diffScope to select the visual scope independently from provider mode.
{ "kind": "screen" }Audits the whole screen first: global placement, major color/appearance changes, broad shape/border/layer differences, and whole-screen diff masks. Target-level recovery is bypassed.
{ "kind": "regions", "regions": ["top", "nav"] }Audits only selected deterministic regions. Current region names are top, middle, bottom, header, content, and nav. Target recovery is restricted to uncovered components inside the selected regions.
{ "kind": "target", "query": "scan button" }Resolves a target by locator label, visible text, and element type, then audits only the best matching pair. If the target cannot be resolved, the report includes a warning and does not pretend the target was checked.
{ "kind": "full" }Default. Runs screen/region summaries, scope-level VLM audit where deterministic triggers fire, and the existing target-level audit/recovery path.
Example MCP payload:
{
"expectedImagePath": "C:/mockups/Today.png",
"actualImagePath": "C:/screenshots/today.png",
"mode": "free",
"diffScope": { "kind": "regions", "regions": ["nav"] }
}All generated images (pixel diff, directional overlay, crop pairs, recovery crops) are machine evidence consumed by audit and recovery models. They are not a manual inspection workflow. Do not rely on visual artifact review as a substitute for structured report.json output.
final-diff-groups-overlay.png shows visual clusters of final diffs. It is not a semantic parent/child hierarchy: a whole-screen cluster and localized clusters can be siblings when they explain different criteria. Use semantic-hierarchy-overlay.png and semantic-hierarchy-legend.json to inspect the detected UI structure such as screen, nav, macro card, macro circle, and repeated cards.
report.json is the slim manifest. Large report sections are written as referenced JSON parts under artifacts/parts/ instead of being duplicated inline:
elements.json
pairs.json
diffs.json
unresolved-regions.json
debug-summary.json
usage-summary.json
scope-summary.json
reportParts[].path is relative to the report.json directory. read_ui_diff_report hydrates these parts before returning the report, so existing MCP consumers still receive a full schema-valid report even though the on-disk manifest keeps elements, pairs, diffs, and unresolved regions compact.
usageSummary is first-class run-level accounting. It records input tokens, output tokens, total tokens, reasoning tokens, successful calls, failed calls, fallbacks, route exhaustion, and duration totals by phase, role, and provider/model route. If a provider reports only total tokens, input/output are left as zero and totalOnlyUsageCalls increments; the MCP does not invent a fake split.
LOCATEANYTHING_MAX_DIMENSION controls the largest image dimension sent to the LocateAnything sidecar. The default remains 1200 for detail.
600 is a local timeout workaround, not a quality default. It shrinks a 1206x2622 Calorix mockup to roughly 276x600 for the locator, which can hide small icons, thin borders, and text. Prefer the highest dimension that fits the sidecar budget, and run the sequential locator benchmark before production sign-off:
export UI_DIFF_LIVE_EXPECTED_IMAGE="/home/agent-runner/projects/calorix/docs/design-handoff/placeholder-app/reference-images/today--dark.png"
export LOCATEANYTHING_SIDECAR_URL="http://127.0.0.1:39731"
export UI_DIFF_LOCATOR_BENCHMARK_DIMENSIONS="600,900,1200"
npm run benchmark:locatorFor Calorix release evidence, UI_DIFF_LIVE_ACTUAL_IMAGE normally stays unset so the gate can prove fresh acquisition. The MCP capture implementation and Calorix live helper now accept adbExecutable/adbSerial options (or UI_DIFF_ADB_EXECUTABLE/UI_DIFF_ADB_SERIAL environment fallbacks) and route every ADB invocation through the resolved executable with optional -s serial prefix. On this Pi, set UI_DIFF_ADB_EXECUTABLE=/home/agent-runner/.local/bin/phone-adb and leave UI_DIFF_ADB_SERIAL unset — the wrapper already pins the serial. For generic ADB executables that do not embed a serial, set UI_DIFF_ADB_SERIAL to the target device serial. Manual /home/agent-runner/.local/bin/phone-adb exec-out screencap -p remains valid for diagnostic comparisons. An explicit UI_DIFF_LIVE_ACTUAL_IMAGE is logged as an override and is not fresh release proof.
The dedicated Samsung SM-G780G, Android 13, serial R58R61161NA is the only default local Android target. Every device command must use /home/agent-runner/.local/bin/phone-adb, which pins the serial. Never use plain adb or implicit device selection. Cuttlefish, android-vm, ReDroid, desktop AVDs, and local emulators are retired; GitHub's x86_64 emulator remains an independent CI gate.
Identity preflight:
/home/agent-runner/.local/bin/phone-adb devices -l
/home/agent-runner/.local/bin/phone-adb shell getprop ro.product.model
/home/agent-runner/.local/bin/phone-adb shell getprop ro.build.version.release
/home/agent-runner/.local/bin/phone-adb get-serialnoCalorix's GitHub Actions workflow .github/workflows/android-build.yml is named Build Android APK. A successful run publishes an artifact named android-apk-<commit-sha> containing the versioned release APK (calorix-<version>-android-release.apk) and its matching .sha256 file.
Use the fetcher only for a committed source revision whose cleanliness has already been established by the caller:
bash scripts/fetch-calorix-actions-apk.sh \
--repo ia23a-lachnita/calorix \
--source-sha 1f538641f5e5f5c4a48c95cdfb97462838187106 \
--source-clean \
--output "$HOME/Downloads/calorix-release.apk"--source-clean is an affirmative trust-boundary attestation. The script intentionally does not inspect any local Calorix checkout, so an omitted clean proof or --source-dirty rejects with uncommitted_source. This prevents the known dirty local .mcp.json from being mistaken for clean, current APK evidence. The script queries only the exact successful Build Android APK run for the requested SHA, downloads the named artifact from its immutable numeric run ID, accepts exactly the APK plus same-basename checksum, verifies the SHA-256, then atomically writes the APK and an adjacent .verified.json provenance record. It never overwrites an existing output or verification record.
Calorix live gates validate the canonical default expected reference before any device or pipeline work: /home/agent-runner/projects/calorix/docs/design-handoff/placeholder-app/reference-images/today--dark.png, its adjacent reference-images-manifest.json, and SHA-256 73BA85F25489C8D45BEAB57DD1B317138870CE8360FE0F4399AB0737A5E505F1. An explicit UI_DIFF_LIVE_EXPECTED_IMAGE remains supported, but must be readable; reference-images-buggy and good-screenshots are never fallback sources.
Calorix reports persist a report-safe inputProvenance record whose expected and actual SHA-256 identities are computed from the exact image bytes by the pipeline. A manifest entry is included only after the pipeline verifies that its recorded hash matches the expected image bytes. Acquisition sources such as auto_capture and env_override are stored separately with verification:"caller_attested"; they are not presented as independently verified facts. Public MCP requests cannot supply computed hashes. On resume, omitted provenance inherits the stored effective record; an explicit replacement is accepted only when the recomputed expected and actual image identities match the resumed report.
Every report records locatorInputSizing, including original image size, sent image size, scale, maxDimension, and whether actual elements were independently located or projected from expected elements. Runs also save the exact image payloads sent to the sidecar as locator-input-expected.png and, in dual-locator mode, locator-input-actual.png; these appear in runArtifacts as locator_input_expected and locator_input_actual.
Locator debugging uses a three-step artifact chain:
locator-input-expected.png: the exact expected-image bytes sent to LocateAnything.
locator-expected-overlay.png: the located target boxes drawn on the normalized expected image.
locator-actual-projected-overlay.png: in default single-pass projection mode, the expected target boxes projected onto the normalized actual screenshot. This does not mean the actual screenshot was sent to LocateAnything; it shows where the expected targets land on the actual image before the diff/audit stages inspect those projected regions.
The overlay files are indexed in runArtifacts as locator_expected_overlay, locator_actual_overlay, and locator_overlay_legend. The legend maps compact overlay labels such as E001 and P001 back to exact element IDs, types, query IDs, source, and boxes.
npm installnpm run verifynpm run buildnode dist/src/index.jsCopy .env.example and fill in the relevant keys.
Variable | Required | Default | Description |
| No |
| Optional OpenCode Zen credential override. The current free route accepts the public credential. |
| No |
| OpenCode Zen API base URL. |
| For Gemini direct mode | — | Gemini API key from AI Studio. Routes are always probe-gated because visible models may have zero free-tier quota. |
| No |
| Override Gemini API base URL. |
| For Mistral direct mode | — | Mistral API key for direct vision routes. |
| No |
| Override Mistral API base URL. |
| For OpenRouter free mode | — | OpenRouter API key. Free-tier account sufficient for |
| For NVIDIA free mode | — | NVIDIA Build/NIM API key for native NVIDIA free VLM endpoints. |
| No |
| Override NVIDIA base URL for self-hosted NIM. |
| For paid mode only | — | Must be exactly |
| No |
| URL of the LocateAnything sidecar. On the Pi this is the broker for the remote GPU host, never a locally started process. |
| No |
| Broker cold-start budget. Must be a finite positive integer, max |
| For the local launcher only (archival/local-development; not permitted on the Pi) | — | Path to Eagle Embodied install. |
| For the local launcher only (archival/local-development; not permitted on the Pi) | Known local venv, then | Python interpreter for local sidecar startup. |
| No | — | Diagnostic mode only. |
| No |
| Image token budget for local sidecar. |
| No |
| Sidecar worker mode: |
| No |
| Sidecar generation cap. |
| No | — | Cap the number of element pairs audited. Bounded runs are marked |
| No |
| Override the ADB executable for mobile capture. Used by |
| No | — | ADB serial for device targeting. Prepends |
This implementation requires no user-authored target map, ROI map, ignore mask, or anchor dump.
No local OpenCode daemon or opencode run process is required. The MCP calls the OpenAI-compatible Zen API directly. The current free route works with the default public credential; set OPENCODE_API_KEY only when OpenCode provides a dedicated key.
$env:RUN_OPENCODE_LIVE="1"
npm run verify:opencode-liveThe gate verifies the live catalog, a one-image structured response, and one deduplicated five-image probe shared across auditor, reviewer, and target recovery.
Set GEMINI_API_KEY to an AI Studio key. The pipeline probes direct Gemini routes before using them; gemini-3.1-pro-preview may be visible but quota-blocked on the free tier, so gemini-3.5-flash is the current live-gate route.
$env:GEMINI_API_KEY="..."
$env:RUN_GEMINI_LIVE="1"
npm run verify:gemini-liveSet MISTRAL_API_KEY to a Mistral API key. The pipeline starts with ministral-14b-2512 because it passed the same five-image role probe used by the auditor/reviewer/recovery gates; ministral-8b-2512 is the next Mistral fallback.
$env:MISTRAL_API_KEY="..."
$env:RUN_MISTRAL_LIVE="1"
npm run verify:mistral-liveSet NVIDIA_API_KEY to a NVIDIA Build API key. The pipeline probes native NVIDIA candidates from CANONICAL_MODEL_RANKING and selects the highest-quality passing model. Use NVIDIA_VLM_BASE_URL to point at a self-hosted NIM instead.
$env:NVIDIA_API_KEY="nvapi-..."
# $env:NVIDIA_VLM_BASE_URL="http://localhost:8000/v1" # for self-hosted NIMRun the NVIDIA live gate to verify:
$env:RUN_NVIDIA_LIVE="1"
npm run verify:nvidia-liveSet OPENROUTER_API_KEY to an OpenRouter key (free-tier account works). The pipeline:
Estimates required calls (probes + audit + recovery + review).
Queries GET https://openrouter.ai/api/v1/key for limit_remaining.
Exits with insufficient_free_quota if estimated calls exceed available quota.
Throttles OpenRouter free calls to ≤ 18 requests/minute.
$env:OPENROUTER_API_KEY="sk-or-..."Run the free live gate to verify:
$env:RUN_FREE_LIVE="1"
npm run verify:free-liveThis section is explicitly operator-managed historical/reference setup for the Windows GPU host itself. Agents must never act on it — no SSH into the Windows host, no remote power management, and no running these local-startup commands from the Pi or any other agent-controlled machine. On the Pi, every locator call goes through the broker (LOCATEANYTHING_SIDECAR_URL) described below; the commands below are kept only as provenance for how the broker's host machine is set up by its human operator.
The MCP calls a sidecar endpoint at POST /v1/locate-ui-elements. Start the local wrapper after installing NVIDIA's Eagle Embodied package:
git clone https://github.com/NVlabs/Eagle.git C:\Users\xursc\projects\Eagle
python -m venv C:\Users\xursc\projects\.venvs\ui-diff-mcp-locateanything
C:\Users\xursc\projects\.venvs\ui-diff-mcp-locateanything\Scripts\python.exe -m pip install --upgrade pip setuptools wheel
C:\Users\xursc\projects\.venvs\ui-diff-mcp-locateanything\Scripts\python.exe -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128
cd C:\Users\xursc\projects\Eagle\Embodied
C:\Users\xursc\projects\.venvs\ui-diff-mcp-locateanything\Scripts\python.exe -m pip install --no-deps -e .
cd C:\Users\xursc\projects\ui-diff-mcp
C:\Users\xursc\projects\.venvs\ui-diff-mcp-locateanything\Scripts\python.exe -m pip install -r sidecars\locateanything\requirements.txt
$env:LOCATEANYTHING_EAGLE_EMBODIED_DIR="C:\Users\xursc\projects\Eagle\Embodied"
$env:LOCATEANYTHING_PYTHON="C:\Users\xursc\projects\.venvs\ui-diff-mcp-locateanything\Scripts\python.exe"
.\scripts\start-locateanything-sidecar.ps1The startup script prefers LOCATEANYTHING_PYTHON, then the known local venv at
C:\Users\xursc\projects\.venvs\ui-diff-mcp-locateanything\Scripts\python.exe, then plain python, and prints the
interpreter it selected. This is the machine that hosts the model behind the broker; consumers (including the Pi's
Calorix live-test helper) never resolve a Python interpreter themselves — they only wait on the broker's /health
contract, and fail fast if it reports an error instead of waiting for the full readiness timeout.
The Pi never starts LocateAnything locally. Every locator call goes through the broker at
LOCATEANYTHING_SIDECAR_URL (default http://127.0.0.1:39731), served by the remote Windows
RTX 3070 host described above. ensureSidecarRunning() in tests/helpers/sidecar-manager.ts is
the broker-only live harness manager: it validates the full pinned /health contract — exact
HTTP 200 with JSON model: "nvidia/LocateAnything-3B", ready: true, error: null (rejecting
both a non-null error and an omitted/undefined error field when ready: true), and
inTokenLimit: 4096 — and never spawns a local process or resolves a Python interpreter. The
checkSidecarHealth() preflight in src/locator/locateanything-client.ts, used by
locateUiElements() on the request path, is a lighter check: it runs within the caller's own
timeoutMs and only requires HTTP success and ready: true — it does not duplicate the
manager's full model/token/exact-null validation. The broker's first response after a cold start
can legitimately hold for minutes; treat any other response (unreachable, non-200, malformed,
wrong model/token contract, or a model-load error) as fail/report, never as a trigger to start a
local sidecar, SSH into the Windows host, or manually power-manage it. Optionally set
LOCATEANYTHING_BROKER_STARTUP_TIMEOUT_MS (finite positive integer, max 600000) to override
the default 600000ms cold-start budget.
The TypeScript client sends image bytes with each locator request, so LOCATEANYTHING_SIDECAR_URL can point to a remote GPU service that exposes the same contract.
The launcher below starts a LocateAnything process directly on the machine that runs it and
binds only to 127.0.0.1:39731. It remains for historical/local-development reference (e.g. a
non-Pi developer machine that hosts the model itself) and is not permitted on this Pi under
the current broker-only topology.
It resolves an explicit
LOCATEANYTHING_PYTHON without fallback, then
/home/agent-runner/projects/.venvs/ui-diff-mcp-locateanything/bin/python,
then python3. It likewise requires an explicit
LOCATEANYTHING_EAGLE_EMBODIED_DIR, or the default
/home/agent-runner/projects/Eagle/Embodied, to contain the
locateanything_worker package supplied by Eagle Embodied.
# Historical/local-development only — do not run on this Pi.
# Validate Python, Eagle Embodied, and the loopback-only configuration only.
bash scripts/start-locateanything-sidecar.sh --check-only
# Historical/local-development only — do not run on this Pi.
# Start or reuse a healthy local sidecar. It prints the child PID and log path.
bash scripts/start-locateanything-sidecar.shThe launcher accepts no host override. UI_DIFF_LOCATEANYTHING_PORT_INTERNAL
is an internal diagnostic/test hook only; production uses port 39731.
LOCATEANYTHING_STARTUP_TIMEOUT_MS (architecture default 600000 on
ARM64/Pi / aarch64, 120000 on all other supported architectures; max
600000) and LOCATEANYTHING_STARTUP_POLL_MS (default 500, max 10000)
configure how long the launcher waits for /health to report "ready": true
before failing; a nonempty error is reported immediately. The longer ARM64
default exists because measured Pi Q4 cold start is about 473 seconds; a 120s
default would kill a healthy Pi load. These are distinct from
LOCATEANYTHING_TIMEOUT_MS, the separate Node-side inference-request timeout
consumed by the MCP client for /v1/locate-ui-elements calls — it does not
control launcher readiness.
LOCATEANYTHING_SKIP_MODEL=1 is only a diagnostic shortcut. It makes /health.ready true without loading the 3B model
and still runs CV/OCR/optional lanes, but a run made with that flag is not full LocateAnything-model release evidence.
The following contract is retained to explain and test historical artifacts. Do not start ReDroid or use this section as current operator guidance or release evidence.
On the C++ backend, if a ui-diff-redroid (or
UI_DIFF_LOCATEANYTHING_REDROID_NAME_INTERNAL-named) Docker/Podman container
is running, the launcher refuses to start the locator unless
LOCATEANYTHING_COLOCATION_EVIDENCE points to a file proving a prior
measured concurrent run stayed within resource bounds. The file is key=value
lines with exactly these 9 keys, no more and no fewer:
Key | Required value |
|
|
| the pinned |
| the pinned Q4_K model SHA-256 |
|
|
|
|
|
|
| a positive integer |
|
|
|
|
Lines containing shell metacharacters ($(, `, ;, |, &, >, <,
or quotes) are rejected without ever being evaluated as shell code; the file
is only ever read with case/parameter-expansion string comparisons.
locateanything-startup.metrics, written by the launcher after the sidecar
becomes ready, is a separate post-ready artifact recording that one
locator process's own RSS/swap. It is not proof of concurrent ReDroid
co-location and must not be substituted for
LOCATEANYTHING_COLOCATION_EVIDENCE.
Parser-only sidecar tests:
/home/agent-runner/projects/.venvs/ui-diff-mcp-locateanything/bin/python -m unittest sidecars.locateanything.test_parser# Verify the dedicated Samsung identity. Use this absolute wrapper for every
# device operation; never use plain adb or implicit device selection.
/home/agent-runner/.local/bin/phone-adb devices -l
/home/agent-runner/.local/bin/phone-adb shell getprop ro.product.model
/home/agent-runner/.local/bin/phone-adb shell getprop ro.build.version.release
/home/agent-runner/.local/bin/phone-adb get-serialno
# Capture a serial-safe diagnostic screenshot.
/home/agent-runner/.local/bin/phone-adb exec-out screencap -p > /absolute/path/to/actual.png
# Check the LocateAnything broker's health directly. Never start a local sidecar on this
# Pi — the launcher scripts below are archival/local-development only.
curl --fail --silent --show-error --max-time 600 http://127.0.0.1:39731/health
# Verify the output body exactly: model: "nvidia/LocateAnything-3B", ready: true,
# error: null, inTokenLimit: 4096. Any other body (missing/non-null error, wrong
# model, wrong inTokenLimit, or a non-200/failed curl) means the broker is not
# ready/release-ready — fail/report and stop rather than proceeding.
# Fetch verified Calorix Actions APK (source-SHA + SHA256 required)
# Workflow: .github/workflows/android-build.yml (named "Build Android APK")
# Artifact: android-apk-<sha>
bash scripts/fetch-calorix-actions-apk.sh \
--repo ia23a-lachnita/calorix \
--source-sha 1f538641f5e5f5c4a48c95cdfb97462838187106 \
--source-clean \
--workflow android-build.yml \
--artifact-name android-apk-1f538641f5e5f5c4a48c95cdfb97462838187106 \
--output "$HOME/Downloads/calorix-release.apk"
# Verify package-lock.json root bin is dist/src/index.js
bash scripts/verify-package-bin-lock.sh
# Full repository verification with the sidecar parser environment
PATH=/home/agent-runner/projects/.venvs/ui-diff-mcp-locateanything/bin:$PATH npm run verifyEvidence boundary: Fresh auto-capture release evidence requires
UI_DIFF_ADB_EXECUTABLEset to the authorized wrapper andUI_DIFF_ADB_SERIALleft unset (the wrapper already pins the serial) — or explicitadbExecutable/adbSerialoptions — so every ADB call routes through the authorized Samsung wrapper. For generic executables, setUI_DIFF_ADB_SERIALto the target device serial. Manual wrapper capture is serial-safe diagnostic input, andUI_DIFF_LIVE_ACTUAL_IMAGEis an override rather than fresh auto-capture proof.
.mcp.json){
"mcpServers": {
"ui-diff": {
"command": "node",
"args": ["/home/agent-runner/projects/ui-diff-mcp/dist/src/index.js"],
"env": {
"OPENROUTER_API_KEY": "<your-key>"
}
}
}
}.codex/config.toml)[mcp_servers.ui-diff]
command = "node"
args = ['/home/agent-runner/projects/ui-diff-mcp/dist/src/index.js']
enabled = truenpm run verify is deterministic and does not call external APIs. Before production use, run the live gates:
Gate | Command | Required env |
Direct Gemini models |
|
|
Direct Mistral models |
|
|
OpenCode Zen MiMo |
|
|
Free OpenRouter models |
|
|
Native NVIDIA models |
|
|
Full pipeline |
|
|
Bounded Calorix smoke |
|
|
Full Calorix all-target |
|
|
A bounded smoke run (UI_DIFF_MAX_AUDIT_PAIRS set) is explicitly not full visual classification:
auditLimited: true in compact output and report.json.
visualClassificationStatus: "incomplete" unless all pairs happened to be within the limit.
auditScope.auditedPairs / auditScope.totalPairs records the actual vs total pair count.
A full all-target run must show auditLimited: false. Use verify:calorix-full-live to confirm.
See docs/release/production-readiness-checklist.md for the complete sign-off sequence.
Captures a screenshot from a connected mobile device using adb or ios-simctl.
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes |
| Name | Required | Description |
|---|---|---|
| capture | Yes | |
| imagePath | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description should disclose behavioral traits but only mentions capture action. No info on side effects, output location, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, 12 words, efficient and front-loaded with the core action and methods.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists (not shown), description need not detail return values, but still lacks setup context (device connection) and output format. Adequate for a simple tool but minimal.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, and description adds context by linking the target parameter to the capture methods (adb/ios-simctl), but does not explain implications like Android vs iOS separation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool captures a screenshot from a connected mobile device using adb or ios-simctl. Differentiates from sibling tools like compare_ui_images, which focus on analysis rather than capture.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Lacks context about prerequisites (e.g., device must be connected) or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Compares an expected mobile mockup image against an actual mobile screenshot using only deterministic methods (no visual models).
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | free | |
| runLabel | No | ||
| projectRoot | No | ||
| actualImagePath | Yes | ||
| expectedImagePath | Yes |
| Name | Required | Description |
|---|---|---|
| runId | Yes | |
| status | Yes | |
| summary | Yes | |
| warnings | Yes | |
| diffCount | Yes | |
| auditScope | No | |
| reportPath | Yes | |
| artifactRoot | Yes | |
| auditLimited | Yes | |
| debugSummary | No | |
| runArtifacts | Yes | |
| recoverySummary | No | |
| locatorCoverageStatus | Yes | |
| unresolvedRegionCount | Yes | |
| visualClassificationStatus | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must cover behavioral traits. It states 'deterministic methods' and 'no visual models', but omits side effects, permissions, performance, or behavior on mismatched images.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that immediately states the action. However, it sacrifices completeness for brevity, lacking structure to cover parameters or context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, an output schema, and sibling tools indicating a workflow, the description is insufficient. It doesn't explain the role of parameters or how this fits into the larger tool suite.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 5 parameters including an enum for 'mode', but the description gives no information about any parameter. With 0% schema description coverage, this is a critical gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool compares expected and actual mobile images using deterministic methods. It does not explicitly differentiate from sibling tools like 'discover_ui_diffs' or 'start_ui_diff_run', but the mention of 'no visual models' hints at its specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description only notes that it uses deterministic methods, implying a use case where visual models are not wanted. No guidance on when to use this over siblings or prerequisites (e.g., captured images). Lacks explicit context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Runs full UI diff analysis, including visual model-based target discovery and classification. Compares expected mockup against actual screenshot.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | free | |
| runLabel | No | ||
| projectRoot | No | ||
| actualImagePath | Yes | ||
| expectedImagePath | Yes |
| Name | Required | Description |
|---|---|---|
| runId | Yes | |
| status | Yes | |
| summary | Yes | |
| warnings | Yes | |
| diffCount | Yes | |
| auditScope | No | |
| reportPath | Yes | |
| artifactRoot | Yes | |
| auditLimited | Yes | |
| debugSummary | No | |
| runArtifacts | Yes | |
| recoverySummary | No | |
| locatorCoverageStatus | Yes | |
| unresolvedRegionCount | Yes | |
| visualClassificationStatus | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions visual model-based discovery but lacks details on side effects, duration, permissions needed, or output summary (despite existence of output schema).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences (20 words) and front-loads the main action, but it could be more structured by enumerating inputs or output. It is concise but lacks completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 5 parameters including an enum, and an output schema, the description is inadequate. It does not mention return values or how the analysis integrates with sibling tools (e.g., read_ui_diff_report).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, meaning no parameter descriptions. The description only hints at expectedImagePath and actualImagePath via 'compares expected mockup against actual screenshot', but omits mode, runLabel, and projectRoot entirely, failing to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs full UI diff analysis including visual model-based target discovery and classification, and compares expected mockup against actual screenshot. This distinguishes it from simpler comparison tools like compare_ui_images.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus its siblings (e.g., compare_ui_images, start_ui_diff_run). There is no mention of prerequisites, exclusions, or alternative tools for specific scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Returns the status of a background UI diff run started by start_ui_diff_run.
| Name | Required | Description | Default |
|---|---|---|---|
| runId | Yes | ||
| projectRoot | Yes |
| Name | Required | Description |
|---|---|---|
| error | No | |
| label | No | |
| runId | Yes | |
| status | Yes | |
| progress | No | |
| startedAt | No | |
| reportPath | No | |
| completedAt | No | |
| heartbeatAt | No | |
| artifactRoot | No | |
| checkpointPath | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states the basic purpose. It does not disclose that this is a read-only operation, any authentication needs, rate limits, or potential error states. Given the lack of annotations, the description should provide more behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that clearly states the core function. It front-loads the purpose with no wasted words, but could benefit from a second sentence providing additional context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 2 required parameters and an output schema present, the description is minimally complete: it states the function but does not elaborate on the status values or the context of the run. The output schema reduces some burden, but the agent lacks guidance on interpreting the response.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage; parameters 'runId' and 'projectRoot' have only type and minLength constraints. The description adds no additional meaning to these parameters, so the agent must infer their purpose from the tool name alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns the status of a background UI diff run, with a specific verb 'Returns' and resource 'status of a background UI diff run started by start_ui_diff_run'. This distinguishes it from siblings like start_ui_diff_run (starting) and read_ui_diff_report (reading the report).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage after starting a run via start_ui_diff_run, but does not explicitly provide when to use or not use this tool, nor does it mention alternatives or prerequisites. It relies on context from the sibling name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Reads and returns a previously generated UI diff report JSON file.
| Name | Required | Description | Default |
|---|---|---|---|
| reportPath | Yes |
| Name | Required | Description |
|---|---|---|
| report | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool reads and returns JSON data, implying a non-destructive read operation. However, with no annotations, it carries full burden and lacks details on error handling (e.g., if the file does not exist) or permissions. It is minimally adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It is front-loaded and immediately conveys the tool's action and subject.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with one parameter and an output schema. The description covers the basic purpose but lacks workflow context—e.g., that the report path likely comes from a previous step like 'start_ui_diff_run'. It does not mention the output schema existence or how to interpret results. Adequate for a basic tool but could be more helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It adds no meaning to 'reportPath' beyond the name and type (string). The description does not specify the path format, how to obtain it, or constraints. The parameter is self-explanatory in name, but the tool description fails to provide useful context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool reads and returns a previously generated UI diff report JSON file. The verb 'reads and returns' and resource 'previously generated UI diff report' are specific, and it is distinct from sibling tools (e.g., 'capture_mobile_screen', 'start_ui_diff_run').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., a report must already exist) or that this should be used after starting a diff run. No exclusions or workflow context are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Starts a UI diff run in the background without holding the MCP request open. Returns a runId to poll with get_ui_diff_run_status. Use this for large or slow audits like Calorix full scans.
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | free | |
| label | No | ||
| projectRoot | No | ||
| resumeRunId | No | ||
| actualImagePath | Yes | ||
| expectedImagePath | Yes |
| Name | Required | Description |
|---|---|---|
| runId | Yes | |
| status | Yes | |
| message | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must disclose all behavioral traits. It correctly states the tool runs in the background and is non-blocking, but omits details such as authentication needs, rate limits, or side effects like creating persistent state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at two sentences, with each sentence providing distinct value: the first states the action and key behavior, the second gives a usage suggestion. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, async) and the presence of an output schema, the description covers the async nature and return value adequately. However, it lacks explanation of parameters, required vs optional, and how this tool differs from synchronous siblings like compare_ui_images.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not mention any parameters or their roles. The required paths (expectedImagePath, actualImagePath) are not explained, nor is the mode enum or other optional fields. This provides no assistance to the agent beyond the schema itself.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool starts a UI diff run asynchronously, returns a runId, and distinguishes from sibling tools like get_ui_diff_run_status and compare_ui_images by emphasizing background execution for large/slow audits.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly recommends use for large/slow audits and mentions the workflow of polling with get_ui_diff_run_status. However, it does not explicitly state when not to use this tool or mention alternatives like compare_ui_images for smaller diffs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Checks health of all visual models used by the diff pipeline.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
| results | Yes | |
| checkedAt | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for behavioral disclosure. It does not state whether the tool is read-only, what happens if models are unhealthy, or any side effects. For a health check, it should at least indicate it has no destructive actions, but this is missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. It is front-loaded and gets straight to the point. Ideal conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so explaining return values is not required. However, the description is very terse and does not provide any context beyond the basic function. For a health check, one might expect additional context like what constitutes health, but given the simplicity, it is minimally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description does not need to elaborate on parameters. It could have explained what 'health' means, but since there are no parameters, a score of 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it checks the health of all visual models used by the diff pipeline. It is a specific verb ('checks') and resource ('health of all visual models'), and it is distinct from sibling tools which focus on capturing screens, comparing images, discovering diffs, etc. No ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as before starting a diff run or after an issue. There is no mention of prerequisites, conditions for use, or exclusions. It only states what it does, not when to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.1.0capture_mobile_screencompare_ui_imagesdiscover_ui_diffsget_ui_diff_run_statusread_ui_diff_reportstart_ui_diff_runui_diff_model_healthScored across 7 tools
Tools are mostly distinct but compare_ui_images and discover_ui_diffs both compare screenshots; descriptions clarify deterministic vs model-based, but could cause temporary confusion.
Naming is inconsistent: some tools start with verbs (capture, compare, discover, get, read, start) while ui_diff_model_health is a noun phrase. Also mixing ui_diff and ui_diffs, and get vs read.
7 tools is well-scoped for UI diffing: capture, two compare modes, background run management (start, status, report), and health check. No unnecessary tools.
Covers core workflow: capture, compare (sync/async), background run, report retrieval, model health. Minor gap: no tool to list or cancel runs, but not critical.
Capture screenshots, detect visual regressions between page versions, and analyze with AI.
Build and edit app screen mockups and clickable prototypes from your AI assistant.
Generate designed, localized App Store screenshot sets from your raw app captures.
Visual regression review: projects, builds, visual diffs, approvals, comments and analytics.
