Skip to main content
Glama

9xvision

Perception for AI agents: video, live input and images. Give any agent (Claude Code, Codex, Cursor, anything that speaks MCP or can run a CLI) a video, a camera, a microphone, a screen or a folder of images, and it gets back structured, timestamped evidence instead of a handful of random frames. Underneath is one substrate: everything becomes the same kind of searchable, cached record, so the same tools work on all of it.

Video files and URLs

  • Screen recordings (apps, websites): every UI state in order, distinct views, the transition between each (tap, scroll, sheet, swipe), OCR of every screen, stitched full pages, and the flow.

  • Everything else (films, music videos, talks, demos): shots split at cuts placed on the exact frame and content beats (slide changes inside one take), camera motion, palettes, colour script, on-screen text and speech per segment, plus the music: tempo, beat grid, and whether the edit is cut to the beat.

  • Visual search: find moments by what is in the picture ("motorcycles with glowing wheels at night", "a pie chart"), not only by on-screen text or speech. Runs locally.

  • Capture QA for games, XR and apps: frame pacing and hitches with timestamps, stereo detection, fast camera turns (comfort), and flashes per second (photosensitivity triage).

  • Drill-down tools: zoom (dense frames of an interval), frame (full-res crop), diff (what changed between two moments), search, qa.

  • Long videos and URLs: watch only --start 1:02:00 --end 1:05:00 of a three-hour stream; for a URL only that part is downloaded. All times stay in full-video time.

Live input (camera, microphone, screen, a stream, or a file played in real time)

  • Motion, cuts, colour and light; sound level, onsets, tempo with a beat phase, voice pitch and speech; faces (51 ARKit-style blendshapes, all but tongueOut, and head pose), hands (gesture, position, pinch) and body; affect signals; and visual triggers by description ("a dog").

  • Streamed as OSC (TouchDesigner, Max, Ableton, Resolume), WebSocket (browsers, games) and JSON lines, with a monitor page. An agent can ask what is happening now, or turn the last minutes into a normal watch.

Images

  • One big image: an overview with a lettered grid, and any cell or box at native resolution, so an agent can read the fine print instead of guessing.

  • A folder: an album with dates, cameras, text, palettes, near-duplicate groups and visual search.

Everything runs locally. There are no API keys and nothing is uploaded.

What you get

App and screen recordings: the flow, step by step, naming where each change happened (+ = that area gained colour, often a selection; − = lost colour):

00:09.3  V7 -> V8    [content change at +'Get recommendations' · +'Female']
00:10.4  V8 -> V9    [screen change]
00:17.7  V13 -> V14  [local change at −'Work' · −'Self-Care']
00:18.4  V14 -> V15  [local change at −'Hobbies']

Every distinct screen is kept with its text, background and palette, and the list compresses repeats ("same as V9; added: 1986 | 1987 | 1988"). The +/− labels are a pointer to where to look, not a verdict: on some apps they name the wrong neighbour, so diff confirms.

Everything else: one labelled cell per shot (or slide / scene change), with camera motion, palette, on-screen text and speech in INDEX.md, plus a colour script of the whole film:

contact sheet colour script

Big Buck Bunny © Blender Foundation, CC BY 3.0.

And the sound, for a music video:

- music: ~128.5 BPM (pulse clarity 0.385); beat times in `sound.json`
- edit vs music: partly on the beat: 45 of 109 cuts within ±63 ms of a beat (41%; chance would be 27%, p=0.00074)
- sudden rises (drops, hits, entries): 00:06.5 (+6 dB), 00:16.0 (+9.8 dB), 02:16.0 (+9.2 dB)

The verdict needs at least 20 cuts and a binomial test against chance; with fewer it says "too few cuts to judge", and a weak pulse (speech, ambience) is flagged as a guess.

Visual search, on the same music video:

$ 9xvision search "motorcycles with glowing wheels at night"
visual moments: 01:07.0-01:08.3 strong 0.98 · 01:09.0-01:09.5 strong 0.93 · 01:13.5-01:14.0 strong 0.82 · ...
$ 9xvision search "a giraffe"
visual moments: 02:11.5-02:12.0 weak 0.0001 · 01:45.0-01:45.5 weak 0.0001 · 00:55.0-00:55.5 weak 0
  all weak: probably not in the video

Scores are SigLIP's calibrated match strength (strong ≥ 0.5, possible ≥ 0.05), so "nothing matches" is a real answer instead of a confident wrong top hit. Short, generic queries ("a cat") score low even on a hit, so a score far above the rest of the video or album also counts as possible. It is reliable for objects, people, clothing, places and colours, looser for actions ("jumping over a treadmill"), and it cannot do negation ("an empty room") — the tool warns when a query uses it. Every search also returns a contact sheet of its best frames to check.

Capture QA, on 40 s of a Half-Life: Alyx playthrough from YouTube (only that window is downloaded):

$ 9xvision qa "https://www.youtube.com/watch?v=9M0_SWkRcMk" --start 1:00:00 --end 1:00:40 --fov 108
- 2400 frames at 60 fps capture; mono
- effective fps while moving: median 60.0, 5th percentile 60.0, worst second 59.0
- hitches: 1 (repeated frames while the picture moved), worst 33.3 ms; at 1:00:31.6 (33.3 ms)
- comfort: peak turn 345.21 deg/s; 13 turns above 90.0 deg/s; at 1:00:13.8-1:00:14.3 ...
- flashes: none above 3 per second

A hitch is a frame shown again while the picture is changing on both sides: the build missed a refresh. A still scene does not count, and a hold of 250 ms or more is reported separately as a freeze (a stall or a deliberate pause). A game capped below the capture rate (30 in 60) is reported as a cadence, and only holds beyond it count. On a 30 s Overwatch capture it reports exactly the three single-frame repeats found by hand. Side-by-side and top-bottom stereo captures are detected and analysed on the left eye. Turn speed is image rotation: in a headset mirror capture it includes the player's own head turns, so it matters most for artificial turning, cutscenes and trailers. The flash check approximates WCAG 2.3.1 for triage. It is not a certified photosensitivity test.

Live: start a session on a camera (camera), the screen, a stream or a file played in real time. Here, a test clip where a hand gives a thumbs up from 3 s to 6 s (real output):

$ 9xvision live start hand-demo.mp4 --hands --watch-for "a thumbs up"
[9xvision] monitor: http://127.0.0.1:8899/?k=...
$ 9xvision live now
live: ./hand-demo.mp4 at 00:07.3 into the session; 15.0 fps, 1.4 ms per frame
- affect: energy 0.0, arousal 0.495 (0.5 = typical for this session) [warming up]
- picture: motion 0.0, brightness 0.428, palette #6B6F6C
- faces: none
- watching for: a thumbs up 0.000
- recent events: gesture_start(Thumb_Up) @00:03.2, seen_start(a thumbs up) @00:03.8, gesture_end(Thumb_Up) @00:06.5, seen_end(a thumbs up) @00:06.9
$ 9xvision live snapshot 30        # the last 30 s become a normal watch, plus a "Live signals" section

Session time starts at the first frame. A gesture registers after a 0.2 s hold (3.0 → 3.2) and ends after a 0.5 s vote (6.0 → 6.5); a visual trigger, scored by SigLIP, needs two checks half a second apart (3.8). The clip has no sound, so there is no sound line. On a MacBook camera with a face in view, a frame takes about 10 ms.

Every message is a JSON state (about 15 a second) or an event. Over OSC the same numbers arrive as addresses:

OSC address

what

/9xv/affect/energy, /arousal, /expression_valence, /engagement

affect (below)

/9xv/audio/db, /onset, /bpm, /beat_phase, /pitch_hz, /speaking

sound

/9xv/video/motion, /brightness, /saturation, /warmth, /hue, /pan

picture

/9xv/face/smile, /jaw_open, /brow_raise, /yaw, /pitch, /face/blend/<name>

face, all 51 blendshapes

/9xv/hands/left/x, /y, /pinch (1 = thumb and index touching), /gesture

hands

/9xv/face/present, /9xv/hands/left/present, /right/present

1 or 0: nothing else is sent for an absent face or hand, so gate on these

/9xv/watch/<query>

visual trigger scores

/9xv/event/beat, /cut, /onset, /smile_start, /nod, /shake, /gesture_start, /seen_start, ...

events

Beat events can fire on the beat rather than after it (the grid is predicted ahead, --beat-lead covers output latency), and beat_phase runs 0→1 between beats for synced visuals.

What the affect numbers mean. energy is absolute (loudness, onsets, motion), so the same scene always gives the same value. arousal is activation compared with this session so far: 0.5 is typical for the session, because people and rooms differ. expression_valence is the face's smile against frown, brow-lowering and sneer, measured against that person's resting face, so someone who smiles the whole time reads near 0. It reads the expression, not the feeling: people smile when uncomfortable and frown when concentrating. Smiles, blinks per minute, nods, head shakes, looking away and gestures come as events; nodding along gives a nod about every 1.5 s. Nothing identifies anyone: there is no face recognition.

Privacy. Live input is processed in memory on your machine. Nothing is written to disk unless you ask: live now keeps its newest frames in the session folder until the session stops; snapshot and --jsonl files stay. The WebSocket listens on 127.0.0.1. Browsers can only connect from the monitor page or an origin you allow with --allow-origin, so a web page you happen to visit can't read your camera's signals. Requests with a non-local Host are refused (no DNS rebinding), and frames and control need the session's token. Agents are told to start the camera or microphone only when the user asked.

Images: a 4200 × 2600 screenshot, where the overview's OCR garbles the fine print:

$ 9xvision image settings.png
{"source_px": [4200, 2600], "grid": {"cols": 5, "rows": 3, "cell_px": [840, 867]},
 "text": [{"text": "Quarterly settings", "cell": "A1"}, {"text": "bulla /T3a-19 ' V2.4.1 (Tine prlnt)", "cell": "E3"}], ...}
$ 9xvision image settings.png --region D3:E3
{"box": [2520, 1733, 1680, 867], "text": ["build 7f3a-19 • v2.4.1 (fine print)"], ...}
$ 9xvision images ~/Pictures/trip           # album: INDEX.md with dates, cameras, text, duplicates
$ 9xvision search "a thumbs up" --watch trip
visual matches in album trip-album-3a57952a: #5 thumb_up.jpg strong 0.83 · #2 fist.jpg weak 0.0015 · ...

HEIC, AVIF, TIFF, WebP, PNG, JPEG and GIF are read, with EXIF orientation applied.

Related MCP server: claude-real-video

Install

uv tool install git+https://github.com/ASGARD-STUDIOS/9xvision
9xvision setup

setup plugs 9xvision into every agent it finds: Claude Code, Codex, Cursor and Claude Desktop, as an MCP server plus the skill. It also offers the optional pieces: on macOS it installs them with Homebrew (it asks first); on Linux it prints the commands. Those pieces are local speech-to-text (whisper.cpp plus a 466 MB model), tesseract for OCR where Apple Vision is missing, the visual search model (SigLIP, 210 MB), and face/hand/body tracking for live input (MediaPipe in its own environment, about 350 MB). Then restart the agent and ask it about a video. 9xvision doctor shows what's installed and connected; 9xvision setup --remove undoes it. Update later with uv tool upgrade ninexvision (the package name; the command is 9xvision).

Requires Python 3.11+ and ffmpeg. Tested on macOS (Apple Silicon), Ubuntu x86-64 and Linux ARM64, Python 3.11 to 3.14. On Linux, face/hand/body tracking also needs sudo apt install libegl1 libgles2 libgl1.

No uv yet? curl -LsSf https://astral.sh/uv/install.sh | sh (macOS/Linux). Manual routes: Claude Code /plugin marketplace add ASGARD-STUDIOS/9xvision then /plugin install 9xvision@9x; any MCP client: command 9xvision-mcp. Any other agent with a shell: put 9xvision on PATH and give it the text from 9xvision skill.

Use

9xvision watch recording.mp4                          # prints INDEX.md, caches everything (~/.9xvision)
9xvision watch "https://youtu.be/..." --start 12:00 --end 15:00
9xvision watch "https://www.instagram.com/reel/..." --cookies-from-browser chrome   # login-gated sites
9xvision search "a red car at night"                  # on-screen text, speech, and what's in the picture
9xvision zoom <watch> 01:20 01:30 -n 16
9xvision frame <watch> 00:43.2 --crop 0,0.6,1,0.4
9xvision diff <watch> 00:17.0 00:18.3
9xvision qa capture.mp4 --fov 110                     # game / XR / app capture QA
9xvision look <watch> view:12
9xvision live devices                                 # cameras, microphones, screens
9xvision live start camera --osc 127.0.0.1:9000       # stream to TouchDesigner / Max / Ableton
9xvision live start screen --no-face                  # watch the screen
9xvision live start "https://youtube.com/watch?v=<live id>"   # a live stream (or rtsp://, srt://)
9xvision live now | snapshot 30 | watch-for "a dog" | stop
9xvision image photo.heic --region B2                 # one part at native resolution
9xvision images ./screenshots                         # an album; then search / look image:N

The session also serves http://127.0.0.1:8899/mirror: a cartoon face driven by the blendshapes, coloured by energy and arousal, pulsing on the beat. It is about 90 lines of plain JavaScript, meant as a template for your own affective visuals (src/ninexvision/live_mirror.html).

On macOS, the first camera, microphone or screen use asks for permission for the terminal (or agent app) running 9xvision. A web game or app can subscribe with new WebSocket("ws://127.0.0.1:8899/") once its dev origin is allowed: --allow-origin http://localhost:5173.

The agent reads INDEX.md, looks at the overview contact sheets, and drills down only where a question needs it. Every image an agent is shown is capped at 1568 px on the long edge, so the model never silently downscales it, and its token cost is printed; frame --crop gives full resolution.

Processing takes about 20–40 s for a short clip, then about 1.5 s per minute for long videos (a 74-minute keynote: ~100 s), once. After that every question is instant.

URLs go through yt-dlp: YouTube, Vimeo, TikTok, X, Instagram and about 1,800 other sites. Playlists are refused with the list of video URLs, live streams need --start/--end, and a failure comes back with the next step (update yt-dlp, use browser cookies, check the URL).

Benchmark

The same agent (Claude Code headless, Claude Sonnet 5.5) answered the same questions about the same videos with and without 9xvision. Both arms had the whole machine: ffmpeg, ffprobe, whisper.cpp, tesseract and python. The kit arm also had 9xvision (v0.1) and the skill text, and started from an empty cache each run, so its processing time counts. Answers were graded blind by Claude Opus 5.5 against answer keys written and verified from raw frames before the runs. 3–4 runs per cell, October 2026.

Video

Task

Cost / run

Agent turns

Accuracy without → with

Wall time

iPhone app onboarding, 97 s

12 questions

$0.27 → $0.25

20 → 15

99% → 99%

84 s → 109 s

same recording

full screen-by-screen teardown, 14-point rubric

$0.49 → $0.29

27 → 13

98% → 96%

157 s → 128 s

music video, 2:37, 110 shots

12 questions

$0.33 → $0.25

28 → 15

92% → 98%

98 s → 99 s

keynote segment, 8 min, captions

12 questions

$0.27 → $0.18

21 → 13

94% → 100%

56 s → 68 s

full keynote, 74 min, captions

8 needle questions (on-screen only)

$0.43 → $0.42

39 → 28

98% → 98%

156 s → 287 s

all

−22%

−38%

96% → 98%

What this means, without spin:

  • The clear win is cost and effort: 22% cheaper and 38% fewer agent turns for the same or better answers, and a cache that makes every later question about the same video instant.

  • Accuracy: a frontier agent with ffmpeg is already a strong video reader. 96% → 98% is within noise at 3–4 runs per cell. The misses the kit avoids are on fast, visual-only detail (0.5 s inserts in a music video, a file path on a slide).

  • Wall time is often worse on a single question, because the first watch processes the whole video. Use --start/--end for a long video when the question is about one stretch.

  • It builds things the baseline never does: the UI flow, every screen's copy, palettes, a colour script, stitched pages, beat alignment, capture QA.

  • The visual search, sound and capture QA added in v0.3 are not in this benchmark yet. They are tested against known answers (a click track, synthetic captures with injected hitches, answer-keyed shots) in tests/.

The answer keys were corrected three times after inspecting failures; each time the key was wrong, not the run, and both arms were re-graded. Harness and grader: bench/. The videos and keys are not in the repo: one recording is private and the others are copyrighted.

How it works

One ffmpeg decode pass at 12 fps (6 fps over an hour) computes per-frame signals: HSV content change, changed-pixel fraction, phase-correlation shift, luma and sharpness. Status bars and home indicators are masked on phone recordings.

  • Screen mode finds stable UI states with an adaptive quiet threshold and keeps screens that never hold still as "moving" states. It clusters revisits into views on sharper 480 px frames, so toggles and checkmarks count. Transitions are classified by motion and change area, scroll groups are merged, and pages are stitched.

  • Film mode detects cuts with adaptive thresholds and fades, then re-decodes a short window around each at the native frame rate to place it on the exact frame (102 of 103 cuts on a music video, checked against native frame differences; a dissolve or whip keeps its analysis time). Content beats are added where the picture changes and holds without a cut. Keyframes are the sharpest frame in the middle of each shot.

  • Sound: spectral-flux onsets, tempo from autocorrelation, Ellis' dynamic-programming beat tracker (beat times within 2 ms on a click track; an outside check found 87–99% of beats within 50 ms of librosa's on pop songs, and Rick Astley at 113.2 BPM vs the published 113), RMS loudness.

  • Visual search: SigLIP base (Google, Apache-2.0) on onnxruntime, CPU. Frames every 0.5–3 s plus every shot keyframe and screen, embedded once per watch.

  • Capture QA: every frame at its native rate, small and grey; repeated frames, phase-correlation catch-up, packet timestamps, luminance swings.

  • Text: Apple Vision OCR on macOS (in a watchdog process), tesseract elsewhere.

  • Live: one ffmpeg process delivers frames and audio together, and both are stamped on the same content clock, so sound events land within ±17 ms of the true time (a click track), and picture events land at the next analysis frame (66 ms at 15 fps). Audio runs per 23 ms hop: spectral flux, YIN pitch, tempo by autocorrelation every 0.5 s, and a beat grid predicted forward (predicted beats land 11 ms after the clicks, range 6–18). Faces, hands and body run through MediaPipe in a separate process that always takes the newest frame, so tracking never falls behind (about 4–5 ms per frame for faces on an M4 Max; 32 ms with hands and body too). A face too small for MediaPipe's short-range detector is found by scanning tiles, then followed with a crop around it. Processing a 640 px frame takes about 10 ms, at 15 fps. Visual triggers run SigLIP on the newest frame twice a second and need two scores in a row, so they react in about 0.7 s.

  • Images: Pillow, with macOS sips or ffmpeg for HEIC/AVIF; the grid keeps each cell under about 1,000 native pixels; small regions are enlarged up to 4× for legibility; duplicates by 64-bit difference hash.

  • Speech: captions first (yt-dlp or a sidecar .vtt), then whisper.cpp locally.

Limits

  • The screen-mode +/− labels are a colour heuristic: right on a journaling app's onboarding toggles, wrong on some iOS settings sheets. Confirm with diff.

  • Scroll stitching is experimental and only runs when consecutive states share content.

  • Sound covers tempo, beats, loudness and speech. It does not describe instruments, mood or sound design.

  • OCR quality follows the source resolution; on 720p screen shares, small UI text needs frame --crop. Stylised titles are often missed.

  • Visual search is as good as SigLIP base: strong on objects, scenes and colours; looser on actions; no negation, counting, fine text or small details.

  • Capture QA works from the video alone, so it sees what the viewer saw: a hitch hidden by the capture tool, or added by its encoder, is invisible or indistinguishable. Engine frame-time logs are the ground truth when you have them.

  • Live affect is expression and activity, not emotion: it can't tell what someone feels, and it should not be used to judge people (hiring, school, policing; the EU AI Act bans emotion recognition at work and in education). It is built for art, music, games and your own sessions.

  • Live picture analysis runs at 15 fps by default, so a one-frame event in 30 or 60 fps footage can fall between samples (--fps 30 if it matters). Faces need to be roughly 50 px wide in the analysis frame; for a room or stage camera use --width 1280.

  • Hands are labelled as the person's own left/right for a normal (unmirrored) camera; pass --mirrored for a selfie view. Head pitch direction follows MediaPipe's convention and has not been checked against a measured head.

  • Visual triggers (--watch-for) judge single frames: use them for things you can see in a still ("a dog", "a red jacket", "a laptop"). Actions such as nodding or waving are guesses for them (an outside test saw "a person nodding" fire on a head shake); use the face and hand events for those.

  • Hand gestures are MediaPipe's seven (Closed_Fist, Open_Palm, Pointing_Up, Thumb_Down, Thumb_Up, Victory, ILoveYou); recognition drops with hand angle and distance (an outside test: a peace sign recognised in about 1 frame in 20 at a side angle).

  • Live cut detection is a light per-frame check (in an outside test it found 69% of the cuts watch found, 75% precise); snapshot the stretch and use its shots for exact cuts.

  • Live sources need a picture: an audio-only file or stream is refused for now.

  • Live input is tested on macOS (camera, microphone, screen, files, streams) and with files on Linux. Linux cameras (v4l2) and Windows are untested.

License

MIT © Koto Murai

Available Tools

15 tools
diffC

Compare two moments: side-by-side plus highlighted changed regions (pixel boxes in source coordinates).

NameRequiredDescriptionDefault
aYes
bYes
watch_idYes

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it only partially does so: it discloses the return shape (side-by-side image plus changed-region pixel boxes in source coordinates), which is genuinely useful. It says nothing about whether this is read-only, whether it captures new frames or reads stored ones, or cost/latency for a visual operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the output description is packed efficiently into the same line. Brevity is appropriate, though the terse phrasing contributes to the ambiguity noted elsewhere.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Three required parameters at 0% schema coverage, no annotations, and no output schema mean the description should be doing much more. Describing the return value helps, but the missing parameter formats and safety profile leave an agent unable to call this correctly without guessing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% across three required params, so the description must compensate and largely doesn't. "Two moments" loosely maps to a and b, but their format (timestamp, frame ID, image reference?) is never stated, and watch_id is entirely unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a verb and resource ("Compare two moments") and even describes the output shape (side-by-side plus highlighted pixel boxes). However, "moments" is undefined and nothing distinguishes this from siblings like frame, look, or image, so an agent can't confidently pick it over those.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use or when-not-to-use guidance, and no mention of alternatives such as frame or look. The verb "compare" implies intent but the agent must infer the scenario and prerequisites entirely.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

frameC

Full-resolution frame at a time. crop = "x,y,w,h" in pixels or 0-1 fractions, for reading small detail.

NameRequiredDescriptionDefault
atYes
cropNo
watch_idYes

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden but says nothing about read-only behavior, side effects, permissions, or failure modes. It confirms that crop exists and is for small detail, but does not describe what state the call changes or what the return looks like.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The text is brief and the crop detail is front-loaded, but 'Full-resolution frame at a time' is a fragment that reads more like a note than a complete purpose statement. It is concise but under-specified, forcing the agent to infer too much from a single sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 3 parameters, 2 required, 0% schema description coverage, no annotations, and no output schema, the description is far too thin. It explains crop but leaves the required timing parameter and the watch identifier completely undocumented.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain all three parameters. It does an excellent job on crop (pixel vs. fractional coordinates), but completely omits the required 'at' and 'watch_id' parameters, leaving their meaning and format unknown.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description implies retrieving a full-resolution frame, but uses a noun phrase rather than a specific verb and never states what resource it operates on (a watch). It is distinct from siblings like zoom or image, but the core action is only inferred, not stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The only usage hint is 'for reading small detail,' which explains why crop exists but not when to choose frame over look, zoom, image, or images. No alternatives or exclusions are named, leaving the agent to guess.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

imageA

One image (file or URL). Without region: an overview with a lettered grid (when the image is larger than you can see at full resolution), plus EXIF, palette and OCR text with grid cells. With region ("B2", "B2:C3", or "x,y,w,h" in px or 0-1 fractions): that part at native resolution, enlarged when small. Use it to read small text in big screenshots and photos instead of guessing.

NameRequiredDescriptionDefault
regionNo
sourceYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses that the no-region path returns a lettered grid plus EXIF, palette, and OCR-with-grid-cells, and that region requests return a native-resolution crop enlarged when small. It omits operational limits like max file size, auth, or whether source accepts data URIs/base64, leaving some behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is front-loaded ('One image (file or URL)') and then cleanly splits the two modes with 'Without region...' / 'With region...'. Every clause contributes either coordinate format, output content, or usage rationale, with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 2-parameter, no-output-schema, no-annotation tool, the description covers both input semantics and the shape of returned content in each mode, which is most of what an agent needs. Minor gaps remain around source formats (data URI vs. path vs. URL) and image size limits.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does: it spells out accepted region syntaxes ('B2', 'B2:C3', 'x,y,w,h' in px or 0-1 fractions) and defines the default behavior when region is omitted. It also clarifies that source is a file or URL, giving meaning to both parameters beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource ('One image (file or URL)') and clearly delineates two operating modes: overview with a lettered grid + EXIF/palette/OCR, and native-resolution region extraction. It differentiates itself behaviorally (region zoom vs. full overview) but never explicitly contrasts against near-name siblings like 'images', 'zoom', or 'look'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear use case: 'Use it to read small text in big screenshots and photos instead of guessing,' which tells the agent when this tool is the right pick. It does not name alternative siblings (e.g., 'zoom', 'look') or state when not to use it, so the guidance stops short of explicit routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

imagesB

Index images (folders, files, globs, URLs) into an album: thumbnails, dates, cameras, OCR text, palettes, near-duplicate groups and a visual index. Returns INDEX.md and the first contact sheet. Then search(query, album_id) finds images by content or text, look(album_id, "image:N") gives an original, image(path, region) reads detail.

NameRequiredDescriptionDefault
sourcesYes
recursiveNo

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It discloses return artifacts such as INDEX.md and a contact sheet, but omits important mutation details like whether an album is created or overwritten, permission requirements, reversibility, or rate limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two dense sentences and front-loads the indexing action and outputs. The second sentence shifts into sibling tool calls, which is useful context but slightly beyond what is needed to invoke this tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, extracted metadata, and return values, which is helpful given there is no output schema. But with no annotations, no output schema, and an undocumented recursive parameter, album lifecycle and side effects remain incomplete for a tool that indexes images into an album.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It usefully explains that sources can be folders, files, globs, or URLs, but it never mentions the recursive boolean, leaving one of two parameters without any semantic guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action, 'Index images ... into an album,' and enumerates what is generated and returned. It also names follow-on sibling tools search, look, and image, making its role fairly clear, though it stops short of explicitly differentiating when this tool is preferred over those siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies a workflow: index sources into an album, then use search, look, or image for retrieval and detail. However, it does not explicitly state when to use this tool versus alternatives, nor does it list prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_watchesB

Previously watched videos (cached; re-watching is instant).

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
resultYes

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does add real value by disclosing that results are cached and re-watching is instant. However, it does not clarify that this is a read-only enumeration, whether results are ordered or limited, or how it relates to starting a watch.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single compact sentence with zero waste, and the core concept (previously watched videos) is front-loaded. It is arguably too terse rather than bloated, but structurally efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be explained, and there are no parameters to document. Still, for a history-listing tool sitting among many viewing siblings, the description omits how it should be chosen over watch or live_watch_for, leaving a meaningful gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline of 4 applies; nothing in the description is needed to explain inputs, and it introduces no misleading parameter expectations.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the resource (previously watched videos) but uses a noun phrase with no explicit verb; the listing action is only inferable from the tool name. An agent can guess it returns watch history, but it isn't stated as 'list all watched videos' and there is no differentiation from siblings like watch or live_watch_for.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no exclusions, and no reference to any of the many siblings (watch, live_watch_for, search, look). The agent must infer that this is the history-listing tool entirely on its own.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

live_nowA

What the running live session sees and hears right now: a summary, the latest frame, and recent events.

NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the return composition (summary, latest frame, recent events), which is useful behavioral context, but omits whether the operation is read-only, how it behaves without a live session, or any permission/rate constraints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single efficient sentence with the key information front-loaded. No wasted words or unnecessary structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, no-output-schema tool, the description adequately explains what is returned. It could be slightly more complete by mentioning behavior when no live session exists, but it covers the essential return contents.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there are no parameter semantics to explain. Baseline 4 applies for a no-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the resource and scope: current state of a running live session, including summary, latest frame, and recent events. However, it does not explicitly distinguish this tool from close siblings like live_snapshot or frame.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'right now' hints at temporal relevance, but there is no explicit guidance on when to use this tool versus live_snapshot, frame, live_watch_for, or other siblings. No exclusions or alternative tools are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

live_snapshotA

Turn the last seconds of the live session (up to the buffer, 120 s by default) into a normal watch: shots, text, speech, sound, plus a 'Live signals' section (affect, smiles, nods, gestures, triggers). All watch tools (zoom, frame, diff, search) then work on it.

NameRequiredDescriptionDefault
secondsNo

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It usefully discloses scope (capped at the buffer, ~120 s) and the produced contents (shots, text, speech, sound, Live signals), but omits whether a live session must be running, whether the snapshot is registered in list_watches, and what happens if `seconds` exceeds the buffer.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense sentences with the core action front-loaded and the downstream capability stated second; little waste, though the parenthetical buffer note adds a mildly confusing default value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description does a good job enumerating what the resulting watch contains and how it can be used downstream. The remaining gap is the lack of preconditions and the buffer/default ambiguity, which matter for a session-dependent tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the single `seconds` parameter depends entirely on the description. It explains that it selects the last N seconds capped by the buffer, but the claim '120 s by default' conflicts confusingly with the schema default of 30 and leaves the actual truncation behavior ambiguous.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: turns the last `seconds` of the live session into a normal watch. That clearly separates it from siblings like live_watch_for and live_now, though those alternatives are never named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for use: convert buffered live footage so that 'All watch tools (zoom, frame, diff, search) then work on it', which names the follow-up tools. It does not state prerequisites (e.g., an active live session) or when to prefer live_now or live_watch_for instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

live_startA

Start live perception in the background: source = "camera" (or "camera:N"/name), "screen" ("screen:N"), a stream URL (rtsp/srt/YouTube live) or a file played in real time. It tracks motion, cuts, colour, sound (level, beat, voice pitch, speech), faces (51 blendshapes, head pose: expression, not feelings; no identities), optional hands (gesture, pinch = thumb and index touching) and body, affect (energy, arousal vs this session, expression valence, engagement) and visual triggers (watch_for: ["a dog"]). It also streams to a monitor page and to OSC if given. Only start the camera or microphone when the user asked for it. Then use live_now, live_snapshot, live_stop.

NameRequiredDescriptionDefault
micNoauto
oscNo
poseNo
handsNo
sourceNocamera
watch_forNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it discloses background execution, optional monitor-page and OSC streaming, opt-in hands/pose cost, and a privacy guarantee (blendshapes and head pose, "no identities"). It omits failure modes such as a camera already in use or what happens on a second start.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the action and source set, then the tracked signals, then the consent rule and next steps. Dense but nearly every clause adds usable information; the parenthetical affect and blendshape detail runs long.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values needn't be spelled out, and the description covers sources, opt-ins, consent, and follow-up tools for a six-parameter tool. The unexplained mic parameter is the notable hole.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate and largely does: source formats (camera, camera:N/name, screen:N, rtsp/srt/YouTube URL, file), watch_for with a concrete example, hands with pinch/gesture meaning, pose as body, and osc as output routing. The mic parameter and its "auto" default are never explained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (start live perception in the background) and enumerates the accepted source types plus everything it tracks. It is clearly distinguished from live_now/live_snapshot/live_stop, which it names as follow-on tools, and from frame/watch style one-shot siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit consent constraint ("Only start the camera or microphone when the user asked for it") and routes to the three follow-up tools. It does not name an alternative for non-realtime or one-shot needs, so it stops short of full when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

live_stopA

Stop the live session (camera, microphone and streams close).

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It does well to disclose the concrete effects (camera, microphone and streams close), but says nothing about idempotence, behavior when no session is active, or whether the session can be resumed afterward.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short sentence, front-loaded with the action and immediately qualified by the affected resources. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists and there are no parameters, so the description only needs to convey the effect, which it does. Only the no-active-session edge case is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing to disambiguate; the baseline of 4 applies. Nothing in the description contradicts the empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Stop') and resource ('the live session') and the parenthetical enumerates exactly what is torn down (camera, microphone, streams). An agent can immediately tell this is the teardown counterpart to live_start without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the sibling set contains live_start, so calling live_stop after live_start is obvious. The description gives no explicit when-to-use or precondition (e.g., whether an active session must exist, or what happens if none does).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

live_watch_forC

Change the running session's visual triggers (descriptions of things to watch for).

NameRequiredDescriptionDefault
addNo
removeNo

Output Schema

NameRequiredDescription
resultYes

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does not disclose whether adding triggers replaces or appends to the existing set, whether removal of a nonexistent trigger errors, or whether changes persist after `live_stop`. "Change" signals mutation but the mechanics and reversibility are left entirely implicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with no filler, and the resource is front-loaded after the verb. It is efficient but so short that it omits necessary operational detail rather than being optimally dense.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return values needn't be explained, but for a mutation tool with zero annotation coverage and 0% parameter documentation, the description leaves the add/remove semantics and session prerequisite unaddressed. It is not complete enough for an agent to invoke this confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% — `add` and `remove` are bare arrays of strings with no descriptions. The description never mentions these two parameters, so it fails to compensate: an agent cannot tell whether `add` is additive or a full replacement, or how strings are matched for `remove`.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ("Change") and resource ("visual triggers") and adds a clarifying parenthetical that these are descriptions of things to watch for. It scopes the change to the running session, which implicitly separates it from a persistent `watch` tool, though it never names a sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this versus `watch`, `list_watches`, or `live_start`. The phrase "running session" hints that a live session must already exist, but no prerequisite or exclusion is stated outright.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

lookC

Show an artefact of a watch: sheet:N, timeline:N, view:N (full-res screen), shot:N (keyframe), page:N (stitched long page), colour (colour script).

NameRequiredDescriptionDefault
whatYes
watch_idYes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. 'Show' implies a read operation and the artefact list adds useful context, but the description does not state permissions, side effects, error behavior, or result format.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single front-loaded sentence with an efficient inline list. It wastes little space, though the dense enumeration could be slightly clearer for an agent unfamiliar with the domain.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations, no output schema, and 0% schema description coverage, the description should do more. It explains the 'what' values but does not explain watch_id semantics, return behavior, or invocation context needed for a fully self-contained tool definition.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It usefully defines the accepted forms for 'what' (sheet:N, timeline:N, view:N, shot:N, page:N, colour), but it leaves 'watch_id' and the meaning of 'N' unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a clear verb and resource: 'Show an artefact of a watch.' It further enumerates the artefact types (sheet, timeline, view, shot, page, colour), so the agent can understand the operation. It does not explicitly differentiate 'look' from similar siblings such as frame, image, images, or zoom.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description lists valid artefact types but provides no when-to-use guidance, no exclusions, and no routing to alternatives. With many visually overlapping siblings, an agent receives no help deciding when 'look' is the right tool versus image, frame, or zoom.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

qaA

Capture QA for a game, XR (VR/AR headset) or app recording: frame pacing (hitches = frames the build failed to deliver while the picture moved, with times and ms; effective fps per second), side-by-side / top-bottom stereo detection, comfort (fast camera turns; pass the headset's horizontal fov_deg for deg/s) and flashes above 3 per second (photosensitivity triage). source = file, URL or watch id. Returns a summary plus a chart.

NameRequiredDescriptionDefault
endNo
startNo
sourceYes
fov_degNo

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden well: it explains what hitches mean, that effective fps is per second, that flashes above 3/sec are flagged for photosensitivity, and that the result is a summary plus chart. It only leaves out side effects such as whether an analysis artifact is persisted or what permissions/limits apply.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the purpose and then packs in the relevant measurement details in a single dense paragraph. Every clause relates to capability, parameter usage, or return value, though the density makes it slightly harder to scan than a bulleted structure.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must explain returns; it does so minimally with 'Returns a summary plus a chart.' For a four-parameter analysis tool with no annotations, the missing definitions for start/end and the sparse return-value detail leave meaningful gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds useful meaning for source (file, URL, or watch id) and fov_deg (horizontal headset field of view for deg/s comfort analysis), but gives no explanation for start or end, leaving half the parameters undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly defines a specific analysis resource: QA of game/XR/app recordings across frame pacing, stereo, comfort, and flash/photosensitivity metrics. It is distinguishable from the sibling tools by its diagnostic focus, though it never explicitly contrasts itself with tools like frame or live_watch_for.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It implies the tool is for analyzing recordings from a file, URL, or watch id, but does not state when to choose this over alternatives such as frame or live_snapshot. Usage is inferable from the metric list but not explicitly routed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

watchA

Analyse a local video file or URL (YouTube, Vimeo, TikTok, X, Instagram...). mode: auto | film | screen (screen = app/screen recordings). start/end ("12:30", "1:02:00", or seconds) watch only a window, and a URL downloads only that part: use it for long videos when the question is about one stretch. Times stay in full-video time. cookies_from_browser ("chrome", "safari", ...) is for login-gated URLs, only with the user's OK. Returns INDEX.md plus the first overview contact sheet. First run takes ~1-2 s per minute.

NameRequiredDescriptionDefault
endNo
modeNoauto
forceNo
startNo
sourceYes
cookies_from_browserNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations at all, the description carries the full burden and does well: it discloses the return value (INDEX.md plus first overview contact sheet), performance cost (~1-2 s per minute on first run), and auth behavior for gated URLs. The 'force' behavior and whether results are cached on subsequent runs are left unexplained.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded in the first sentence and the remaining lines each add a distinct fact (mode, windowing, cookies, return value, cost). It is dense but not padded; the middle sentence is a long run-on that slightly hurts scannability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema and no annotations, so the description correctly compensates by naming the returned artifacts and noting the initial processing time. The unexplained 'force' flag is the one remaining gap for an agent deciding how to call it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate, and it does for 5 of 6 parameters: mode values (auto | film | screen with screen explained), start/end accepted formats plus the key semantic that 'times stay in full-video time', and cookies_from_browser values/conditions. Only 'force' receives no explanation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence gives a specific verb and resource ('Analyse a local video file or URL') and enumerates supported platforms, which makes the opaque name 'watch' unambiguous. It does not explicitly distinguish itself from siblings like frame, look, or live_watch_for, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Offers concrete when-to-use guidance: start/end windowing is recommended 'for long videos when the question is about one stretch', and cookies_from_browser is restricted to login-gated URLs 'only with the user's OK'. No sibling alternatives are named for choosing between watch and frame/zoom/look, so it is clear context without explicit routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zoomB

Dense timestamped grid of frames between start and end (mm:ss.s or seconds), up to 30 frames.

NameRequiredDescriptionDefault
nNo
endYes
startYes
watch_idYes

TDQS

B3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It usefully states a hard limit ('up to 30 frames') and timestamp format, but does not explicitly confirm read-only behavior, permissions, or what happens with invalid ranges.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single tight sentence with no filler. Information is front-loaded around output shape and scope, and every phrase adds useful meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a four-parameter tool with no annotations, no output schema, and 0% schema description coverage, the description is too sparse. It omits the meaning of `watch_id`, the role of `n`, and enough return-value detail to compensate for the missing output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for four undocumented parameters. It explains the `start` and `end` time formats and implies a maximum frame count, but never explains `watch_id` or clearly defines `n` as the requested number of frames.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the resource and scope: a dense timestamped grid of frames between two timestamps, capped at 30 frames. It lacks an explicit action verb such as 'returns' or 'retrieves', but an agent can still infer the tool's function. It does not explicitly differentiate itself from the sibling `frame` tool or other frame-related siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives such as `frame`, `images`, or `look`. The interval and frame cap imply it is for multi-frame sampling, but there are no explicit when-to-use or when-not-to-use conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 15 tool updatesv0.4.1
    • First observeddiff
    • First observedframe
    • First observedimage
    • First observedimages
    • First observedlist_watches
    • First observedlive_now
    • First observedlive_snapshot
    • First observedlive_start
    • First observedlive_stop
    • First observedlive_watch_for
    • First observedlook
    • First observedqa
    • First observedsearch
    • First observedwatch
    • First observedzoom

TDQS

B3.3/5.0

Scored across 15 tools

Disambiguation4/5

Most tools target distinct actions/resources: watch analyses video, frame extracts a single frame, zoom builds a dense grid, diff compares moments, search finds moments, and the live_* tools handle real-time perception. There is some overlap among frame, look, zoom, and image for inspecting visual detail, but the descriptions mostly clarify when to use each.

Naming Consistency3/5

The set mixes noun names (frame, image, images, qa) with verb names (watch, look, zoom, diff, search) and a prefixed live_ group plus list_watches. The live_ tools are consistently prefixed, but overall there is no single predictable verb_noun or noun pattern.

Tool Count5/5

Fifteen tools fit a rich visual/video analysis domain with distinct areas: video analysis, image albums, live sessions, search, comparison, and QA. The count is well within a reasonable scope and each tool appears to serve a specific workflow.

Completeness4/5

The surface covers core workflows for video/image/live analysis: ingest, inspect, search, compare, QA, and live snapshots. Minor gaps exist around managing cached watches/albums (e.g. no delete or rename operations), but these are not central to the stated analysis purpose.

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    C
    maintenance
    Let AI agents watch videos: local transcripts, speaker labels, scenes, chapters and exact-moment search from any video URL or file. Fully local, no API keys.
    4
    3
    AGPL 3.0
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI agents to inspect UI screen recordings as token-efficient image sheets with changed regions boxed, single-frame flicker flagged, and speech aligned to frames. Exposes watch_video and zoom_video tools so agents can pull dense keyframes across any time range or one sharp full-resolution frame, all processed locally.
    25 npm
    MIT