video-sense

Overview Schema Source Related Servers Discussions Admin

video_start

Starts adaptive visual sampling of a local video and returns a job ID for incrementally fetching selected frames and timestamps as evidence.

Instructions

Start adaptive visual sampling of an absolute local video path. Returns a job ID immediately. Read video_next incrementally and video_sheet (zero-based) as needed for the host-chosen evidence range; video_status reports separate visual/text states and mediaId. Saves actual selected images and timestamps. Does not implicitly start audio/text; add transcript_start when needed. Default quality low, pace normal. scene=true gives one SELECTED frame per image, not every second.

The HOST chooses evidence depth from the user request and actual subtitle/ASR quality and coverage: text alone, selected storyboards, all storyboards, or targeted higher-quality follow-up. Subtitles, ASR and images are independently optional; choose any combination. For text-only media use audio_start/audio_url_start (audio-bearing video files/pages are accepted) without a visual job. No mandatory image retrieval, all-images rule, fixed answer format or mandatory ASR. Use video_review action=plan to record visual=none|selected|all; revise as evidence changes. Missing subtitles do NOT block usable images, and a video-only stream does not prove missing source audio. Each image response has a pending batch. Return images AND timestamp metadata to the model, inspect the chosen images, then call video_review action=ack with batchId and explicit viewedIndices (empty [] releases uninspected images without credit). ONLY THEN fetch another batch. Never accumulate many video_read calls inside one code/exec result: return control to the model after each bounded batch. Transport errors leave the batch pending: video_review replay retries it; status/reviewId support resume. Acknowledgement is a caller assertion, not proof of comprehension. Keep any semantic working notes in the Agent workspace, not in MCP evidence. Before finishing, video_review finish checks only the HOST-CHOSEN visual scope and pending image receipts, not text reading completion. The host must still judge if that scope answers the user, and state relevant gaps. Honest incomplete-work disclaimers do not justify abandoning feasible work. session_stop cleanup cancels processing but does not complete reading; user_cancelled is only for an explicit user cancellation. Audio-only workflows remain independent.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "file": "<file>"
    },
    "name": "video_start"
  }
}

Input Schema

Name	Required	Description	Default
`end`	No
`file`	Yes
`pace`	No
`scene`	No
`start`	No
`quality`	No

Schema Changelog

Changes observed during successful MCP inspections.

First observedv0.9.0Nov 5, 2026

TDQS

A4.1/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare a non-read-only, non-idempotent, non-destructive local tool, and the description adds a great deal beyond them: immediate async job ID, persisted selected images and timestamps, defaults of quality=low and pace=normal, the semantics of scene=true as 'one SELECTED frame per image, not every second', and no implicit audio/transcript start. It also discloses failure/resume behavior ('Transport errors leave the batch pending: video_review replay retries it'). Nothing here contradicts readOnlyHint=false; writing evidence files is consistent with a non-read-only tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose is front-loaded, but the bulk of the text is workflow policy for other tools and for the host's reasoning process (batch acknowledgement loop, 'Never accumulate many video_read calls inside one code/exec result', finish-check semantics, disclaimers about honest incompleteness). Those rules belong to video_read/video_review or a top-level instruction, not to the start tool. Every sentence about job startup could survive alone; the rest dilutes it.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a stateful, multi-tool workflow entry point with no output schema, the description does a good job: it explains the immediate return (job ID), the follow-up tools by name, the mediaId/status split, and how audio and visual tracks relate. What is missing is the time-range semantics of start/end, which is the one caller-facing gap. The apparent completeness is partly borrowed from instructions that arguably belong elsewhere.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the full burden, and it only partially meets it: it documents defaults for quality ('low') and pace ('normal') and explains scene=true, plus that file is an 'absolute local video path'. The start and end parameters — which define the evidence range and are the main source of caller error — are never explained (units, inclusivity, defaults). Three of six parameters gain real meaning, the rest do not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening line names a specific verb and resource: 'Start adaptive visual sampling of an absolute local video path', and adds the immediate outcome ('Returns a job ID immediately'), which tells an agent this is an asynchronous job-starter. It implicitly rather than explicitly separates itself from video_url_start by scoping input to a 'local video path'. The core purpose is unambiguous, though the signal is diluted by the long procedural body that follows.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit alternatives with conditions are given: 'Does not implicitly start audio/text; add transcript_start when needed' and 'For text-only media use audio_start/audio_url_start ... without a visual job'. It also states that images and text are independently optional and that a video-only stream does not prove missing source audio, which prevents a common mis-selection. Downstream routing to video_review action=plan/ack and video_next/video_sheet is spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server