Skip to main content
Glama
Sign In
video-sense

video-sense

by znplusW
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Local

video_url_start

Starts visual analysis of a public video or video-page URL, extracting timestamped frames from a chosen time range with optional stream fallback for retrieval failures.

Instructions

Analyze a public HTTP(S) video or video-page URL. Prefers streaming through one FFmpeg decoder. videoFallback=on_failure (default): after two source-stream attempts with bounded HTTP reconnects fail, downloads the complete same resolved video track (may be video-only), checks duration, then feeds the local file into the same visual analysis for unfinished ranges. never disables complete-download fallback. Prior partial images remain at their existing indices; retries may repeat media times. videoTransfer exposes mode/reason/progress/local file without signed URLs. Cancellation, source rejection, policy/disk/output/network limits must not trigger a fallback. Original mediaId and separate subtitle/audio binding remain intact. Supported media include MP4, HLS, DASH; page resolution uses yt-dlp and depends on site availability. Default analysis=continuous, quality=low (page streams prefer <=480p); overview samples 8 short windows across the range, NOT full coverage. start/end are source seconds. sourceType=auto recognizes media extensions; use direct for extensionless media or page to force extraction. A multi-video page/playlist analyzes videoIndex (1-based, default 1), not all videos. No browser cookies, DRM or live-web-stream support. Returns task metadata only by default; firstImage=true explicitly opts into waiting up to 5s for a first selected image. If an image is returned, inspect or release its batch before video_next/video_read. Use audio_url_start for text-only media processing without creating images. Limits: 1GiB network, 30min processing, 5000 images. Network analysis is independent of browser playback. Images default to format=jpeg (FFmpeg q:v4, 4:4:4); request format=png for original PNG detail. Encoding is cancellable with 2 active conversions per MCP process, at most 24 queued, a 30s queue deadline and a 10s execution deadline. Every image response is limited to 8MiB encoded bytes. Saved storyboard PNGs are retained. Acknowledge the pending batch before switching formats; replay preserves its original format and bytes.

The HOST chooses evidence depth from the user request and actual subtitle/ASR quality and coverage: text alone, selected storyboards, all storyboards, or targeted higher-quality follow-up. Subtitles, ASR and images are independently optional; choose any combination. For text-only media use audio_start/audio_url_start (audio-bearing video files/pages are accepted) without a visual job. No mandatory image retrieval, all-images rule, fixed answer format or mandatory ASR. Use video_review action=plan to record visual=none|selected|all; revise as evidence changes. Missing subtitles do NOT block usable images, and a video-only stream does not prove missing source audio. Each image response has a pending batch. Return images AND timestamp metadata to the model, inspect the chosen images, then call video_review action=ack with batchId and explicit viewedIndices (empty [] releases uninspected images without credit). ONLY THEN fetch another batch. Never accumulate many video_read calls inside one code/exec result: return control to the model after each bounded batch. Transport errors leave the batch pending: video_review replay retries it; status/reviewId support resume. Acknowledgement is a caller assertion, not proof of comprehension. Keep any semantic working notes in the Agent workspace, not in MCP evidence. Before finishing, video_review finish checks only the HOST-CHOSEN visual scope and pending image receipts, not text reading completion. The host must still judge if that scope answers the user, and state relevant gaps. Honest incomplete-work disclaimers do not justify abandoning feasible work. session_stop cleanup cancels processing but does not complete reading; user_cancelled is only for an explicit user cancellation. Audio-only workflows remain independent.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>"
    },
    "name": "video_url_start"
  }
}

Input Schema

NameRequiredDescriptionDefault
endNo
urlYes
paceNo
sceneNo
startNo
formatNo
qualityNo
samplesNo
analysisNo
firstImageNo
sourceTypeNo
videoIndexNo
videoFallbackNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • addedInput schema / properties / firstImage
      Added value: +{
+  "type": "boolean"
+}
  2. First observedv0.9.0

TDQS

A3.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare non-read-only, open-world, non-idempotent. The description adds substantial context beyond annotations: streaming fallback logic, retry semantics, cancellation behavior, queue limits (2 active, 24 queued), 8MiB image cap, and the full batch acknowledgement protocol.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Extremely long and dense, with multiple topics interleaved without clear sectioning. Key usage guidance is buried in a wall of text. While many sentences carry necessary detail for a complex tool, the lack of structure and front-loading makes it hard to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers many complex behaviors and workflow steps, including fallback, limits, batch protocol, and the audio alternative. Missing parameter definitions for pace/scene/samples and no explicit contrast with video_start. No output schema exists, so the described return value ('task metadata only') is sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must carry parameter meaning. It explains videoFallback, sourceType, videoIndex, start/end, analysis, quality, firstImage, and format. But it leaves 'pace', 'scene', and 'samples' (2–16) undefined, and only indirectly references samples via 'overview samples 8 short windows'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Analyze a public HTTP(S) video or video-page URL.' It distinguishes from audio_url_start for text-only processing, but does not contrast with sibling video_start, leaving ambiguity about URL vs local media.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names audio_url_start as the alternative for text-only media and gives sourceType guidance (auto/direct/page). However, it provides no guidance on when to choose this over video_start or video_frame, and offers no exclusions for unsupported cases beyond the stated limits.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools