Skip to main content
Glama

Sample Frames

sample_frames
Read-onlyIdempotent

Extract representative PNG frames from a video by FPS, timestamps, or scene changes, then return the first few inline so a model can inspect what happened on screen.

Instructions

Sample PNG frames from a video (by fps, timestamps, or scene changes) and return the first few inline so the model can see what happened.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "input": "<input>"
    },
    "name": "sample_frames"
  }
}

Input Schema

NameRequiredDescriptionDefault
fpsNoFrames per second (mode fps).
modeNoscenes: only frames where the screen changed (strongest first), usually the cheapest review. Default: fps or timestamps, whichever is given.
inputYesPath to the source video file.
dedupeNoMode fps: drop near-duplicate frames (default false).
maxFramesNoFrame cap (default 100); fps is lowered to fit while still spanning the video.
outputDirNoOptional folder for the frames (a relative path resolves under SCREENCAST_HOME/frames). Defaults to SCREENCAST_HOME/frames/<id>.
timestampsNoSeconds to grab a frame at (mode timestamps).
returnImagesNoReturn the first maxInlineImages frames inline (default true).
maxImageWidthNoMax inline image width (default 1280).
sceneThresholdNoMode scenes: change needed to count, 0-1 (default 0.1).
maxInlineImagesNoFrames to embed inline (default 8); all are listed by path.

Output Schema

NameRequiredDescriptionDefault
modeYes
framesYes
frameDirYes
frameCountYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed11 schema fields changedv0.47.0
    • changedInput schema / properties / dedupe / description
      Previous value: -"Mode 'fps': drop frames that are nearly identical to the previous one (default false)."New value: +"Mode fps: drop near-duplicate frames (default false)."
    • changedInput schema / properties / fps / description
      Previous value: -"Sample this many frames per second (mode 'fps')."New value: +"Frames per second (mode fps)."
    • changedInput schema / properties / maxFrames / description
      Previous value: -"Most frames this call may write (default 100, max 1000). In fps mode a rate that would exceed it is lowered so the frames still span the whole video."New value: +"Frame cap (default 100); fps is lowered to fit while still spanning the video."
    • changedInput schema / properties / maxImageWidth / description
      Previous value: -"Downscale inline images to at most this width (default 1280)."New value: +"Max inline image width (default 1280)."
    • changedInput schema / properties / maxInlineImages / description
      Previous value: -"How many frames to embed inline (default 8, max 20). Every frame is still written to disk and listed by path."New value: +"Frames to embed inline (default 8); all are listed by path."
    • changedInput schema / properties / mode / description
      Previous value: -"How to pick frames. 'scenes' returns only frames where the screen changed (plus the first frame), strongest changes first up to maxFrames, which is usually the cheapest way to review a recording. Defaults to 'fps' or 'timestamps' depending on which of those is given."New value: +"scenes: only frames where the screen changed (strongest first), usually the cheapest review. Default: fps or timestamps, whichever is given."
    • changedInput schema / properties / outputDir / description
      Previous value: -"Optional folder for the frames. Defaults to SCREENCAST_HOME/frames/<id>."New value: +"Optional folder for the frames (a relative path resolves under SCREENCAST_HOME/frames). Defaults to SCREENCAST_HOME/frames/<id>."
    • changedInput schema / properties / returnImages / description
      Previous value: -"Also return the frames inline as images the model can view directly (default true). Only the first maxInlineImages frames are embedded."New value: +"Return the first maxInlineImages frames inline (default true)."
    • changedInput schema / properties / sceneThreshold / description
      Previous value: -"Mode 'scenes': how much the frame must change to count, 0-1 (default 0.1; raise it to keep only big changes)."New value: +"Mode scenes: change needed to count, 0-1 (default 0.1)."
    • changedInput schema / properties / timestamps / description
      Previous value: -"Explicit second offsets to grab one frame at each (mode 'timestamps')."New value: +"Seconds to grab a frame at (mode timestamps)."
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
      +  "$schema": "http://json-schema.org/draft-07/schema#",
      +  "additionalProperties": {},
      +  "properties": {
      +    "frameCount": {
      +      "maximum": 9007199254740991,
      +      "minimum": -9007199254740991,
      +      "type": "integer"
      +    },
      +    "frameDir": {
      +      "type": "string"
      +    },
      +    "frames": {
      +      "items": {
      +        "type": "string"
      +      },
      +      "type": "array"
      +    },
      +    "mode": {
      +      "type": "string"
      +    }
      +  },
      +  "required": [
      +    "frameDir",
      +    "frameCount",
      +    "frames",
      +    "mode"
      +  ],
      +  "type": "object"
      +}
  2. First observedv0.32.0

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive/closed-world, so the safety bar is low. The description still adds real value beyond them by disclosing the inline-image return behavior and its purpose ("so the model can see what happened"), which tells the agent to expect image content in the response. It does not mention that frames are also written to disk, a mild tension with readOnlyHint that stops short of a contradiction because the schema documents outputDir.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence covering what is sampled, the three strategies, and what comes back. No wasted clauses, no restatement of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present and full parameter coverage, the description needn't explain return values and is complete enough to call correctly. The only gap is that the on-disk side effect and default frame caps are left entirely to the schema, which is acceptable but means the description alone would mislead a reader into thinking this is purely a read.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and every parameter carries a substantive description (defaults, caps, mode scoping), so the schema does the heavy lifting. The description reinforces the three modes but adds no syntax or default detail beyond what the schema already states; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource ("Sample PNG frames from a video") and enumerates the three sampling strategies (fps, timestamps, scene changes), which maps directly onto the mode enum. An agent can tell immediately what this does and how the output differs from a plain screenshot or contact_sheet tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The parenthetical list of modes implies usage, and the schema's mode description adds the crucial guidance ("scenes ... usually the cheapest review"), but the description itself never says when to prefer this over siblings like contact_sheet, screenshot, or screen_diff, nor what preconditions apply. Usage is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.