extractKeyframes

Extract keyframe images from downloaded videos at regular intervals to analyze visual content without classification. Specify interval, format, and dimensions for raw frame output.

Instructions

Extract keyframe images from a locally downloaded video at regular intervals using ffmpeg. Requires the video to be downloaded first via downloadAsset. Does NOT do visual search or classification — produces raw frame images.

Input Schema

TableJSON Schema

Name	Required	Description
`videoIdOrUrl`	Yes	Video ID or URL (must have a local video asset)
`intervalSec`	No	Extract one frame every N seconds (default 30)
`maxFrames`	No	Maximum frames to extract (default 20)
`imageFormat`	No	Output image format (default jpg)
`width`	No	Image width in pixels, height auto-scaled (default 640)

Implementation Reference

src/lib/thumbnail-extractor.ts:49-111 (handler)

The extractKeyframes method in ThumbnailExtractor class, which performs the keyframe extraction by calculating timestamps, creating output directories, and running ffmpeg commands via extractFrames.

async extractKeyframes(options: ExtractKeyframesOptions): Promise<ExtractKeyframesResult> {
  const startMs = Date.now();
  const intervalSec = options.intervalSec ?? 30;
  const maxFrames = options.maxFrames ?? 20;
  const imageFormat = options.imageFormat ?? "jpg";
  const width = options.width ?? 1280;

  // Resolve video file path
  const videoPath = options.videoPath ?? this.findVideoFile(options.videoId);
  if (!videoPath || !existsSync(videoPath)) {
    throw new Error(
      `No local video file found for ${options.videoId}. Download the video first with downloadAsset.`,
    );
  }

  const videoProbe = await this.probeVideo(videoPath);
  const durationSec = videoProbe.durationSec;
  if (!durationSec || durationSec <= 0) {
    throw new Error(`Could not determine duration for ${videoPath}`);
  }

  // Calculate timestamps
  const timestamps: number[] = [];
  for (let t = 0; t < durationSec && timestamps.length < maxFrames; t += intervalSec) {
    timestamps.push(t);
  }

  if (timestamps.length === 0) {
    return {
      videoId: options.videoId,
      framesExtracted: 0,
      assets: [],
      durationMs: Date.now() - startMs,
    };
  }

  // Create output directory
  const framesDir = join(this.store.videoDir(options.videoId), "keyframes");
  mkdirSync(framesDir, { recursive: true });

  // Pre-fetch existing assets once for skip checks (avoids N DB queries)
  const existingByPath = new Map(
    this.store.listAssetsForVideo(options.videoId).map((a) => [a.filePath, a]),
  );

  // Extract frames using seek-based approach (fast — seeks directly to each timestamp)
  const assets = await this.extractFrames({
    videoId: options.videoId,
    videoPath,
    timestamps,
    framesDir,
    imageFormat,
    width,
    existingByPath,
  });

  return {
    videoId: options.videoId,
    framesExtracted: assets.length,
    assets,
    durationMs: Date.now() - startMs,
  };
}

src/lib/types.ts:1174-1181 (schema)

Input type definition for extractKeyframes.

export interface ExtractKeyframesInput {
  videoIdOrUrl: string;
  intervalSec?: number;
  maxFrames?: number;
  imageFormat?: "jpg" | "png" | "webp";
  width?: number;
}

src/lib/types.ts:1182-1184 (schema)

Output type definition for extractKeyframes.

export interface ExtractKeyframesOutput {
  videoId: string;
  framesExtracted: number;

vidlens-mcp