Skip to main content
Glama

framedeck

Coding agents can't watch video. This makes video legible to them.

Point framedeck at a TikTok, Reel, Short, or local file. It downloads the video, works out which frames are actually different, writes them to disk, and hands back a manifest of file paths the agent reads as images.

framedeck extract "https://vt.tiktok.com/ZSVKJNPcM/" -o ./out -n 10
tikwm  ยท  editmateproject  ยท  36.0s
  Cover undangan nggak harus foto prewed ๐Ÿ‘€ Foto masa kecil kalian juga bisa โ€ฆ

10 distinct frames โ†’ ./out
  [first ] t=   0.20s  frame-01-t0000.20.jpg
  [scene ] t=   6.50s  frame-02-t0006.50.jpg
  [scene ] t=  11.27s  frame-03-t0011.27.jpg
  [scene ] t=  13.07s  frame-04-t0013.07.jpg
  โ€ฆ
contact sheet: ./out/contact-sheet.jpg
manifest:      ./out/manifest.json

14 seconds end to end for that clip.

How it works

framedeck pipeline

Source: docs/pipeline.mmd โ€” Mermaid, rendered with line9: line9 render docs/pipeline.mmd --out docs/pipeline.svg --theme blueprint

Related MCP server: glideit

Why not just yt-dlp and ffmpeg

You can absolutely yt-dlp | ffmpeg -vf fps=1/2 yourself. Three things go wrong when you do, and framedeck exists for exactly those three:

1. Uniform sampling is the wrong sampler. A 36-second scroll-through of a UI holds maybe nine distinct screens. Sampling every two seconds gives you eighteen frames, half of them mid-scroll blur, and still misses the screen that was only on camera for 800 ms. framedeck lets ffmpeg's scene filter nominate candidates, adds uniform anchors so a slow continuous scroll isn't skipped entirely, then drops near-duplicates by perceptual hash. You read ten frames instead of eighteen and see more.

2. Social extractors break. yt-dlp's TikTok path was returning Unable to extract universal data for rehydration the day this was written โ€” including on nightly. framedeck tries providers in order (local file โ†’ yt-dlp โ†’ tikwm โ†’ oembed cover) and only fails when all of them do. When the video itself is unreachable it still returns the caption and cover image, which is often enough to answer the question.

3. The output isn't shaped for an agent. framedeck writes a manifest.json of absolute paths, timestamps, hashes, and why each frame was kept โ€” plus one contact sheet for a cheap overview before reading frames individually.

Install

pip install framedeck          # or: git clone && pip install -e .

Needs ffmpeg and ffprobe on PATH. yt-dlp is optional but widens the supported sites considerably.

No Python dependencies. Not "few" โ€” zero. The perceptual hashing runs on a 9ร—8 grayscale buffer that ffmpeg produces and Python bit-twiddles, so there is no Pillow, no numpy, and nothing to conflict with in an agent sandbox.

mkdir -p ~/.local/bin && cd /tmp
curl -sSL -o ff.tar.xz https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz
tar xf ff.tar.xz && cp ffmpeg-*-static/{ffmpeg,ffprobe} ~/.local/bin/
curl -sSL -o ~/.local/bin/yt-dlp https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp_linux
chmod +x ~/.local/bin/yt-dlp

Use it as an MCP server

This is the part that matters. Wire it in once and any question about a video becomes answerable.

Claude Code

claude mcp add framedeck -- python3 -m framedeck mcp

opencode โ€” in opencode.json:

{
  "mcp": {
    "framedeck": {
      "type": "local",
      "command": ["python3", "-m", "framedeck", "mcp"],
      "enabled": true
    }
  }
}

Then just ask:

what's the layout of this invitation? https://vt.tiktok.com/ZSVKJNPcM/

The agent calls framedeck_extract, reads the returned frame paths as images, and describes the eight sections it sees.

Tool

Does

framedeck_extract

download + extract distinct frames, return the manifest

framedeck_info

caption, author, duration โ€” no download

The server speaks JSON-RPC 2.0 over stdio directly, without an MCP SDK. The stdio framing is small enough that vendoring one would cost more than it saves, and a dependency-free server drops into any sandbox unchanged.

CLI

framedeck extract <url|path> [options]
  -o, --out DIR          output directory        (./framedeck-out)
  -n, --max-frames N     frame budget            (12)
  -w, --width PX         frame width             (720)
  -t, --threshold F      scene sensitivity, lower finds more  (0.12)
  -d, --distance N       dedupe strictness, lower keeps more  (10)
      --no-sheet         skip the contact sheet
      --keep-video       keep the downloaded video
      --json             print the manifest

framedeck info <url>     metadata only
framedeck mcp            run as an MCP server over stdio

Two knobs cover almost all tuning. Getting near-duplicates anyway? Raise -d. Missing a screen that was briefly on camera? Lower -t.

Manifest

{
  "source": { "provider": "tikwm", "author": "editmateproject", "duration": 36.0,
              "title": "Cover undangan nggak harus foto prewed โ€ฆ" },
  "frame_count": 10,
  "contact_sheet": "/abs/out/contact-sheet.jpg",
  "frames": [
    { "index": 1, "time": 0.2, "path": "/abs/out/frame-01-t0000.20.jpg",
      "phash": "3c1e0f078381c0e0", "reason": "first" }
  ]
}

reason is first, scene (the picture changed here) or anchor (uniform sample, kept because nothing else covered this stretch).

Good for

  • Reading a UI or design walkthrough posted as a video

  • Pulling structure out of a demo reel a client sent instead of a spec

  • Turning a screen recording of a bug into frames you can point at

  • Any "what happens in this video" question aimed at an agent

Limits

  • Frames only โ€” no audio, no transcript. Pipe the audio to whisper if you need words; framedeck deliberately stops at the picture.

  • Downloads whatever the URL points at. Respect the platform's terms and other people's copyright: reference material is not the same as redistribution, and neither is it a licence to clone someone's work.

MIT.

Available Tools

2 tools
framedeck_extractA

Download a video (TikTok, Instagram, YouTube, X, or a local file path) and extract the handful of visually distinct frames, skipping near-duplicates. Returns a manifest with an absolute file path per frame โ€” read those paths as images to actually see the video. Use this whenever you are asked about the contents, design, or UI of a video.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesVideo URL, or an absolute path to a local video file.
widthNoFrame width in px. Lower it to save tokens when reading.
out_dirNoWhere to write frames. Defaults to a temp directory.
thresholdNoScene-change sensitivity; lower finds more cuts.
max_framesNoFrame budget. 8โ€“14 suits most short videos.
contact_sheetNoAlso render one grid image of every frame.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden. It discloses that the tool downloads videos (including from local paths), skips near-duplicates, returns absolute file paths, and that those paths must be read as images to see the video. It does not mention potential network/auth requirements or failure modes, but the key behavior is clearly conveyed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with no fluff: the first states the core action, the second explains the output and how to use it, and the third gives usage context. Purpose is front-loaded, and every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having no output schema and 6 parameters, the description plus schema provide enough to call the tool correctly. It explains the manifest return value and that frames are returned as paths to read. It could mention the contact-sheet output, but that is captured in the schema, so the gap is minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-specific meaning; it simply gives overarching context about frame extraction, which is already supported by the schema's field-level descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('extract') and resource ('visually distinct frames from a video'), and names supported sources. It also tells the agent when to use it ('whenever you are asked about the contents, design, or UI of a video'), which distinguishes it from the sibling framedeck_info by use case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit when-to-use condition: 'Use this whenever you are asked about the contents, design, or UI of a video.' It does not explicitly say when not to use it or compare with the sibling framedeck_info, so it stops short of a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

framedeck_infoA

Fetch a video's metadata โ€” caption, author, duration โ€” without downloading it. Cheap first call when the caption alone may answer the question.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present, the description carries the behavioral burden and does so well by disclosing that it does not download the video and is inexpensive. It could add more about response format or error behavior, but the non-destructive nature is clearly conveyed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, each earning its place: the first states the operation and scope, the second gives the cost and routing rationale. No filler or redundant restatement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter info tool with no output schema, the description supplies the return fields and a clear use case, which is largely complete. It leaves minor gaps such as exact URL format and explicit alternative mention, but nothing blocks correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has zero description coverage and only a string 'url'. The description gives the URL its semantic role โ€” identifying a video whose metadata will be fetched โ€” and previews the returned fields, adding meaning beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Starts with a specific verb and object ('Fetch a video's metadata') and lists concrete fields (caption, author, duration). The 'without downloading it' phrasing distinguishes it from a potential extract/download sibling, so an agent can tell it apart from framedeck_extract.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear selection rule: use this as a cheap first call when the caption alone may answer the question. It does not explicitly name the alternative or state when not to use it, but the intended routing is strongly implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

A4.4/5.0
Disambiguation5/5

framedeck_extract and framedeck_info are cleanly separated: one downloads video content and returns visual frames, while the other fetches metadata without downloading. There is no realistic overlap in what they are used for.

Naming Consistency4/5

Both tools share the clear framedeck_ prefix and snake_case convention. The only minor inconsistency is that 'extract' is a verb while 'info' is a noun, so the action pattern is not perfectly uniform.

Tool Count4/5

Two tools is slightly below the typical 3-15 range, but it fits the server's narrow purpose: obtaining video metadata and extracting representative frames. Each tool earns its place, so the count feels reasonable rather than thin.

Completeness5/5

For the apparent domain of video understanding, the surface is complete: framedeck_info covers quick metadata lookups and framedeck_extract covers the visual content workflow. There is no obvious missing operation or dead end.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/nurhikam/framedeck'

If you have feedback or need assistance with the MCP directory API, please join our Discord server