shortform-breakdown
Uses OpenAI's API for audio transcription (whisper-1) and scene/video decision requests, powering short-form video analysis; supports custom OpenAI base URLs including Azure OpenAI.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
shortform-analyzer
One fixed record per short-form video: what it is, what it sells, and what each scene does
Features • Time and cost • Get started • Usage • How it works • The record • Building blocks
Give it a TikTok post, or a TikTok, Reels or Shorts file, and it returns one record: the video's topic and format, whether it sells something and what, tags, what it's about and how it's made, and scene by scene its role (hook, body, payoff, cta), its footage, what fills the frame, how the frame is laid out (split screen, overlay, text bar), whether it has captions or a message, whether the product shows, the words said and the text on screen. For a post that sells or asks for a comment, it also reads the top comments and gives each a kind. One line in Node, or a CLI, an MCP server and an agent skill.
import { analyzeVideo, summarize } from "shortform-analyzer";
const a = await analyzeVideo("https://www.tiktok.com/@studyfetch/video/7500726429689204010");
if (a.ok) console.log(summarize(a));tutorial about education, sells software ("StudyFetch"). 19.4 s, 6 scenes, words: TikTok's subtitles.
Tags: studyfetch, studyapp, education, lecture, studytools, notes, flashcards, practicetests, recommendation.
About: The video promotes StudyFetch as a way to turn a lecture video into detailed notes, flashcards, and practice tests. It shows a phone playing a neuroscience lecture, then generating study materials from it.
Made: A handheld phone demo filmed over a wooden floor. It opens with the phone showing a lecture video, then cuts to a closer view as the user copies and pastes material into the StudyFetch app. Subsequent shots show generated notes, flashcards, and a multiple-choice practice question. Large text overlays narrate the benefits, ending with a sticker naming the app.
hook 0.0-3.0 s, 1 scene (camera body): "Look at this. Oh," [on screen: "I don’t have 1 hour 💀💀💀"]
body 3.0-19.4 s, 5 scenes (camera body, product shown): "perfect. Oh, my god. Isn't it great, Chris? I know. It's my dream. I have goosebumps everywhere. This is out of this world. What a birthday!..." [on screen: "I just copy + paste / watch this... / detailed notes in seconds / it even makes ..."]
Comments: 50 read, 31 negative, 9 other, 7 positive, 3 intent.
Cost: $0.00481.A real run on @studyfetch's post (the video is theirs), made with 0.4.1. From 0.5.0 each scene line also gives, after the footage, what fills the frame and the layout and text fields that are yes.
Features
One fixed record. Every field, value and rule is written down in docs/video-analysis.md and is the same for every video, so records compare across videos and teams. You add questions; the record stays.
Read from the content. The hook ends where the video's opening ends, not at a fixed 3 seconds. Whether it sells, each scene's role and the format come from what is said and shown, with TikTok's own facts about the post (the account, caption, hashtags, sticker text, topic labels, commercial tags) as evidence.
TikTok posts in one call, through apinoa: the post, its video or photos, its own subtitles, and for a post that sells or asks, up to 50 top comments, each labelled
cta_reply,intent,negative,positiveorother.Cheap. Two model requests per video and no transcription by default (TikTok's subtitles are free): about $0.0037 per post, the post read included.
flexhalves the price of the describe request.Reuse. Every stage has a version. Give an earlier result and only what changed runs again; a failure keeps what finished.
Never throws. A bad input or a missing key comes back as
{ ok: false, error }; a refused answer falls back to a stated default.Yours to extend. Up to 20 questions of your own, about the video or each scene, ride in the same request. The transcriber, the describer and the decider can each be your own.
Four ways in: a library, a CLI, an MCP server and an agent skill. The output is
analysis.jsonwith a JSON Schema, a one-pagereport.html, andsheet.jpgwith every frame on one image.
Related MCP server: clipspeed-mcp
Time and cost
23 public TikToks (20 videos of 6 to 95 s and 3 photo posts; 11 with TikTok subtitles), one at a time from the post URL to the record, on a busy Apple Silicon laptop, October 2026, on the default settings:
p50 | p90 | |
Time per post | 8.5 s | 12.3 s |
↳ post read (apinoa) | 0.1 s | 0.3 s |
↳ download and scenes | 1.8 s | 3.1 s |
↳ frames | 0.3 s | 0.9 s |
↳ words (subtitles) | 0.0 s | 0.2 s |
↳ describe request | 5.5 s | 7.8 s |
↳ decide request | 0.5 s | 1.0 s |
↳ comments, when read (8 posts) | 1.8 s | 2.4 s |
Cost per post | $0.0029 | $0.0060 |
These are 0.4.0's numbers. 0.5.0 asks the decide request six more questions per scene: on 36 TikToks it read 16,415 input tokens per post on average instead of 8,092, about $0.0008 more per post, so about $0.0037 at p50.
Most of the time is the describe request; the download and scene detection grow with the video's length. The post reads
were quick because apinoa had read these posts earlier that day: a first read took 1.1 s at p50 and 1.7 s at p90. Most
of the cost is the decide request and the apinoa reads; a post you already read ({ post }) skips the first read. With
flex, the describe request costs half and can take minutes. How accurate each field is: eval/. The
comparison with other tools in bench/ was measured on 0.3.1.
Get started
You need:
Node.js 20 or later;
ffmpegandffprobeon PATH (brew install ffmpeg,apt install ffmpeg), or passffmpegPathandffprobePath;an OpenAI key in
OPENAI_API_KEY, and for TikTok posts an apinoa key inAPINOA_API_KEY.
npm install shortform-analyzerOr run it without installing:
npx shortform-analyzer analyze https://www.tiktok.com/@handle/video/<id>A TikTok post URL (/video/<id> or /photo/<id>) is read through apinoa. Any other URL is refused, TikTok short
links included: download a Reel or a Short and pass the file. A file has no words unless transcription is on.
Usage
Library
import { writeFile } from "node:fs/promises";
import { analyzeVideo, reportHtml, summarize } from "shortform-analyzer";
const a = await analyzeVideo("https://www.tiktok.com/@studyfetch/video/7500726429689204010", {
questions: [{ name: "face_on_camera", question: "Is a person's face on camera at any point?" }],
});
if (!a.ok) throw new Error(a.error);
a.topic; // "education"
a.format; // "tutorial"
a.sells; // "software", and a.product is "StudyFetch"
a.scenes[0]; // { index, start, end, cut, role: "hook", footage, showsProduct, subject, splitScreen, overlay, textBar,
// captions, message, spoken, onScreenText, frame }
a.comments; // [{ id, text, likes, kind: "intent" }, …] when it sells or asks
a.answers; // { face_on_camera: false }
a.costUsd; // what this run cost
console.log(summarize(a)); // text for an agent
await writeFile("report.html", reportHtml(a)); // one self-contained pageExport | Does |
| The record. |
| Text for an agent: the video's fields, then one line per run of scenes with the same role |
| A self-contained HTML page: the role strip, then one row per scene, then the comments |
| The default engines, to configure or to replace with your own |
| The price table behind |
| The value lists |
The main options: transcribe (off; true runs whisper-1 for a file, or a post without subtitles), flex,
comments (on), questions, reuse, and describer, decider and transcribe for engines of your own. Every option
and output field is in docs/reference.md; the output's schema is
schema/analysis.schema.json (schemaVersion: 2).
CLI
npx shortform-analyzer analyze https://www.tiktok.com/@handle/video/<id>
npx shortform-analyzer analyze clip.mp4 --transcribe --question face="Is a person's face on camera?" --jsonanalyze prints the summary, or with --json the record without frames and raw, and writes <post id>.analysis/
(or <file>.analysis/ next to a file):
analysis.jsonandreport.html;frames/01.jpg…: the 512 px frames the model saw;sheet.jpg: every frame tiled 4 wide, so an agent can look at one image.
Running it again on the same video reuses the folder: only the stages whose version changed run. Flags: --out,
--transcribe, --flex, --no-comments, --question name="question" (yes/no, repeatable), --json. Exit codes: 0
ok, 1 the analysis failed (the reason is on stderr), 2 a usage error.
MCP server
Claude Code:
claude mcp add shortform-analyzer -e OPENAI_API_KEY=sk-... -e APINOA_API_KEY=... -- npx -y shortform-analyzer mcpClaude Desktop, Cursor and other clients:
{
"mcpServers": {
"shortform-analyzer": {
"command": "npx",
"args": ["-y", "shortform-analyzer", "mcp"],
"env": { "OPENAI_API_KEY": "sk-...", "APINOA_API_KEY": "..." }
}
}
}Tool | Input | Returns |
|
| The summary, the output folder, and the record without frames and |
Agent skill
skills/shortform-analyzer/SKILL.md teaches an agent when to run it and how to
read the result. It ships in the npm package too.
npx skills add trevkang/shortform-analyzerHow it works
Stage | What happens | Calls |
| Reads the TikTok post through apinoa, or takes the post you pass | apinoa post read |
| ffmpeg finds the cuts. Flashes under 0.4 s join the scene before; a take over 5 s is split, so a one-take video still has a beginning, middle and end | none |
| The middle frame of every scene, 512 px | none |
| The post's own subtitles, each word given to the scene its midpoint falls in. Transcription only when it's on | the subtitle file |
| One request writes the product, the tags, what the video is about and how it's made, and each scene's text on screen | OpenAI Responses |
| One request picks every value, for the video and each scene, reading what describe wrote next to the frames and words, and answers your questions | OpenAI Decisions |
| For a post that sells or asks: its top comments, and one request gives each a kind | apinoa comments read, OpenAI Decisions |
Then code applies the rules: the fallback for a refused answer, the hook rule, and product following sells.
Describing and deciding are two requests: one writes what the video shows in its own words, the other picks each value from a closed list, with a probability. A wrong description and a wrong label are then different bugs, and a change to the labels reruns only the decide request. The decide request covers the whole video at once, because a scene's role depends on its neighbours: the same shot is a hook at 0 s and a payoff at 20 s. The stages and layers are explained in docs/concepts.md.
The record
Field | Values |
|
|
|
|
|
|
| yes or no |
A scene's |
|
A scene's |
|
A scene's | yes or no, when the video sells |
A scene's |
|
A scene's | yes or no: how the frame is laid out |
A scene's | yes or no: what the text on screen does |
A comment's |
|
The model writes the rest in its own words: product, tags, description, formatDescription and each scene's
onScreenText. Each value's test, example and near misses, and what happens when an answer is missing, are in
docs/video-analysis.md.
Building blocks
Every step is public from shortform-analyzer/primitives.
This subpath is experimental: a minor version may change a signature. Every change is inCHANGELOG.md.
TikTok subtitles aligned to the cuts, with no model call:
import { alignWords, cueWords, detectScenes, download, fetchSubtitles, parseVtt, postMedia, probe, readPost } from "shortform-analyzer/primitives";
const post = await readPost("7500726429689204010"); // one apinoa call, APINOA_API_KEY
const media = postMedia(post); // the video link and the ASR subtitle track
await download(media.video!, "clip.mp4");
const { duration } = await probe("clip.mp4");
const spans = await detectScenes("clip.mp4", { duration });
const said = alignWords(cueWords(parseVtt(await fetchSubtitles(media.subtitles!.url))), spans); // one string per sceneA question of your own, asked of every scene:
import { decide, extractFrames, sceneEvidence, type DecisionQuestion } from "shortform-analyzer/primitives";
const frames = await extractFrames("clip.mp4", spans);
const logo: DecisionQuestion = { type: "predicate", name: "logo", scope: "scene", instructions: "Is the brand logo visible in scene {scene}?" };
const d = await decide(sceneEvidence(spans, frames, said, { duration, words: "subtitles" }), [logo]);
const logoShown = d.scenes.map((s) => (s.logo?.probability ?? 0.5) >= 0.5);decide takes predicate (a probability), choice (one of your values, with probabilities) and score questions,
for the whole video or per scene, up to 400 per request and more in several requests. It returns only what the model answered: a refused answer is
absent, and you choose the fallback. The building blocks throw an Error that names the stage (source:, video:,
speech:, describe:, decide:); analyzeVideo never throws.
Keys and costs
OpenAI, on
OPENAI_API_KEY: one Responses and one Decisions request per video, and one more Decisions request when comments are read.whisper-1per minute of audio when transcription is on.apinoa, on
APINOA_API_KEY: $0.001 per post read and per comments read.costUsdsays what each run cost, from thePRICEStable. There is no telemetry; the keys go only to OpenAI and apinoa.
Analyse your own videos, or videos you have the right to analyse.
Development
pnpm install
pnpm test # builds, then runs every test (videos are made with ffmpeg lavfi; no network)
pnpm typecheck
pnpm test:consumer # installs the packed tarball into a blank project and imports itStart with docs/concepts.md; a change starts as a doc edit, then a test, then the code. A change to a prompt or a model runs eval/ before and after.
Credits
Built by Trevor and Viktor. It started as the video analysis module of viral-oracle, their team project at the Omnara hackathon (October 2026).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
Hosted video MCP server for branded explainers and reference-driven edits.
TikTok transcripts, comment sentiment by moment, AI breakdowns and director-level shot analysis.
AI video clipping MCP: long videos and podcasts into scored, captioned 9:16 shorts.
Short-form video analytics and trend intelligence for TikTok, YouTube, and Instagram
Related MCP Servers
FlicenseNot gradedqualityBmaintenanceMCP server enabling video analysis via scene detection, audio transcription, visual description, and stylistic fingerprinting, with tools for full pipeline execution and storyboard generation.-- AlicenseNot gradedqualityCmaintenanceEnables turning any video into scored, captioned, ready-to-post vertical clips directly from your MCP client, with capabilities to download finished MP4s, schedule posts, and discover trending content.451 npmMIT
- FlicenseNot gradedqualityAmaintenanceEnables MCP clients to ingest and analyze video footage, then receive license-safe BGM recommendations with hook windows, beat/cut guidance, attribution, and ffmpeg ducking or mix specs.-
- AlicenseNot gradedqualityCmaintenanceEnables MCP clients to turn direct video URLs and chosen start/end times into ready-to-post vertical 9:16 MP4 clips with animated burned-in captions, thumbnails, and SRT files, including batch processing for multiple highlights.MIT