Skip to main content
Glama

shortform-analyzer

One fixed record per short-form video: what it is, what it sells, and what each scene does

npm version Build Status Node.js TypeScript License

Features • Time and cost • Get started • Usage • How it works • The record • Building blocks

Give it a TikTok post, or a TikTok, Reels or Shorts file, and it returns one record: the video's topic and format, whether it sells something and what, tags, what it's about and how it's made, and scene by scene its role (hook, body, payoff, cta), its footage, what fills the frame, how the frame is laid out (split screen, overlay, text bar), whether it has captions or a message, whether the product shows, the words said and the text on screen. For a post that sells or asks for a comment, it also reads the top comments and gives each a kind. One line in Node, or a CLI, an MCP server and an agent skill.

import { analyzeVideo, summarize } from "shortform-analyzer";

const a = await analyzeVideo("https://www.tiktok.com/@studyfetch/video/7500726429689204010");
if (a.ok) console.log(summarize(a));
tutorial about education, sells software ("StudyFetch"). 19.4 s, 6 scenes, words: TikTok's subtitles.
Tags: studyfetch, studyapp, education, lecture, studytools, notes, flashcards, practicetests, recommendation.
About: The video promotes StudyFetch as a way to turn a lecture video into detailed notes, flashcards, and practice tests. It shows a phone playing a neuroscience lecture, then generating study materials from it.
Made: A handheld phone demo filmed over a wooden floor. It opens with the phone showing a lecture video, then cuts to a closer view as the user copies and pastes material into the StudyFetch app. Subsequent shots show generated notes, flashcards, and a multiple-choice practice question. Large text overlays narrate the benefits, ending with a sticker naming the app.
  hook 0.0-3.0 s, 1 scene (camera body): "Look at this. Oh," [on screen: "I don’t have 1 hour 💀💀💀"]
  body 3.0-19.4 s, 5 scenes (camera body, product shown): "perfect. Oh, my god. Isn't it great, Chris? I know. It's my dream. I have goosebumps everywhere. This is out of this world. What a birthday!..." [on screen: "I just copy + paste / watch this... / detailed notes in seconds / it even makes ..."]
Comments: 50 read, 31 negative, 9 other, 7 positive, 3 intent.
Cost: $0.00481.

A real run on @studyfetch's post (the video is theirs), made with 0.4.1. From 0.5.0 each scene line also gives, after the footage, what fills the frame and the layout and text fields that are yes.

Features

  • One fixed record. Every field, value and rule is written down in docs/video-analysis.md and is the same for every video, so records compare across videos and teams. You add questions; the record stays.

  • Read from the content. The hook ends where the video's opening ends, not at a fixed 3 seconds. Whether it sells, each scene's role and the format come from what is said and shown, with TikTok's own facts about the post (the account, caption, hashtags, sticker text, topic labels, commercial tags) as evidence.

  • TikTok posts in one call, through apinoa: the post, its video or photos, its own subtitles, and for a post that sells or asks, up to 50 top comments, each labelled cta_reply, intent, negative, positive or other.

  • Cheap. Two model requests per video and no transcription by default (TikTok's subtitles are free): about $0.0037 per post, the post read included. flex halves the price of the describe request.

  • Reuse. Every stage has a version. Give an earlier result and only what changed runs again; a failure keeps what finished.

  • Never throws. A bad input or a missing key comes back as { ok: false, error }; a refused answer falls back to a stated default.

  • Yours to extend. Up to 20 questions of your own, about the video or each scene, ride in the same request. The transcriber, the describer and the decider can each be your own.

  • Four ways in: a library, a CLI, an MCP server and an agent skill. The output is analysis.json with a JSON Schema, a one-page report.html, and sheet.jpg with every frame on one image.

Related MCP server: clipspeed-mcp

Time and cost

23 public TikToks (20 videos of 6 to 95 s and 3 photo posts; 11 with TikTok subtitles), one at a time from the post URL to the record, on a busy Apple Silicon laptop, October 2026, on the default settings:

p50

p90

Time per post

8.5 s

12.3 s

↳ post read (apinoa)

0.1 s

0.3 s

↳ download and scenes

1.8 s

3.1 s

↳ frames

0.3 s

0.9 s

↳ words (subtitles)

0.0 s

0.2 s

↳ describe request

5.5 s

7.8 s

↳ decide request

0.5 s

1.0 s

↳ comments, when read (8 posts)

1.8 s

2.4 s

Cost per post

$0.0029

$0.0060

These are 0.4.0's numbers. 0.5.0 asks the decide request six more questions per scene: on 36 TikToks it read 16,415 input tokens per post on average instead of 8,092, about $0.0008 more per post, so about $0.0037 at p50.

Most of the time is the describe request; the download and scene detection grow with the video's length. The post reads were quick because apinoa had read these posts earlier that day: a first read took 1.1 s at p50 and 1.7 s at p90. Most of the cost is the decide request and the apinoa reads; a post you already read ({ post }) skips the first read. With flex, the describe request costs half and can take minutes. How accurate each field is: eval/. The comparison with other tools in bench/ was measured on 0.3.1.

Get started

You need:

  • Node.js 20 or later;

  • ffmpeg and ffprobe on PATH (brew install ffmpeg, apt install ffmpeg), or pass ffmpegPath and ffprobePath;

  • an OpenAI key in OPENAI_API_KEY, and for TikTok posts an apinoa key in APINOA_API_KEY.

npm install shortform-analyzer

Or run it without installing:

npx shortform-analyzer analyze https://www.tiktok.com/@handle/video/<id>
NOTE

A TikTok post URL (/video/<id> or /photo/<id>) is read through apinoa. Any other URL is refused, TikTok short links included: download a Reel or a Short and pass the file. A file has no words unless transcription is on.

Usage

Library

import { writeFile } from "node:fs/promises";
import { analyzeVideo, reportHtml, summarize } from "shortform-analyzer";

const a = await analyzeVideo("https://www.tiktok.com/@studyfetch/video/7500726429689204010", {
  questions: [{ name: "face_on_camera", question: "Is a person's face on camera at any point?" }],
});
if (!a.ok) throw new Error(a.error);

a.topic;        // "education"
a.format;       // "tutorial"
a.sells;        // "software", and a.product is "StudyFetch"
a.scenes[0];    // { index, start, end, cut, role: "hook", footage, showsProduct, subject, splitScreen, overlay, textBar,
                //   captions, message, spoken, onScreenText, frame }
a.comments;     // [{ id, text, likes, kind: "intent" }, …] when it sells or asks
a.answers;      // { face_on_camera: false }
a.costUsd;      // what this run cost
console.log(summarize(a));                       // text for an agent
await writeFile("report.html", reportHtml(a));   // one self-contained page

Export

Does

analyzeVideo(input, options?)

The record. input: a file path, a Buffer, a TikTok post URL, or { post } for a post you already read through apinoa

summarize(analysis)

Text for an agent: the video's fields, then one line per run of scenes with the same role

reportHtml(analysis, title?)

A self-contained HTML page: the role strip, then one row per scene, then the comments

openaiTranscriber, openaiDescriber, openaiDecider

The default engines, to configure or to replace with your own

PRICES

The price table behind costUsd and every stage's cost

TOPICS, FORMATS, SELLS, ROLES, FOOTAGE, COMMENT_KINDS

The value lists

The main options: transcribe (off; true runs whisper-1 for a file, or a post without subtitles), flex, comments (on), questions, reuse, and describer, decider and transcribe for engines of your own. Every option and output field is in docs/reference.md; the output's schema is schema/analysis.schema.json (schemaVersion: 2).

CLI

npx shortform-analyzer analyze https://www.tiktok.com/@handle/video/<id>
npx shortform-analyzer analyze clip.mp4 --transcribe --question face="Is a person's face on camera?" --json

analyze prints the summary, or with --json the record without frames and raw, and writes <post id>.analysis/ (or <file>.analysis/ next to a file):

  • analysis.json and report.html;

  • frames/01.jpg…: the 512 px frames the model saw;

  • sheet.jpg: every frame tiled 4 wide, so an agent can look at one image.

Running it again on the same video reuses the folder: only the stages whose version changed run. Flags: --out, --transcribe, --flex, --no-comments, --question name="question" (yes/no, repeatable), --json. Exit codes: 0 ok, 1 the analysis failed (the reason is on stderr), 2 a usage error.

MCP server

Claude Code:

claude mcp add shortform-analyzer -e OPENAI_API_KEY=sk-... -e APINOA_API_KEY=... -- npx -y shortform-analyzer mcp

Claude Desktop, Cursor and other clients:

{
  "mcpServers": {
    "shortform-analyzer": {
      "command": "npx",
      "args": ["-y", "shortform-analyzer", "mcp"],
      "env": { "OPENAI_API_KEY": "sk-...", "APINOA_API_KEY": "..." }
    }
  }
}

Tool

Input

Returns

analyze_video

input (a file path or a TikTok post URL), questions (optional: yes/no or a choice, about the video or each scene), transcribe (optional)

The summary, the output folder, and the record without frames and raw

Agent skill

skills/shortform-analyzer/SKILL.md teaches an agent when to run it and how to read the result. It ships in the npm package too.

npx skills add trevkang/shortform-analyzer

How it works

Stage

What happens

Calls

source

Reads the TikTok post through apinoa, or takes the post you pass

apinoa post read

scenes

ffmpeg finds the cuts. Flashes under 0.4 s join the scene before; a take over 5 s is split, so a one-take video still has a beginning, middle and end

none

frames

The middle frame of every scene, 512 px

none

speech

The post's own subtitles, each word given to the scene its midpoint falls in. Transcription only when it's on

the subtitle file

describe

One request writes the product, the tags, what the video is about and how it's made, and each scene's text on screen

OpenAI Responses

decide

One request picks every value, for the video and each scene, reading what describe wrote next to the frames and words, and answers your questions

OpenAI Decisions

comments

For a post that sells or asks: its top comments, and one request gives each a kind

apinoa comments read, OpenAI Decisions

Then code applies the rules: the fallback for a refused answer, the hook rule, and product following sells.

Describing and deciding are two requests: one writes what the video shows in its own words, the other picks each value from a closed list, with a probability. A wrong description and a wrong label are then different bugs, and a change to the labels reruns only the decide request. The decide request covers the whole video at once, because a scene's role depends on its neighbours: the same shot is a hook at 0 s and a payoff at 20 s. The stages and layers are explained in docs/concepts.md.

The record

Field

Values

topic

beauty · fashion · gaming · tech · education · business_money · entertainment · music_dance · food · fitness_sports · shopping · lifestyle · other

format

tutorial · listicle · before_after · review · storytime · skit · performance · reaction · highlight · interview · vlog · talking_head · other

sells

none · software · physical_goods · service · digital_content · other

asksForComment

yes or no

A scene's role

hook · body · payoff · cta

A scene's footage

camera_face · camera_body · camera_no_person · screen · graphics · other

A scene's showsProduct

yes or no, when the video sells

A scene's subject

face · full_body · people · hands · object · animal · place · slide · interface

A scene's splitScreen, overlay, textBar

yes or no: how the frame is laid out

A scene's captions, message

yes or no: what the text on screen does

A comment's kind

cta_reply · intent · negative · positive · other

The model writes the rest in its own words: product, tags, description, formatDescription and each scene's onScreenText. Each value's test, example and near misses, and what happens when an answer is missing, are in docs/video-analysis.md.

Building blocks

Every step is public from shortform-analyzer/primitives.

WARNING

This subpath is experimental: a minor version may change a signature. Every change is inCHANGELOG.md.

TikTok subtitles aligned to the cuts, with no model call:

import { alignWords, cueWords, detectScenes, download, fetchSubtitles, parseVtt, postMedia, probe, readPost } from "shortform-analyzer/primitives";

const post = await readPost("7500726429689204010");   // one apinoa call, APINOA_API_KEY
const media = postMedia(post);                         // the video link and the ASR subtitle track
await download(media.video!, "clip.mp4");
const { duration } = await probe("clip.mp4");
const spans = await detectScenes("clip.mp4", { duration });
const said = alignWords(cueWords(parseVtt(await fetchSubtitles(media.subtitles!.url))), spans); // one string per scene

A question of your own, asked of every scene:

import { decide, extractFrames, sceneEvidence, type DecisionQuestion } from "shortform-analyzer/primitives";

const frames = await extractFrames("clip.mp4", spans);
const logo: DecisionQuestion = { type: "predicate", name: "logo", scope: "scene", instructions: "Is the brand logo visible in scene {scene}?" };
const d = await decide(sceneEvidence(spans, frames, said, { duration, words: "subtitles" }), [logo]);
const logoShown = d.scenes.map((s) => (s.logo?.probability ?? 0.5) >= 0.5);

decide takes predicate (a probability), choice (one of your values, with probabilities) and score questions, for the whole video or per scene, up to 400 per request and more in several requests. It returns only what the model answered: a refused answer is absent, and you choose the fallback. The building blocks throw an Error that names the stage (source:, video:, speech:, describe:, decide:); analyzeVideo never throws.

Keys and costs

  • OpenAI, on OPENAI_API_KEY: one Responses and one Decisions request per video, and one more Decisions request when comments are read. whisper-1 per minute of audio when transcription is on.

  • apinoa, on APINOA_API_KEY: $0.001 per post read and per comments read.

  • costUsd says what each run cost, from the PRICES table. There is no telemetry; the keys go only to OpenAI and apinoa.

IMPORTANT

Analyse your own videos, or videos you have the right to analyse.

Development

pnpm install
pnpm test           # builds, then runs every test (videos are made with ffmpeg lavfi; no network)
pnpm typecheck
pnpm test:consumer  # installs the packed tarball into a blank project and imports it

Start with docs/concepts.md; a change starts as a doc edit, then a test, then the code. A change to a prompt or a model runs eval/ before and after.

Credits

Built by Trevor and Viktor. It started as the video analysis module of viral-oracle, their team project at the Omnara hackathon (October 2026).

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers