Turn any video into LLM-ready data.
A klaket is a clapperboard β the tool that syncs sound and image on a film set. Klaket syncs video with LLMs.
LLMs read text. The web became readable with scrapers β but video, the largest store of human knowledge, is still locked away. Klaket unlocks it: give it a video URL or file, get back structured, timestamped, LLM-ready data.
pip install klaket
klaket ingest "https://youtube.com/watch?v=..." --wait{
"transcript": [
{ "start": 14.32, "end": 19.80, "speaker": "S1", "text": "So let's deploy this with docker compose..." }
],
"scenes": [
{ "start": 190.0, "end": 342.5, "keyframes": ["scene_004_01.jpg"] }
],
"chapters": [...],
"summary": "..."
}π Transcript β timestamped speech-to-text in ~100 languages (auto-detected) with word-level timestamps; pick the model per job ("model": "medium")
ποΈ Podcasts too β pass an audio file/URL (mp3, m4aβ¦) and Klaket skips the visual stages, deriving chapters from speech pauses
π£οΈ Speaker diarization β who said what (S1/S2/β¦), local & keyless (sherpa-onnx)
π¬ Subtitles β ready-to-use .srt / .vtt files with speaker labels
ποΈ Scene detection β content-aware scene boundaries + keyframes per scene
π On-screen text (OCR) β reads slides, terminals and captions per scene, local & keyless
𧩠One JSON timeline β transcript, scenes, frames and on-screen text aligned on a single timeline
π Works offline, no API key required β the core pipeline uses zero LLM calls
π§ Pluggable model layer β optional scene descriptions via local VLMs (Ollama) or any OpenAI-compatible endpoint (KLAKET_VLM=off by default)
π€ MCP server β let coding agents "watch" any video and find moments inside it
π In-video search β GET /v1/jobs/{id}/search?q=β¦ finds the exact moment
βΆοΈ Playground β the dashboard plays the video with a click-to-seek, live-highlighted transcript
Related MCP server: vidtheque
# pip install klaket
from klaket import Klaket
result = Klaket().process("https://youtube.com/watch?v=...", num_speakers=2)// npm i klaket-sdk
import { Klaket } from "klaket-sdk";
const result = await new Klaket().process("https://youtube.com/watch?v=...");# Claude Code
claude mcp add klaket -- npx klaket-mcp # KLAKET_API_URL defaults to localhost:8484Then: "Watch https://youtube.com/watch?v=β¦ and summarize the commands the presenter runs."
The agent gets klaket_ingest, klaket_job_status and klaket_get_result tools.
git clone https://github.com/huseyinstif/klaket.git && cd klaket
docker compose up --build
# API on :8484, dashboard on :5180
curl -X POST localhost:8484/v1/ingest \
-H "Content-Type: application/json" \
-d '{"url": "https://youtube.com/watch?v=..."}'That's it β no API keys, no GPUs required. make help lists developer shortcuts (make up, make test, make e2e).
client βββΊ Go API βββΊ Redis queue βββΊ Python worker (ffmpeg Β· faster-whisper Β· scenedetect)
β β
dashboard ββββββββββββββββββββββ /data/jobs/<id>/result.jsonapps/api β Go, job orchestration
apps/worker β Python, media pipeline
apps/dashboard β React dashboard
Klaket is open source (AGPL-3.0) and fully self-hostable. A hosted, pay-per-minute cloud API with managed GPUs is planned β join the waitlist (coming soon).
π§ v0.7 β pre-1.0, moving fast. Star the repo to follow along.
AGPL-3.0. SDKs and clients will be MIT.
Built by HΓΌseyin TΔ±ntaΕ β X (@1337stif) Β· LinkedIn
Search inside a processed video: finds the moments (timestamped transcript lines, on-screen text or scene descriptions) matching a query. Job must be 'done'.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Job id of a completed job | |
| query | Yes | What to look for, e.g. 'docker compose command' or 'pricing slide' |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description fully describes the tool's behavior: it searches within a processed video and returns moments. It does not mention side effects, but as a search operation this is acceptable. There is no contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no wasted words. The first sentence describes what the tool does, the second adds a crucial precondition. Ideal length for this tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the main purpose, search targets, and precondition. It does not describe the output format in detail, but for a simple search tool with two parameters, the information is adequate. The list of moment types provides sufficient completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds value with query examples ('docker compose command') and clarifies the 'id' parameter as a job id from a completed job, enhancing understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'search/find' and the resource 'processed video moments'. It lists the types of moments (timestamped transcript lines, on-screen text, scene descriptions), and the precondition 'Job must be done' distinguishes from sibling tools like klaket_get_result or klaket_job_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly requires the job to be 'done', providing a clear precondition. Although it doesn't name sibling tools, the context implies when to use this tool (after processing) vs. alternatives (e.g., check status first). The guidance is clear but could be more explicit about alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get the processed video as LLM-ready markdown: timestamped transcript, scene list with descriptions and on-screen text. Only works when the job status is 'done'.
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Job id of a completed job |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Discloses prerequisite (job done) and output format (markdown). Could mention idempotency or failure cases, but adequately informs agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose and output, then condition. No unnecessary words, highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with one parameter and no output schema, the description fully explains what it does, what it returns, and when to use it. Sibling tools are distinct enough to avoid confusion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameter with description 'Job id of a completed job'. Description adds no extra semantics beyond schema; baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves processed video as LLM-ready markdown, specifying content (timestamped transcript, scene list). It distinguishes from siblings like klaket_ingest (ingestion) and klaket_job_status (status check).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit prerequisite: 'Only works when the job status is done.' Does not mention alternatives or when not to use, but the condition is clear enough for usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Start processing a video (URL or server-side file path) into LLM-ready data. Returns a job id β poll klaket_job_status until status is 'done', then call klaket_get_result.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Video URL (YouTube, direct file) or a path visible to the Klaket worker |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral disclosure. It correctly indicates the tool is non-blocking and returns a job id, but fails to mention potential side effects (e.g., compute usage, storage requirements, error handling). The transparency is adequate but not thorough for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with zero waste. The first sentence states the purpose and output, the second provides the follow-up workflow. Information is front-loaded and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one param, async, no output schema), the description covers the essential workflow and references sibling tools. It lacks details on error states or job id format, but these are minor omissions for a straightforward ingestion tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already describes the single 'url' parameter thoroughly (100% coverage). The tool description adds that the URL can be a YouTube link, direct file, or server-side path, which is useful but not critical. Overall, the description does not significantly augment the schema's parameter documentation beyond context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Start processing' and the resource 'a video', specifying the transformation into 'LLM-ready data'. It distinguishes from siblings by outlining the async workflow that follows, making the tool's role in the pipeline explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides direct guidance: poll klaket_job_status until done, then call klaket_get_result. It also clarifies valid inputs (URL or server-side path). However, it does not explicitly state when not to use this tool or scenarios where alternatives like klaket_find_moment would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Check the status of a Klaket ingest job (queued | processing | done | failed).
| Name | Required | Description | Default |
|---|---|---|---|
| id | Yes | Job id returned by klaket_ingest |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the full burden. It discloses the possible statuses but lacks details about side effects, permissions, or whether the operation is read-only. The description is minimal beyond the status list.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one sentence, 11 words) with clear front-loading of verb and resource. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema, the description lists the possible statuses, which is helpful. However, it does not specify the exact response format (e.g., whether it returns a plain string or an object), leaving minor ambiguity for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter 'id', which already describes it as the job id returned by klaket_ingest. The description adds no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Check' and the resource 'status of a Klaket ingest job', listing the possible statuses. It effectively distinguishes from sibling tools like klaket_ingest (which starts a job) and klaket_get_result (which retrieves results).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit usage guidance is provided, but the parameter description references klaket_ingest, implying it should be used after that tool. The description does not specify when to avoid using this tool or alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.7.1klaket_find_momentklaket_get_resultklaket_ingestklaket_job_statusScored across 4 tools
Each tool has a distinct purpose: ingest starts processing, job_status checks progress, get_result retrieves markdown, and find_moment searches within processed video. There is no functional overlap.
All tools use the 'klaket_' prefix and snake_case, with three following a verb_noun pattern (ingest, get_result, find_moment) and one being noun_noun (job_status). The pattern is mostly consistent but has a minor deviation.
With 4 tools, the server is well-scoped for its purpose of video ingestion and querying. Each tool serves a necessary step in the workflow without unnecessary bloat.
The tool set covers the core lifecycle (ingest, status, retrieve, search) but lacks deletion or listing capabilities. This is acceptable for a focused server, though minor gaps exist.
YouTube transcripts, search, channel/playlist listings and upload tracking for AI agents. No signup.
15 media & data tools for AI agents: search, transcribe, subtitles, voiceover, translate & more.
Summarize and transcribe videos, audio, documents and web pages; subtitles; search your library.
Video knowledge base for agents: search your library's transcripts, keyframes and on-screen text.
