Skip to main content
Glama
video-research-mcp

video-research-mcp

by Galbaz1

video-research-mcp

Ask questions across recordings and documents, then inspect the source frames, transcripts and clips behind the answer. The server exposes 120 MCP tools for video, audio and image operations, document analysis, web and academic research, and saved knowledge. Agent workflows connect those tools into a research or production task.

Use it in Codex, Claude Code, or any client that supports stdio Model Context Protocol. Gemini handles the main analysis and research routes; local tools inspect and transform source files. Optional providers and runtimes extend that core.

Open the interactive user guide (Dutch): from your problem to an approach. Search the function map or choose a task to see its tools, prerequisites and steps. The English user documentation also covers setup problems and examples. Describe your problem in your MCP client in ordinary language to begin.

Release · npm · PyPI · Source

Release candidate: 0.8.0-rc.6 (PyPI 0.8.0rc6). The links above identify the exact packages and immutable source tag. Check the release notes and each route's prerequisites before installing. The npm prerelease channel is next. Local-model qualification is outside this API-first release. Stable 0.7.1 predates the native Codex plugin and expanded media surface.

From source material to a useful result

Work

Tools to start with

What to inspect

Question a recording

video_analyze, video_create_session, video_continue_session

Source timestamps, the answer and supporting material

Analyze a long recording in stages

media_info, video_analyze_windows, job_status

Requested intervals, dry-run plan, execution limits and retained outcomes

Prepare audio or transcripts

audio_transcribe, audio_clip_export, audio_dsp_analyze

Caption or model provenance, source timing and exported audio

Inspect images or export media

image_read, image_crop, image_ocr, video_frame, video_clip_export

Source bytes, selected region or interval, output files and manifests

Compare documents or investigate a topic

content_analyze, research_document, research_plan, research_web

Citations, contradictions, missing evidence and job status

Generate images or short video

Separate explainer image and video APIs

Pinned sources, a current operator quote, explicit operations and saved decoded files; provider quality remains unqualified

Search previous work

knowledge_ingest, knowledge_search, knowledge_ask

Stored records and retrieval provenance; requires Weaviate

For example, ask your client:

Compare ~/recordings/design-review.mp4 with ~/docs/requirements.pdf.
Identify decisions that change a requirement. Cite recording timestamps and
relevant document pages, distinguish explicit decisions from interpretation,
and flag anything the sources do not establish.

Workflows combine tools; individual MCP calls return their own results. Source hashes identify bytes, and export manifests retain how an artifact was produced. Model timestamps, citations and interpretations still need checking against the original material. Sampling a recording does not establish complete coverage.

Related MCP server: media-mcp

Install

You need Python 3.11+, uv with uvx on your client's PATH, and a Gemini API key for Gemini requests. Plugin installation also needs Node.js 22+ and npm. Install FFmpeg/FFprobe for local media inspection, frame extraction and clip exports. Other operations may need an optional dependency or configured backend.

By default, the server reads process environment variables first, then ~/.config/video-research-mcp/.env, then defaults. Project installations can select a separate credential file. Add your key to the shared file for the routes below, keeping any existing settings:

GEMINI_API_KEY=your-gemini-api-key

Keep the file private (chmod 600 ~/.config/video-research-mcp/.env). Codex plugin servers receive a filtered environment, so this file is the reliable credential route there. Provider analysis sends source material to the configured service and can incur usage charges. A selected analysis interval does not necessarily limit the uploaded file.

Codex: native plugin

The plugin supplies 26 skills and the version-pinned research server. Historical RC3 observation: on 2026-10-06, the selected Codex 0.160.1 RC3 installation and fresh-session restart loaded the research tools and packaged skills. This is historical RC3 evidence. Follow the verification steps below for the version you install. The native route does not run the Claude workflow installer.

For a new installation, save this catalog as ~/.local/share/video-research-mcp/.agents/plugins/marketplace.json, creating its parent directories if needed:

{
  "name": "video-research",
  "interface": { "displayName": "Video Research" },
  "plugins": [
    {
      "name": "video-research",
      "source": {
        "source": "npm",
        "package": "video-research-mcp",
        "version": "0.8.0-rc.6"
      },
      "policy": { "installation": "AVAILABLE", "authentication": "ON_INSTALL" },
      "category": "Productivity"
    }
  ]
}
codex plugin marketplace add ~/.local/share/video-research-mcp
codex plugin add video-research@video-research
codex plugin list --json

Start a fresh Codex session. Check that video-research@video-research is enabled, its source is npm at 0.8.0-rc.6, and the server's tools are available.

For an existing installation, use the native plugin and migration guide. Back up local plugin edits before reinstalling: Codex replaces its managed cache. A manual mcp_servers.video-research entry can hide the plugin server. In the historical RC1 check on Codex 0.160.0, this also occurred when the entry was disabled; the guide describes the specific configuration to remove.

Claude Code: workflow installer

npx video-research-mcp@0.8.0-rc.6 --global

This installs slash commands, skills and agents into ~/.claude/, registers the research server in ~/.claude.json, and creates a credential template if needed. Set the key, restart Claude Code and inspect /mcp. Use /gr:advisor to select a workflow, or /gr:video, /gr:research and /gr:analyze to start directly.

Use --local for project scope and set the key in ./.config/video-research-mcp/.env; this replaces the shared credential-file route. Use --global --check to inspect the global installation. Updates preserve modified workflow files and custom configuration. See installer options and recovery for scope, checkpoints and rollback.

Other MCP clients: server only

Use your client's stdio registration format. A typical JSON entry is:

{
  "mcpServers": {
    "video-research": {
      "command": "uvx",
      "args": ["video-research-mcp==0.8.0-rc.6"]
    }
  }
}

This connects the same tools; agent workflows are installed separately. Python package filenames use the normalized spelling 0.8.0rc6.

Verify your first connection

Ask the client to call infra_configure with no arguments. It returns the running configuration without making an analysis request. Then try a tool on material you can inspect yourself. A successful connection verifies setup; checking the returned evidence verifies the particular result.

Configure only what the task needs

Setting or addition

Use it for

GEMINI_MODEL, GEMINI_FLASH_MODEL

Select supported primary and auxiliary models

YOUTUBE_API_KEY

YouTube metadata, comments and playlists; otherwise uses the Gemini key

GEMINI_SESSION_DB

Persist video sessions in SQLite; unset means in-memory sessions

WEAVIATE_URL, WEAVIATE_API_KEY

Store and retrieve research across sessions

MLFLOW_TRACKING_URI and the tracing extra

Trace tool execution; GEMINI_TRACING_ENABLED=false disables optional tracing

LOCAL_FILE_ACCESS_ROOT

Restrict supported local-file operations to a directory

Research works without Weaviate. When storage is configured, write-through errors are non-fatal; verify the stored record when persistence matters. The knowledge-store guide covers embeddings, collections and optional query dependencies. For a manually configured server, retain the version pin when adding extras, for example uvx 'video-research-mcp[tracing,agents]==0.8.0-rc.6'. The full runtime configuration lives in ServerConfig.

Image edits, OCR, speech inference, Blender, FreeCAD and local model services have additional runtime or backend requirements. Installing the plugin does not install those runtimes, model weights or provider accounts. Inspect the relevant skill and its prerequisites before using an optional integration.

For video production, the separate explainer companion exposes 39 tools, including direct HTTP image and text/frame-video generation and an optional external CLI/rendering pipeline; the scene-agent companion exposes two tools for scene-code generation. Neither is required for research. Production skills also provide workflows for narration, image generation, clip generation and assembly.

Generate an image or a short video

Connect the explainer companion separately. Its RC4 candidate (0.2.2rc4 on PyPI) adds direct HTTP routes for text-to-image, image editing, image translation and video from text, a first frame, or first plus last frames. These routes need the companion's generation extra, an existing project with pinned script/scene files, explicit regional DashScope configuration and current pinned price/access and quote declarations. Video output checks also need FFmpeg/ffprobe. They run without the upstream video_explainer CLI.

Follow the image lifecycle or video lifecycle. Obtain actual human authority for source submission and spending before setting spend_authorized=true; that field records an input acknowledgement and does not authenticate anyone or impose a provider billing cap. Keep the returned job ID. Use explainer_image_generation_finalize for captured synchronous images, explainer_image_generation_poll for translation and explicit image recovery, or explainer_generation_poll for video, each with a bounded authorized operation. Core job_status rejects both generated-media job kinds.

Saved hashes and complete decode checks qualify the selected file bytes. Live provider access, image identity/style fidelity, creative quality and full security review remain open. S2V/HappyHorse successor modes are optional and await separate source acceptance; the disabled raw Qwen process integration is a separate route. Stock search, local ASR intake timing and the retained renderer timing failure remain limits.

Inspect and extend

The selected RC3 native check covered the installed baseline and restart, with 124 cache files. Fresh npm network acquisition versus reuse of the same-version cache remains unknown. Earlier RC1 native and RC2 archive/installer checks remain historical results. Tool discovery does not qualify every optional runtime; live-provider quality and complete comparative acceptance remain open.


Created by Fausto Albers · Wonder Why. Built with Google Gemini, FastMCP and Pydantic, with optional knowledge and tracing integrations. Project code is MIT licensed; third-party notices cover bundled components with their own terms.

Available Tools

90 tools
audio_clip_exportAudio Clip ExportA

Export a whole or one-sided audio selection to a unique private 16kHz mono WAV.

Originals and prior exports remain unchanged. Caller-selected output paths and overwriting are rejected. Use image_manifest_read for verified restart.

Inputs: request (required). Returns: object

audio_deduplicateAudio DeduplicateA

Compare local audio candidates while retaining silence, errors and uncertainty.

Requires the optional audio extra and local FFmpeg. Similarity does not establish equal speech, speaker identity or permission to delete evidence.

Inputs: request (required). Returns: object

audio_dsp_analyzeAudio Dsp AnalyzeC

Measure audio or invoke one separately installed operator-pinned native DSP workflow.

Local analysis preserves stereo, finite silence/abstentions and absolute clipping intervals. A/B uses reference-minus-primary differences with units. Optional Juzzy modes preserve harmonic/rhythm/masking/section observations; Ferrous receives an explicit mono derivation and retains heuristic quality, content and fingerprint output. Standards and semantic claims are unverified. Reusing an exact job_id reads its attested artifacts without launching again. Use job_status for durable status, including interrupted/failed work.

Inputs: request (required), include_image. Returns: object

audio_transcribeAudio TranscribeA

Extract exact caption intervals or infer speech through an explicit backend.

Supplied cue and word times remain source assertions. Model-derived words and speaker labels remain inference. Caption selection precedes ASR; local-only selection forbids cloud requests. Readback requires the retained receipt digest and exact source identity. No local model is installed.

Inputs: request (required). Returns: object

content_analyzeContent AnalyzeA
Read-only

Analyze content (file, URL, or text) with any instruction.

Provide exactly one of file_path, url, or text. Uses Gemini's structured output for reliable JSON responses. Pass a custom output_schema to control the response shape, or use the default ContentResult schema.

Inputs: url, text, file_path, instruction, +2 more. Returns: object

content_batch_analyzeContent Batch AnalyzeA
Read-only

Analyze multiple content files from a directory or explicit file list.

Rejects discovery over 5,000 entry visits and compare payloads over DOC_MAX_DOWNLOAD_BYTES. Supports two modes: 'compare' sends files in a single call for cross-document analysis, 'individual' analyzes each file separately with bounded concurrency (3 parallel calls).

Inputs: mode, directory, max_files, file_paths, +4 more. Returns: object

content_extractContent ExtractA
Read-only

Extract structured data from content using a JSON Schema.

Uses Gemini's response_json_schema for guaranteed structured output.

Inputs: schema (required), content (required). Returns: object

image_cropImage CropA

Crop local PNG pixels without model inference or provider uploads.

Requires an independently installed FFmpeg and respects LOCAL_FILE_ACCESS_ROOT. A source hash binds the crop to bytes; it does not establish factual support or whether the input was synthetic. Large outputs retain metadata/path only.

Inputs: crop_box (required), file_path (required), output_path (required), include_image. Returns: object

image_editImage EditA

Edit a source image or decoded video frame and persist its exact manifest.

Coordinates use the oriented source grid before crop and resize. Requires the optional images extra; frame selection also requires local FFmpeg. Source-derived edits and annotations retain explicit provenance.

Inputs: request (required), include_image. Returns: object

image_manifest_readImage Manifest ReadA
Read-onlyIdempotent

Read an exact manifest after restart and verify its source and artifact bytes.

Inputs: manifest_path (required), expected_sha256 (required), include_image. Returns: object

image_ocrImage OcrA

Observe text and geometry using an explicitly selected local OCR engine.

Source OCR and detected geometry do not establish semantic table structure or factual support. Missing local runtimes return an explicit limitation.

Inputs: request (required). Returns: object

image_readImage ReadB

Return a bounded local still image as inspectable PNG and exact source metadata.

Animated inputs expose one still with an explicit limitation. Native output may be processed by the host's model; this server invokes no model or upload.

Inputs: file_path (required), max_pixels, include_image. Returns: object

image_segmentImage SegmentA

Plan or submit one image and preserve actual mask and overlay artifacts.

Scores and masks are unverified proposals. Service selection and checkpoint declarations do not attest a loaded model or establish redistribution rights.

Inputs: request (required), include_image. Returns: object

infra_cacheInfra CacheA
DestructiveIdempotent

Manage the analysis cache — stats, list, clear, or inspect context cache state.

Inputs: action, auth_token, content_id. Returns: object

infra_configureInfra ConfigureA
Idempotent

Reconfigure the server at runtime — preset, model, thinking level, or temperature.

Changes take effect immediately for all subsequent tool calls.

Inputs: model, preset, auth_token, temperature, +1 more. Returns: object

job_cancelJob CancelA

Cancel queued video work or request its owner stop at the next checkpoint.

Inputs: job_id (required). Returns: object

job_statusJob StatusB
Read-only

Read retained job identity, outcomes and current byte attestation.

Inputs: job_id (required). Returns: object

knowledge_askKnowledge AskA
Read-onlyIdempotent

Ask a question and get an AI-generated answer grounded in stored knowledge.

Uses Weaviate AsyncQueryAgent in ask mode to synthesize an answer from objects across knowledge collections, with source citations.

Inputs: query (required), collections. Returns: object

knowledge_fetchKnowledge FetchA
Read-onlyIdempotent

Fetch a single object by UUID from a knowledge collection.

Inputs: object_id (required), collection (required). Returns: object

knowledge_ingestKnowledge IngestA

Manually insert data into a knowledge collection.

Properties are validated against the collection schema — unknown keys are rejected with allowed name:type pairs.

Tip: call knowledge_schema(collection=...) first to see expected properties.

Inputs: collection (required), properties (required). Returns: object

knowledge_queryKnowledge QueryA
Read-onlyIdempotent

[DEPRECATED] Search knowledge store using natural language.

Deprecated: Use knowledge_search instead, which now includes Cohere reranking and Flash summarization for better results with lower token usage.

Uses Weaviate AsyncQueryAgent in search mode for intelligent object retrieval.

Inputs: query (required), limit, collections. Returns: object

knowledge_schemaKnowledge SchemaA
Read-onlyIdempotent

Return property schemas for knowledge collections.

Reads from local CollectionDef objects — no Weaviate connection needed. Use this before knowledge_ingest to discover expected property names and types.

Inputs: collection. Returns: object

knowledge_statsKnowledge StatsB
Read-onlyIdempotent

Get object counts per collection, optionally grouped by a property.

Inputs: group_by, collection. Returns: object

media_acquireMedia AcquireC

Acquire bounded media into private owned storage with exact byte identity.

Inputs: source (required). Returns: object

media_analyze_musicMedia Analyze MusicA

Infer timed musical sections and properties from actual decoded audio.

Instruments, mood, tags, tempo, key and meter remain model interpretations. Submitted PCM clocks and hashes identify the evidence for each section.

Inputs: request (required). Returns: object

media_asset_removeMedia Asset RemoveA
Destructive

Remove one verified owned asset and invalidate its source-dependent caches.

Inputs: asset_id (required). Returns: object

media_assets_listMedia Assets ListB
Read-onlyIdempotent

List a bounded page with fresh byte checks and explicit unhealthy states.

Inputs: limit, offset. Returns: object

media_caption_eventsMedia Caption EventsC

Caption timed audio, visual and combined occurrences with submitted support.

Inputs: request (required). Returns: object

media_count_eventsMedia Count EventsB

Count retained inferred occurrences of a target in bounded source windows.

Counts concern admitted records in submitted evidence. Overlapping records remain distinct, and sampled frames leave gaps where events may be missed.

Inputs: request (required). Returns: object

media_edit_footageMedia Edit FootageA

Edit bounded local footage through a measured prepare and approval workflow.

Prepare retains source intervals, actual frame clocks, grade/audio decisions and scene previews. Assemble requires the exact prepared manifest and every scene artifact approval, then gates full decoding, timeline, black spans and required audio loudness before delivery. Declared beat grids and approvals remain caller assertions; technical checks do not establish visual semantics. FFmpeg and Pillow must already be installed. No provider is invoked.

Inputs: request (required), include_image. Returns: object

media_ground_eventsMedia Ground EventsC

Ground a query in source-supported intervals with uncalibrated scores.

Inputs: request (required). Returns: object

media_infoMedia InfoA

Inspect exact local media bytes, stream metadata, rotation, audio and chapters.

Uses independently installed FFprobe and a bounded private stable snapshot. This performs no provider upload and does not establish watched coverage.

Inputs: file_path (required). Returns: object

media_metadataMedia MetadataB
Read-onlyIdempotent

Retrieve explicit metadata without downloading video, inference or upload.

Inputs: source (required). Returns: object

media_perceiveMedia PerceiveA

Infer separate spoken and visible source evidence across bounded AV windows.

Local dry plans prepare exact sampled frames and decoded PCM with no provider calls. Submission requires the exact workflow grant and configured account. Model intervals map back to source seconds; sampled images leave visual gaps.

Inputs: request (required). Returns: object

media_source_inspectMedia Source InspectA
Read-onlyIdempotent

Inspect source syntax and capabilities without contacting or downloading it.

Inputs: source (required). Returns: object

mhs_discoverMhs DiscoverB
Read-onlyIdempotent

Discover registered simulated devices and their explicit capabilities.

Inputs: tag, device_type. Returns: object

mhs_health_checkMhs Health CheckC
Idempotent

Inspect fresh simulated health and acknowledge a failed-write health gate.

Inputs: device_id. Returns: object

mhs_meta_infoMhs Meta InfoB
Read-onlyIdempotent

Inspect current device values, directions and effective simulated limits.

Inputs: device_id (required). Returns: object

mhs_readMhs ReadC

Read declared simulated sensors or retain independently generated image bytes.

Inputs: params (required), device_id (required), capability (required). Returns: object

mhs_resetMhs ResetB
Idempotent

Latch a simulated emergency stop or perform separately authorized recovery.

Inputs: mode (required), device_id (required), command_id (required). Returns: object

mhs_writeMhs WriteA
Idempotent

Apply one atomic simulated write or return its exact authority proposal.

Soft overrides and confirmation-required operations need separate host approval bound to this command, state and limits. confirm=True grants no approval. Terminal IDs replay their receipt. An explicitly resumed pending proposal may attempt its first action after exact host approval.

Inputs: params (required), device_id (required), capability (required), command_id (required), +1 more. Returns: object

provider_capabilitiesProvider CapabilitiesA
Read-only

Inspect configured operation support and reject unsupported selections.

Profiles declare support; inspection performs no connectivity or model test.

Inputs: backend, required. Returns: object

research_assess_evidenceResearch Assess EvidenceB
Read-only

Assess a claim against sources, returning evidence tier and confidence.

Inputs: claim (required), sources (required), context. Returns: object

research_deepResearch DeepA
Read-only

Run three-phase model-only synthesis without external source retrieval.

Phases: Scope Definition -> Evidence Collection -> Synthesis. Model tiers remain proposals. CONFIRMED and STRONG INDICATOR proposals become UNKNOWN because this route neither retrieves nor verifies sources.

Inputs: topic (required), scope, thinking_level. Returns: object

research_documentResearch DocumentA
Read-only

Run multi-phase deep research grounded in source documents.

Phases: Document Mapping -> Evidence Extraction -> Cross-Reference -> Synthesis. Every claim is labeled with evidence tiers and cited back to source documents.

Inputs: instruction (required), urls, scope, file_paths, +1 more. Returns: object

research_executeResearch ExecuteA

Prepare or execute source-preserving research under one global allowance.

Dry runs make no provider or URL calls. Supplied originals are byte checked. URL execution requires source-access authorization; inference requires submission authorization. Google hosted research remains a separate route. Unknown price ceilings fail closed. Model tiers never establish CONFIRMED.

Inputs: request (required). Returns: object

research_paper_citationsResearch Paper CitationsC
Read-only

Get citations or references for a paper.

Inputs: paper_id (required), limit, direction. Returns: object

research_paper_detailsResearch Paper DetailsA
Read-only

Get detailed metadata for a specific paper.

Accepts Semantic Scholar IDs, DOIs (prefix with DOI:), or ArXiv IDs (prefix with ArXiv:).

Inputs: paper_id (required). Returns: object

research_paper_recommendationsResearch Paper RecommendationsB
Read-only

Get paper recommendations based on seed papers.

Uses Semantic Scholar's recommendation engine to find related papers.

Inputs: seed_paper_ids (required), limit. Returns: object

research_planResearch PlanA
Read-only

Generate a multi-agent research orchestration plan.

Returns a phased blueprint with task decomposition and model assignments. Does NOT spawn agents — provides the blueprint for the caller.

Inputs: topic (required), scope, available_agents. Returns: object

research_webResearch WebA

Launch a Gemini Deep Research Agent for autonomous web-grounded research.

The agent plans its own research, searches the web (~80-160 queries), reads sources, and produces a cited markdown report. Runs in background; poll with research_web_status. Provider usage is billed; runtime varies.

Inputs: topic (required), job_id, output_format. Returns: object

research_web_cancelResearch Web CancelA

Cancel a running Deep Research task.

Sends a cancel request to the Interactions API and cleans up local tracking state. Useful for stopping a task before further provider usage.

Inputs: interaction_id (required). Returns: object

research_web_followupResearch Web FollowupA

Ask a follow-up question about a completed Deep Research report.

Uses previous_interaction_id to maintain context from the original research. Synchronous — follow-ups are fast (no background needed).

Inputs: question (required), interaction_id (required), job_id. Returns: object

research_web_statusResearch Web StatusB
Read-only

Poll or retrieve a Deep Research task.

Returns the full report with sources when completed, or current status if still in progress. Auto-stores completed reports to Weaviate.

Inputs: interaction_id (required). Returns: object

reverse_search_frameReverse Search FrameA

Plan or reverse-search one captured frame using Serper Lens.

Execution publicly uploads the PNG to Uguu, verifies hosted bytes, then submits its URL to Lens. It requires both explicit grants and an enabled Serper credential. Similarity suggestions never assert entity identity; compare appearance, text and source context before reporting an identity. Publication retention/deletion and live service schemas remain unqualified.

Inputs: request (required). Returns: object

session_memorySession MemoryA
Destructive

Inspect original history or read/edit/delete one explicitly scoped derived profile.

History remains recoverable after active-session expiry when SQLite persistence is configured. Large inline-media pages export exact bounded private JSON. Memory edits require the current revision; deletion clears only derived text. Caller-selected scope is isolation, not an authenticated multi-user identity. No provider, inference, upload or global-memory search occurs.

Inputs: request (required). Returns: object

source_ingestSource IngestA
Idempotent

Retain original bytes and extract bounded elements with explicit source positions.

PDF requires separately installed Poppler; DOCX, Markdown, HTML, plain text and PCM16 WAV use local parsers. Unsupported OCR, PDF table reconstruction, image pixels and speech are reported explicitly. Empty/error extraction fails. Identical source/revision/parser work reuses its durable job. Extraction does not index, call a model, upload to a parser service or prove factual claims.

Inputs: request (required). Returns: object

source_ingest_cancelSource Ingest CancelB
Idempotent

Request cancellation without promoting an unfinished extraction to success.

The owner acknowledges cancellation after its bounded parser has joined; this request alone does not prove that a running native process has stopped.

Inputs: job_id (required). Returns: object

source_ingest_readSource Ingest ReadC
Read-onlyIdempotent

Recheck a retained ingestion job and read exact located extraction bytes.

Inputs: job_id (required). Returns: object

text_generateText GenerateA

Generate structured text using one selected compatible HTTP endpoint.

No default Gemini/vision account is borrowed. Sources are untrusted text; this operation cannot accept files, media, tools or grounded search. Missing input-token/currency meters cause requested hard ceilings to fail.

Inputs: request (required). Returns: object

twelvelabs_callTwelvelabs CallA
Destructive

Plan or perform one bounded TwelveLabs asset/index/search/embed/entity/analysis operation.

Disabled by default. Dry plans read only selected fenced local media and make no DNS, provider or database calls. Execution requires explicit BYOK/network authority; media disclosure and deletion/cancellation have separate grants. Each call makes at most one HTTP request with no retries or automatic polling. Reusing an identical job_id reads the attested local receipt without resending. Remote readiness, exact clip seconds and IDs stay provider observations.

Inputs: request (required). Returns: object

video_analyzeVideo AnalyzeA
Idempotent

Analyze a video (YouTube URL or local file) with any instruction.

Provide exactly one of url or file_path. Uses Gemini's structured output for reliable JSON responses. Pass a custom output_schema to control the response shape, or use the default VideoResult schema.

When strict_contract=True, runs the full contract pipeline: analysis with strict Pydantic models, parallel strategy/concept-map generation, artifact rendering, and structural quality gates. Factual/media review and observed coverage remain explicit pending/unknown fields. Returns richer output but takes longer.

Inputs: fps, url, dry_run, file_path, +11 more. Returns: object

video_analyze_windowsVideo Analyze WindowsC

Plan or execute contiguous source windows with retained partial outcomes.

Inputs: request (required), execution_budget (required), job_id, dry_run, +1 more. Returns: object

video_batch_analyzeVideo Batch AnalyzeA
Read-onlyIdempotent

Analyze all video files in a directory concurrently.

Scans the directory for supported video files (mp4, webm, mov, avi, mkv, mpeg, wmv, 3gpp), then analyzes each with the given instruction using bounded concurrency (3 parallel Gemini calls). Discovery rejects more than 5,000 entry visits; narrow the directory or pattern to stay within that bound.

Inputs: directory (required), job_id, max_files, instruction, +4 more. Returns: object

video_clip_exportVideo Clip ExportA

Export a bounded source clip and verify its encoded output and manifest.

Requested bounds, selected original PTS and actual encoded timing remain separate. This operation preserves the original and invokes no provider.

Inputs: request (required). Returns: object

video_clip_selectVideo Clip SelectC

Export a whole or one-sided bounded video selection using measured clip timing.

Inputs: request (required). Returns: object

video_commentsVideo CommentsA
Read-onlyIdempotent

Fetch top YouTube comments sorted by relevance.

Returns comment text, like count, and author for each comment. Costs 1+ YouTube API units, 0 Gemini units.

Inputs: url (required), max_comments. Returns: object

video_continue_sessionVideo Continue SessionB

Continue analysis within an existing video session.

Inputs: prompt (required), session_id (required), scope. Returns: object

video_create_sessionVideo Create SessionA

Create a persistent session for multi-turn video exploration.

Provide exactly one of url or file_path. When download=True and the source is YouTube, the video is downloaded via yt-dlp, uploaded to the Gemini File API, and context-cached for fast multi-turn use.

Inputs: url, scope, download, file_path, +1 more. Returns: object

video_deduplicate_framesVideo Deduplicate FramesC

Compare bounded source frames without dropping similar or failed candidates.

Inputs: request (required), include_image. Returns: object

video_detect_scenesVideo Detect ScenesC

Detect hard visual cuts and return contiguous intervals with actual source PTS.

Inputs: request (required). Returns: object

video_frameVideo FrameB

Retrieve a source frame with requested time, original decoded PTS and actual delta.

Times are relative to the measured container presentation origin. Precise selection returns the first decoded frame at or after the request. Keyframe selection reports its measured approximation and selection method.

Inputs: file_path (required), time_seconds (required), crop_box, selection, +2 more. Returns: object

video_frame_by_queryVideo Frame By QueryB

Select a frame by lexical transcript match, requiring the exact video digest.

Segment times use the container-relative presentation clock. Ties choose the earliest segment. A zero-token match abstains. The supplied transcript remains unverified; the extracted frame carries measured PTS.

Inputs: query (required), file_path (required), transcript (required), max_pixels, +1 more. Returns: object

video_framesVideo FramesC

Inspect a bounded burst/window or timestamped contact sheet without a watched claim.

Dense filmstrips require a narrow window and explicit FPS. Selection retains changed text; it performs no similarity pruning or OCR. Sheet tiles carry actual source times/digests. Scene/keyframe sampling may miss short events.

Inputs: file_path (required), fps, mode, crop_box, +5 more. Returns: object

video_metadataVideo MetadataA
Read-onlyIdempotent

Fetch YouTube video metadata without Gemini analysis.

Returns title, description, view/like/comment counts, duration, tags, channel info, category, and language. Costs 1 YouTube API unit, 0 Gemini units.

Inputs: url (required). Returns: object

video_note_createVideo Note CreateA
Destructive

Create a source-linked illustrated tutorial PDF or validate a read-only dry plan.

Actual generation retains exact source PNGs, page rasters and a manifest/receipt. Missing illustrations or AV inference preserve supplied text with warnings. Deterministic PDF checks do not establish factual or human review.

Inputs: request (required). Returns: object

video_playlistVideo PlaylistA
Read-onlyIdempotent

Get video IDs and titles from a YouTube playlist.

Results can be passed to video_analyze for batch analysis. Costs 1 YouTube API unit per page (max 50 items/page).

Inputs: url (required), max_items. Returns: object

video_storyboardVideo StoryboardC

Create a measured source storyboard with literal absolute source-clock labels.

Inputs: request (required), include_image. Returns: object

vision_chatVision ChatA

Analyze, identify or compare exact images and bounded source video samples.

Dry plans perform local preparation with zero provider calls. A live request requires explicit workflow authorization and a configured model/account. Interpretation remains model inference, separately from source byte lineage.

Inputs: request (required). Returns: object

vision_groundingVision GroundingB

Locate objects with strict inferred boxes and optional actual original-pixel crops.

Normalized1000 boxes map through the exact transmitted preparation to original stored/oriented pixels. Crop byte/geometry verification does not verify that the model identified the object correctly. Empty detections remain valid abstentions.

Inputs: request (required). Returns: object

vision_ocrVision OcrB

Infer visible text from exact prepared images or precise source frames.

Model transcription is labeled inference. The separate image_ocr operation selects local Tesseract/Vision observations. No automatic fallback occurs.

Inputs: request (required). Returns: object

web_extractWeb ExtractA

Acquire one checked UTF-8 page or a labeled provider-returned representation.

Dry plans make no DNS/HTTP requests. Direct redirects stay within the source allowlist. Provider services' remote page peers, redirects and original-byte identity remain unverified; returned text cannot authorize further actions.

Inputs: request (required). Returns: object

web_search_providerWeb Search ProviderA

Plan or perform one explicitly selected optional search request.

Automatic selection uses configured credential-ready Serper, Tavily, Exa, then Serply. Missing pinned keys and failed requests never cause fallback. Provider snippets remain untrusted data and do not establish factual support.

Inputs: request (required). Returns: object

youtube_channel_catalogYoutube Channel CatalogA
Read-onlyIdempotent

List one channel uploads page and retain any continuation token.

A channel video count and playlist reported total are provider metadata. The returned items establish this page's coverage; no video bytes are fetched.

Inputs: reference (required), max_items, page_token. Returns: object

youtube_channel_inspectYoutube Channel InspectB
Read-onlyIdempotent

Inspect channel metadata through YouTube Data API without downloading media.

Inputs: reference (required). Returns: object

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3/5.0

    Scored across 90 tools

    Disambiguation2/5

    With 90 tools across knowledge, research, video, audio, image, web, and device simulation, many boundaries overlap: video_analyze/content_analyze/media_perceive/vision_chat, research_web/research_deep/research_document/research_execute, and video_frame/video_frames/video_frame_by_query are especially confusable. The detailed descriptions help, but an agent still faces high misselection risk because many distinctions are workflow-specific and subtle.

    Naming Consistency4/5

    All names use snake_case and largely follow a domain-prefixed operation pattern (knowledge_ingest, video_analyze, research_web_status), which is readable and predictable by area. However, there are mixed noun-only and verb styles (knowledge_stats, media_info, job_status) and brand/acronym prefixes (mhs_, twelvelabs_), so it is not a perfectly uniform verb_noun convention.

    Tool Count1/5

    90 tools is far beyond the typical 3-15 scoped set and exceeds even the 25+ heavy threshold. The server spans many unrelated domains, making the surface unwieldy and creating extreme cognitive load for tool selection.

    Completeness4/5

    The surface covers research, knowledge, video/image/audio processing, web, and media assets with lifecycle operations such as ingest, search, acquire, export, edit, and cancel. Minor gaps exist, including knowledge object update/delete and collection management, but most workflows have supporting tools.

    Maintenance

    ActivityMaintained
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers