Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@mcp-video-recognition-bilibili Analyze https://www.bilibili.com/video/BV1xx and summarize its content."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP (Model Context Protocol) server that analyzes images, audio, and video using Google's Gemini AI.
This is a fork of mario-andreschak/mcp_video_recognition. All original recognition logic and credit belong to the upstream author. This fork adds the changes listed below. Licensed under MIT, same as upstream.
Compared to the upstream project, this fork adds:
URL input for video recognition / 视频识别支持网址输入
The video_recognition tool's filepath argument now accepts either a local file path OR a video URL (e.g. a Bilibili or YouTube link). When a URL is given, the server downloads the video with yt-dlp first, then analyzes it with Gemini, and deletes the temp file afterwards.
video_recognition 的 filepath 参数现在既能填本地路径，也能直接填视频网址（B站 / YouTube 等）。传网址时服务端先用 yt-dlp 下载，再交给 Gemini 分析，用完自动删除临时文件。
Auto quality selection / 自动选清晰度
Uses yt-dlp -S res:480 to pick the format closest to 480p (works for both landscape and portrait videos), keeping downloads small and fast to save tokens and memory.
用 -S res:480 自动选最接近 480p 的档（横屏竖屏都适配），省流量、省 token、省内存。
Cookie & User-Agent support for anti-bot sites / 支持 Cookie 和 UA 绕过风控
If a Netscape-format cookie file exists at /app/bili-cookies.txt, it is passed to yt-dlp automatically (Bilibili and some sites return HTTP 412 without login cookies). A desktop User-Agent is always sent.
若 /app/bili-cookies.txt 存在（Netscape 格式的 cookie 文件），会自动带给 yt-dlp（B站等站点无登录 Cookie 会返回 412）。同时固定发送桌面版 User-Agent。
Dockerfile bundles yt-dlp + ffmpeg / Dockerfile 内置 yt-dlp 和 ffmpeg。
Image and audio recognition are unchanged from upstream. 图片和音频识别与上游一致，未改动。
Related MCP server: AI Vision MCP Server
image_recognition — analyze an image (local file path)
audio_recognition — analyze / transcribe audio (local file path)
video_recognition — analyze a video; filepath accepts a local path or a URL
Docker (recommended), or Node.js 22+
A Google Gemini API key
git clone <your-fork-url>.git video-mcp
cd video-mcp
# 1. Create env file (see .env.example)
cp .env.example video.env
# then edit video.env and put in your real GOOGLE_API_KEY
# 2. (Optional, for Bilibili) put a Netscape-format cookie file next to the project
# Export it with a browser extension like "Cookie-Editor" while logged in to Bilibili.
# Name it bili-cookies.txt
# 3. Build
docker build -t video-mcp .
# 4. Run (mount cookie file if you have one)
docker run -d --name video --restart unless-stopped \
-p 18014:3000 \
--env-file ./video.env \
-v "$(pwd)/bili-cookies.txt:/app/bili-cookies.txt" \
video-mcpMCP endpoint: http://<host>:3000/mcp (Streamable HTTP).
Variable | Meaning |
| Required. Google Gemini API key. |
|
|
| Port for HTTP transport. Default |
|
|
When your MCP client (or its model) calls video_recognition, pass the Bilibili/YouTube link as the filepath argument. Example intent:
Call
video_recognitionwithfilepath=https://www.bilibili.com/video/BVxxxxxxxxx/and tell me what's in the video.
Keep videos short (Gemini free tier has size/quota limits). Cookies expire — re-export when Bilibili starts returning 412 / login errors.
Never commit your real video.env (contains the API key) or bili-cookies.txt (contains your login session). Both are gitignored.
Downloaded videos are stored in a temp dir and deleted after analysis.
这份教程假设你要把它部署到一台**自己的云服务器（Linux）**上，用 Docker 运行，让远程的 MCP 客户端（如各类聊天平台）通过网址调用，实现「发一个 B 站链接，AI 就能看懂视频画面」。
一台装了 Docker 的 Linux 服务器（1核1G 也能跑，处理视频那几秒会吃点内存，建议有 2G 内存或配了 swap）。
一个 Google Gemini API Key：去 aistudio.google.com/apikey 免费申请。
（下 B 站视频需要）一份你自己的 B 站登录 Cookie，下面第 4 步会讲怎么导出。
cd ~
git clone <你的仓库地址>.git video-mcp
cd video-mcpcp .env.example video.env
nano video.env把 GOOGLE_API_KEY= 后面换成你自己的 Gemini Key。其它保持默认即可（TRANSPORT_TYPE=sse 表示走 HTTP，适合远程用）。保存退出：Ctrl+O 回车 Ctrl+X。
B 站对没有登录 Cookie 的请求会返回 HTTP 412，导致下载失败。解决办法是带上你自己的登录 Cookie：
电脑浏览器登录 B 站（bilibili.com）。
装浏览器扩展 Cookie-Editor，在 B 站页面点开它。
点 Export → Export as Netscape（⚠️ 一定要选 Netscape 格式，不是 JSON）。
把导出的内容保存成服务器上的 ~/video-mcp/bili-cookies.txt：
nano ~/video-mcp/bili-cookies.txt粘贴进去保存。文件开头应该是 # Netscape HTTP Cookie File。
Cookie 会过期。哪天视频又下不了（报 412 或要登录），重新导出覆盖这个文件、再
docker restart video即可。
docker build -t video-mcp .第一次会下载 Node 基础镜像、安装 ffmpeg 和 yt-dlp、编译代码，需要一两分钟。看到 naming to ... video-mcp 就成功了。
docker run -d --name video --restart unless-stopped \
-p 18014:3000 \
--env-file ~/video-mcp/video.env \
-v ~/video-mcp/bili-cookies.txt:/app/bili-cookies.txt \
video-mcp说明：
-p 18014:3000：把容器的 3000 端口映射到服务器的 18014（对外端口你可以改）。
-v ...bili-cookies.txt...：把 Cookie 文件挂进容器。没做第 3 步（没 Cookie）就删掉这一行 -v。
注意挂载 Cookie 不要加 :ro（只读），因为 yt-dlp 运行时会回写更新 Cookie。
docker logs video看到 Server started with Streamable HTTP transport on port 3000 就正常了。再确认 yt-dlp 装好：
docker exec video yt-dlp --version能打印版本号（如 2026.07.04）即可。
MCP 端点是：
http://你的服务器IP:18014/mcp如果你用了域名 + 反向代理（如 Nginx / Caddy）转发到 localhost:18014，就用你的 https://域名/mcp。传输方式选 Streamable HTTP。
在你的聊天客户端里，明确要求调用 video_recognition 工具，把链接作为 filepath 参数。例如对 AI 说：
请调用 video_recognition 工具，filepath 填 https://www.bilibili.com/video/BVxxxxxxxxx/ ，帮我看看视频里是什么。
⚠️ 如果你同时接了「网页读取」类工具（如 jina），AI 可能会把链接拿去读网页而不是下载视频。这时要明确说「不要读网页，用 video_recognition 下载视频看画面」。
现象 | 原因 | 解决 |
下载报 | B 站风控，没带登录 Cookie | 按第 3 步准备 |
报 | 挂载 Cookie 时加了 | 去掉 |
报 | 视频没有对应清晰度档 | 本 fork 已用 |
报 | Gemini Key 免费额度用完 / 被限流 | 换一个 Gemini Key（改 |
AI 不调用视频工具，去读网页了 | 客户端优先用了别的工具 | 对话里点名 |
视频识别很吃 Gemini 的 token，建议只处理 1~3 分钟以内的短视频。
Gemini 有免费额度，个人偶尔用足够；高频使用会触发限流或产生费用。
本 fork 默认下载 480p 左右画质，已经尽量省流量和 token。
Upstream project: mario-andreschak/mcp_video_recognition — original image/audio/video recognition MCP server.
This fork only adds URL download / cookie / quality-selection features on top.
Licensed under the MIT License (see LICENSE). The original copyright
Copyright (c) 2025 mario-andreschak is retained.
Analyze and transcribe audio using Google Gemini AI
| Name | Required | Description | Default |
|---|---|---|---|
| prompt | No | Custom prompt for the recognition | Describe this content |
| filepath | Yes | Path to the media file to analyze | |
| modelname | No | Gemini model to use for recognition | gemini-2.0-flash |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the full burden of behavioral disclosure. It only states 'analyze and transcribe' but does not detail output format, processing behavior, authentication needs, or limitations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, front-loaded with the core purpose. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should explain what the tool returns (e.g., transcribed text or analysis). It does not, nor does it cover edge cases or prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all parameters. The description does not add additional meaning beyond what the schema provides, meeting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: analyze and transcribe audio using Google Gemini AI. It explicitly mentions 'audio' which distinguishes it from sibling tools image_recognition and video_recognition.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs alternatives (e.g., image_recognition, video_recognition). The description only states what it does without context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Analyze and describe images using Google Gemini AI
| Name | Required | Description | Default |
|---|---|---|---|
| prompt | No | Custom prompt for the recognition | Describe this content |
| filepath | Yes | Path to the media file to analyze | |
| modelname | No | Gemini model to use for recognition | gemini-2.0-flash |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only mentions 'using Google Gemini AI' but does not disclose safety (e.g., read-only vs destructive), API costs, file size limits, or the nature of the analysis (e.g., real-time, batch).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb and resource. No wasted words. Efficient and scannable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is provided, yet the description does not explain what the tool returns (e.g., text description, confidence scores). For a tool with 3 parameters and no annotations, this leaves the agent guessing about the response format and behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter described. The description adds no additional meaning beyond the schema; it only names the AI provider. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Analyze and describe images') and the technology ('using Google Gemini AI'). It distinguishes from sibling tools (audio_recognition, video_recognition) by specifying the media type (images).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., audio_recognition, video_recognition). No mention of prerequisites, limitations, or scenarios where it is not appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Analyze and describe a video using Google Gemini AI. The "filepath" argument accepts EITHER a local file path OR a video URL (e.g. a Bilibili or YouTube link). URLs are downloaded automatically (<=480p) before analysis.
| Name | Required | Description | Default |
|---|---|---|---|
| prompt | No | Custom prompt for the recognition | Describe this content |
| filepath | Yes | Path to the media file to analyze | |
| modelname | No | Gemini model to use for recognition | gemini-2.0-flash |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that URLs are downloaded automatically and limited to 480p, and mentions the use of Gemini AI. It does not detail output format, auth, or rate limits, but for a analysis tool the basic behavior is well covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no superfluous information. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and 3 parameters (1 required), the description covers purpose, input variants, and a key behavioral detail (480p download). It does not explain return values, but that is acceptable without an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage, baseline is 3. The description adds significant value by clarifying that filepath accepts both local paths and URLs, and that URLs are downloaded automatically, which is not evident from the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool analyzes and describes a video using Google Gemini AI, and distinguishes it from sibling tools (audio_recognition, image_recognition) by focusing on video input.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explains when to use the tool (to analyze a video) and provides guidance on input types (local file or URL) and automatic download limitation (<=480p). It does not explicitly mention when not to use, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v1.0.0audio_recognitionimage_recognitionvideo_recognitionScored across 3 tools
Each tool targets a distinct media type (audio, image, video) with no overlap, making selection unambiguous.
All tools follow a consistent 'type_recognition' pattern, providing predictable naming.
Three tools cover the core recognition capabilities (audio, image, video) without excess; each tool serves a clear purpose.
The set covers the main recognition tasks for the domain, but lacks operations like downloading videos or managing results, though the video tool handles URL input.
Analyze images and videos with Gemini to get fast, reliable visual insights. Handle content from U…
Multimodal video analysis MCP — transcription, vision, and OCR for any video URL.
Transform video, audio and images, and generate media from prompts. FFmpeg, captions, models.
Analyze images from multiple angles to extract detailed insights or quick summaries. Describe visu…
