OpenDaemon ComfyUI MCP
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| FFMPEG | No | A specific ffmpeg executable. Defaults to being found automatically; one comes with this package. | |
| COMFYUI_DIR | No | ComfyUI's folder (the one with main.py). Defaults to being looked for in the usual places. | |
| COMFYUI_URL | No | ComfyUI's address. | http://127.0.0.1:8188 |
| PIPELINE_URL | No | The pipeline server's URL/address. | |
| PIPELINE_PORT | No | The pipeline server's port. | 8190 |
| COMFYUI_PYTHON | No | The Python that runs ComfyUI, used to start it and to install node packs. Defaults to the portable build's own Python. | |
| COMFY_MCP_DATA | No | Where videos, your workflows and settings are kept. | ~/OpenDaemon-ComfyUI-MCP |
| GEMINI_API_KEY | No | A free Gemini vision model key, used to write prompts and to judge clips. If unset, those two tools say so. | |
| NVIDIA_API_KEY | No | A free NVIDIA vision model key, used to write prompts and to judge clips. If unset, those two tools say so. | |
| PIPELINE_AUTOSTART | No | '0' = never start the pipeline server itself (for an app that runs it); '1' = allow autostart. | 1 |
| PIPELINE_DOWN_HINT | No | The message the model gets when PIPELINE_AUTOSTART is 0 and the pipeline server is down. | |
| OPENDAEMON_COMFY_TOOLS | No | Tool exposure: 'full' (67 tools), 'core' (15 tools), or a comma list of tool names. | full |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
How to make a video with this server
comfyui_status - is ComfyUI running, which graphics card, how much memory. start_comfyui if it is down.
list_models - what is installed. On a 4 GB card start with model "small".
upload_image (optional) - a picture to start from (image to video). Use the name it returns. No picture? render with the prompt alone (text to video): describe the whole scene then.
write_prompt (optional, needs a picture) - turns a short idea into a good video prompt, looking at the picture.
render - picks every setting from a style and a quality. Returns a job id and a rough time.
job_status - waits up to 50 s per call and reports progress. Call it again until status is done. The video's path is in the answer. One video renders at a time; others wait in the queue. If a job fails: diagnose_job. Out of memory: quality "Draft", fewer seconds, or a smaller model.
Prompt tips (LTX-Video): describe ONE simple action in the present tense, then the look and the camera. Say what moves and how ("slowly raises one hand", "the antennae wobble softly"). Keep it under 120 words. Long clips (5 s+) drift at the end: make short clips instead. The camera choice is added for you. Without a picture the model invents everything, so also say who or what is there, what it looks like, the place, the light and the style ("a fluffy white kitten on a grey rug in a sunny living room, soft daylight, realistic"). A picture gives steadier, sharper clips; ask the user for one when the look matters.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| comfyui_statusA | Is ComfyUI running, which graphics card and how much memory it has, free disk space, and what is rendering. |
| start_comfyuiA | Starts ComfyUI if it is not running (takes 20-60 s). Does nothing if it already runs. |
| restart_comfyuiA | Stops and starts ComfyUI again. Fixes a stuck ComfyUI. Refused while a video renders. Ask the user first, then call with user_confirmed=true. |
| free_gpu_memoryA | Unloads models from the graphics card. Use after an out-of-memory error or before another program needs the GPU. Refused while a video renders. |
| get_logsA | The last lines of the ComfyUI log (max 500). contains= keeps only lines with that text, e.g. "error". |
| get_defaultsA | Your saved defaults for render (e.g. model, quality, style). They apply when render leaves a setting out. |
| set_defaultsA | Saves render defaults so later renders can leave them out. clear=true forgets all of them. |
| list_modelsA | What models are installed and what can be downloaded, with size, the card they need, and if we tested them. "best" marks the best one for this PC's graphics card. Use a model's "use_as" value in render(model=...). |
| download_modelA | Downloads a model from our list (ids from list_models), with any helper files it needs. Big downloads: tell the user the size and call again with user_confirmed=true. |
| download_statusC | Progress of the running model download (one at a time). |
| cancel_downloadA | Stops the running model download. What was downloaded is kept, so starting it again resumes. |
| search_modelsA | Searches Hugging Face for video models that fit this PC's graphics card (bigger ones are hidden). These are NOT tested by us; check_model reads a file's header first. Empty query = LTX-Video and Wan VACE models. |
| check_modelA | Says if a model file can run here and why not if it can't, without loading it. A file on this PC: file="C:/path/model.gguf" or a file name in ComfyUI's models folder. A file on Hugging Face (from search_models): repo="owner/name" and file="path/in/repo.gguf". |
| add_model_fileA | Adds a model file that is already on this PC, where it is (a link, no copy). It is checked first. After this, render(model=) uses it. |
| remove_modelA | Removes a model the user added (from list_models "added_by_user"). Only our link is removed, never the user's file. Ask the user first, then call with user_confirmed=true. |
| upload_imageA | Puts a picture into ComfyUI's input folder. Give ONE of: path (a file on this PC), url (http/https), base64_data (the image bytes; give a name like "cat.png"). Returns the name to use in render(image=...). |
| write_promptB | Turns a short idea ("the cat waves hello") into a full video prompt, looking at the uploaded picture. Uses the AI keys set up in OpenDaemon (Gemini or NVIDIA). You can also write the prompt yourself. |
| write_avoid_listB | Writes the avoid list (negative prompt) for a prompt. Only vace and LTX dev models use it. |
| render_optionsA | The models, styles, qualities, cameras and speeds render() accepts, with what each one does. |
| renderA | Makes a video from a prompt, starting from a picture (image = a name from upload_image) or, with no image, from the prompt alone (text to video: describe the whole scene). Everything else is optional: model: small (default, fast) | ltx13 | vace | an added model's file name (list_models) style: e.g. "Balanced", "Soft and calm", "Lively", "Slow motion" (render_options lists all) quality: "Draft" (fastest) | "Normal" (default) | "High" | "Best" (1080p, very slow) | "Manual" (then width, height) seconds: 1-7 (2-3 holds together best). camera: still | zoom | zoomout | panr | panl. speed: slow | normal | fast movement: 10-70, how much things move (style sets it). seed: -1 = new each time; same seed + settings = same video. avoid: what must not appear (vace and LTX dev models only). priority: "user" (default, goes first) or "batch" (waits while the user's own renders go first). Returns a job id right away; then call job_status(id). |
| rerenderB | Makes a video again with the settings of an earlier one (a name from list_videos). seed: leave out for the same seed (same video, e.g. after a crash), -1 for a new variation, or a number. |
| job_statusA | Waits until the job is done (up to wait_seconds, max 55) and reports progress, then returns its status. Call again while status is queued or rendering. No id = the last 10 jobs. Also takes a batch id ("batch-...", from render_batch), a fix id ("fix-...", from auto_fix) or an edit id ("edit-..."). |
| cancelC | Stops a queued or rendering job. |
| diagnose_jobB | Why a job failed, in plain words, with what to try and the ComfyUI log lines. |
| list_videosB | The finished videos, newest first, with their file path and short settings. |
| video_infoB | Everything about one video: all its settings, size, length, path and a link to watch it. |
| delete_videoA | Moves a video and its settings to the out/trash folder (not deleted for good). Ask the user first, then call with user_confirmed=true. |
| estimateA | How long a render (same settings as render) or a whole batch (items = a list of render settings) would take on this PC, and how long the queue is, WITHOUT starting anything. |
| render_batchA | Many clips at once. items = a list of render settings, e.g. [{"image": "cat.png", "prompt": "...", "style": "Lively"}, ...] (same keys as render). All items are checked before any starts. Batches wait while the user's own renders go first. Over 30 minutes in total: tell the user the estimate and call again with user_confirmed=true. Returns a batch id; job_status(batch id) reports progress. |
| render_variationsA | Several versions of an earlier video to pick from (2-6). vary: "seed" (same settings, different luck: the usual choice) or "movement" (less and more movement around the original, same seed). Returns job ids for job_status. |
| queueA | The render queue. action: list (default) | pause (nothing new starts; the current render finishes) | resume |
clear (cancels waiting jobs; priority="batch" clears only batch jobs) | front (move waiting job |
| review_videoA | Shows you the clip: frames spread over it (start, middle, end for 3) side by side in ONE picture, after the start picture, so you can SEE if it worked. judge=true also asks a vision model for a score and problems (uses the Gemini/NVIDIA key from OpenDaemon). Look for: the character changing, the action not happening, extra hands, melting, the end falling apart. |
| auto_fixA | The self-check loop for one video: a vision model judges it, the settings are fixed by the kind of problem
(drift = less movement, no action = more movement, end melts = shorter, hands/melting = avoid list) and it renders
again, at most |
| suggest_settingsA | Settings to start with for a goal ("calm portrait", "lively dance", "quick test"), plus the best-scored videos made on this PC so far (scores come from review_video(judge=true) and auto_fix). |
| build_videoA | ONE call for a finished video. clips: video names in order, or {"video", "start", "end", "speed"} to cut/speed one. transition: cut | fade | crossfade. shape: "9:16" (Shorts/Reels/TikTok) | "1:1" | "4:5" | "16:9". voice: the words to say (or {"text", "voice": andrew|emma|ava|brian|es|pt..., "engine": edge|local, "sample"} or {"audio": file}). captions: "auto" (timed to the voice) or your own text. music: a music file path (or {"file", "volume"}); it gets quieter under the voice. title: text for a card at the start. end_card: text for the last card (true = "Follow for more"). export: shorts | reels | tiktok | facebook | youtube | x | square (right size and format to post). Only the final video stays in the gallery. |
| join_clipsB | Clips into one video, in order. transition: cut | fade (through black) | crossfade; seconds = its length. |
| trim_clipB | Keeps only start..end seconds. Good for cutting the end of a clip where it starts to melt. |
| extend_clipB | GPU. Makes a video longer without it falling apart: the last frame becomes the start of a new clip (same model and settings, your prompt for what happens next), then both are joined. Takes a render's time. |
| change_speedC | 0.5 = half speed (slow motion), 2 = twice as fast. 0.1 to 8. |
| reverse_clipC | Plays the clip backwards. |
| loop_clipC | Repeats the clip (2-20 times). |
| boomerangC | Forwards then backwards, like a boomerang. times = how many back-and-forths. |
| smooth_motionB | Smoother motion: makes in-between frames, 24 -> 30/48/50/60 fps (on the processor, about 2 s per second of video). |
| upscale_clipA | Bigger video. method "simple": resize to |
| crop_zoomA | A slow camera move on a finished clip. direction: in | out | left | right | up | down | none (a still crop). zoom 1.01-4. x, y (0-1) = the point to zoom on (0.5, 0.5 = the middle). |
| color_adjustC | brightness -1..1 (0 = same), contrast 0.5..2 (1 = same), saturation 0..3 (1 = same), warmth -1 cool .. 1 warm. |
| fadeA | Fades from black at the start and to black at the end (seconds; 0 = none). Sound fades too. |
| add_musicA | Background music (a file path), looped or cut to the video's length, faded in and out. If the video has a voice, the music gets quieter while it speaks (duck). |
| make_voiceA | A voice line from text, as an audio file for add_voice. engine "edge" (default): free online voices, fast,
with word timings for auto captions; voice: andrew, emma, ava, brian, es, es-f, pt, pt-f, or any edge voice name.
engine "local": Chatterbox on this PC, offline, CLONES the voice in |
| add_voiceB | Puts a voice line (from make_voice, or any sound file) on a clip, starting at |
| lipsyncB | Moves a real human face's lips to a voice (LatentSync). Needs a graphics card with 8 GB or more. |
| add_captionsA | Burned-in captions. text "auto" = timed to the voice line put on with add_voice (edge voices). Or your own text, spread over the clip ("|" forces a break). style: bold | impact | clean | box. position: bottom | lower | center | top. highlight = yellow text. |
| add_textC | A title or any text on the video, from |
| add_logoC | A logo or watermark (a picture file, PNG with transparency is best). position: top-right | top-left | bottom-right | bottom-left | center. scale = its width as part of the video's width. |
| title_cardB | A card at the START: text and/or a picture on a dark background, faded into the video. |
| end_cardC | A card at the END ("Follow for more", your logo...), faded in after the video. |
| reformatA | Changes the shape: 9:16 (Shorts, Reels, TikTok) | 1:1 | 4:5 | 16:9. mode: blur (blurred copy behind, nothing is cut off) | crop (fills the frame, cuts the sides) | pad (black bars). |
| exportB | Ready to post: the platform's shape, H.264 + AAC, file size under the limit. platform: shorts | reels | tiktok | facebook (10 MB) | youtube | x | square. max_mb overrides the size limit. |
| thumbnailA | A still picture (JPG) from the video at |
| make_gifD | An animated GIF of the video. |
| extract_audioA | Saves the video's sound as an .m4a file. |
| remove_audioC | The same video without sound. |
| list_workflowsB | Your own ComfyUI workflows (API-format .json files in the workflows folder) and the inputs each one lets you change (prompt, image, seed, steps... found automatically, or nodes titled "$name" in ComfyUI). |
| run_workflowA | Runs one of your workflows. inputs = {"prompt": "...", "image": "C:/pics/cat.png", "steps": 20} using the names from list_workflows; a file path for image/video/audio is copied into ComfyUI for you. A new random seed each run unless you pass seed or keep_seed=true. Checks the workflow first and refuses if it would fail (force=true runs anyway). Returns a job id: wait with job_status; the saved files come back in "files". |
| check_workflowA | Before a slow run: finds missing node types (and which node pack has them), missing model files and wrong values in a saved workflow (name) or one you give (workflow, API format). Also lists its changeable inputs. |
| install_custom_nodeA | Installs a ComfyUI node pack: a GitHub link, a pack name, or a node type from check_workflow. Node packs are programs from other people, so ASK THE USER FIRST, then call again with user_confirmed=true. ComfyUI needs a restart after (restart_comfyui). |
| save_workflowB | Keeps a workflow (ComfyUI API format) under a name, so run_workflow can use it. Returns its inputs. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
| prompt-guide | How to use this server and write prompts that work. |
TDQS
Scored across 67 tools
Tools are grouped into clear domains (render, queue/system, model management, editing, workflows, audio) and most have distinct, well-explained purposes. The main overlaps are intentional convenience layers: render vs rerender vs render_batch vs render_variations, and build_video vs the individual join_clips/add_voice/add_captions/export tools, which could make an agent hesitate about which path to take. add_text vs title_card vs add_captions vs add_logo are also similar enough to require reading descriptions carefully.
The dominant pattern is verb_noun (render_batch, delete_video, list_models, add_voice, check_workflow), applied consistently across most of the surface. Some noun-only outliers exist (queue, estimate, render, thumbnail, boomerang, export, fade, title_card, job_status), but they are readable and each denotes a single unambiguous operation.
67 tools is heavy for a single server, even one covering a full video pipeline from model download to platform export. Several tools could be folded into aggregates that already exist (build_video already combines join/caption/voice/export), so the set feels broader than strictly necessary. The breadth is somewhat justified by the domain, but it is at the upper edge of what an agent can navigate comfortably.
Coverage is impressive: model install/check, rendering, batching, queue control, diagnostics, editing, audio/voice, captions, workflows, and platform export are all present with no obvious dead ends for the core workflow. Minor gaps remain, such as deleting or renaming saved workflows, and job history beyond the last 10 entries.