Skip to main content
Glama
sena-labs

Replicate-MCP-Server

by sena-labs
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Hybrid

Lipsync / talking avatar with Replicate

replicate_lipsync

Animate a portrait image to speak from a text script or driving audio, producing an MP4 video for avatar narration or audio-driven lipsync.

Instructions

Animate a portrait image to speak — either from a text script (model does TTS + lipsync) or from a driving audio file. Produces an MP4 video.

DISPLAY REQUIREMENT — after this tool returns successfully, include the URL(s) so the user can open the video. URLs expire in ~24h.

Args:

  • image_url (URL): Portrait or face image to animate. Use replicate_upload_file for local files.

  • text (string, optional): Script for the avatar to speak. Used by video-avatar (maps to voice_script). At least one of text or audio_url is required.

  • audio_url (URL, optional): Driving audio for lipsync. Required for sadtalker; optional override for video-avatar. At least one of text or audio_url is required.

  • model (string, default "video-avatar"): Curated key (video-avatar, sadtalker) or "owner/name[:version]".

  • extra_input (object, optional): Model-specific extras (e.g. {voice_prompt: "speak slowly"} for video-avatar).

  • download (boolean, default true): Download the MP4 locally.

  • timeout_ms: Default 300000.

Returns: PredictionResult. local_paths contain .mp4 files.

Examples:

  • image_url="<portrait.jpg>", text="Hello! Welcome to our product demo." → video-avatar (TTS + lipsync)

  • image_url="<face.jpg>", audio_url="<speech.wav>", model="sadtalker" → audio-driven lipsync

Input Schema

NameRequiredDescriptionDefault
textNoText script for the avatar to speak. Required for models that do TTS+lipsync (video-avatar). Ignored when audio_url is provided.
modelNoLipsync model. Curated: video-avatar, sadtalker. Or "owner/name".video-avatar
downloadNo
audio_urlNoURL of the driving audio. Required for audio-only lipsync models (sadtalker). Optional override when model can do TTS.
image_urlYesURL of the portrait or face image to animate. Use replicate_upload_file for local files.
timeout_msNoMax ms to wait for the prediction. If exceeded, returns the prediction ID so you can poll via replicate_get_prediction. Default: 300000 (5min).
extra_inputNoAdditional model-specific inputs.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv3.3.1
    • addedInput schema / properties / extra_input / propertyNames
      Added value: +{
+  "type": "string"
+}
  2. First observedv3.2.3

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the safe-but-mutating, open-world profile; the description adds real value on top: URL expiry (~24h), a display requirement for the returned links, local download by default, and the timeout fallback that returns a prediction ID for polling via replicate_get_prediction. It stops short of describing GPU cost or queue/wait behavior, but the operational traits that matter are covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose, then a display requirement, then args and examples — a sensible order with no filler sentences. The Args block largely restates schema descriptions, which is mild duplication, but it keeps the definition self-contained for an agent that reads only the prose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter tool with a nested extra_input and no output schema, the description supplies the return shape (PredictionResult, local_paths contain .mp4), the failure/poll path, and the artifacts' lifetime. Nothing an agent needs in order to call and use the result is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 86%, so baseline is 3, but the description goes beyond it: it documents the text→voice_script mapping for video-avatar, the sadtalker audio requirement, the curated-vs-owner/name model form, and gives an example of a valid extra_input key. Only timeout_ms and download are restated rather than enriched.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource ('Animate a portrait image to speak') and names the two operating modes (text script vs. driving audio) plus the MP4 output. An agent can distinguish this from replicate_generate_video or replicate_generate_speech without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the either/or constraint ('At least one of text or audio_url is required'), which model needs audio (sadtalker), and points to replicate_upload_file for local files. The two examples map concrete argument combinations to concrete model behavior.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools