post-controlnet-ip-adapter-inferences
Generate images using ControlNet for structural guidance and IP-Adapter for style or character reference. Combines pose, depth, or edge control with visual style transfer.
Instructions
Trigger a new image generation in ControlNet + IpAdapter mode. The control image is used to guide the generation; it can be a pose, canny map, or similar. The second reference image is used as an IPAdapter to guide the generation process.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| originalAssets | No | If set to true, returns the original asset without transformation | |
| dryRun | No | ||
| ipAdapterImageIds | No | ||
| controlEnd | No | Specifies how long the ControlNet guidance should be applied during the inference process. Only available for Flux.1-dev based models. The value represents the percentage of total inference steps where the ControlNet guidance is active. For example: - 1.0: ControlNet guidance is applied during all inference steps - 0.5: ControlNet guidance is only applied during the first half of inference steps Default values: - 0.5 for Canny modality - 0.6 for all other modalities | |
| modality | Yes | The modality associated with the control image used for the generation: it can either be an object with a combination of maximum For models of SD1.5 family: - up to 3 modalities from `canny`, `pose`, `depth`, `lines`, `seg`, `scribble`, `lineart`, `normal-map`, `illusion` - or one of the following presets: `character`, `landscape`, `city`, `interior`. For models of the SDXL family: - up to 3 modalities from `canny`, `pose`, `depth`, `seg`, `illusion`, `scribble` - or one of the following presets: `character`, `landscape`. For models of the FLUX schnell or dev families: - one modality from: `canny`, `tile`, `depth`, `blur`, `pose`, `gray`, `low-quality` Optionally, you can associate a value to these modalities or presets. The value must be within `]0.0, 1.0]`. Examples: - `canny` - `depth:0.5,pose:1.0` - `canny:0.5,depth:0.5,lines:0.3` - `landscape` - `character:0.5` - `illusion:1` Note: if you use a value that is not supported by the model family, this will result in an error. | |
| seed | No | Used to reproduce previous results. Default: randomly generated number. | |
| modelId | Yes | The model id to use for the inference | |
| ipAdapterType | No | The type of IP Adapter model to use. Must be one of [`style`, `character`], default to `style`` | |
| ipAdapterImage | No | Deprecated for type txt2img-ip-adapter and img2img-ip-adapter, use `ipAdapterImages` instead. The IpAdapter image as a data url. Will be ignored if the `ipAdapterImages` parameter is provided. | |
| scheduler | No | The scheduler to use to override the default configured for the model. See detailed documentation for more details. | |
| disableModalityDetection | No | If false, the process uses the given image to detect the modality. If true (default), the process will not try to detect the modality of the given image. For example: with `pose` modality and `false` value, the process will detect the pose of people in the given image with `depth` modality and `false` value, the process will detect the depth of the given image with `scribble` modality and `true`value, the process will use the given image as a scribble ⚠️ For models of the FLUX schnell or dev families, this parameter is ignored. The modality detection is always disabled. ⚠️ | |
| ipAdapterImages | No | ||
| imageParentId | No | Specifies the parent asset Id for the image when provided as a dataurl. | |
| baseModelId | No | The base model to use for the inference. Only Flux LoRA models can use this parameter. Allowed values are available in the model's attribute: `compliantModelIds` | |
| controlStart | No | Specifies the starting point of the ControlNet guidance during the inference process. Only available for Flux.1-dev based models. The value represents the percentage of total inference steps where the ControlNet guidance starts. For example: - 0.0: ControlNet guidance starts at the beginning of the inference steps - 0.5: ControlNet guidance starts at the middle of the inference steps | |
| height | No | The height of the generated images, must be a 8 multiple (within [64, 2048], default: 512) If model.type is `sd-xl`, `sd-xl-lora`, `sd-xl-composition` the height must be within [512, 2048] If model.type is `sd-1_5`, the height must be within [64, 1024] If model.type is `flux.1.1-pro-ultra`, you can use the aspectRatio parameter instead | |
| imageHide | No | Toggles the hidden status of the image when provided as a dataurl. | |
| controlImageId | No | The controlnet input image as an AssetId. Will be ignored if the `controlnet` parameter is provided | |
| image | No | The input image as a data URL (example: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVQYV2NgYAAAAAMAAWgmWQ0AAAAASUVORK5CYII=") or the asset ID (example: "asset_GTrL3mq4SXWyMxkOHRxlpw") | |
| imageId | No | Deprecated: The input image as an AssetId. Prefer to use image with the asset ID instead. | |
| ipAdapterImageId | No | Deprecated for type txt2img-ip-adapter and img2img-ip-adapter, use `ipAdapterImageIds` instead. The IpAdapter image as an AssetId. Cannot be set if `ipAdapterImage` is provided. Will be ignored if the `ipAdapterImageIds` parameter is provided. | |
| ipAdapterScale | No | Deprecated for type txt2img-ip-adapter and img2img-ip-adapter, use `ipAdapterScales` instead. IpAdapter scale factor (within [0.0, 1.0], default: 0.9). Will be ignored if the `ipAdapterScales` parameter is provided | |
| modelEpoch | No | The epoch of the model to use for the inference. Only available for Flux Lora Trained models. | |
| hideResults | No | If set, generated assets will be hidden and not returned in the list of images of the inference or when listing assets (default: false) | |
| ipAdapterScales | No | ||
| negativePrompt | No | The prompt not to guide the image generation, ignored when guidance < 1 (example: "((ugly face))") For Flux based model (not Fast-Flux): requires negativePromptStrength > 0 and active only for inference types txt2img / img2img / controlnet. | |
| intermediateImages | No | Enable or disable the intermediate images generation (default: false) | |
| concepts | No | ||
| controlImage | No | The controlnet input image as a data URL (example: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVQYV2NgYAAAAAMAAWgmWQ0AAAAASUVORK5CYII=") | |
| guidance | No | Controls how closely the generated image follows the prompt. Higher values result in stronger adherence to the prompt. Default and allowed values depend on the model type: - For Flux dev models, the default is 3.5 and allowed values are within [0, 10] - For Flux pro models, the default is 3 and allowed values are within [2, 5] - For SDXL models, the default is 6 and allowed values are within [0, 20] - For SD1.5 models, the default is 7.5 and allowed values are within [0, 20] | |
| numInferenceSteps | No | The number of denoising steps for each image generation (within [1, 150], default: 30) | |
| numSamples | No | The number of images to generate (within [1, 128], default: 4) | |
| width | No | The width of the generated images, must be a 8 multiple (within [64, 2048], default: 512) If model.type is `sd-xl`, `sd-xl-lora`, `sd-xl-composition` the width must be within [512, 2048] If model.type is `sd-1_5`, the width must be within [64, 1024] If model.type is `flux.1.1-pro-ultra`, you can use the aspectRatio parameter instead | |
| negativePromptStrength | No | Only applicable for flux-dev based models for `txt2img`, `img2img`, and `controlnet` inference types. Controls the influence of the negative prompt. Default 0 means the negative prompt has no effect. Higher values increase negative prompt influence. Must be > 0 if negativePrompt is provided. | |
| prompt | Yes | Full text prompt including the model placeholder. (example: "an illustration of phoenix in a fantasy world, flying over a mountain, 8k, bokeh effect") |