smart_generate
Generate responses to prompts by routing to a fleet host that already has the model in memory, skipping cold-load latency. Falls back to installed-but-unloaded hosts and errors if no host has the model.
Instructions
Generate a response for model, routed to whichever fleet host already has it loaded
in memory (skips cold-load latency). Falls back to any host that has it installed but not
loaded, and errors if no host has the model at all.
Args:
model: model name (e.g. "qwen3:32b").
prompt: the prompt text to send.
timeout_sec: max seconds to wait for the generate call (default 120).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| model | Yes | ||
| prompt | Yes | ||
| timeout_sec | No |