llm_generate
Generate text with an open-source model hosted on Groq, on the server's key — no API key needed. Models: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b (default "openai/gpt-oss-20b"). HuggingFace models are not served on the server key. A reasoning model that spends max_tokens thinking is retried once at 4096. Rate-limited.
Example tool call
This call uses example arguments supplied in the tool’s input schema.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"prompt": "Reply with the single word OK."
},
"name": "llm_generate"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | Model ID (default: "openai/gpt-oss-20b"). Server-keyed whitelist only — Groq: openai/gpt-oss-20b, openai/gpt-oss-120b, qwen/qwen3.8-27b. Other ids from list_llm_models, HuggingFace ones included, are BYOK-only and will be rejected. | |
| prompt | Yes | The user prompt / instruction to send to the model | |
| system | No | Optional system prompt to set context or persona | |
| max_tokens | No | Maximum tokens to generate (default: 2048, max: 4096) | |
| temperature | No | Sampling temperature 0.0–1.5 (default: 0.7) |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | ||
| usage | No | ||
| content | No | ||
| provider | No | ||
| latency_ms | No |