Smoke-test a running server
smoke_testMeasure response latency and generation speed of a running llama-server by sending batch chat prompts through its OpenAI-compatible API.
Instructions
Send a batch of chat prompts to a running llama-server over its OpenAI-compatible API and report each response with latency and generation speed (tokens/second as measured by llama.cpp).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| port | Yes | Port of a running llama-server. | |
| prompts | No | Prompts to send. Defaults to three short sanity prompts. | |
| maxTokens | No | max_tokens per completion. | |
| timeoutMs | No | Per-request timeout in milliseconds. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| port | Yes | ||
| prompts | Yes | ||
| results | Yes | ||
| meanLatencyMs | Yes | ||
| meanTokPerSec | Yes |