Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| OPENAI_API_KEY | No | OpenAI API key (required for OpenAI provider) | |
| ANTHROPIC_API_KEY | No | Anthropic API key (required for Claude provider) | |
| GOOGLE_CLOUD_PROJECT | No | GCP project ID for Vertex AI (required for Gemini provider) |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| describe | Get an AI-generated description of an image. Supports multiple providers (Gemini, OpenAI, Claude). |
| detect | Detect objects in an image and return bounding boxes. Uses Gemini for native bounding box support. Coordinates are normalized 0-1000 as [ymin, xmin, ymax, xmax]. |
| describe_region | Crop an image to a bounding box and describe that region in detail. Use this after detect() to zoom in on specific objects. |
| analyze_colors | Extract dominant colors from an image region using K-Means clustering in LAB color space. Returns colors sorted by frequency with human-readable names from color.pizza. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |