local-agent-senses
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| OLLAMA_EXE | No | Full path to the ollama executable | |
| OLLAMA_HOST | No | Ollama address | http://localhost:11434 |
| VISION_FONT | No | Font used by the health-check image | |
| VISION_YTDLP | No | Full path to yt-dlp | |
| OLLAMA_MODELS | No | Model dir (disk check) | ~/.ollama/models |
| VISION_CONFIG | No | Path to the configuration file | |
| VISION_FFMPEG | No | Full path to ffmpeg | |
| VISION_API_KEY | No | API key for the endpoint | |
| VISION_API_BASE | No | OpenAI-compatible endpoint, e.g. http://localhost:11434/v1 | |
| VISION_MCP_CACHE | No | Persist MCP results to disk (hits across server restarts) | true |
| VISION_KEEP_ALIVE | No | Model residency (0 = unload immediately) | 10m |
| VISION_MAX_TOKENS | No | Output budget for transcription mode | 98304 |
| VISION_SPEECH_ENV | No | Conda env name for speech | funasr |
| VISION_TEXT_MODEL | No | OCR / transcription model | haervwe/GLM-4.6V-Flash-9B |
| VISION_QUICK_MODEL | No | Fast image/video model | qwen3.5:4b |
| VISION_QUICK_THINK | No | Enable Qwen thinking mode | false |
| VISION_MAX_IMAGE_MB | No | Local/stdin image size cap (MB) | 20 |
| VISION_BUDGET_PIXELS | No | Pixel budget; frames drop automatically | 20000000 |
| VISION_MCP_CACHE_DIR | No | MCP disk-cache directory | |
| VISION_SPEECH_PYTHON | No | Full path to the speech interpreter | |
| VISION_MAX_DURATION_H | No | Video/audio duration cap (hours); longer inputs rejected; 0 disables | 6 |
| VISION_MAX_DOWNLOAD_MB | No | RAM cap for unknown-type URL buffering | 500 |
| VISION_SINGLE_RESIDENT | No | Keep only one model resident (VRAM) | true |
| VISION_QUICK_MAX_TOKENS | No | Output budget for quick mode | 16384 |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| describe_imageA | Describe or understand one or more images with a local vision model (scene, objects, colors, UI, etc.). Returns JSON: {"text": ..., "mode": "image", "media": [...]}. |
| transcribeA | Verbatim transcription of text in images/documents/screenshots/video subtitles (no summary, judgment, or interpretation). Returns JSON with text and, for videos, frame timestamps. |
| analyze_videoA | Analyze a video with a local vision model: scenes, actions, time-window deep reading, contact sheets, etc. Returns JSON: {"text": ..., "mode": ..., "frames": [{"t": ..., "w": ..., "h": ...}], "duration": ...}. |
| transcribe_audioA | Speech-to-text (FunASR SenseVoice, with timestamps; embedded subtitle tracks are preferred when available). Returns JSON: {"text": ..., "source": "asr"|"subtitle"}. |
| vision_statusA | Show the vision backend status (models, backend type, ffmpeg, speech environment, GPU, watchdog). |
| vision_checkA | Run the environment health check (image reading, transcription, video sampling; can be slow). |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 6 tools
Each tool targets a distinct medium or task: image understanding, text extraction, video analysis, audio transcription, and system diagnostics. Minor overlap exists between describe_image and analyze_video for video content, but descriptions clearly separate by media type.
Tool names mix conventions: three use verb_noun (describe_image, analyze_video, transcribe_audio), one is a bare verb (transcribe), and two use a vision_ prefix (vision_status, vision_check). This inconsistency makes the set feel less predictable.
Six tools is well-scoped for a multimodal sensing server. Each tool has a clear purpose and none feel redundant or unnecessary.
The set covers the core sensing lifecycle: image description, video analysis, audio/speech transcription, and health checks. Minor gaps like general audio analysis or dedicated frame extraction are absent but not essential for the stated purpose.