Cellar provides a structured, multi-source computer use platform for AI agents to perceive, interact with, and automate tasks across desktop and browser environments through 4 composable tools.

cel_see — Screen Observation

• Read structured UI elements with types, labels, bounds, and confidence scores

• Capture screenshots, list windows/monitors, get cursor position

• Inspect elements by ID or coordinates, create resilient element references

• Retrieve browser (CDP) page content, debug targets

• Wait for elements to appear (with timeout) or screen to stabilize (idle detection)

• Watch for 18 event types: tree changes, network idle, focus/value/window/menu/selection changes, and more

cel_act — Screen Interaction

• Click (single, right, double), move mouse by coordinates or element reference

• Type text, press keys, execute keyboard combos with modifiers

• Perform native accessibility API actions (ax_action) and set values directly on form fields/checkboxes

• Write and read spreadsheet cells deterministically (Numbers app)

• Scroll, drag & drop, execute JavaScript in browser via CDP

• Batch up to 4 sequential actions in one call

cel_think — Planning, Memory & Autonomous Execution

• Delegate a natural language goal for fully autonomous see→plan→act execution (run_goal)

• Generate LLM-powered step-by-step plans (with optional vision/screenshot)

• Store and search persistent knowledge via FTS5 full-text search

• Get/set per-workflow working memory (scratchpad)

• Record observations with priority levels, track runs with confidence scores

• Send text or image+text directly to the LLM (passthrough)

cel_perceive — Always-On Continuous Perception (Cortex)

• Start a continuously-updated mental model of the screen with a goal

• Get instant snapshots kept warm by background event streams

• Report actions and receive verification (screen diff + settlement detection)

• Checkpoint completed work and reset action history between task phases

• Check Cortex health: confidence score, uptime, cycle count, element stability

• Stop the Cortex and receive a session summary

Additional Capabilities

• Multi-source fusion: Combines accessibility tree, CDP, vision, network, and app-specific adapters (Apple Calendar, Mail, Numbers; planned for Excel, Slack, Figma) into structured JSON with confidence scoring

• Agent & model agnostic: Works with LangGraph, Mastra, Claude Code, Cursor, n8n and LLMs including OpenAI, Anthropic, Gemini, and local Ollama

• Hybrid runtime: Prioritizes structured sources (accessibility, native APIs) and escalates to vision only when needed; prevents infinite loops via impossible-action detection

• CLI: Commands for init, setup, context viewing, screen capture, actions, MCP server, workflows, and training mode

• Platform support: Full macOS support (AXUIElement, Cortex, MCP); Linux via AT-SPI2; Windows UI Automation planned

Provides a first-party adapter for SAP GUI, enabling structured interaction with SAP applications through accessibility and input injection.