Ask ONE short English question about what a window on this computer shows; get short JSON back (windows_seen, status, summary, confidence, details, model_used, duration_seconds, memory_mb). details is filled by code: project, session, branch, next_step, failures, files, background_tasks when found. Read-only: never clicks or types. Pass app when known. For multi-part checks call step by step (status first, then details). On error, follow error.suggestion. client and session_id are OPTIONAL. Without them every call is independent: the full screen is re-read, no 'what changed' is possible and no file is written. With them the server keeps context between calls (more accurate, shorter answers, plus changed_since_last). Reuse the same pair for one task; sending a new session_id replaces the old one for that client. keep_warm (optional bool) keeps the model loaded 10 min instead of 3 when you will ask again soon. Call release_model() when your task is done.

Heartbeat: checks that this bridge works end to end. Confirms the server is up, makes sure the local Ollama is reachable (auto-starts it if needed) and asks the model for 'pong'. No screen reading. Returns {ok, server, ollama, model_reply_ok, model, seconds, cold, error?}. The first ping may take 30-50 s if the model is cold (not loaded); later pings take a few seconds.

Unload the local model now and return freed memory_mb. Call it at the end of your task (after your last look_at_screen) so the RAM is free again. Safe to call when nothing is loaded.

Check the backend before/after errors: Ollama reachable, model pulled and loaded, memory, free RAM, current keep_alive policy, agent-cu installed. Call this first if look_at_screen returns OLLAMA_UNREACHABLE or MODEL_NOT_FOUND.