Skip to main content
Glama

Clippy Vision

Clippy Vision watches your work on this computer, stores a searchable activity history locally, and lets you ask about it through a connected MCP client such as Cursor, Claude Desktop, or VS Code. The desktop app captures, summarizes, and shows the timeline.

Platform License Models MCP All Contributors Open Source Helpers

Ask in a connected MCP client, for example:

  • "What was I working on before lunch?"

  • "Which hotels did I look at yesterday?"

  • "What was the error I encountered in my terminal?"


Download

Windows is tested for 2.0. macOS validation is pending, so macOS 2.0 installers are not included yet. macOS downloads below are the previous release (v1.3.1). Linux is experimental and X11 only; an unsigned x86_64 AppImage is built in CI (see docs/linux.md). The README describes current main; some installer features differ from 1.3.1.

The installers are unsigned: on Windows choose More info → Run anyway, and on macOS right-click → Open the first time. First run needs internet for dependencies and for the chat model you pick in setup (for example qwen3:8b is ~4.7 GB).

Trying Clippy? Share feedback (5 min).

Requirements

Minimum

Recommended

OS

Windows 10 / 11 (64-bit), macOS 12+, or Linux (Ubuntu 22.04+, experimental, X11 only)

Windows 11 or a current macOS

System RAM

8 GB

16 GB

Graphics memory

Not required. Integrated graphics or Apple unified memory is enough

4 GB+ dedicated on Windows, or 16 GB unified memory on Apple silicon

Free disk

8 GB

10 GB+

Python runtime

System Python 3.11+ on PATH (not bundled). Windows auto-install uses 3.11; CI uses 3.12

Python 3.12

Python is not bundled in the Windows installer or Linux AppImage. requirements.txt does not pin a Python version. The packaged app runs CLIPPY_PYTHON if set, otherwise python3 (Linux/macOS) or python (Windows) from PATH (electron-ui/electron/lib/paths.js). CI unit tests and the release dry-run install Python 3.12. The setup wizard parses python --version and requires Python 3.11+.

When Python is missing, the setup wizard (electron-ui/electron/lib/setup.js) does this:

  • It parses --version on that interpreter and requires Python 3.11+. Preflight text says "Clippy needs Python 3.11+ on PATH." It then re-resolves known install locations before declaring failure.

  • Windows: tries winget install Python.Python.3.11 --silent. If winget is missing or the install cannot be found on PATH, it shows python.org/downloads and probes %LOCALAPPDATA%\Programs\Python\Python311\python.exe.

  • macOS: tries Homebrew brew install python when Homebrew is present. If Homebrew is missing, it shows a manual-install message with the python.org link.

  • Linux: stays a manual install. The wizard does not try Homebrew or apt; it shows a python.org link.

Advanced: point Clippy at a specific interpreter before launch:

$env:CLIPPY_PYTHON = "C:\path\to\python.exe"
export CLIPPY_PYTHON=/path/to/venv/bin/python3

Linux support is experimental and X11 only. Wayland is not supported. On X11, screenshots use mss (optional crop via xdotool). Window titles use xdotool, clipboard uses wl-paste/xclip/xsel. AT-SPI accessibility text is implemented; it is covered by CI unit tests, not a real desktop. An unsigned Linux AppImage is built in CI. Install and run steps: docs/linux.md.

The setup wizard checks these numbers before installing. Below minimum → setup is blocked. Between minimum and recommended → you can continue with a warning that summaries may feel slower. Homebrew, dependency, and permission troubleshooting: docs/usage.md. Lower-spec machines: CONTRIBUTING.md.


Related MCP server: macos-screen-mcp

Quick Start

  1. Download and install Clippy.

  2. Complete the setup wizard and grant required permissions.

  3. Leave Clippy running to build your activity history.

  4. Open Settings → Connect apps, connect your agent, and ask about your activity.

More detail: docs/usage.md · docs/mcp.md · client guides: Claude Desktop · LM Studio
Build from source: CONTRIBUTING.md


Features

  • Searchable activity history across apps

  • Local summaries and long-term memory

  • Access through MCP-compatible agents

  • A timeline for reviewing and deleting activity

  • Capture and privacy controls


Privacy

  • Clippy captures foreground windows, clipboard, typing metrics, and screenshots.

  • Capture, summarization, and storage run locally on this computer.

  • Connected cloud agents can receive retrieved memory through MCP tool replies.

  • Connected clients have access to all exposed Clippy tools.

  • Redaction is best effort. Private-browsing detection is on by default: Chrome, Edge, and Brave on Windows and macOS; Firefox and Opera on Windows (and attempted but untested on macOS); Vivaldi on Windows. Vivaldi, Safari, and Linux are not covered. See the coverage table.

  • You can pause capture and delete stored activity from the tray or timeline.

Full guide (retention defaults and detailed behavior): docs/usage.md.


Documentation & Roadmap

Guide

Contents

docs/usage.md

Capture controls, privacy, accessibility, install troubleshooting

docs/linux.md

Experimental Linux install and run (X11 only; AppImage, Python, capture tools)

docs/mcp.md

Supported clients, Copy JSON, tool reference, access permissions

docs/mcp-clients/claude-desktop.md

Step-by-step Claude Desktop setup: config location, restart, verification

docs/mcp-clients/lm-studio.md

Step-by-step LM Studio setup: MCP configuration, local model, test query, privacy

docs/architecture.md

Tech stack, capture pipeline, summarization, distillation, schema

CONTRIBUTING.md

Build from source and contribution guide

PROJECT_VISION.md

Roadmap, principles, and known limitations


Contributing

See CONTRIBUTING.md for setup steps, building from source, and good first issues.


Contributors

Thanks to everyone who has shipped code, docs, design, or ideas.

All Contributors Contributors

Contributor

What they built

@protocorn💻 📖 🎨 🤔 🚧

Designed the core app: agent, vision pipeline, memory system, and the Electron desktop shell.

@rusetiq💻 📦

Brought Clippy Vision to macOS: native screen capture, permissions, and Apple Silicon + Intel packaging.

@cursoragent💻

See their commits →

@Afrin-banu-A💻

See their commits →

@ABarpanda💻

See their commits →

@vitorparras💻

See their commits →

@Furox-Art💻

See their commits →

@vaishn4vi💻

See their commits →

@adity982💻

See their commits →

@Draoui-Haroun💻

See their commits →

@KeuqS💻

See their commits →

@shaurya703💻

See their commits →

@cyforkk💻

Made errors readable: replaced bare HTTP status codes with real API error messages in chat.

@tadaokatzuki💻

See their commits →

@Buzepool💻

See their commits →

@BitsRevenge💻

See their commits →

@icn5381💻

See their commits →

How to get on this wall: CONTRIBUTING.md.


License

AGPL-3.0. See LICENSE. Releases already published through v1.3.1 stay under MIT. A commercial license for companies that cannot use AGPL is not written yet.

Available Tools

17 tools
activity_coverage_toolB

Hourly (or custom) event-count buckets so capture gaps are honest. Empty buckets mean no events that hour — often capture was off. start required (epoch or YYYY-MM-DD); end defaults to now.

Inputs: start (required), end, bucket_hours. Returns: result

app_time_summary_toolB

Approximate per-app foreground time from event gaps in a window. start/end: epoch or YYYY-MM-DD. Durations are approximate (gaps capped at 5 min).

Inputs: start (required), end. Returns: result

delete_note_toolA

Delete a note or memory fact the user wants forgotten. Use when the user says 'forget', 'delete', 'remove', or 'don't remember that'. Matches by substring — pass the key phrase or exact text from the note.

Inputs: note_text (required). Returns: result

fetch_cluster_toolA

Get all facts stored in a named memory cluster (with freshness annotations). Use after recall_memory_tool to get the full content of a specific topic. Pass the cluster label exactly as returned by recall_memory_tool.

Inputs: label (required). Returns: result

find_files_toolC

Search trusted workspace roots for a filename or substring.

Inputs: name (required), limit, under. Returns: result

get_screenshot_toolB

Fetch one screenshot by filename or unix timestamp. Returns redacted screen text. The image file path is not included. Screenshots use adaptive TTL (important frames kept longer, capped).

Inputs: filename, timestamp. Returns: result

list_screenshots_toolB

List screenshots (or OCR-backed stubs) in a time window. image_available=false means the JPEG expired; OCR preview may still be present.

Inputs: end, limit, start. Returns: result

list_sessions_toolA

Chronological deduped session list for a time window. Prefer this over search_sessions_tool for day/week timelines. start/end: epoch or YYYY-MM-DD.

Inputs: end, limit, start, offset. Returns: result

list_urls_toolB

Distinct URLs seen in active_url / OCR within a time window. pattern filters substring (e.g. 'greenhouse'). Deduped with counts.

Inputs: end, limit, start, pattern. Returns: result

list_workspace_roots_toolA

List folders Clippy may search with find_files_tool.

Inputs: no named parameters. Returns: result

recall_memory_toolA

List long-term memory clusters ranked by freshness score. Pass query to run semantic recall instead of listing. Clusters below min_freshness are hidden unless include_stale=true.

Inputs: query, include_stale, min_freshness. Returns: result

remember_turn_toolA

Store durable facts from the user's own words. Call after the user tells you something about themselves, their preferences, relationships, plans, or anything they would want recalled later. Pass only the user's message, not your reply. Skip for activity questions, file search, and one-off tasks. Secrets, passwords, and API keys are not stored.

Inputs: user_message (required). Returns: result

save_identity_toolA

Save a personal fact about the user. op='set' for scalar facts (name, location, job). op='add_items' with items=[] for adding to a list (hobbies, skills). op='override' only when the user explicitly corrects a previous fact. op='remove_items' with items=[] to remove from a list.

Inputs: op (required), field (required), items, value. Returns: result

save_note_toolB

Save a free-form note or reminder the user wants remembered.

Inputs: note (required). Returns: result

search_bounded_toolA

Keyword search with explicit time bounds (no LLM date guessing). start/end: epoch seconds or YYYY-MM-DD (local). table: 'events' or 'sessions'. Use when you know the calendar window and need pagination via limit/offset.

Inputs: question (required), end, limit, start, +2 more. Returns: result

search_events_toolA

Search individual events in the activity database. Use for: specific messages, OCR screen text, exact URLs, clipboard content, app usage, WhatsApp/email content, fine-grained timestamps, copy-paste history. Returns raw event rows with screen/OCR data. If the result says the info isn't there, call search_sessions_tool next. For exact date ranges prefer search_bounded_tool.

Inputs: question (required). Returns: result

search_sessions_toolA

Search session summaries in the activity database. Use for: broad time windows (yesterday, this week), daily/weekly overviews, what-did-I-work-on questions, project topics, task recaps. Returns paragraph summaries — NOT granular event detail. If the result says the info isn't there, call search_events_tool next. For exact date ranges prefer list_sessions_tool or search_bounded_tool.

Inputs: question (required). Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3.4/5.0

    Scored across 17 tools

    Disambiguation3/5

    Several tools have overlapping search purposes: search_sessions_tool, search_events_tool, search_bounded_tool, and list_sessions_tool all query the activity database, and descriptions only partially disambiguate (e.g. 'if result says not there, call the other one'). Memory tools recall_memory_tool, fetch_cluster_tool, save_identity_tool, remember_turn_tool, save_note_tool, and delete_note_tool overlap around memory storage/retrieval.

    Naming Consistency5/5

    All 17 tools follow a consistent snake_case verb_noun or noun_verb pattern (get_screenshot_tool, recall_memory_tool, find_files_tool, list_urls_tool), with the uniform '_tool' suffix making the set predictably uniform.

    Tool Count3/5

    17 tools is on the heavy side for a screen/memory assistant, and several pairs (search_sessions vs list_sessions, search_events vs search_bounded) could likely be collapsed without losing capability.

    Completeness4/5

    The surface covers capture (screenshots), memory storage/retrieval, note lifecycle (save/delete), file search, and rich activity querying with coverage/app-time/URL listing. Minor gaps like note editing/updating or explicit screenshot deletion exist but agents can work around them.

    Maintenance

    ActivityActive
    ResponsivenessLimited data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      GUI automation MCP server that enables AI agents to see and control the Windows desktop using a local Vision LLM (Ollama), supporting screenshot analysis, mouse/keyboard actions, and autonomous task execution.
      5
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      An MCP server that gives any AI assistant eyes and hands on your desktop — screenshots, clicking, typing, OCR, window management, accessibility-tree queries, workflow recording.
      5
      Apache 2.0