Skip to main content
Glama
KieranK07

claudeyes

by KieranK07
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

claudeyes

A macOS perception daemon for Claude Code that only tells the agent about screen changes it did not cause.

       t  what happened                                 obs  self  cons  world  verdict
    0.00  agent moves the cursor                         66    66     0      0  quiet
    0.62  button depresses, toolbar spinner             108    12    96      0  consequence of my action
    1.30  page repaints after the click                1392    95  1297      0  consequence of my action
 *  2.40  *** Slack message arrives ***                  48     0     0     48  WAKE  <- 1 region(s)
    3.00  agent scrolls                                1392  1392     0      0  quiet
 *  5.60  *** tests fail on their own ***               136     0     0    136  WAKE  <- 1 region(s)
 *  6.70  *** popup during the click's wide window ***   48     0     0     48  WAKE  <- 1 region(s)
  13 frames -> 3 wake-ups

Excerpt of tools/demo.py output on a synthetic trace.

An agent is the loudest thing on its own screen, so naive change detection wakes it on its own echo. claudeyes borrows the efference copy from motor neuroscience: every action registers a predicted screen delta, the capture loop reports the observed delta, and only the residual can wake the agent. Each changed region is labelled SELF (suppressed), CONSEQUENCE (downstream of an action, used for verification) or WORLD (the interrupt). See docs/design.md for the full model.

Try it without a Mac

pip install -e .    # installs numpy
python3 tools/demo.py

Runs a synthetic 13-frame trace (cursor moves, hover, click, repaint, scroll, typing) with three outside events dropped in. A correct run wakes on exactly those three.

Related MCP server: screen-memory-mcp

Run it for real

cd capture && swift build -c release        # macOS 13+, Xcode toolchain
cd .. && ./run.sh                           # capture | daemon -> ~/.claudeyes/events.db
python3 -m claudeyes.install --apply --editor "Code"
python3 -m claudeyes.doctor

Grant Screen Recording permission when asked, or ScreenCaptureKit returns empty frames without erroring. install merges the PreToolUse/PostToolUse hooks into ~/.claude/settings.json (with a backup) and prints the claude mcp add command to run. doctor checks every link in the chain, since most failures here are silent.

MCP tools

tool

for

what_changed(since_seconds)

Ambient context: what happened that the agent did not cause.

watch_for(timeout_seconds, app)

Blocks until the world does something. Replaces sleep-and-screenshot while waiting on a build.

perception_status()

Whether it is watching, and whether the suppression rate is healthy.

Layout

path

what

capture/

Swift ScreenCaptureKit shim: dirty rects plus owning app, one JSON line per frame.

claudeyes/actionbus.py

Predicted-delta envelopes.

claudeyes/residual.py

Observed minus predicted, and the three-way attribution.

claudeyes/daemon.py

Capture JSONL in, events to an append-only SQLite log out.

claudeyes/mcp/

MCP server.

hooks/

Claude Code tool hook.

bridge/

Unrelated: a small allowlisted stdio MCP shell for running swift build from a cloud session.

Status

Prototype. The model, event log, MCP server, hooks, installer and doctor work, and 12 unit tests pass (python3 -m unittest discover -s tests). The Swift shim compiles but has not yet run a real session with Screen Recording granted, so frame rate and dirty-rect fidelity are unproven on a live desktop.

License

MIT. See LICENSE.

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers