screen-context
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@screen-contextfind the error message I was looking at in the browser a moment ago"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
ScreenContext
ScreenContext records the foreground window, reads its text with the operating system's built-in OCR, keeps an encrypted local history, and exposes that history to AI assistants through the Model Context Protocol (MCP). Any MCP client can use it: Claude Code, Claude Desktop, Cursor, VS Code, Windsurf, Codex CLI, Gemini CLI and others.
Ask your assistant things like "find the error message I was looking at in the browser a moment ago" or "what was the spec page I read this morning?".
Platform | Status |
macOS 14+ | Supported (menu bar app + CLI) |
Windows 10/11 | Beta (control window + CLI). Not yet validated on a wide range of hardware. |
License: MIT
How it works
Three processes, each with a narrow job:
Capture (menu bar app on macOS, control window on Windows) captures only the foreground window at native resolution. Similar frames are skipped with a perceptual hash. Frames go to an encrypted spool.
Indexer runs OCR (Apple Vision on macOS,
Windows.Media.Ocron Windows), applies your exclusion policy, and stores text in an SQLCipher database with a trigram FTS5 index.MCP server (
screen-context serve) is started by your MCP client. It never imports capture code, only reads the database, and labels every result as untrusted observed data.
More detail: docs/ARCHITECTURE.md.
Related MCP server: ContextPulse
Requirements
Python 3.11 or later and uv
macOS 14 or later, or Windows 10/11 (beta)
To build the macOS
.appwith py2app, use a python.org or Homebrew Python. Some standalone Python builds fail in py2app becausezlibis built in.
Quick start (macOS)
git clone https://github.com/ikeikeikeda66/screen-context-agent.git
cd screen-context-agent
uv sync --locked --extra macos --extra encrypted --extra dev
.venv/bin/screen-context initinit creates ~/Library/Application Support/ScreenContext and stores a random encryption key in the macOS Keychain. If you lose the key, the history cannot be decrypted.
Start the indexer in one terminal:
.venv/bin/screen-context index --watchBuild and start the menu bar app. macOS grants Screen Recording permission to the app bundle, so capture runs from the app, not from the terminal:
# "-" makes an ad-hoc signature for local use. Use a Developer ID identity
# to keep the Screen Recording permission across rebuilds.
SCREEN_CONTEXT_SIGN_IDENTITY=- sh packaging/build_mac.sh
open dist/ScreenContext.appAllow ScreenContext in System Settings > Privacy & Security > Screen Recording, then open the app again. To start the indexer at login, see packaging/macos/README.md.
For a one-time test with the terminal's own permission: .venv/bin/screen-context capture --once.
Menu bar

The menu bar shows SC Rec while recording and SC Paused while paused.
Pause Capture / Resume Capture: use this while watching video or showing private content. The state is saved.
Capture Interval: 5 s, 15 s (default), 30 s, 1 min, 2 min or 5 min. The change applies without a restart. Unchanged screens are not saved again.
Language: System Default, English or 日本語. All menu text, dialogs, and generated material follow this setting.
Connect an MCP client
Print a ready-to-paste entry for your client:
.venv/bin/screen-context mcp-config --client claude-code # prints a `claude mcp add` command
.venv/bin/screen-context mcp-config --client claude-desktop
.venv/bin/screen-context mcp-config --client cursor
.venv/bin/screen-context mcp-config --client vscode
.venv/bin/screen-context mcp-config --client windsurf
.venv/bin/screen-context mcp-config --client codex # TOML for ~/.codex/config.toml
.venv/bin/screen-context mcp-config --client gemini
.venv/bin/screen-context mcp-config --client generic # plain mcpServers JSONThe client starts the server itself over stdio. Per-client instructions and the HTTP transport are in docs/MCP-CLIENTS.md.
Profiles and tools
Each server process runs with one fixed profile. A tool call cannot raise it.
Tool |
|
|
| yes | yes |
| yes | yes |
| yes | yes |
| yes | yes |
| yes | |
| yes | |
| yes | |
| yes | |
| yes | |
| yes | |
| yes | |
| yes |
standardis for coding assistants. It hides frames from IDEs and terminals (ide_appsin the policy), because the assistant already has the source.fullis for a personal agent that you trust with screenshots and timelines.The names
claude_codeandopenclawfrom earlier versions still work as aliases forstandardandfull.
Every response and record carries source=observed_screen and trust=untrusted. This is a label, not a defense against prompt injection: treat screen text as data, never as instructions.
Privacy and storage
Edit policy.json in the data folder. It is read again on every capture and every query, so a new exclusion also hides past records from search, activity summaries and images. An invalid policy makes processing fail; it never disables exclusions.
Key | Meaning |
| Bundle IDs (macOS) or process names (Windows) never captured. Defaults include password managers and video-call apps. |
| Frames whose OCR text shows these domains are dropped. Detection depends on the URL being visible and read correctly. |
| Regular expressions matched against window titles. |
| Hidden from the |
| Frames showing an assistant's own output. Proposals cannot use them as evidence. |
Spool files and preview images use AES-GCM. The database and full-text index use SQLCipher. The key lives in the OS credential store (Keychain or Windows Credential Manager), or in
SCREEN_CONTEXT_KEYfor headless use.The spool stops accepting frames at 100 files or 512 MB. Unprocessed frames older than 24 hours are deleted by
maintain.After 90 days, preview images and OCR bounding boxes are deleted. Searchable OCR text and daily rollups are kept.
audit.jsonlrecords the client, time, tool, a SHA-256 of the arguments, and the result count. Queries themselves are not stored.SCREEN_CONTEXT_PLAINTEXT=1is for development tests only. ScreenContext never falls back to plaintext on its own.
Configuration
Environment variable | Purpose |
| Data folder. Default: |
| 64 hex characters. Replaces the OS credential store (headless use). |
|
|
| Comma-separated OCR languages, for example |
| Client name written to the audit log. |
| Bearer token (32+ characters) for the HTTP transport. |
The language can also be set with screen-context language en|ja|system.
CLI reference
screen-context init create the data folder, key and database (also migrates)
screen-context index [--watch] OCR and store spooled frames
screen-context capture [--once] capture from the terminal (development)
screen-context pause | resume stop or restart new captures
screen-context status | health queue and worker state
screen-context maintain retention and daily rollups
screen-context serve [--profile standard|full] [--transport stdio|http] [--port 8765]
screen-context mcp-config [--client NAME] [--profile standard|full]
screen-context language [system|en|ja]
screen-context diary-material DATE [--budget 6000] [--lang en|ja]
screen-context proposal prepare|simulate|finish
screen-context export DATE | push DATEOptional: diary material and periodic proposals
diary-material DATEprints a compact, bounded Markdown summary of one day, for use as input to a diary or daily report prompt.proposal prepareis designed as a pre-run script for a scheduler (cron or an agent framework). It prints material only when there are new observations. Otherwise its last line is{"wakeAgent": false, ...}, so the scheduler can skip starting the agent. The agent registers a suggestion withsubmit_proposal; evidence must be a quote from a non-assistant frame, and the same conclusion is suppressed for 24 hours. Close the run withproposal finish --run-id ID --response-file FILE.
Optional: OpenViking export
export DATE writes a daily rollup JSON to exports/ (plaintext). push DATE sends it to a local OpenViking server at http://127.0.0.1:1933 (VIKING_API_KEY if needed). Nothing is pushed automatically. Exported data is not removed when you later add exclusions.
Windows (beta)
The Windows version has a control window (start, pause, stop, language, MCP client setup) and the same CLI. It is beta: it passes automated tests with simulated Windows APIs, but real-device coverage (DPI, multiple monitors, lock and resume, credential store) is still limited. Please report problems in Issues.
Setup, portable build, and the acceptance checklist: docs/WINDOWS.md.
Development
uv sync --locked --extra macos --extra encrypted --extra dev # use --extra windows on Windows
.venv/bin/python -m pytest -q
.venv/bin/python packaging/verify_bundle.py # after building the macOS appContributions are welcome. See CONTRIBUTING.md. Report security issues as described in SECURITY.md.
Available Tools
4 toolsget_context_aroundBRead-only
Get activity blocks before and after a Unix timestamp. Returns untrusted observed data.
| Name | Required | Description | Default |
|---|---|---|---|
| n | No | ||
| ts | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark readOnlyHint=true and destructiveHint=false, and the description adds the important caveat that the returned data is untrusted observed data. This goes beyond the annotations, though it doesn't disclose return shape, pagination, or invalid timestamp behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, no filler, with the core action front-loaded and the trust caveat appended. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema and a 0% schema coverage, so the description must carry more weight. It omits the semantics of the 'n' parameter and gives only a vague indication of the return type ('activity blocks'), leaving an agent under-informed for a correct call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It clarifies that 'ts' is a Unix timestamp, but 'n' is left unexplained; an agent cannot infer its meaning or bounds from the text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description states a specific verb ('Get'), resource ('activity blocks'), and temporal scope ('before and after a Unix timestamp'). It is clear on what the tool does, though it does not explicitly discuss sibling tools to distinguish from get_recent_activity or get_day_material.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a timestamp-centric use case but provides no explicit guidance on when to prefer this tool over search_screen_history, get_recent_activity, or get_day_material, nor any exclusions. This is adequate but relies on inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_day_materialARead-only
Get bounded screen observations for a specified local calendar day to draft a report or diary. Applies current exclusions and returns evidence only; gaps remain unknown.
| Name | Required | Description | Default |
|---|---|---|---|
| date | Yes | ||
| limit | No | ||
| cursor | No | ||
| purpose | No | report | |
| timezone | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare read-only/non-destructive behavior; the description adds that results are bounded, subject to current exclusions, and that gaps remain unknown. This is meaningful context beyond the structured annotations and sets accurate expectations about incompleteness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler. The primary action and scope are front-loaded, and the key caveat about exclusions and unknown gaps is delivered compactly in the second sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only list-type tool with no output schema, the description covers purpose, date scoping, exclusion behavior, and a key output limitation. It still leaves 'current exclusions' undefined and does not describe the result shape or pagination semantics clearly, so it is adequate but has gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description only partially compensates: it connects 'local calendar day' to date/timezone and 'report or diary' to purpose. It does not clarify limit or cursor beyond their titles, leaving pagination semantics mostly inferred.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Get'), resource ('bounded screen observations'), and scope ('specified local calendar day'), plus a clear intended use ('draft a report or diary'). The bounded/day framing distinguishes it from broader sibling tools like search_screen_history and get_recent_activity without needing to open the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a clear use context ('to draft a report or diary') and hints at filtering behavior ('Applies current exclusions'), but does not explicitly name alternatives or state when not to use this tool. It leaves sibling routing to inference rather than spelling out the choice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_recent_activityBRead-only
Summarize recent screen activity as blocks of consecutive frames (app, window title, time range, text). Returns untrusted observed data.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| minutes | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds valuable context beyond that by warning 'Returns untrusted observed data' and by describing the grouped frame blocks, which informs the agent about output shape and data reliability.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with the core action and output shape front-loaded, followed by a concise data-reliability warning. Every word earns its place; no duplication or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description conveys purpose, output format, and a data-reliability caveat, which is decent for a simple tool with annotations. However, it omits parameter semantics (limit, minutes), which are central to invoking the tool correctly, leaving an agent to guess how to control the query.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description never mentions the 'limit' or 'minutes' parameters. The agent is given no explanation of what these knobs control, so the description fails to compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Summarize recent screen activity') and output structure ('blocks of consecutive frames (app, window title, time range, text)'), making the tool's purpose clear. It does not explicitly contrast with sibling tools like search_screen_history, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for obtaining a summary of recent screen activity, but it gives no explicit guidance on when to prefer this tool over search_screen_history, get_context_around, or get_day_material, and no exclusions. The context is clear but the routing is only implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_screen_historyARead-only
Full-text search over OCR text of previously captured windows. Returns matching frames, newest first. Returns untrusted observed data.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | ||
| query | Yes | ||
| since_minutes | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and destructiveHint, and the description adds meaningful context: results are untrusted observed data and are returned newest first. This is useful behavioral information beyond the schema, and it does not contradict the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short sentences with no filler. Each sentence adds value: search scope, ordering, and trustworthiness of results. It is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Core behavior, ordering, and data trustworthiness are covered, and annotations handle the safety profile. However, with no output schema and no parameter semantics, the agent must infer the meaning of limit, since_minutes, and the exact structure of a matching frame.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate by explaining the parameters. It does not: query, limit, and since_minutes are left to their names and defaults. The only indirect hint is that 'full-text search' implies the query parameter, which is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a specific verb and resource: full-text search over OCR text of previously captured windows. It also states the result ordering (newest first) and clearly differentiates this from the sibling retrieval tools by scoping it to OCR screen history.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when this tool applies: when an agent needs to find previously observed screen text via full-text search. However, it does not explicitly name alternatives such as get_context_around or get_recent_activity, nor state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
v0.1.0- First observed
get_context_around - First observed
get_day_material - First observed
get_recent_activity - First observed
search_screen_history
TDQS
Scored across 4 tools
Each tool has a distinct retrieval intent: search by text, explore around a timestamp, summarize recent activity, and bound a full calendar day. There is some potential overlap between get_recent_activity and get_day_material when an agent asks about 'yesterday', but the descriptions largely clarify the temporal scoping.
Three tools follow a clean get_<something> pattern while search_screen_history uses search_ instead of get_. All names are snake_case and verb-first, so the deviation is minor and does not hurt readability.
Four tools is well-scoped for a screen-history context server. Each tool covers a distinct query mode without unnecessary redundancy or bloat.
The set covers the main ways someone would query screen history: full-text search, temporal context, recent summaries, and day-level report material. Missing features like arbitrary custom time ranges or app/window filtering are workable gaps but not critical for the apparent purpose.
Maintenance
Related MCP Connectors
Cross-device AI memory with encrypted activity capture and context handoff between AI tools
- DoneThatOAuthai.donethat
Privacy-first work tracking with summaries, reports, coaching, and AI-ready long-term memory.
Gives your AI assistant persistent memory and intelligence about your work patterns.
- EngramOAuthapp.getengram
Persistent, verbatim, searchable memory for AI assistants — one memory across every MCP client.
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceEnables AI assistants to search and analyze Chrome browser history and bookmarks data locally, including keyword searches, date range filtering, recent browsing activity, and usage statistics across all platforms.2-

ContextPulseofficial
AlicenseAqualityBmaintenanceLets AI assistants understand what you're working on — current screen content, recent dictation, clipboard, and saved notes — running entirely on your own machine with nothing sent to the cloud.36AGPL 3.0- AlicenseAqualityDmaintenanceEnables AI assistants to query your local browsing history using keyword search, RAG Q&A, and activity stats.6MIT
- FlicenseNot gradedqualityBmaintenanceEnables AI assistants to search and retrieve memories from your Mac, including screen captures, meeting transcripts, and browsing history, all locally and privately.1-