SnapAction

Turn a saved event or booking screenshot into a source-backed checklist and calendar file.

SnapAction reads one screenshot, generates an action card, asks a separate AI reviewer to check the original image, and revises once if the review fails. It never books anything or silently adds events. Ambiguous details become questions; calendar export requires a passed review, a complete date/time/timezone, and explicit user confirmation.

Status: local first-release candidate. Source, UI, MCP tools and automated tests are included. Not yet published to GitHub/PyPI or hosted remotely. Live AI quality has not been tested here because no API key was available. Independent reviews and exact verification are in docs/VALIDATION.md.

Try it locally

Python 3.11+ is required. Download/extract the source ZIP and open a terminal inside snapaction:

python -m venv .venv
# macOS/Linux:
.venv/bin/python -m pip install .
.venv/bin/snapaction web

Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\python.exe -m pip install .
.\.venv\Scripts\snapaction.exe web

Open http://127.0.0.1:8765. “Try the sample” works without credentials. You can also open src/snapaction/web/index.html directly for the standalone fictional sample prototype; upload processing needs the Python server. Open the interactive prototype to inspect the design and sample flow.

Related MCP server: Google Calendar MCP Server

Enable real screenshot analysis

Set an OpenAI API key in the terminal before starting the app. Do not paste it into chat, source code, a screenshot, or the browser. A ChatGPT subscription does not supply this app's API key.

export OPENAI_API_KEY='YOUR_API_KEY'
# Optional: use another model supporting image input and Responses structured output.
export SNAPACTION_MODEL='gpt-4.1-mini'
export SNAPACTION_REVIEW_MODEL='gpt-4.1-mini'
.venv/bin/snapaction web

Windows PowerShell:

$env:OPENAI_API_KEY = 'YOUR_API_KEY'
.\.venv\Scripts\snapaction.exe web

The .env.example file documents settings; .env is not loaded automatically. Both agents default to gpt-4.1-mini; independent requests use separate prompts and the same original image, not separate providers. You can configure a different reviewer model. Each analysis makes two API calls, or four if the first draft fails review. SDK automatic retries are disabled; each call has a 60-second timeout. Follow-up analysis repeats those calls. Provider usage is charged to your key. Model availability and cost depend on your API account.

1. Select one PNG/JPEG/WebP, up to 8 MiB and 20 megapixels.

2. Enter an IANA timezone only if you explicitly want to supply it. The field starts blank.

3. Consent to sending the screenshot and answers to OpenAI, then analyze.

4. Inspect source quotes, questions and tasks. Answer missing details and run again.

5. Confirm the event details, then download/import the .ics file.

There is no reminder delivery, calendar account integration, autonomous booking or URL fetching. Registration deadlines appear as checklist items, not a substitute event start. Missing duration is not invented; exported ICS omits DTEND. Unresolved questions conservatively block export. DST-ambiguous or nonexistent local times are blocked; ask the organizer for an unambiguous time.

Install as an MCP server

Works in clients that launch local stdio MCP servers. Web-only clients that require a remote URL cannot use this version. There is no universal install link. After installing, configure your client with the absolute path to the installed executable:

{
  "mcpServers": {
    "snapaction": {
      "command": "/ABSOLUTE/PATH/snapaction/.venv/bin/snapaction",
      "args": ["mcp"],
      "env": {
        "OPENAI_API_KEY": "YOUR_API_KEY",
        "SNAPACTION_MODEL": "gpt-4.1-mini"
      }
    }
  }
}

This is a common configuration shape, not a guarantee every client uses the same format. For Windows, use the absolute .venv\\Scripts\\snapaction.exe path and JSON-escape backslashes. Keep any credential-containing config outside Git. Use your client's secret/environment controls where supported. examples/mcp-config.json is a credential-free template.

The client must be able to supply plain base64 image bytes, without a data-URL prefix. Image attachment support varies by client; MCP itself does not guarantee automatic conversion of a chat attachment into a tool parameter. A local path is deliberately not accepted, avoiding arbitrary file access and avoiding shared-filesystem assumptions. See examples/process_image.py for a real MCP client that reads a user-selected image locally, base64-encodes it, obtains consent, and calls the tool.

Architecture and source layout

snapaction/
├── src/snapaction/
│   ├── agent.py        # Extractor, independent reviewer, bounded repair
│   ├── images.py       # Byte/pixel validation and metadata stripping
│   ├── models.py       # Shared typed schemas
│   ├── calendar.py     # Deterministic gate and ICS encoding
│   ├── demo.py         # Fictional fixture
│   ├── server.py       # Three MCP tools; stdio only
│   ├── webapp.py       # Loopback UI/API
│   ├── cli.py          # web / mcp / demo
│   └── web/index.html  # Prototype and actual UI, no frontend build
├── tests/              # Core, calendar, image, HTTP and real MCP tests
├── examples/           # MCP config and image client
├── docs/               # Product, independent reviews, verification, prototype
├── .github/workflows/ci.yml
└── pyproject.toml

The UI and MCP adapters call the same core. The workflow is bounded orchestration; it does not autonomously browse, keep a memory store, or delegate outside actions.

Privacy and limits

Images stay in app memory and are not intentionally saved, logged, or added to a gallery. Input image metadata is removed before sending normalized pixels. This is not a promise of secure memory erasure: OS/process/provider retention still applies. Answers and images go to OpenAI only after consent; store=False is sent, which does not mean zero provider retention. See OpenAI API data controls. Use images you are comfortable sending; crop private information beforehand.

The local web server binds only 127.0.0.1, enforces its exact Host and browser Origin on writes, limits image/request sizes and simultaneous processing, and never returns API secrets. It is not a public hosting server. Do not expose it through a tunnel or bind it publicly. A hosted release needs authentication, rate limits, billing controls and a separate remote MCP transport.

AI can misread pixels or accept plausible errors. Evidence quotes are AI transcriptions; review is a second check, not proof. Runtime export checks validate consistency, not provenance or identity. A caller can fabricate a passed Result; this tool does not provide signed approvals. Review the source yourself. No task completion, booking, or reminder is implied by a generated card.

Test and build

python -m pip install '.[dev]'
ruff check src tests
python -m pytest -q
node --test tests/prototype_state.mjs
python -m build

Tests use a controlled provider for AI orchestration; they do not certify live extraction. Real stdio tests launch the installed server and exercise tools. Browser checks need the optional browser extra and a Playwright Chromium installation; see tests/browser_smoke.py. GitHub Actions is configured for Python 3.11–3.13 on Linux and Windows. Only the local execution record can be claimed until those hosted jobs actually run.

Publish and give people an install link

The download includes snapaction.git.bundle with local Git history. Restore it with git clone snapaction.git.bundle snapaction-repo, or initialize Git in the extracted source folder. Create an empty GitHub repository under your own account, then run inside this directory (replace YOUR_ACCOUNT with your account; never paste credentials):

git remote add origin https://github.com/YOUR_ACCOUNT/snapaction.git
git push -u origin main

After the repository is public, users can install directly from its actual Git URL:

python -m pip install 'git+https://github.com/YOUR_ACCOUNT/snapaction.git'

Those are templates, not existing public URLs. For a stable installation, use an actual release tag or commit rather than a moving branch. Check package-name availability before publishing to PyPI; snapaction-mcp is a proposed distribution name, not a reserved name. Build and inspect dist/, perform the live smoke check below, then upload with your own registry credentials using python -m twine upload dist/* (install twine first). Once successfully published under the chosen name, users can pip install it. No registry upload is automated.

Before public release, run at least 10 representative real screenshots (clear, blurry, ambiguous date/year, missing timezone, conflicting timezone, unsupported and injection text), record supported facts/questions and verify every timestamp/quote. Check quota/timeouts with a real key. Run CI, inspect the package, and review the independent validation report.