ScreenLane

Talk to your screen.

Open-source screen-aware voice command layer for AI agents. Capture what’s on your screen, take an instruction, build an agent-ready command, and send it to tools like Tera, Codra, GateLane, Codex, OpenCode, MCP, clipboard, or stdout.

Voice alone is not the product. Voice + screen context is the product.

Install

npm install -g @talocode/screenlane
# or
npx @talocode/screenlane@latest demo

Requires Node.js ≥ 18.

Published versions

The packages have intentionally different patch versions because the npm patch fixes its executable mapping only.

Related MCP server: markupR MCP Server

Quickstart

screenlane init
screenlane doctor
screenlane demo

screenlane command \
  --text "Fix this error" \
  --context-file ./error.txt \
  --target codra \
  --out prompt

screenlane capture --source text --text "Stack trace here..." --save
screenlane dictate --text "Explain this page" --out text
screenlane serve    # local API: http://127.0.0.1:3070
screenlane mcp      # MCP over stdio

Android Workflow Foundation

ScreenLane v0.2.0 adds the local safety foundation for Android workflows. It records what an agent intends to inspect or do before a future Android companion bridge executes anything.

screenlane android doctor
screenlane android plan --action inspect --app com.example.notes
screenlane android plan --action tap --app com.example.notes --target "New note"
screenlane android plan --action tap --app com.example.notes --target "New note" --approved
screenlane android receipts

• Inspection can be planned without approval.

• Tap, type, and scroll actions require explicit approval.

• Sensitive app classes are blocked by policy.

• Every planned action saves a local receipt.

• Device execution is unavailable until a paired Android companion bridge is installed.

Auth & cloud

One key. One cloud base.

export TALOCODE_API_KEY=your_key

# Optional: gate the local HTTP API
export SCREENLANE_REQUIRE_AUTH=true
# Then clients send: Authorization: Bearer <TALOCODE_API_KEY>

screenlane auth set --key your_key
screenlane auth status
screenlane auth clear

• Local capture / command / demo work without a key.

• Cloud send (Tera, Codra, GateLane, …) uses TALOCODE_API_KEY + https://api.talocode.site.

CLI

SDK

import { ScreenLaneClient } from "@talocode/screenlane";

// Local (no server)
const local = new ScreenLaneClient();
const cmd = await local.createCommand({
  text: "Fix this error",
  contextText: "TypeError: ...",
  target: "codra",
});

// Talk to local server
const api = new ScreenLaneClient({
  baseUrl: "http://127.0.0.1:3070",
  apiKey: process.env.TALOCODE_API_KEY,
});

Local HTTP API

screenlane serve → http://127.0.0.1:3070

• GET /health

• GET /v1/screenlane/doctor

• POST /v1/screenlane/capture|dictate|command|send|demo

• GET /v1/screenlane/contexts · commands

When SCREENLANE_REQUIRE_AUTH=true:

Authorization: Bearer <TALOCODE_API_KEY>

MCP

screenlane mcp

Tools: screenlane_capture, screenlane_dictate, screenlane_command, screenlane_send, screenlane_doctor, screenlane_demo, screenlane_list_contexts, screenlane_list_commands

How it works

Screen context + instruction
        ↓
Agent command (deterministic templates in v0.1)
        ↓
Tera / Codra / Codex / OpenCode / GateLane / MCP / clipboard / stdout

Cloud path uses TALOCODE_API_KEY against https://api.talocode.site.

Skills

• skills/screen-aware-command/

• skills/screen-aware-debugging/

• skills/screen-aware-writing/

Python

pip install talocode-screenlane
screenlane-py --help

Notes

• Prefer --text for dictate (no bundled live mic in v0.1).

• Screenshots use OS tools when available; OCR via tesseract when installed (--no-ocr to skip).

• Text / file / url / clipboard always work.

• Demo is text-mode voice simulation (deterministic).

• More docs: docs/ · Demo: screenlane demo

Talocode ecosystem

Part of Talocode. Sibling projects:

More: github.com/talocode · talocode.site · docs.talocode.site

License

MIT © Talocode