Skip to main content
Glama

jharokha

jha·ro·kha  /dʒʱəˈroː.kʰaː/  noun

An overhanging window of Indian palaces, used to watch the street without being seen.

From Hindi jharokhā (झरोखा).

jharokha lets AI agents (Claude Code, Claude Desktop / Cowork, any MCP client) work in a Windows remote-desktop session they can't otherwise reach: Windows 365 or Azure Virtual Desktop in Windows App, an mstsc RDP window. It reads the session window as text with local OCR and, only if started with --allow-input, scrolls, clicks, types and presses keys in it. Nothing is installed in the session; it works from the pixels you already see.

  • Reads what an agent needs: a Teams chat as messages (time, sender, text, quote), an inbox as mails, a screen as text in reading order; not screenshots.

  • Small and fixed: eight tools whatever apps and actions are loaded; a new app or action is data, never a tool. About 3k tokens of tool descriptions a session.

  • Safe by default: read-only unless you opt in; input only to the app it was meant for, never an app's sending keys unless asked, never while you or another agent use the session. Screen text is untrusted data.

  • Private: nothing on disk unless asked (--trace, --capture), no network, no telemetry.

Windows 10 2004+ / 11. Needs an OCR language pack (any Windows display language has one).

Install

claude mcp add --transport stdio --scope user jharokha -- uvx --from git+https://github.com/ibrarwtf/jharokha jharokha

Add --allow-input at the end to let it act. From a checkout: uv run --no-sync --project PATH jharokha (without --no-sync, a second client session fails to start while the first holds jharokha.exe).

Related MCP server: opencode-gui-bridge

Tools

Screen text comes back inside <screen_text source="ocr" untrusted="true">. Coordinates are client-area pixels.

Tool

What it does

jharokha_status

Is there a session window and can it be read; which app and view is in front, with a confidence; what would stop input.

jharokha_apps

The apps, their views and named regions, and every action with its arguments and risk.

jharokha_read

A region as text or items: the app's focus region by default (a Teams thread), screen, a named region (teams.list), a panel or a box.

jharokha_find

One thing by text, regex or where it is ({"right_of": "To"}, {"spot": "taskbar.teams"}), with its click point.

jharokha_wait

Until a text appears or goes, or the screen holds still.

jharokha_do

An action in one call with a structured result (below). Without --allow-input, its plan only.

jharokha_act

With --allow-input: one click, type, paste, press or scroll, guarded.

jharokha_screenshot

The last resort: a WebP image.

The same verbs run in a shell, each the tool called in-process: jharokha status, jharokha read --region teams.thread, jharokha --allow-input do teams.messages --args '{"from": "18/03", "to": "21/03"}'. --window-title picks the window when more than one session is open.

Apps and actions

An app is data (src/jharokha/apps/*.toml): the text and panes it is recognised by, its views, their named regions and how each is read, and its input policy. Built in: Windows Terminal, Teams, Outlook, a browser, Windows' search panel; anything else is generic, read by panels. Calibration for your screen (zoom variants, the taskbar icons as spots so actions can switch apps) goes in a git-ignored apps.toml (--apps PATH; see apps.example.toml). So does a people map, [people.<id>] and [chats.<id>] with the names the screen shows each by: the Teams actions then take an id (chat = "team") and find that chat in the list and header under any of its names, cut short or not. The messages jharokha_read and the Teams actions return then also carry who (the sender's id), day (ISO) and time as HH:MM (~HH:MM when borrowed from the message above).

An action is a checked sequence of steps run by jharokha_do (src/jharokha/apps/actions/*.toml; your own go in apps.toml):

Action

What it does

teams.messages {chat, from, to}

The messages from one day to another, oldest first, scrolling back to from and down past to.

teams.open_chat {name}, teams.draft {chat, text}

Open a chat; type a draft and never send.

outlook.unread {folder}, outlook.open {subject}

A folder's unread mails; one mail's reading pane.

terminal.run {command}

A command's output, exit code and status, read up to the shell's next state prompt; long output paged back from the shell.

terminal.read_file {path, from, count}, terminal.write_file {path, text}

A page of a file's lines, numbered; a file written from this machine's clipboard and checked by its SHA-256, nothing of it typed.

terminal.setup

Prepares the PowerShell tab: a prompt that shows the command counter, date, time, status, exit code and directory, no colours or predictions, and the font learned from the alphabet it prints. Only this action prepares a tab: the others refuse a tab without the prompt (not_prepared) or running a program (not_a_shell), sending nothing.

terminal.result {timeout_s}, terminal.output {first, count}

A run that outlived its timeout, waited for up to 1800 s; a page of the last command's output.

terminal.edit_once {path, old, new}

Replace text in a file only where it occurs exactly once, through the clipboard.

teams.new {chat}, teams.latest {chat, n}

What came into a chat since its last check; the newest messages on screen.

<app>.switch {window}

Bring an app forward by its taskbar icon; when the icon opens a picker of the app's windows, the one titled like window.

browser.goto, browser.click_sequence

Go to a URL; click buttons in order.

Input safety

Every input, from act, an action or the shell, passes these checks first; a refusal sends nothing and says what to do next.

Check

Refusal

Prevents

The expected app is in front (expect; actions expect their own)

wrong_app

A command typed into a browser and searched for on the web

The app's policy: sending keys (Enter in Teams, Ctrl+Enter in Outlook, Enter in a browser) need allow="irreversible"; a typed newline in Teams is Shift+Enter

irreversible

A half-written message sent

One agent at a time per session window

input_leased

Two agents typing into one session

You haven't used this computer's keyboard or mouse in the last 3 s, and stop it while text is sent

user_active

Your keystrokes mixing into the agent's

The point is not covered by another window

covered

Clicks lost under this computer's taskbar

--trace PATH records every input (typed text as a hash unless --trace-text). --capture DIR saves each screen an action or tool reads, for replaying readers offline; it holds real screen content, so keep it out of any repository.

Testing

An action does in one call what an agent would otherwise do in many scrolls and reads: teams.messages reads a chat from one day to another in a single call. How it is tested: DEVELOPMENT.md.

Limits

  • RDP sends no frames to a minimized window; covered is fine.

  • Windows OCR confuses lookalikes (0/O, 1/l); verify re-reads lines with a second engine. A terminal is read by matching its cells against glyphs learned from the screen (terminal.setup, once per tab), so it has no lookalike problem; a glyph it has not seen is read by the second engine and counted as unknown.

  • Keep the session window one size: calibration holds at the size and zoom it was measured at. Taskbar spots follow a resize when [calibration.session] has the size they were measured at.

  • Apps are recognised by their English UI text.

  • Windows only; a port would sit beside src/jharokha/win/ and reuse core/.

Licence

MIT.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables comprehensive Windows desktop automation including screen capture, OCR text extraction, mouse/keyboard control, window management, process control, and clipboard operations through 25+ tools for AI agents.
      55 PyPI
      8
      MIT
    • F
      license
      Not graded
      quality
      D
      maintenance
      Enables AI assistants to control Windows GUI by listing and focusing windows, capturing element snapshots via UIA/OCR/CDP, performing clicks/inputs/scrolls, verifying changes, waiting for screen updates, taking screenshots, and obtaining visual descriptions.
      2
      -