type_into
Automatically locate text input fields using OCR, click to focus, type specified text, and optionally submit with Enter for desktop automation in Hyprland environments.
Instructions
Find a text input field, click it, type text, and optionally submit.
Combines focus + OCR + click + type + Enter into one action. Searches for placeholder text in the active window to find the input field.
Args: text: The text to type input_hint: Placeholder or label text near the input field to click on (e.g. "Type a message", "Search", "Message"). If omitted, tries common placeholders. submit: Whether to press Enter after typing (default False) window: Target a specific window (e.g. "class:signal"). Default: active window.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| input_hint | No | ||
| submit | No | ||
| window | No |
Output Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |
Implementation Reference
- hyprland_mcp/server.py:557-638 (handler)Handler function for the 'type_into' MCP tool. It uses OCR to locate an input field, clicks it, types the provided text, and optionally submits.
async def type_into( text: str, input_hint: str | None = None, submit: bool = False, window: str | None = None, ) -> str: """Find a text input field, click it, type text, and optionally submit. Combines focus + OCR + click + type + Enter into one action. Searches for placeholder text in the active window to find the input field. Args: text: The text to type input_hint: Placeholder or label text near the input field to click on (e.g. "Type a message", "Search", "Message"). If omitted, tries common placeholders. submit: Whether to press Enter after typing (default False) window: Target a specific window (e.g. "class:signal"). Default: active window. """ from . import screenshot as ss, ocr, input as inp import asyncio # Focus the window if specified if window: await hyprctl.dispatch("focuswindow", window) await asyncio.sleep(0.1) png_bytes, origin_x, origin_y = await _auto_scope_capture( None, window, None, ) boxes = ocr.extract_boxes(png_bytes) # Try to find the input field by hint text or common placeholders hints = [input_hint] if input_hint else [ "Type a message", "Message", "Search", "Type here", "Write a message", "Enter message", "Say something", ] match = None used_hint = None for hint in hints: matches = ocr.find_text(boxes, hint) if matches: match = matches[0] used_hint = hint break if not match: all_text = ocr.extract_text(png_bytes) preview = all_text[:500] + "..." if len(all_text) > 500 else all_text tried = ", ".join(f'"{h}"' for h in hints) return ( f"Could not find input field. Tried: {tried}\n\n" f"OCR detected text:\n{preview}" ) # Click the center of the matched text screen_x = match["x"] + origin_x + match["w"] // 2 screen_y = match["y"] + origin_y + match["h"] // 2 await inp.move_cursor(screen_x, screen_y) await inp.click("left") await asyncio.sleep(0.1) # Type the text await inp.type_text(text) result = f"Found '{used_hint}' at ({screen_x},{screen_y}), typed {len(text)} chars" # Submit if requested if submit: await asyncio.sleep(0.05) await hyprctl.dispatch("sendshortcut", ", Return, ") result += ", pressed Enter" return result