[{"_1":2,"_385":-5,"_386":-5},"loaderData",{"_3":4,"_22":23,"_24":25,"_281":282},"root",{"_5":6,"_7":8},"toasts",[],"visitor",{"_9":10,"_11":12},"viewportWidth",1280,"visitorSession",{"_13":14,"_16":17,"_18":-5,"_19":20,"_21":-5},"attributes",[15],"BOT","id",8780354,"membership","referenceId","0197c85f-34b4-7020-804a-1f56d7353f19","userAccount","routes/_public/_layout",{},"routes/_public/mcp/servers/~namespace/~slug/_pages/_layout",{"_26":27,"_66":-5,"_67":28,"_29":68,"_199":200,"_280":130},"availableLocales",[28,40,46,53,60],{"_29":30,"_33":37,"_38":39},"mcpServer",{"_31":32,"_33":34,"_35":36},"description","A server that provides rich UI context and interaction capabilities to AI models, enabling deep understanding of user interfaces through visual analysis and precise interaction via Model Context Protocol.","name","OmniMCP","readme","# OmniMCP\n\n[![CI](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml/badge.svg)](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Python Version](https://img.shields.io/badge/python-3.10%20|%203.11%20|%203.12-blue)](https://www.python.org/)\n[![Code style: ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n\nOmniMCP provides rich UI context and interaction capabilities to AI models through [Model Context Protocol (MCP)](https://github.com/modelcontextprotocol) and [microsoft/OmniParser](https://github.com/microsoft/OmniParser). It focuses on enabling deep understanding of user interfaces through visual analysis, structured planning, and precise interaction execution.\n\n## Core Features\n\n- **Visual Perception:** Understands UI elements using OmniParser.\n- **LLM Planning:** Plans next actions based on goal, history, and visual state.\n- **Agent Executor:** Orchestrates the perceive-plan-act loop (`omnimcp/agent_executor.py`).\n- **Action Execution:** Controls mouse/keyboard via `pynput` (`omnimcp/input.py`).\n- **CLI Interface:** Simple entry point (`cli.py`) for running tasks.\n- **Auto-Deployment:** Optional OmniParser server deployment to AWS EC2 with auto-shutdown.\n- **Debugging:** Generates timestamped visual logs per step.\n\n## Overview\n\n`cli.py` uses `AgentExecutor` to run a perceive-plan-act loop. It captures the screen (`VisualState`), plans using an LLM (`core.plan_action_for_ui`), and executes actions (`InputController`).\n\n### Demos\n\n- **Real Action (Calculator):** `python cli.py` opens Calculator and computes 5*9.\n ![OmniMCP Real Action Demo GIF](images/omnimcp_demo.gif)\n- **Synthetic UI (Login):** `python demo_synthetic.py` uses generated images (no real I/O). *(Note: Pending refactor to use AgentExecutor).*\n ![OmniMCP Synthetic Demo GIF](images/omnimcp_demo_synthetic.gif)\n\n## Prerequisites\n\n- Python >=3.10, <3.13\n- `uv` installed (`pip install uv`)\n- **Linux Runtime Requirement:** Requires an active graphical session (X11/Wayland) for `pynput`. May need system libraries (`libx11-dev`, etc.) - see `pynput` docs.\n\n*(macOS display scaling dependencies are handled automatically during installation).*\n\n### For AWS Deployment Features\n\nRequires AWS credentials in `.env` (see `.env.example`). **Warning:** Creates AWS resources (EC2, Lambda, etc.) incurring costs. Use `python -m omnimcp.omniparser.server stop` to clean up.\n\n```.env\nAWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY\nAWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY\nANTHROPIC_API_KEY=YOUR_ANTHROPIC_KEY\n# OMNIPARSER_URL=http://... # Optional: Skip auto-deploy\n```\n\n## Installation\n\n```bash\ngit clone [https://github.com/OpenAdaptAI/OmniMCP.git](https://github.com/OpenAdaptAI/OmniMCP.git)\ncd OmniMCP\n./install.sh # Creates .venv, installs deps incl. test extras\ncp .env.example .env\n# Edit .env with your keys\n# Activate: source .venv/bin/activate (Linux/macOS) or relevant Windows command\n```\n\n## Quick Start\n\nEnsure environment is activated and `.env` is configured.\n\n```bash\n# Run default goal (Calculator task)\npython cli.py\n\n# Run custom goal\npython cli.py --goal \"Your goal here\"\n\n# See options\npython cli.py --help\n```\nDebug outputs are saved in `runs//`.\n\n**Note on MCP Server:** An experimental MCP server (`OmniMCP` class in `omnimcp/mcp_server.py`) exists but is separate from the primary `cli.py`/`AgentExecutor` workflow.\n\n## Architecture\n\n1. **CLI** (`cli.py`) - Entry point, setup, starts Executor.\n2. **Agent Executor** (`omnimcp/agent_executor.py`) - Orchestrates loop, manages state/artifacts.\n3. **Visual State Manager** (`omnimcp/visual_state.py`) - Perception (screenshot, calls parser).\n4. **OmniParser Client & Deploy** (`omnimcp/omniparser/`) - Manages OmniParser server communication/deployment.\n5. **LLM Planner** (`omnimcp/core.py`) - Generates action plan.\n6. **Input Controller** (`omnimcp/input.py`) - Executes actions (mouse/keyboard).\n7. **(Optional) MCP Server** (`omnimcp/mcp_server.py`) - Experimental MCP interface.\n\n## Development\n\n### Environment Setup & Checks\n```bash\n# Setup (if not done): ./install.sh\n# Activate env: source .venv/bin/activate (or similar)\n# Format/Lint: uv run ruff format . && uv run ruff check . --fix\n# Run tests: uv run pytest tests/\n```\n\n### Debug Support\nRunning `python cli.py` saves timestamped runs in `runs/`, including:\n* `step_N_state_raw.png`\n* `step_N_state_parsed.png` (with element boxes)\n* `step_N_action_highlight.png` (with action highlight)\n* `final_state.png`\n\nDetailed logs are in `logs/run_YYYY-MM-DD_HH-mm-ss.log` (`LOG_LEVEL=DEBUG` in `.env` recommended).\n\n

Example Log Snippet (Auto-Deploy + Agent Step)

\n\n```log\n# --- Initialization & Auto-Deploy ---\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.client:... - No server_url provided, attempting discovery/deployment...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.server:... - Creating new EC2 instance...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.server:... - Instance i-... is running. Public IP: ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.server:... - Setting up auto-shutdown infrastructure...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.server:... - Auto-shutdown infrastructure setup completed...\n... (SSH connection, Docker setup) ...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.client:... - Auto-deployment successful. Server URL: http://...\n... (Agent Executor Init) ...\n\n# --- Agent Execution Loop Example Step ---\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - --- Step N/10 ---\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Perceiving current screen state...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.visual_state:update:... - VisualState update complete. Found X elements. Took Y.YYs.\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - Perceived state with X elements.\n... (Save artifacts) ...\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Planning next action...\n... (LLM Call) ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - LLM Plan: Action=..., TargetID=..., GoalComplete=False\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Added to history: Step N: Planned action ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - Executing action: ...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.agent_executor:run:... - Action executed successfully.\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Step N duration: Z.ZZs\n... (Loop continues or finishes) ...\n```\n*(Note: Details like timings, counts, IPs, instance IDs, and specific plans will vary)*\n

\n\n## Roadmap & Limitations\n\nKey limitations & future work areas:\n\n* **Performance:** Reduce OmniParser latency (explore local models, caching, etc.) and optimize state management (avoid full re-parse).\n* **Robustness:** Improve LLM planning reliability (prompts, techniques like ReAct), add action verification/error recovery, enhance element targeting.\n* **Target API/Architecture:** Evolve towards a higher-level declarative API (e.g., `@omni.publish` style) and potentially integrate loop logic with the experimental MCP Server (`OmniMCP` class).\n* **Consistency:** Refactor `demo_synthetic.py` to use `AgentExecutor`.\n* **Features:** Expand action space (drag/drop, hover).\n* **Testing:** Add E2E tests, broaden cross-platform validation, define evaluation metrics.\n* **Research:** Explore fine-tuning, process graphs (RAG), framework integration.\n\n## Project Status\n\nCore loop via `cli.py`/`AgentExecutor` is functional for basic tasks. Performance and robustness need significant improvement. MCP integration is experimental.\n\n## Contributing\n\n1. Fork repository\n2. Create feature branch\n3. Implement changes & add tests\n4. Ensure checks pass (`uv run ruff format .`, `uv run ruff check . --fix`, `uv run pytest tests/`)\n5. Submit pull request\n\n## License\n\nMIT License\n\n## Contact\n\n- Issues: [GitHub Issues](https://github.com/OpenAdaptAI/OmniMCP/issues)\n- Questions: [Discussions](https://github.com/OpenAdaptAI/OmniMCP/discussions)\n- Security: security@openadapt.ai\n","English","tag","en-US",{"_29":41,"_33":44,"_38":45},{"_31":42,"_33":34,"_35":43},"Un servidor que proporciona un rico contexto de UI y capacidades de interacción para los modelos de IA, lo que permite una comprensión profunda de las interfaces de usuario a través del análisis visual y una interacción precisa mediante el Protocolo de Contexto de Modelo.","# OmniMCP\n\n[![CI](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml/badge.svg)](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml) [![Licencia: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Versión de Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org/) [![Estilo de código: ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n\nOmniMCP proporciona un contexto de interfaz de usuario completo y capacidades de interacción a los modelos de IA mediante [el Protocolo de Contexto de Modelo (MCP)](https://github.com/modelcontextprotocol) y [Microsoft/OmniParser](https://github.com/microsoft/OmniParser) . Se centra en facilitar una comprensión profunda de las interfaces de usuario mediante el análisis visual, la planificación estructurada y la ejecución precisa de las interacciones.\n\n## Características principales\n\n* **Percepción visual:** comprende los elementos de la interfaz de usuario mediante OmniParser.\n* **Planificación LLM:** planifica las próximas acciones en función del objetivo, el historial y el estado visual.\n* **Agente Ejecutor:** Orquesta el ciclo percibir-planificar-actuar ( `omnimcp/agent_executor.py` ).\n* **Ejecución de la acción:** controla el mouse/teclado a través de `pynput` ( `omnimcp/input.py` ).\n* **Interfaz CLI:** punto de entrada simple ( `cli.py` ) para ejecutar tareas.\n* **Implementación automática:** implementación opcional del servidor OmniParser en AWS EC2 con apagado automático.\n* **Depuración:** genera registros visuales con marca de tiempo por paso.\n\n## Descripción general\n\n`cli.py` usa `AgentExecutor` para ejecutar un bucle de percepción-planificación-acción. Captura la pantalla ( `VisualState` ), planifica usando un LLM ( `core.plan_action_for_ui` ) y ejecuta acciones ( `InputController` ).\n\n### Población\n\n* **Acción real (Calculadora):** `python cli.py` abre la Calculadora y calcula 5\\*9.![GIF de demostración de acción real de OmniMCP](images/omnimcp_demo.gif)\n* **Interfaz de usuario sintética (Inicio de sesión):** `python demo_synthetic.py` utiliza imágenes generadas (sin E/S real). *(Nota: Refactorización pendiente para usar AgentExecutor).*![GIF de demostración sintética de OmniMCP](images/omnimcp_demo_synthetic.gif)\n\n## Prerrequisitos\n\n* Python >=3.10, <3.13\n* `uv` instalado ( `pip install uv` )\n* **Requisitos de ejecución de Linux:** Requiere una sesión gráfica activa (X11/Wayland) para `pynput` . Puede requerir bibliotecas del sistema ( `libx11-dev` , etc.); consulte la documentación `pynput` .\n\n*(Las dependencias de escala de pantalla de macOS se manejan automáticamente durante la instalación).*\n\n### Para funciones de implementación de AWS\n\nRequiere credenciales de AWS en `.env` (consulte `.env.example` ). **Advertencia:** Crea recursos de AWS (EC2, Lambda, etc.) con costos. Use `python -m omnimcp.omniparser.server stop` para limpiar.\n\n```.env\nAWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY\nAWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY\nANTHROPIC_API_KEY=YOUR_ANTHROPIC_KEY\n# OMNIPARSER_URL=http://... # Optional: Skip auto-deploy\n```\n\n## Instalación\n\n```bash\ngit clone [https://github.com/OpenAdaptAI/OmniMCP.git](https://github.com/OpenAdaptAI/OmniMCP.git)\ncd OmniMCP\n./install.sh # Creates .venv, installs deps incl. test extras\ncp .env.example .env\n# Edit .env with your keys\n# Activate: source .venv/bin/activate (Linux/macOS) or relevant Windows command\n```\n\n## Inicio rápido\n\nAsegúrese de que el entorno esté activado y que `.env` esté configurado.\n\n```bash\n# Run default goal (Calculator task)\npython cli.py\n\n# Run custom goal\npython cli.py --goal \"Your goal here\"\n\n# See options\npython cli.py --help\n```\n\nLas salidas de depuración se guardan en `runs//` .\n\n**Nota sobre el servidor MCP:** existe un servidor MCP experimental (clase `OmniMCP` en `omnimcp/mcp_server.py` `AgentExecutor` pero está separado del flujo de trabajo principal `cli.py` .\n\n## Arquitectura\n\n1. **CLI** ( `cli.py` ): punto de entrada, configuración, inicia el ejecutor.\n2. **Agente Ejecutor** ( `omnimcp/agent_executor.py` ): orquesta el bucle y administra el estado y los artefactos.\n3. **Visual State Manager** ( `omnimcp/visual_state.py` ): Percepción (captura de pantalla, analizador de llamadas).\n4. **OmniParser Client & Deploy** ( `omnimcp/omniparser/` ): administra la comunicación/implementación del servidor OmniParser.\n5. **LLM Planner** ( `omnimcp/core.py` ) - Genera un plan de acción.\n6. **Controlador de entrada** ( `omnimcp/input.py` ): ejecuta acciones (mouse/teclado).\n7. **(Opcional) Servidor MCP** ( `omnimcp/mcp_server.py` ): interfaz MCP experimental.\n\n## Desarrollo\n\n### Configuración y comprobaciones del entorno\n\n```bash\n# Setup (if not done): ./install.sh\n# Activate env: source .venv/bin/activate (or similar)\n# Format/Lint: uv run ruff format . && uv run ruff check . --fix\n# Run tests: uv run pytest tests/\n```\n\n### Soporte de depuración\n\nAl ejecutar `python cli.py` se guardan las ejecuciones con marca de tiempo en `runs/` , incluidos:\n\n* `step_N_state_raw.png`\n* `step_N_state_parsed.png` (con cuadros de elementos)\n* `step_N_action_highlight.png` (con resaltado de acción)\n* `final_state.png`\n\nLos registros detallados se encuentran en `logs/run_YYYY-MM-DD_HH-mm-ss.log` (se recomienda `LOG_LEVEL=DEBUG` en `.env` ).\n\n```log\n# --- Initialization & Auto-Deploy ---\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.client:... - No server_url provided, attempting discovery/deployment...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.server:... - Creating new EC2 instance...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.server:... - Instance i-... is running. Public IP: ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.server:... - Setting up auto-shutdown infrastructure...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.server:... - Auto-shutdown infrastructure setup completed...\n... (SSH connection, Docker setup) ...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.client:... - Auto-deployment successful. Server URL: http://...\n... (Agent Executor Init) ...\n\n# --- Agent Execution Loop Example Step ---\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - --- Step N/10 ---\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Perceiving current screen state...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.visual_state:update:... - VisualState update complete. Found X elements. Took Y.YYs.\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - Perceived state with X elements.\n... (Save artifacts) ...\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Planning next action...\n... (LLM Call) ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - LLM Plan: Action=..., TargetID=..., GoalComplete=False\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Added to history: Step N: Planned action ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - Executing action: ...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.agent_executor:run:... - Action executed successfully.\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Step N duration: Z.ZZs\n... (Loop continues or finishes) ...\n```\n\n*(Nota: Los detalles como tiempos, conteos, direcciones IP, ID de instancias y planes específicos variarán)*\n\n## Hoja de ruta y limitaciones\n\nLimitaciones clave y áreas de trabajo futuras:\n\n* **Rendimiento:** reduzca la latencia de OmniParser (explore modelos locales, almacenamiento en caché, etc.) y optimice la gestión del estado (evite el reanálisis completo).\n* **Robustez:** Mejorar la confiabilidad de la planificación de LLM (indicaciones, técnicas como ReAct), agregar verificación de acciones/recuperación de errores, mejorar la focalización de elementos.\n* **API/Arquitectura de destino:** evolucionar hacia una API declarativa de nivel superior (por ejemplo, estilo `@omni.publish` ) y potencialmente integrar la lógica de bucle con el servidor MCP experimental (clase `OmniMCP` ).\n* **Consistencia:** refactorice `demo_synthetic.py` para utilizar `AgentExecutor` .\n* **Características:** Ampliar el espacio de acción (arrastrar y soltar, pasar el cursor).\n* **Pruebas:** agregar pruebas E2E, ampliar la validación multiplataforma, definir métricas de evaluación.\n* **Investigación:** Explorar el ajuste fino, los gráficos de procesos (RAG) y la integración del marco.\n\n## Estado del proyecto\n\n`AgentExecutor` bucle principal mediante `cli.py` funciona para tareas básicas. El rendimiento y la robustez requieren mejoras significativas. La integración con MCP es experimental.\n\n## Contribuyendo\n\n1. Repositorio de bifurcaciones\n2. Crear rama de características\n3. Implementar cambios y agregar pruebas\n4. Asegúrese de que las comprobaciones pasen ( `uv run ruff format .` , `uv run ruff check . --fix` , `uv run pytest tests/` )\n5. Enviar solicitud de extracción\n\n## Licencia\n\nLicencia MIT\n\n## Contacto\n\n* Problemas: [Problemas de GitHub](https://github.com/OpenAdaptAI/OmniMCP/issues)\n* Preguntas: [Discusiones](https://github.com/OpenAdaptAI/OmniMCP/discussions)\n* Seguridad: ","Spanish","es-ES",{"_29":47,"_33":51,"_38":52},{"_31":48,"_33":49,"_35":50},"AI モデルに豊富な UI コンテキストとインタラクション機能を提供するサーバー。��デルコンテキストプロトコルを介した視覚的な分析と正確なインタラクションを通じて、ユーザーインターフェイスを深く理解できます。","オムニMCP","# オムニMCP\n\n[![CI](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml/badge.svg)](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml) [![ライセンス: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Pythonバージョン](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org/) [![コードスタイル: ラフ](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n\nOmniMCPは[、モデルコンテキストプロトコル（MCP）](https://github.com/modelcontextprotocol)と[microsoft/OmniParser](https://github.com/microsoft/OmniParser)を通じて、AIモデルに豊富なUIコンテキストとインタラクション機能を提供します。視覚的な分析、構造化された計画、そして正確なインタラクション実行を通じて、ユーザーインターフェイスの深い理解を可能にすることに重点を置いています。\n\n## コア機能\n\n* **��覚認識:** OmniParser を使用して UI 要素を理解します。\n* \\*\\*LLM 計画:\\*\\*目標、履歴、視覚的な状態に基づいて次のアクションを計画します。\n* \\*\\*エージェントエグゼキューター:\\*\\*認識-計画-行動ループ ( `omnimcp/agent_executor.py` ) を調整します。\n* **アクション実行:** `pynput` ( `omnimcp/input.py` ) を介してマウス/キーボードを制御します。\n* \\*\\*CLI インターフェース:\\*\\*タスクを実行するためのシンプルなエントリポイント ( `cli.py` )。\n* \\*\\*自動デプロイメント:\\*\\*自動シャットダウン付きの AWS EC2 へのオプションの OmniParser サーバーデプロイメント。\n* \\*\\*デバッグ:\\*\\*ステップごとにタイムスタンプ付きのビジュアルログを生成します。\n\n## 概要\n\n`cli.py` `AgentExecutor`を使用して、認識・計画・行動のループを実行します。画面をキャプチャし ( `VisualState` )、LLM を使用してプランニングを行い ( `core.plan_action_for_ui` )、アクションを実行します ( `InputController` )。\n\n### デモ\n\n* **実際のアクション (電卓):** `python cli.py`電卓を開き、5\\*9 を計算します。![OmniMCP リアルアクションデモ GIF](images/omnimcp_demo.gif)\n* **合成 UI (ログイン):** `python demo_synthetic.py`生成された画像を使用します (実際の I/O はありません)。 *(注: AgentExecutor を使用するためのリファクタリングが保留中です)。*![OmniMCP 合成デモ GIF](images/omnimcp_demo_synthetic.gif)\n\n## 前提条件\n\n* Python >=3.10、<3.13\n* `uv`がインストールされました（ `pip install uv` ）\n* **Linux ランタイム要件:** `pynput`にはアクティブなグラフィカルセッション (X11/Wayland) が必要です。システムライブラリ ( `libx11-dev`など) が必要になる場合があります - `pynput`ドキュメントを参照してください。\n\n*(macOS ディスプレイのスケーリング依存関係はインストール中に自動的に処理されます)。*\n\n### AWS デプロイメント機能について\n\n`.env`にAWS認証情報が必要です（ `.env.example`を参照）。**警告：** AWSリソース（EC2、Lambdaなど）が作成され、コストが発生します。クリーンアップするには、 `python -m omnimcp.omniparser.server stop`を使用してください。\n\n```.env\nAWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY\nAWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY\nANTHROPIC_API_KEY=YOUR_ANTHROPIC_KEY\n# OMNIPARSER_URL=http://... # Optional: Skip auto-deploy\n```\n\n## インストール\n\n```bash\ngit clone [https://github.com/OpenAdaptAI/OmniMCP.git](https://github.com/OpenAdaptAI/OmniMCP.git)\ncd OmniMCP\n./install.sh # Creates .venv, installs deps incl. test extras\ncp .env.example .env\n# Edit .env with your keys\n# Activate: source .venv/bin/activate (Linux/macOS) or relevant Windows command\n```\n\n## クイックスタート\n\n環境がアクティブ化され、 `.env`が構成されていることを確認します。\n\n```bash\n# Run default goal (Calculator task)\npython cli.py\n\n# Run custom goal\npython cli.py --goal \"Your goal here\"\n\n# See options\npython cli.py --help\n```\n\nデバッグ出力は`runs//`に保存されます。\n\n\\*\\*MCP サーバーに関する注意:\\*\\*実験的な MCP サーバー ( `omnimcp/mcp_server.py`の`OmniMCP`クラス) が存在しますが、プライマリ`cli.py` / `AgentExecutor`ワークフローとは別です。\n\n## 建築\n\n1. **CLI** ( `cli.py` ) - エントリポイント、セットアップ、Executor の起動。\n2. **エージェントエグゼキューター**( `omnimcp/agent_executor.py` ) - ループをオーケストレーションし、状態/アーティファクトを管理します。\n3. **Visual State Manager** ( `omnimcp/visual_state.py` ) - 認識 (スクリーンショット、パーサーの呼び出し)。\n4. **OmniParser クライアント & デプロイ**( `omnimcp/omniparser/` ) - OmniParser サーバーの通信/デプロイメントを管理します。\n5. **LLM プランナー**( `omnimcp/core.py` ) - アクションプランを生成します。\n6. **入力コントローラ**( `omnimcp/input.py` ) - アクション (マウス/キーボード) を実行します。\n7. **(オプション) MCP サーバー**( `omnimcp/mcp_server.py` ) - 実験的な MCP インターフェース。\n\n## 発達\n\n### 環境設定とチェック\n\n```bash\n# Setup (if not done): ./install.sh\n# Activate env: source .venv/bin/activate (or similar)\n# Format/Lint: uv run ruff format . && uv run ruff check . --fix\n# Run tests: uv run pytest tests/\n```\n\n### デバッグサポート\n\n`python cli.py`を実行すると、次の内容を含むタイムスタンプ付きの実行が`runs/`に保存されます。\n\n* `step_N_state_raw.png`\n* `step_N_state_parsed.png` （要素ボックス付き）\n* `step_N_action_highlight.png` (アクションハイライト付き)\n* `final_state.png`\n\n詳細なログは`logs/run_YYYY-MM-DD_HH-mm-ss.log`にあります ( `.env`では`LOG_LEVEL=DEBUG`推奨されます)。\n\n```log\n# --- Initialization & Auto-Deploy ---\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.client:... - No server_url provided, attempting discovery/deployment...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.server:... - Creating new EC2 instance...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.server:... - Instance i-... is running. Public IP: ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.server:... - Setting up auto-shutdown infrastructure...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.server:... - Auto-shutdown infrastructure setup completed...\n... (SSH connection, Docker setup) ...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.client:... - Auto-deployment successful. Server URL: http://...\n... (Agent Executor Init) ...\n\n# --- Agent Execution Loop Example Step ---\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - --- Step N/10 ---\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Perceiving current screen state...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.visual_state:update:... - VisualState update complete. Found X elements. Took Y.YYs.\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - Perceived state with X elements.\n... (Save artifacts) ...\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Planning next action...\n... (LLM Call) ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - LLM Plan: Action=..., TargetID=..., GoalComplete=False\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Added to history: Step N: Planned action ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - Executing action: ...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.agent_executor:run:... - Action executed successfully.\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Step N duration: Z.ZZs\n... (Loop continues or finishes) ...\n```\n\n*(注: タイミング、カウント、IP、インスタンス ID、特定のプランなどの詳細は異なります)*\n\n## ロードマップと制限事項\n\n主な制限事項と将来の作業領域:\n\n* **パフォーマンス:** OmniParser のレイテンシを削減し (ローカルモデル、キャッシュなどを探索)、状態管理を最適化します (完全な再解析を回避します)。\n* **堅牢性:** LLM 計画の信頼性 (プロンプト、ReAct などのテクニック) を向上させ、アクション検証/エラー回復を追加し、要素のターゲティングを強化します。\n* \\*\\*ターゲット API/アーキテクチャ:\\*\\*より高レベルの宣言型 API (例: `@omni.publish`スタイル) に向けて進化し、ループロジックを実験的な MCP サーバー ( `OmniMCP`クラス) と統合する可能性があります。\n* **一貫性:** `AgentExecutor`使用するように`demo_synthetic.py`をリファクタリングします。\n* \\*\\*機能:\\*\\*アクションスペースを拡張します (ドラッグ/ドロップ、ホバー)。\n* **テスト:** E2E テストを追加し、クロスプラットフォーム検証を拡大し、評価メトリックを定義します。\n* \\*\\*調査:\\*\\*微調整、プロセスグラフ (RAG)、フレームワークの統合を調査します。\n\n## プロジェクトのステータス\n\n`cli.py` / `AgentExecutor`経由のコアループは基本的なタスクには機能します。パフォーマンスと堅牢性には大幅な改善が必要です。MCP との統合は試験段階です。\n\n## 貢献\n\n1. フォークリポジトリ\n2. 機能ブランチを作成する\n3. 変更を実装し、テストを追加する\n4. チェックが合格したことを確認します ( `uv run ruff format .` 、 `uv run ruff check . --fix` 、 `uv run pytest tests/` )\n5. プルリクエストを送信する\n\n## ライセンス\n\nMITライセンス\n\n## 接触\n\n* 問題: [GitHub の問題](https://github.com/OpenAdaptAI/OmniMCP/issues)\n* 質問：[ディスカッション](https://github.com/OpenAdaptAI/OmniMCP/discussions)\n* セキュリティ: ","Japanese","ja-JP",{"_29":54,"_33":58,"_38":59},{"_31":55,"_33":56,"_35":57},"AI 모델에 풍부한 UI 컨텍스트와 상호작용 기능을 제공하는 서버로, 시각적 분석을 통해 사용자 인터페이스를 심층적으로 이해하고 모델 컨텍스트 프로토콜을 통해 정확한 상호작용을 구현할 수 있습니다.","옴니MCP","# 옴니MCP\n\n[![씨아이](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml/badge.svg)](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml) [![라이센스: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![파이썬 버전](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org/) [![코드 스타일: ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n\nOmniMCP는 [Model Context Protocol(MCP)](https://github.com/modelcontextprotocol) 과 [microsoft/OmniParser를](https://github.com/microsoft/OmniParser) 통해 AI 모델에 풍부한 UI 컨텍스트 및 상호작용 기능을 제공합니다. 시각적 분석, 체계적인 계획, 그리고 정밀한 상호작용 실행을 통해 사용자 인터페이스에 대한 심층적인 이해를 지원하는 데 중점을 둡니다.\n\n## 핵심 기능\n\n* **시각적 인식:** OmniParser를 사용하여 UI 요소를 이해합니다.\n* **LLM 계획:** 목표, 기록, 시각적 상태를 기반으로 다음 작업을 계획합니다.\n* **에이전트 실행자:** 인식-계획-행동 루프를 조율합니다( `omnimcp/agent_executor.py` ).\n* **동작 실행:** `pynput` ( `omnimcp/input.py` )을 통해 마우스/키보드를 제어합니다.\n* **CLI 인터페이스:** 작업 실행을 위한 간단한 진입점( `cli.py` ).\n* **자동 배포:** 자동 종료 기능을 갖춘 AWS EC2에 대한 선택적 OmniParser 서버 배포.\n* **디버깅:** 단계별로 타임스탬프가 포함된 시각적 로그를 생성합니다.\n\n## 개요\n\n`cli.py` `AgentExecutor` 사용하여 인식-계획-행동 루프를 실행합니다. 화면( `VisualState` )을 캡처하고, LLM( `core.plan_action_for_ui` )을 사용하여 계획을 수립하고, 동작( `InputController` )을 실행합니다.\n\n### 시민\n\n* **실제 작업(계산기):** `python cli.py` 계산기를 열고 5\\*9를 계산합니다.![OmniMCP 실제 동작 데모 GIF](images/omnimcp_demo.gif)\n* **합성 UI(로그인):** `python demo_synthetic.py` 생성된 이미지를 사용합니다(실제 I/O 없음). *(참고: AgentExecutor를 사용하기 위한 리팩토링이 진행 중입니다).*![OmniMCP 합성 데모 GIF](images/omnimcp_demo_synthetic.gif)\n\n## 필수 조건\n\n* 파이썬 3.10 이상, 3.13 미만\n* `uv` 설치됨 ( `pip install uv` )\n* **Linux 런타임 요구 사항:** `pynput` 사용하려면 활성 그래픽 세션(X11/Wayland)이 필요합니다. 시스템 라이브러리( `libx11-dev` 등)가 필요할 수 있습니다. `pynput` 문서를 참조하세요.\n\n*(macOS 디스플레이 크기 조정 종속성은 설치 중에 자동으로 처리됩니다).*\n\n### AWS 배포 기능\n\n`.env` 에서 AWS 자격 증명이 필요합니다( `.env.example` 참조). **경고:** AWS 리소스(EC2, Lambda 등)가 생성되어 비용이 발생합니다. `python -m omnimcp.omniparser.server stop` 사용하여 리소스를 정리하세요.\n\n지엑스피1\n\n## 설치\n\n```bash\ngit clone [https://github.com/OpenAdaptAI/OmniMCP.git](https://github.com/OpenAdaptAI/OmniMCP.git)\ncd OmniMCP\n./install.sh # Creates .venv, installs deps incl. test extras\ncp .env.example .env\n# Edit .env with your keys\n# Activate: source .venv/bin/activate (Linux/macOS) or relevant Windows command\n```\n\n## 빠른 시작\n\n환경이 활성화되어 있고 `.env` 구성되어 있는지 확인하세요.\n\n```bash\n# Run default goal (Calculator task)\npython cli.py\n\n# Run custom goal\npython cli.py --goal \"Your goal here\"\n\n# See options\npython cli.py --help\n```\n\n디버그 출력은 `runs//` 에 저장됩니다.\n\n**MCP 서버에 대한 참고 사항:** 실험적 MCP 서버( `omnimcp/mcp_server.py` 의 `OmniMCP` 클래스)가 있지만 기본 `cli.py` / `AgentExecutor` 워크플로와 별개입니다.\n\n## 건축학\n\n1. **CLI** ( `cli.py` ) - 진입점, 설정, Executor 시작.\n2. **에이전트 실행자** ( `omnimcp/agent_executor.py` ) - 루프를 조정하고 상태/아티팩트를 관리합니다.\n3. **시각적 상태 관리자** ( `omnimcp/visual_state.py` ) - 인식(스크린샷, 파서 호출).\n4. **OmniParser 클라이언트 및 배포** ( `omnimcp/omniparser/` ) - OmniParser 서버 통신/배포를 관리합니다.\n5. **LLM Planner** ( `omnimcp/core.py` ) - 작업 계획을 생성합니다.\n6. **입력 컨트롤러** ( `omnimcp/input.py` ) - 동작(마우스/키보드)을 실행합니다.\n7. **(선택 사항) MCP 서버** ( `omnimcp/mcp_server.py` ) - 실험적 MCP 인터페이스.\n\n## 개발\n\n### 환경 설정 및 확인\n\n```bash\n# Setup (if not done): ./install.sh\n# Activate env: source .venv/bin/activate (or similar)\n# Format/Lint: uv run ruff format . && uv run ruff check . --fix\n# Run tests: uv run pytest tests/\n```\n\n### 디버그 지원\n\n`python cli.py` 실행하면 다음을 포함하여 타임스탬프가 지정된 실행이 `runs/` 에 저장됩니다.\n\n* `step_N_state_raw.png`\n* `step_N_state_parsed.png` (요소 상자 포함)\n* `step_N_action_highlight.png` (액션 강조 표시 포함)\n* `final_state.png`\n\n자세한 로그는 `logs/run_YYYY-MM-DD_HH-mm-ss.log` 에 있습니다( `.env` 에서 `LOG_LEVEL=DEBUG` 권장).\n\n```log\n# --- Initialization & Auto-Deploy ---\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.client:... - No server_url provided, attempting discovery/deployment...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.server:... - Creating new EC2 instance...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.server:... - Instance i-... is running. Public IP: ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.server:... - Setting up auto-shutdown infrastructure...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.server:... - Auto-shutdown infrastructure setup completed...\n... (SSH connection, Docker setup) ...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.client:... - Auto-deployment successful. Server URL: http://...\n... (Agent Executor Init) ...\n\n# --- Agent Execution Loop Example Step ---\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - --- Step N/10 ---\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Perceiving current screen state...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.visual_state:update:... - VisualState update complete. Found X elements. Took Y.YYs.\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - Perceived state with X elements.\n... (Save artifacts) ...\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Planning next action...\n... (LLM Call) ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - LLM Plan: Action=..., TargetID=..., GoalComplete=False\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Added to history: Step N: Planned action ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - Executing action: ...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.agent_executor:run:... - Action executed successfully.\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Step N duration: Z.ZZs\n... (Loop continues or finishes) ...\n```\n\n*(참고: 타이밍, 개수, IP, 인스턴스 ID, 특정 계획과 같은 세부 사항은 다를 수 있습니다)*\n\n## 로드맵 및 제한 사항\n\n주요 제한 사항 및 향후 작업 영역:\n\n* **성능:** OmniParser 지연 시간을 줄이고(로컬 모델 탐색, 캐싱 등) 상태 관리를 최적화합니다(전�� 재구문을 방지).\n* **견고성:** LLM 계획의 안정성(프롬프트, ReAct와 같은 기술)을 개선하고, 작업 검증/오류 복구를 추가하고, 요소 타겟팅을 강화합니다.\n* **대상 API/아키텍처:** 더 높은 수준의 선언적 API(예: `@omni.publish` 스타일)로 발전하고 잠재적으로 루프 논리를 실험적 MCP 서버( `OmniMCP` 클래스)와 통합합니다.\n* **일관성:** `AgentExecutor` 사용하도록 `demo_synthetic.py` 리팩토링합니다.\n* **특징:** 작업 공간 확장(드래그/드롭, 호버)\n* **테스트:** E2E 테스트를 추가하고, 플랫폼 간 검증을 확대하고, 평가 지표를 정의합니다.\n* **연구:** 미세 조정, 프로세스 그래프(RAG), 프레임워크 통합을 탐구합니다.\n\n## 프로젝트 상태\n\n`cli.py` `AgentExecutor` 통한 코어 루프는 기본 작업에 적합합니다. 성능과 견고성은 크게 개선되어야 합니다. MCP 통합은 실험 중입니다.\n\n## 기여하다\n\n1. 포크 저장소\n2. 기능 브랜치 생성\n3. 변경 사항 구현 및 테스트 추가\n4. 검사 통과 보장( `uv run ruff format .` , `uv run ruff check . --fix` , `uv run pytest tests/` )\n5. 풀 리퀘스트 제출\n\n## 특허\n\nMIT 라이센스\n\n## 연락하다\n\n* 문제: [GitHub 문제](https://github.com/OpenAdaptAI/OmniMCP/issues)\n* 질문: [토론](https://github.com/OpenAdaptAI/OmniMCP/discussions)\n* 보안: ","Korean","ko-KR",{"_29":61,"_33":64,"_38":65},{"_31":62,"_33":34,"_35":63},"为AI模型提供丰富的UI上下文和交互能力的服务器，通过可视化分析实现对用户界面的深度理解，并通过模型上下文协议实现精准的交互。","# OmniMCP\n\n[![CI](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml/badge.svg)](https://github.com/OpenAdaptAI/OmniMCP/actions/workflows/ci.yml) [![许可证：MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Python 版本](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue)](https://www.python.org/) [![代码风格：ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n\nOmniMCP 通过[模型上下文协议 (MCP)](https://github.com/modelcontextprotocol)和[microsoft/OmniParser](https://github.com/microsoft/OmniParser)为 AI 模型提供丰富的 UI 上下文和交互能力，专注于通过可视化分析、结构化规划和精准交互执行，实现对用户界面的深度理解。\n\n## 核心功能\n\n* \\*\\*视觉感知：\\*\\*使用 OmniParser 理解 UI 元素。\n* \\*\\*LLM 规划：\\*\\*根据目标、历史和视觉状态规划下一步行动。\n* \\*\\*代理执行器：\\*\\*协调感知-计划-行动循环（ `omnimcp/agent_executor.py` ）。\n* \\*\\*动作执行：\\*\\*通过`pynput` ( `omnimcp/input.py` ) 控制鼠标/键盘。\n* \\*\\*CLI 接口：\\*\\*运行任务的简单入口点（ `cli.py` ）。\n* \\*\\*自动部署：\\*\\*可选将 OmniParser 服务器部署到 AWS EC2 并自动关闭。\n* \\*\\*调试：\\*\\*每一步生成带有时间戳的可视日志。\n\n## 概述\n\n`cli.py`使用`AgentExecutor`运行感知-计划-执行循环。它捕获屏幕（ `VisualState` ），使用 LLM（ `core.plan_action_for_ui` ）进行规划，并执行操作（ `InputController` ）。\n\n### 演示\n\n* **实际动作（计算器）：** `python cli.py`打开计算器，计算 5\\*9。![OmniMCP 真实动作演示 GIF](images/omnimcp_demo.gif)\n* **合成 UI（登录）：** `python demo_synthetic.py`使用生成的图像（无实际 I/O）。 *（注：待重构以使用 AgentExecutor）。*![OmniMCP 合成演示 GIF](images/omnimcp_demo_synthetic.gif)\n\n## 先决条件\n\n* Python >=3.10，<3.13\n* `uv`安装（ `pip install uv` ）\n* **Linux 运行时要求：** `pynput`需要一个活动的图形会话 (X11/Wayland)。可能需要系统库（ `libx11-dev`等）——请参阅`pynput`文档。\n\n*（macOS 显示缩放依赖关系在安装期间自动处理）。*\n\n### 对于 AWS 部署功能\n\n需要在`.env`中提供 AWS 凭证（参见`.env.example` ）。\\*\\*警告：\\*\\*创建 AWS 资源（EC2、Lambda 等）会产生费用。请使用`python -m omnimcp.omniparser.server stop`进行清理。\n\n```.env\nAWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY\nAWS_SECRET_ACCESS_KEY=YOUR_SECRET_KEY\nANTHROPIC_API_KEY=YOUR_ANTHROPIC_KEY\n# OMNIPARSER_URL=http://... # Optional: Skip auto-deploy\n```\n\n## 安装\n\n```bash\ngit clone [https://github.com/OpenAdaptAI/OmniMCP.git](https://github.com/OpenAdaptAI/OmniMCP.git)\ncd OmniMCP\n./install.sh # Creates .venv, installs deps incl. test extras\ncp .env.example .env\n# Edit .env with your keys\n# Activate: source .venv/bin/activate (Linux/macOS) or relevant Windows command\n```\n\n## 快速入门\n\n确保环境已激活并且`.env`已配置。\n\n```bash\n# Run default goal (Calculator task)\npython cli.py\n\n# Run custom goal\npython cli.py --goal \"Your goal here\"\n\n# See options\npython cli.py --help\n```\n\n调试输出保存在`runs//`中。\n\n\\*\\*关于 MCP 服务器的说明：\\*\\*存在一个实验性的 MCP 服务器（ `omnimcp/mcp_server.py`中的`OmniMCP`类），但与主要的`cli.py` / `AgentExecutor`工作流程是分开的。\n\n## 建筑学\n\n1. **CLI** （ `cli.py` ）——入口点，设置，启动执行器。\n2. **代理执行器**（ `omnimcp/agent_executor.py` ）——协调循环，管理状态/工件。\n3. **视觉状态管理器**（ `omnimcp/visual_state.py` ） - 感知（屏幕截图，调用解析器）。\n4. **OmniParser 客户端和部署**（ `omnimcp/omniparser/` ） - 管理 OmniParser 服务器通信/部署。\n5. **LLM 规划器**（ `omnimcp/core.py` ）——生成行动计划。\n6. **输入控制器**（ `omnimcp/input.py` ） - 执行操作（鼠标/键盘）。\n7. **（可选）MCP 服务器**（ `omnimcp/mcp_server.py` ） - 实验性 MCP 接口。\n\n## 发展\n\n### 环境设置和检查\n\n```bash\n# Setup (if not done): ./install.sh\n# Activate env: source .venv/bin/activate (or similar)\n# Format/Lint: uv run ruff format . && uv run ruff check . --fix\n# Run tests: uv run pytest tests/\n```\n\n### 调试支持\n\n运行`python cli.py`会将带有时间戳的运行保存在`runs/`中，包括：\n\n* `step_N_state_raw.png`\n* `step_N_state_parsed.png` （带元素框）\n* `step_N_action_highlight.png` （带有动作突出显示）\n* `final_state.png`\n\n详细日志位于`logs/run_YYYY-MM-DD_HH-mm-ss.log`中（建议在`.env`中使用`LOG_LEVEL=DEBUG` ）。\n\n```log\n# --- Initialization & Auto-Deploy ---\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.client:... - No server_url provided, attempting discovery/deployment...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.server:... - Creating new EC2 instance...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.server:... - Instance i-... is running. Public IP: ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.omniparser.server:... - Setting up auto-shutdown infrastructure...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.server:... - Auto-shutdown infrastructure setup completed...\n... (SSH connection, Docker setup) ...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.omniparser.client:... - Auto-deployment successful. Server URL: http://...\n... (Agent Executor Init) ...\n\n# --- Agent Execution Loop Example Step ---\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - --- Step N/10 ---\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Perceiving current screen state...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.visual_state:update:... - VisualState update complete. Found X elements. Took Y.YYs.\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - Perceived state with X elements.\n... (Save artifacts) ...\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Planning next action...\n... (LLM Call) ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - LLM Plan: Action=..., TargetID=..., GoalComplete=False\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Added to history: Step N: Planned action ...\n2025-MM-DD HH:MM:SS | INFO | omnimcp.agent_executor:run:... - Executing action: ...\n2025-MM-DD HH:MM:SS | SUCCESS | omnimcp.agent_executor:run:... - Action executed successfully.\n2025-MM-DD HH:MM:SS | DEBUG | omnimcp.agent_executor:run:... - Step N duration: Z.ZZs\n... (Loop continues or finishes) ...\n```\n\n*（注意：时间、数量、IP、实例 ID 和具体计划等详细信息会有所不同）*\n\n## 路线图和局限性\n\n主要限制和未来工作领域：\n\n* \\*\\*性能：\\*\\*减少 OmniParser 延迟（探索本地模型、缓存等）并优化状态管理（避免完全重新解析）。\n* \\*\\*稳健性：\\*\\*提高 LLM 规划可靠性（提示、ReAct 等技术），添加动作验证/错误恢复，增强元素定位。\n* \\*\\*目标 API/架构：\\*\\*向更高级别的声明式 API 发展（例如， `@omni.publish`样式），并可能将循环逻辑与实验性 MCP 服务器（ `OmniMCP`类）集成。\n* \\*\\*一致性：\\*\\*重构`demo_synthetic.py`以使用`AgentExecutor` 。\n* \\*\\*特点：\\*\\*扩展操作空间（拖放、悬停）。\n* \\*\\*测试：\\*\\*添加 E2E 测试，扩大跨平台验证，定义评估指标。\n* \\*\\*研究：\\*\\*探索微调、流程图（RAG）、框架集成。\n\n## 项目状态\n\n通过`cli.py` / `AgentExecutor`核心循环可以完成基本任务。性能和稳健性需要显著改进。MCP 集成尚处于实验阶段。\n\n## 贡献\n\n1. Fork 存储库\n2. 创建功能分支\n3. 实施变更并添加测试\n4. 确保检查通过（ `uv run ruff format .` , `uv run ruff check . --fix` , `uv run pytest tests/` ）\n5. 提交拉取请求\n\n## 执照\n\nMIT 许可证\n\n## 接触\n\n* 问题： [GitHub 问题](https://github.com/OpenAdaptAI/OmniMCP/issues)\n* 问题：[讨论](https://github.com/OpenAdaptAI/OmniMCP/discussions)\n* 安全： ","Chinese","zh-CN","githubUser","locale",{"_69":70,"_71":72,"_13":92,"_100":32,"_101":32,"_102":103,"_106":107,"_127":-5,"_16":128,"_129":130,"_131":132,"_33":34,"_177":119,"_178":-5,"_179":-5,"_180":181,"_182":-5,"_183":184,"_189":-5,"_139":34,"_190":191,"_195":-5,"_120":196,"_197":198},"addedAt","2025-03-18T20:00:39.533533931Z","argumentsJsonSchema",{"_73":74,"_89":90,"_80":91},"properties",{"_75":76,"_82":83,"_86":87},"OMNIMCP_DEBUG",{"_77":78,"_31":79,"_80":81},"default","1","Enable debug mode","type","string","OMNIMCP_LOG_LEVEL",{"_77":84,"_31":85,"_80":81},"DEBUG","Log level","OMNIMCP_PARSER_URL",{"_31":88,"_80":81},"Custom parser URL","required",[],"object",[93,94,95,96,97,98,99],"category:autonomous-agents","category:browser-automation","category:os-automation","environment:local","hosting:hybrid","language:python","license_grade:f","descriptionMarkdown","descriptionPlainText","githubProject",{"_35":36,"_104":105},"url","https://github.com/OpenAdaptAI/OmniMCP","githubRepository",{"_108":109,"_110":111,"_112":113,"_114":115,"_33":34,"_116":117,"_122":123,"_124":-5,"_125":126},"defaultBranch","main","fullName","OpenAdaptAI/OmniMCP","githubTreeId",106070,"language","Python","owner",{"_118":119,"_120":121},"login","OpenAdaptAI","uid","G2wpezUg43","recentStargazers",3,"spdxLicense","stargazers",24,"glamaJson",1946,"inspectable",false,"integrations",[133,143,150,157,163,170],{"_134":135,"_31":141,"_120":142},"brand",{"_136":137,"_33":138,"_139":140},"logoSvgPath","M24 0v24H0V0h24ZM10.933 15.89H6.84v5.52h4.198v-.93H7.955v-1.503h2.77v-.93h-2.77v-1.224h2.978v-.934Zm2.146 0h-1.084v5.52h1.035v-3.6l2.226 3.6h1.118v-5.52h-1.036v3.686l-2.259-3.687Zm5.117 0h-1.208l1.973 5.52h1.19l1.976-5.52h-1.182l-1.352 4.085-1.397-4.086ZM5.4 19.68H3.72v1.68H5.4v-1.68Z",".ENV","slug","dotenv","Supports configuration via .env files for settings like debug mode, parser URL, and log levels","dkv4yvvth0",{"_134":144,"_31":148,"_120":149},{"_136":145,"_33":146,"_139":147},"M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12","GitHub","github","Integrates with GitHub for repository management, including cloning repositories and accessing project files","17y9hbdh8y",{"_134":151,"_31":155,"_120":156},{"_136":152,"_33":153,"_139":154},"M2.6152 0v.8867h3.8399V0zm5.0215 0v.8867h3.8418V0zm4.957 0v.8867h3.8418V0zm4.9356 0v.8867h3.8418V0zM2.4473 1.8945a.935.935 0 0 0-.9356.9356c0 .517.4185.9375.9356.9375h19.1054c.5171 0 .9356-.4204.9356-.9375a.935.935 0 0 0-.9356-.9356zm.168 2.8477V24H6.455V4.7422zm5.0214 0V20.543h3.8418V4.7422zm4.957 0V15.291h3.8497V4.7422zm4.9356 0v6.4941h3.8418V4.7422z","Pytest","pytest","Provides testing infrastructure for validating OmniMCP functionality","u4uc5neqf4",{"_134":158,"_31":161,"_120":162},{"_136":159,"_33":115,"_139":160},"M14.25.18l.9.2.73.26.59.3.45.32.34.34.25.34.16.33.1.3.04.26.02.2-.01.13V8.5l-.05.63-.13.55-.21.46-.26.38-.3.31-.33.25-.35.19-.35.14-.33.1-.3.07-.26.04-.21.02H8.77l-.69.05-.59.14-.5.22-.41.27-.33.32-.27.35-.2.36-.15.37-.1.35-.07.32-.04.27-.02.21v3.06H3.17l-.21-.03-.28-.07-.32-.12-.35-.18-.36-.26-.36-.36-.35-.46-.32-.59-.28-.73-.21-.88-.14-1.05-.05-1.23.06-1.22.16-1.04.24-.87.32-.71.36-.57.4-.44.42-.33.42-.24.4-.16.36-.1.32-.05.24-.01h.16l.06.01h8.16v-.83H6.18l-.01-2.75-.02-.37.05-.34.11-.31.17-.28.25-.26.31-.23.38-.2.44-.18.51-.15.58-.12.64-.1.71-.06.77-.04.84-.02 1.27.05zm-6.3 1.98l-.23.33-.08.41.08.41.23.34.33.22.41.09.41-.09.33-.22.23-.34.08-.41-.08-.41-.23-.33-.33-.22-.41-.09-.41.09zm13.09 3.95l.28.06.32.12.35.18.36.27.36.35.35.47.32.59.28.73.21.88.14 1.04.05 1.23-.06 1.23-.16 1.04-.24.86-.32.71-.36.57-.4.45-.42.33-.42.24-.4.16-.36.09-.32.05-.24.02-.16-.01h-8.22v.82h5.84l.01 2.76.02.36-.05.34-.11.31-.17.29-.25.25-.31.24-.38.2-.44.17-.51.15-.58.13-.64.09-.71.07-.77.04-.84.01-1.27-.04-1.07-.14-.9-.2-.73-.25-.59-.3-.45-.33-.34-.34-.25-.34-.16-.33-.1-.3-.04-.25-.02-.2.01-.13v-5.34l.05-.64.13-.54.21-.46.26-.38.3-.32.33-.24.35-.2.35-.14.33-.1.3-.06.26-.04.21-.02.13-.01h5.84l.69-.05.59-.14.5-.21.41-.28.33-.32.27-.35.2-.36.15-.36.1-.35.07-.32.04-.28.02-.21V6.07h2.09l.14.01zm-6.47 14.25l-.23.33-.08.41.08.41.23.33.33.23.41.08.41-.08.33-.23.23-.33.08-.41-.08-.41-.23-.33-.33-.23-.41-.08-.41.08z","python","Built on Python with async support for UI automation and interaction","eoovk6e1x3",{"_134":164,"_31":168,"_120":169},{"_136":165,"_33":166,"_139":167},"M9.419 10.32v-.645c0-.531.437-.968.968-.968h3.225c.204 0 .395.065.552.175a2.295 2.295 0 0 0-.904.47h-2.873c-.177 0-.323.146-.323.323v3.224c0 .177.146.323.323.323h2.921l.048.067v.578h-2.969c-.531 0-.968-.437-.968-.968v-1.934h-5.13l.08.967h3.115v.645H4.423l.058.698c.651.192 1.323.309 2.001.347.901.038 1.789.229 2.625.564.138.04.244.146.288.276a4.183 4.183 0 0 0-1.771-.395H6.88c-.25 0-.485.065-.688.18a9.4449 9.4449 0 0 1-1.654-.286l.147 1.772c.267-.044.533-.095.798-.15.018.224.09.434.203.616-.314.067-.63.127-.947.179l.09 1.09c.03.332.31.589.643.591h2.101c-.335.14-.626.364-.846.645H5.472c-.666-.003-1.226-.519-1.285-1.183l-.548-6.556H2.358l.758 7.578c.081.821.78 1.453 1.605 1.451h1.565c.001.121.012.239.033.355-.293.04-.566.141-.806.29h-.792c-1.155.004-2.134-.882-2.246-2.032l-.794-7.933c-.001-.01-.001-.021-.001-.032 0-.088.036-.169.094-.228l2.58-2.58a.324.324 0 0 1 .228-.094h1.023a.9628.9628 0 0 1-.056-.323c0-.435.294-.819.713-.933a.9928.9928 0 0 1-.068-.357c0-.53.437-.967.968-.967h1.29c.531 0 .967.437.967.967 0 .099-.015.198-.045.293a.976.976 0 0 1 .436-.258.9719.9719 0 0 1-.068-.357c0-.531.437-.968.967-.968h1.29c.531 0 .968.437.968.968 0 .11-.019.219-.055.322h1.667c.331 0 .639.17.816.449a.9527.9527 0 0 1 .474-.126h2.58c.531 0 .968.436.968.967 0 .11-.019.219-.056.323h.461l-.282-.174 1.876-3.051h-2.645V3.87h.637a5.077 5.077 0 0 0-.027-.323c-.64-.002-.853-.011-1.57-.04l-.09-.004a2.6858 2.6858 0 0 1-.673-.089.4482.4482 0 0 1-.267-.189.4067.4067 0 0 1-.041-.282.455.455 0 0 1 .238-.274c.059-.032.12-.062.181-.089h-3.87v1.29h4.837v.645H2.579l1.087 1.766-.549.338-1.389-2.258a.3145.3145 0 0 1-.048-.169C1.68 1.802 3.205 0 5.227 0h.078c.403 0 .747.275.855.645h11.679c.107-.37.451-.645.854-.645h.078c2.023 0 3.548 1.802 3.548 4.192a.318.318 0 0 1-.067.196L20.191 7.74h.193c.53 0 .967.437.967.967 0 .224-.078.441-.22.614l1.083 1.082c.065.06.106.145.106.239 0 .011 0 .022-.001.032l-.794 7.933c-.11 1.131-1.059 2.006-2.188 2.032a2.053 2.053 0 0 0-.805-.29c.021-.116.032-.234.032-.355h.714c.824.002 1.524-.63 1.604-1.451l.76-7.578h-1.283l-.546 6.556c-.059.664-.62 1.18-1.287 1.183h-.403c-.094-.12-.201-.23-.319-.328v-.317h.722c.333-.002.613-.259.643-.591l.004-.054c-.578 0-1.099-.002-1.636-.018v-.645c.553.017 1.091.018 1.69.018l.08-.96c-.666.022-1.243.098-1.77.237v-.665c.549-.13 1.145-.201 1.824-.219l.081-.971c-.637.008-1.272.043-1.905.105v-.647a23.275 23.275 0 0 1 1.958-.103l.081-.969h-1.521c.134-.2.234-.417.298-.645h1.276l.081-.967h-1.276a2.2538 2.2538 0 0 0-.129-.645h2.079c.177 0 .322-.146.322-.323a.323.323 0 0 0-.322-.322.3248.3248 0 0 1-.323-.323c0-.177.146-.322.323-.322.177 0 .322-.146.322-.323a.323.323 0 0 0-.322-.322h-1.935a.3248.3248 0 0 1-.323-.323c0-.177.146-.322.323-.322.177 0 .322-.146.322-.323a.323.323 0 0 0-.322-.322h-2.58a.321.321 0 0 0-.277.161c-.057.1-.164.161-.279.161h-.089a.323.323 0 0 1-.322-.322c0-.177-.146-.323-.323-.323h-2.58a.323.323 0 0 1-.322-.322c0-.177.145-.323.322-.323.177 0 .323-.145.323-.322 0-.177-.146-.323-.323-.323h-1.29c-.177 0-.322.146-.322.323 0 .177.145.322.322.322.177 0 .323.146.323.323 0 .177-.146.322-.323.322h-.645a.323.323 0 0 0-.302.218.3227.3227 0 0 1-.305.215.327.327 0 0 1-.179-.054.3172.3172 0 0 0-.181-.056h-.645a.3248.3248 0 0 1-.323-.323c0-.177.146-.322.323-.322.177 0 .322-.146.322-.323a.323.323 0 0 0-.322-.322h-1.29c-.177 0-.323.145-.323.322 0 .177.146.323.323.323.177 0 .322.145.322.322 0 .177-.145.323-.322.323h-.645c-.177 0-.323.145-.323.322 0 .177.146.323.323.323.177 0 .322.145.322.322 0 .177-.145.323-.322.323H4.904c-.176 0-.322.145-.322.322 0 .177.146.323.322.323.177 0 .323.145.323.322 0 .177-.146.323-.323.323H4.26c-.177 0-.323.145-.323.322 0 .177.146.323.323.323h1.289a.3117.3117 0 0 0 .081-.013.3177.3177 0 0 1 .162 0c.026.007.053.012.08.013h2.58c.177 0 .322-.146.322-.323a.323.323 0 0 0-.322-.322h-.968V9.03h.968c.531 0 .967.436.967.967 0 .113-.019.222-.056.323h.056Zm3.225-6.45V1.29h-1.29v2.58h1.29Zm-10.309 0h1.609c.089-1.854.936-2.52 1.452-2.754A.2513.2513 0 0 0 5.55.89c0-.134-.11-.245-.244-.245h-.079C3.662.645 2.46 2.01 2.335 3.87Zm2.255 0h6.119V2.257c-1.154 0-2.459.001-3.508.003h-.077c.065.041.136.079.208.119.127.066.25.141.368.223a.43.43 0 0 1 .18.536.4527.4527 0 0 1-.258.228c-.223.079-.458.121-.696.123-.793.043-1.475.055-2.309.058-.012.102-.021.21-.027.323Zm14.663-.969c-.226-.759-.637-1.071-.917-1.198a.903.903 0 0 1-.434-.413h-4.613v.645c1.274 0 3.003 0 4.155.002h.055c.129-.006.259.004.385.032.259.037.451.267.44.528a.5407.5407 0 0 1-.265.373c.494.021.727.029 1.194.031ZM6.096 1.29a.8943.8943 0 0 1-.434.413c-.279.127-.691.439-.917 1.198.751-.003 1.383-.016 2.103-.054-.104-.06-.203-.129-.295-.207a.6246.6246 0 0 1-.235-.405c-.008-.263.164-.5.417-.572.132-.04.271-.056.409-.048h.055c1.051-.002 2.356-.003 3.51-.003V1.29H6.096Zm-3.315 9.03h.566a.9616.9616 0 0 1-.056-.323c0-.072.008-.143.024-.212l-.534.535Zm17.273-6.45h1.609C21.538 2.01 20.337.645 18.771.645h-.078a.245.245 0 0 0-.244.244c.002.099.063.188.154.226.516.235 1.362.901 1.451 2.755Zm-6.698 12.638c-.345.098-.701.162-1.062.19-.197.025-.395.047-.593.069-.056.004-.111.008-.167.014a3.995 3.995 0 0 0-.289-.618c.129-.015.257-.027.386-.037.193-.022.387-.043.58-.068.391-.027.776-.102 1.145-.222v.672Zm-.284-3.609h-2.363a.323.323 0 0 1-.322-.322v-2.58c0-.177.145-.322.322-.322h2.24c-.153.192-.275.409-.359.645h-1.558v1.934h1.627c.064.141.143.276.236.403l.177.242Zm-5.467 5.147.005-.01v.008l-.005.002ZM3.292 11.61v.644h-.645v-.644h.645Zm1.29 7.094v.645h-.645v-.645h.645Zm16.769-7.417v.645h-.645v-.645h.645Zm-1.29 7.417v.645h-.645v-.645h.645ZM8.774 11.932v.645h-.645v-.645h.645Zm3.548-1.29v1.29h-.645v-1.29h.645ZM4.064 6.927l.395.642-.549.338-.395-.642.549-.338Zm14.18 13.902c.875 0 1.586.712 1.586 1.585 0 .875-.711 1.586-1.586 1.586H6.606c-.875 0-1.585-.711-1.585-1.586 0-.873.71-1.585 1.585-1.585h.427c-.155-.246-.247-.534-.247-.845 0-.785.574-1.437 1.324-1.563v-.786c0-.385-.172-.744-.473-.986l-.034-.028a1.158 1.158 0 0 0-.723-.254c-.497 0-.902-.404-.902-.9s.405-.9.902-.9h.744c.836 0 1.653.288 2.304.812.866.697 1.363 1.733 1.363 2.842v.177h2.565v-5.272l-.557-.764c-.223-.306-.342-.666-.342-1.045v-.233c0-.979.799-1.775 1.781-1.775h1.418c.982 0 1.781.796 1.781 1.775v.233c0 .379-.118.739-.341 1.045l-.559.764v5.376c.599.226 1.027.805 1.027 1.482 0 .311-.092.599-.246.845h.426Zm-3.385-5.232a.263.263 0 0 1-.021-.372l.345-.389v-.794a1.7787 1.7787 0 0 1-.586-.196v4.552h1.7v-4.552c-.181.099-.379.165-.585.196v.794l.345.389a.264.264 0 0 1-.023.372.2624.2624 0 0 1-.175.068.2668.2668 0 0 1-.198-.089l-.214-.241-.214.241a.2668.2668 0 0 1-.198.089.2649.2649 0 0 1-.176-.068Zm-6.758.475c.476.384.75.953.75 1.563v.763h1.7v-.177a2.98 2.98 0 0 0-.076-.662l-.406-.062c-.733-.114-1.307-.704-1.398-1.438l-.072-.585a2.9598 2.9598 0 0 0-.975-.167H6.88c-.089 0-.162.072-.162.16 0 .088.073.159.162.159.431 0 .852.149 1.186.418l.035.028Zm8.342 7.188c.402-.819 1.152-1.415 2.057-1.616l.06-.012a.8101.8101 0 0 0-.316-.063H6.606c-.466 0-.845.38-.845.845 0 .467.379.846.845.846h.558l-.007-.006a.8244.8244 0 0 1-.265-.604c0-.409.307-.76.714-.816l.985-.136c.343-.047.696.071.942.315.309.309.41.773.255 1.182l-.025.065h6.68Zm-8.817-3.672a.836.836 0 0 0-.099.396c0 .466.379.845.846.845h4.31c.082-.551.464-1.022 1.001-1.211.419-.147.881-.106 1.266.114l.199.113c.374.214.636.574.73.984h.599c.466 0 .846-.379.846-.845 0-.467-.38-.846-.846-.846h-4.935l-.125.202c-.405.654-1.108 1.017-1.831 1.017-.348 0-.701-.085-1.028-.261l-.933-.508Zm6.072-8.271c0 .22.068.431.199.608l.709.971c.195.267.509.427.841.427.331 0 .646-.16.841-.427l.71-.971c.13-.177.199-.388.199-.608v-.233c0-.451-.293-.835-.698-.976v.301c0 .146-.119.265-.265.265a.2646.2646 0 0 1-.264-.265v-.359h-.258v.684c0 .146-.119.265-.265.265a.2646.2646 0 0 1-.264-.265v-.684h-.258v.359c0 .146-.119.265-.265.265a.2646.2646 0 0 1-.264-.265v-.301c-.406.141-.698.525-.698.976v.233Zm1.157.309c0 .166-.135.301-.303.301a.3015.3015 0 0 1-.302-.301.3026.3026 0 0 1 .605 0Zm1.672.012c0 .166-.136.302-.303.302a.3024.3024 0 0 1-.302-.302c0-.166.135-.301.302-.301.167 0 .303.135.303.301Z","Rich","rich","Implements rich debug information and context visualization for UI automation","mep6ng0wj3",{"_134":171,"_31":175,"_120":176},{"_136":172,"_33":173,"_139":174},"M3 0C1.338 0 0 1.338 0 3v18c0 1.662 1.338 3 3 3h18c1.662 0 3-1.338 3-3V3c0-1.662-1.338-3-3-3Zm4.2 7.2h8.641c.53 0 .959.43.959.959v3.266c0 .53-.43.959-.959.959h-.961v.768h1.92V16.8h-4.416v-2.88h-.768v2.88H7.2Zm3.648 3.648v.768h2.304v-.768z","Ruff","ruff","Includes linting support using Ruff for code quality assurance","2qbnd1y9p5","namespace","npmPackage","prompts","redditPostForeignId","1jfzv84","resources","scores",{"_185":186,"_187":-5,"_188":-5},"license",0,"quality","security","simpleIcon","supportedPlatforms",[192,193,194],"WINDOWS","LINUX","MACOS","tools","gf61tqx1fu","updatedAt","2025-04-08T07:03:20.672672165Z","meta",[201,204,206,213,216,218,221,224,227,230,233,235,238,241,243,245,247,249,251],{"_202":203},"title","OmniMCP | Glama",{"_205":32,"_33":31},"content",{"_207":208,"_209":210,"_211":212},"href","https://glama.ai/mcp/servers/@OpenAdaptAI/OmniMCP","rel","canonical","tagName","link",{"_205":34,"_214":215},"property","og:title",{"_205":32,"_214":217},"og:description",{"_205":219,"_214":220},"https://glama.ai/logo.png","og:logo",{"_205":222,"_214":223},"https://glama.ai/mcp/servers/@OpenAdaptAI/OmniMCP/og-image","og:image",{"_205":225,"_214":226},"630","og:image:height",{"_205":228,"_214":229},"1200","og:image:width",{"_205":231,"_214":232},"website","og:type",{"_205":208,"_214":234},"og:url",{"_205":236,"_214":237},"Glama – MCP Hosting Platform","og:site_name",{"_205":239,"_33":240},"summary_large_image","twitter:card",{"_205":236,"_33":242},"twitter:title",{"_205":32,"_33":244},"twitter:description",{"_205":222,"_33":246},"twitter:image",{"_205":225,"_33":248},"twitter:image:height",{"_205":228,"_33":250},"twitter:image:width",{"_252":253},"script:ld+json",{"_254":255,"_256":257},"@context","http://schema.org","@graph",[258],{"_254":259,"_260":261,"_262":263},"https://schema.org","@type","BreadcrumbList","itemListElement",[264,271,275,278],{"_260":265,"_266":267,"_33":268,"_269":270},"ListItem","item","https://glama.ai/","Glama","position",1,{"_260":265,"_266":272,"_33":273,"_269":274},"https://glama.ai/mcp","MCP",2,{"_260":265,"_266":276,"_33":277,"_269":123},"https://glama.ai/mcp/servers","Servers",{"_260":265,"_266":208,"_33":34,"_269":279},4,"visitorIsMaintainer","routes/_public/mcp/servers/~namespace/~slug/_pages/_explorer/blob/~branch/~/_route",{"_283":284,"_288":109,"_29":289,"_199":315,"_339":340,"_341":342},"blob",{"_205":285,"_286":287},".env\n*.pem\n*.sw[m-p]\nomnimcp.egg-info/\nomnimcp.log\n__pycache__\nruns/\nlogs/\nimages/*/\n","fileName",".gitignore","branch",{"_69":70,"_71":290,"_13":296,"_100":32,"_101":32,"_102":297,"_106":298,"_127":-5,"_16":128,"_129":130,"_131":300,"_33":34,"_177":119,"_178":-5,"_179":-5,"_180":181,"_182":-5,"_183":313,"_189":-5,"_139":34,"_190":314,"_195":-5,"_120":196,"_197":198},{"_73":291,"_89":295,"_80":91},{"_75":292,"_82":293,"_86":294},{"_77":78,"_31":79,"_80":81},{"_77":84,"_31":85,"_80":81},{"_31":88,"_80":81},[],[93,94,95,96,97,98,99],{"_35":36,"_104":105},{"_108":109,"_110":111,"_112":113,"_114":115,"_33":34,"_116":299,"_122":123,"_124":-5,"_125":126},{"_118":119,"_120":121},[301,303,305,307,309,311],{"_134":302,"_31":141,"_120":142},{"_136":137,"_33":138,"_139":140},{"_134":304,"_31":148,"_120":149},{"_136":145,"_33":146,"_139":147},{"_134":306,"_31":155,"_120":156},{"_136":152,"_33":153,"_139":154},{"_134":308,"_31":161,"_120":162},{"_136":159,"_33":115,"_139":160},{"_134":310,"_31":168,"_120":169},{"_136":165,"_33":166,"_139":167},{"_134":312,"_31":175,"_120":176},{"_136":172,"_33":173,"_139":174},{"_185":186,"_187":-5,"_188":-5},[192,193,194],[316,318,320,322,324,325,326,328,329,330,331,332,333,334,335,336,337,338],{"_202":317},"MCP Server | Glama",{"_205":319,"_33":31},"Enterprise-grade security, privacy, and features like agents, MCP, prompt templates, and more.",{"_207":321,"_209":210,"_211":212},"https://glama.ai/mcp/servers/@OpenAdaptAI/OmniMCP/blob/main/",{"_205":323,"_214":215},"MCP Server",{"_205":319,"_214":217},{"_205":219,"_214":220},{"_205":327,"_214":223},"https://glama.ai/generated-images/og?title=MCP+Server",{"_205":225,"_214":226},{"_205":228,"_214":229},{"_205":231,"_214":232},{"_205":321,"_214":234},{"_205":236,"_214":237},{"_205":239,"_33":240},{"_205":236,"_33":242},{"_205":319,"_33":244},{"_205":327,"_33":246},{"_205":225,"_33":248},{"_205":228,"_33":250},"path","","treeMembers",[343,346,348,350,352,354,356,358,360,362,364,366,368,370,371,373,375,377,379,381,383],{"_339":344,"_80":345},"demo_output","tree",{"_339":347,"_80":345},"demo_output_multistep",{"_339":349,"_80":345},"docs",{"_339":351,"_80":345},".github",{"_339":353,"_80":345},"images",{"_339":355,"_80":345},"omnimcp",{"_339":357,"_80":345},"paper",{"_339":359,"_80":345},"test_images",{"_339":361,"_80":345},"tests",{"_339":363,"_80":283},"CLAUDE.md",{"_339":365,"_80":283},"cli.py",{"_339":367,"_80":283},"demo_synthetic.py",{"_339":369,"_80":283},".env.example",{"_339":287,"_80":283},{"_339":372,"_80":283},"install.bat",{"_339":374,"_80":283},"install.sh",{"_339":376,"_80":283},"make_gif.py",{"_339":378,"_80":283},"pyproject.toml",{"_339":380,"_80":283},"README.md",{"_339":382,"_80":283},"run_omnimcp.py",{"_339":384,"_80":283},"uv.lock","actionData","errors"]

MCP directory API