App Adapter — Control Any Software via Commands

The AI-to-App Bridge. Give any AI agent the power to control real software — and let agents contribute back to a shared ecosystem.

"zoom.send_chat|||text=Hello" → message typed into Zoom
"weibo.trends"                 → real-time trending topics  
"terminal.run|||cmd=dir"       → shell command executed

Why

Current AI agents are trapped in the browser. They can read web pages but can't do things in real apps. Codex's Record & Replay watches your screen — visual, slow, fragile.

App Adapter translates human GUI actions into machine-executable commands. No screen recording. No pixel matching. Just app_do("app.action", params).

And it's a living ecosystem. Agents don't just use adapters — they discover, learn, test, and publish them back to the community.

How It Works

┌──────────┐     ┌──────────────┐     ┌──────────┐
│ Any Agent │ ──▶ │ App Adapter  │ ──▶ │ Real App │
│ (Claude,  │     │ (MCP Server) │     │ (Zoom,   │
│  Codex,   │     │              │     │  WeChat, │
│  Cursor)  │     │ Strategies:  │     │  Excel,  │
│           │     │ • http/api   │     │  etc.)   │
│           │     │ • cdp/browser│     │          │
│           │     │ • uia/desktop│     │          │
│           │     │ • shell/cmd  │     │          │
└──────────┘     └──────────────┘     └──────────┘

Strategy Priority

Strategy	Use When	Speed	Reliability
`http`	App has a REST API	Fast	High
`cdp`	App has a web interface	Medium	Medium
`uia`	Desktop-only app, no API	Slow	Low
`shell`	CLI tool or script	Fast	High
`startfile`	Just need to open the app	Instant	High

Prefer http over visual strategies. Always. Visual automation is the last resort.

Quick Start

# One-line install (recommended)
pip install git+https://github.com/codenoob-jacky/app-adapter.git

# Or clone + dev install
git clone https://github.com/codenoob-jacky/app-adapter.git
cd app-adapter
pip install -e .

# With all optional deps (browser + desktop automation):
pip install "app-adapter[all] @ git+https://github.com/codenoob-jacky/app-adapter.git"

# Windows: scripts\install.bat   |   Mac/Linux: bash scripts/install.sh

CLI (install once, use everywhere)

app-adapter                          # Quick overview
app-adapter list                     # List all 26 apps
app-adapter scan                     # Find installable apps
app-adapter do "zoom.send_chat|||text=Hello"
app-adapter prompt --style brief     # Get contribution prompt for your agent
app-adapter server --port 8080       # Start HTTP server
app-adapter server --mcp             # Start MCP stdio server

Python

from app_adapter import app_do, app_list, app_scan, get_agent_prompt

print(app_list())
app_do("terminal.run|||cmd=echo Hello from Python")
print(get_agent_prompt("brief"))  # Inject this into your agent

HTTP Server

app-adapter server --port 8080
  -d '{"tool":"app_do","arguments":{"action":"zoom.send_chat|||text=Hello from AI"}}'

The Agent Ecosystem

This isn't just a tool — agents can discover, learn, and contribute:

# 1. Discover what's on this system
app_scan()                     # → "Found Zoom, Slack, VS Code, Spotify..."

# 2. Register a new app  
app_register("spotify|||Spotify|||https://open.spotify.com|||cdp|||entertainment")

# 3. Teach it actions
app_learn("spotify|||search|||Search tracks|||cdp|||input[data-testid='search-input']|||fill")
app_learn("spotify|||play|||Play/pause|||cdp|||button[data-testid='play-button']|||click")

# 4. Test before sharing
app_test("spotify.search|||query=Bohemian Rhapsody")

# 5. Publish to community
app_publish("spotify")

# Other agents can then discover and install it:
app_search("spotify")          # → Found in registry
app_install("spotify")         # → Downloaded and merged
app_do("spotify.play")

Full API

Function	Description	Example
`app_do(spec)`	Execute an action	`app_do("zoom.send_chat\|\|\|text=Hi")`
`app_list(category?)`	List all apps & actions	`app_list("social")`
`app_register(spec)`	Register new application	`app_register("slack\|\|\|Slack\|\|\|https://slack.com\|\|\|cdp\|\|\|communication")`
`app_learn(spec)`	Teach new action to app	`app_learn("slack\|\|\|send\|\|\|Send msg\|\|\|cdp\|\|\|selector\|\|\|fill_then_click")`
`app_scan()`	Discover installed apps	`app_scan()`
`app_search(query?)`	Search community registry	`app_search("discord")`
`app_install(name)`	Install from registry	`app_install("spotify")`
`app_publish(name)`	Publish to registry	`app_publish("my_slack_adapter")`
`app_export(name?)`	Export adapter as JSON	`app_export("zoom")`
`app_import(json)`	Import adapter from JSON	`app_import('{"app":{...}}')`
`app_test(spec)`	Test action with report	`app_test("zoom.send_chat\|\|\|text=Test")`

Built-in Apps (26 apps, 80+ actions)

Category	Apps
Communication	Zoom, Slack, Discord, Teams, Outlook
Social Media	WeChat MP, Weibo, Zhihu, Bilibili, Xiaohongshu, Douyin, X/Twitter
Productivity	Excel, Word, PowerPoint, PDF, Notion, GitHub
Development	VS Code, Terminal, Postman
Design	Figma, Canva
Finance	TradingView
Utility	Browser (navigate, search, screenshot)

Make Your Agent a Contributor

The killer feature: agents that voluntarily improve the ecosystem.

# Get the contribution prompt
app-adapter prompt --style full

# Inject it into your agent's system message.
# Now your agent will:
#   - Discover unregistered apps with app_scan()
#   - Check the registry with app_search()
#   - Register + teach + test + publish autonomously

Every contribution helps every AI agent worldwide. The registry grows exponentially.

Connect to Your Agent

Claude Desktop

{
  "mcpServers": {
    "app-adapter": {
      "command": "app-adapter",
      "args": ["server", "--mcp"]
    }
  }
}

Codex

# codex.yaml or .codex/config.json
mcp:
  app-adapter:
    type: stdio
    command: app-adapter
    args: [server, --mcp]

Cursor

// .cursor/mcp.json
{
  "mcpServers": {
    "app-adapter": {
      "command": "app-adapter",
      "args": ["server", "--mcp"]
    }
  }
}

LangChain / OpenAI Agents SDK

from app_adapter import app_do, get_agent_prompt
# Inject the prompt + wrap tools. See docs/INTEGRATION.md for full examples.

Full integration guide: docs/INTEGRATION.md

Community Registry

The registry/ folder in this repo is the community adapter registry. Anyone (human or agent) can contribute:

Create your adapter with app_register() + app_learn()
Test with app_test()
Export with app_export("your_app")
Save the JSON to registry/your_app.json
Open a PR to this repo

Agents discover these via app_search() → app_install().

Current Community Adapters

Adapter	Actions
Slack	open, send_message, search, jump_to_channel
Discord	open, send_message, mute_toggle
Spotify	open, search, play, next_track, get_current

Be the next contributor. → registry/

Design Philosophy

Humans use GUI. Machines need commands. The adapter is the translation layer. Don't make AI learn to see — teach apps to listen. Then let agents share what they've learned.

License

MIT

App Adapter