browsercontrol
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@browsercontrolgo to github.com and star the browsercontrol repo"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
✨ What it is
BrowserControl is an MCP server that gives your AI agent a real browser it can see, click, type, and debug — using a vision-first approach based on the Set of Marks (SoM) pattern.
Instead of fragile CSS selectors, XPath, or DOM trees, the agent sees an annotated screenshot with numbered red boxes over every interactive element, then just calls:
click(5) → clicks the 5th element
type_text(3, "hello world") → types into the 3rd element
upload_file(7, "/path/to/file.pdf") → uploads a file via the native browser inputNo selectors to guess. No selectors to break when the page changes. Just point at numbers.
Related MCP server: @playwright/mcp
🖼️ How it looks
Every tool that interacts with the page returns an annotated screenshot plus a textual element map:
┌─────────────────────────────────────────────────┐
│ GitHub [Sign in] │
├─────────────────────────────────────────────────┤
│ │
│ ┌──────────────────────┐ ┌──────────────┐ │
│ │ 1 🔍 Search… │ │ 2 Pulls │ │
│ └──────────────────────┘ │ 3 Issues │ │
│ │ 4 Codespace │ │
│ ┌───────────────────────┐ └──────────────┘ │
│ │ 5 Sign in │ │
│ └───────────────────────┘ │
└─────────────────────────────────────────────────┘
Found 5 interactive elements:
[1] input - Search or jump to...
[2] a - Pulls
[3] a - Issues
[4] a - Codespaces
[5] button - Sign inThe model receives the image and the list, so it can either reason over the pixels or read the text — whichever is cheaper.
🚀 Quick start
1 · Install
# pip
pip install browsercontrol
# uv (recommended — much faster)
uv add browsercontrolNo extra setup needed. Chromium auto-installs on first run. If the auto-install fails for any reason, run
python -m playwright install chromiumonce and you're set.
2 · Run the server
# As a CLI
browsercontrol
# As a module
python -m browsercontrol
# With FastMCP (for `fastmcp dev` / inspector UI)
fastmcp run browsercontrol.server:mcp3 · Connect it to your AI
Pick your client — full configs in Connect your AI below.
// Claude Desktop — claude_desktop_config.json
{
"mcpServers": {
"browsercontrol": {
"command": "browsercontrol"
}
}
}Restart Claude Desktop, then try:
"Open Hacker News and tell me the top story."
That's it — your agent now has a browser.
🧠 Why BrowserControl?
A dozen MCP servers can open a web page. Here's what BrowserControl does that most don't:
🟢 Set of Marks (SoM) — Numbered red boxes on every interactive element. The agent picks a number. No selectors, ever. | 🟢 Shadow DOM + iframe aware — Recursively descends into open shadow roots and same-origin iframes (with proper coordinate offsets). Modern web apps just work. |
🟢 True persistent sessions — Uses | 🟢 Built-in devtools — Console logs, network requests (with timing), JS errors, page performance, element inspection, computed styles. No second tool needed. |
🟢 Native file uploads — Uses Playwright's | 🟢 Multi-tab orchestration — Open, switch, close, list tabs. Clicking a link that opens a new tab doesn't lose the agent. |
🟢 Smart localhost routing — Auto-falls-back | 🟢 100% local & private — No LLM API key, no cloud, no telemetry, no usage cap. Your browsing stays on your machine. |
🟢 Zero marginal cost — Runs on your hardware, no per-action fees. | 🟢 Cross-platform — CI tests on Ubuntu, Windows, and macOS. |
🧩 Connect your AI
BrowserControl speaks MCP stdio, so it works with anything that speaks MCP.
Add to claude_desktop_config.json:
macOS:
~/Library/Application Support/Claude/claude_desktop_config.jsonLinux:
~/.config/Claude/claude_desktop_config.jsonWindows:
%APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"browsercontrol": {
"command": "browsercontrol"
}
}
}Configure MCP servers in Gemini's settings, or via environment:
export MCP_SERVERS='{"browsercontrol": {"command": "browsercontrol"}}'For Google AI Studio, configure in the MCP settings panel.
Install Cline
Open Cline settings → MCP Servers → Add:
{
"browsercontrol": {
"command": "browsercontrol"
}
}Add to ~/.continue/config.json:
{
"mcpServers": [
{ "name": "browsercontrol", "command": "browsercontrol" }
]
}Cursor Settings → Features → Model Context Protocol → Add:
{ "browsercontrol": { "command": "browsercontrol" } }Add to ~/.config/zed/settings.json:
{
"context_servers": {
"browsercontrol": {
"command": { "path": "browsercontrol" }
}
}
}import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
async def main() -> None:
params = StdioServerParameters(command="browsercontrol", args=[])
async with stdio_client(params) as (read, write):
async with ClientSession(read, write) as session:
await session.initialize()
tools = await session.list_tools()
result = await session.call_tool(
"navigate_to", {"url": "https://news.ycombinator.com"}
)
asyncio.run(main()){
"mcpServers": {
"browsercontrol": { "command": "uvx", "args": ["browsercontrol"] }
}
}{
"mcpServers": {
"browsercontrol": { "command": "pipx", "args": ["run", "browsercontrol"] }
}
}Pass environment variables to tune the browser:
{
"mcpServers": {
"browsercontrol": {
"command": "browsercontrol",
"env": {
"BROWSER_HEADLESS": "false",
"BROWSER_VIEWPORT_WIDTH": "1920",
"BROWSER_VIEWPORT_HEIGHT": "1080",
"LOG_LEVEL": "DEBUG"
}
}
}
}See Configuration for the full list.
🛠️ Tools
Every action returns an annotated screenshot + element map (when relevant), so the model always has the latest visual context.
Navigation
Tool | Description |
| Open a URL. Auto-falls back to |
| Standard history controls. |
|
|
Interaction
Tool | Description |
| Click by SoM number. Resolves the actual DOM element at the coordinates first, so overlays don't fool it. |
| Click raw coordinates. |
| Uses |
| Any keyboard key: |
| Hover for tooltips/menus. |
| Scroll the element into view. |
| Sleep — for animations or lazy-loaded content. |
Tabs
Tool | Description |
| Open a new tab, optionally navigated to a URL. |
| Switch by 0-based index. |
| Close a tab. Auto-falls-back to another open page. |
| List all tabs with title, URL, active marker. |
Forms
Tool | Description |
| Pick from a |
| Toggle a checkbox. |
| Native file upload via |
Content
Tool | Description |
| Whole page as Markdown (script/style stripped, 30 KB cap). |
| Read text from a specific element. |
| Current URL + title. |
| Run JS, return serialized result + screenshot. |
| Annotated viewport, clean viewport, or clean full-page. |
DevTools
Tool | Description |
| Last 50 captured console messages, with level + source location. |
| Method, URL, status, duration (ms). Requests paired by identity (no URL-collision bugs). |
| Uncaught JS exceptions with stack traces. |
| Eval JS with structured error handling; returns JSON-stringified result + screenshot. |
| Computed styles, dimensions, attributes, classes, href, src, value. |
| TTFB, FCP, DOMContentLoaded, load time, resource count, JS heap. |
| Full cookie control. |
| Targeted or total cookie wipe. |
| Change viewport on the fly (great for responsive testing). |
Recording
Tool | Description |
| Begin a Playwright trace (screenshots + DOM snapshots + sources). |
| Save the trace to |
| Save PNG + HTML + URL triplet to |
| List recent recordings and snapshots. |
View recordings with:
npx playwright show-trace ~/.browsercontrol/recordings/<name>.zip
⚙️ Configuration
All settings are environment variables. Sensible defaults out of the box.
Variable | Default | Description |
|
| Run without a visible window. Set |
|
| Viewport width in pixels. |
|
| Viewport height in pixels. |
|
| Navigation timeout (ms). |
|
| Browser profile dir (cookies, history, extensions persist here). |
| — | Path to a |
|
|
|
# Headful + phone-sized viewport for mobile testing
BROWSER_HEADLESS=false BROWSER_VIEWPORT_WIDTH=375 BROWSER_VIEWPORT_HEIGHT=812 browsercontrol
# Verbose logs for debugging the MCP server itself
LOG_LEVEL=DEBUG browsercontrol📚 Examples
🔍 Web research
"Find the Wikipedia article on Python and tell me who created it."
→ navigate_to("https://wikipedia.org")
→ type_text(2, "Python programming language")
→ press_key("Enter")
→ get_page_content()
→ "Python was created by Guido van Rossum, first released in 1991."🐛 Debug a web app
"Check localhost:3000 for any errors and tell me what's broken."
→ navigate_to("http://localhost:3000") # auto-fallback to 127.0.0.1 if needed
→ get_console_logs()
→ "2 errors:
[ERROR] Uncaught TypeError: Cannot read property 'map' of undefined (app.js:42)
[ERROR] Failed to load resource: 404 /api/users"
→ get_network_requests()
→ "GET /api/users -> 404 (12ms) ← check your API route"🧪 Record a test run
"Walk through the login flow on staging.example.com while recording."
→ start_recording("login_flow")
→ navigate_to("https://staging.example.com/login")
→ type_text(3, "user@example.com")
→ type_text(4, "hunter2")
→ click(5) # Sign In button
→ take_snapshot("after_login")
→ stop_recording()
→ "Saved ~/.browsercontrol/recordings/login_flow.zip"📝 Fill out a form
"Fill out the contact form at example.com/contact with a polite message."
→ navigate_to("https://example.com/contact")
→ type_text(2, "Alex Doe")
→ type_text(3, "alex@example.com")
→ type_text(4, "Hi! Just saying hello.")
→ click(5) # Submit🗂️ Multi-tab research
"Open GitHub in one tab and Hacker News in another, then tell me the top repo and the top story."
→ create_tab("https://github.com/trending")
→ create_tab("https://news.ycombinator.com")
→ list_tabs() # → returns both tabs with indices
→ switch_tab(0) # focus GitHub
→ get_page_content()
→ switch_tab(1) # focus HN
→ get_page_content()📂 Upload a file
"Upload my resume to the job application form."
→ navigate_to("https://jobs.example.com/apply")
→ upload_file(8, "/Users/me/Documents/resume.pdf")📱 Mobile responsive check
"Switch to iPhone 14 viewport and screenshot the homepage."
→ set_viewport(390, 844)
→ navigate_to("https://my-site.com")
→ screenshot(annotate=False, full_page=True)🏗️ Architecture
┌─────────────────┐ ┌────────────────────┐ ┌─────────────┐
│ AI Agent │────▶│ BrowserControl │────▶│ Chromium │
│ (Claude, Gemini,│◀────│ MCP Server │◀────│ (Playwright)│
│ GPT, local…) │ │ │ │ │
└─────────────────┘ └────────────────────┘ └─────────────┘
│ │ │
│ call_tool("click",5) │ page.click(handle) │
│ ◀─────────────────────│ ◀─────────────────────│
│ annotated PNG + │ screenshot.png │
│ element map │ + SoM overlay │Action loop
Agent calls a tool — e.g.
click(5).Server resolves the element — Looks up element 5 from the last SoM pass, then calls
document.elementFromPoint(x, y)to get the actual DOM element (handles overlays, sticky headers, and shadow boundaries).Browser executes — Playwright drives the real Chromium.
Server re-screenshots — New screenshot, fresh element detection (with shadow DOM + iframe recursion), numbered boxes drawn with Pillow.
Annotated result returns — Both image and textual map are sent back so the model can pick its preferred representation.
Why SoM over selectors?
No DOM drift — If the site gets redesigned, the numbered elements still point at the right pixels.
No hallucinated selectors — The model never invents
div.flex-container > button.btn-primary:nth-child(3).Tiny tokens — A list of
{"id": 7, "tag": "button", "text": "Submit"}is far cheaper than a full DOM tree.Vision + text — The model can use either or both. Vision for visual reasoning, text for fast selection.
🆚 How it compares
A few notes, kept honest:
Capability | BrowserControl | Playwright MCP | Stagehand / Browser-Use | AgentQL |
Set of Marks (numbered boxes) | ✅ | ❌ text-tree only | ❌ | ❌ |
Shadow DOM traversal | ✅ recursive | ⚠️ partial | ⚠️ varies | ⚠️ varies |
Same-origin iframe support | ✅ with offset coords | ⚠️ basic | ⚠️ varies | ❌ |
Native | ✅ | ⚠️ manual | ❌ | ❌ |
Built-in devtools (console/network/errors/perf) | ✅ 11 tools | ⚠️ partial | ❌ | ❌ |
Multi-tab control | ✅ | ⚠️ implicit | ⚠️ varies | ❌ |
True persistent profile | ✅ | ⚠️ | ❌ | ❌ |
No external LLM API required | ✅ | ✅ | ❌ needs API key | ❌ cloud |
Runs 100% locally / offline | ✅ | ✅ | ❌ | ❌ |
Cost per 1k actions | $0 | $0 | API spend | API spend |
⚖️ We try to be fair: Playwright MCP is excellent if you're fine with the accessibility tree approach. Stagehand and Browser-Use are great if you specifically want an LLM in the loop. BrowserControl's niche is vision-first, fully local, zero-LLM-API browser control with built-in devtools.
🧪 Troubleshooting
BrowserControl defaults to headless. If you want to see the browser, run on a desktop session:
xvfb-run browsercontrol # Linux without a display
BROWSER_HEADLESS=false browsercontrol # with a displayBrowserControl auto-installs Chromium on first run. If that fails:
python -m playwright install chromium --with-depsBrowserControl auto-tries 127.0.0.1 when localhost fails. If it still fails:
Confirm the dev server is running (
curl http://localhost:3000from the shell).If your dev server binds only to
127.0.0.1, navigate directly:navigate_to("http://127.0.0.1:3000").Corporate proxies can interfere — BrowserControl already passes
--proxy-bypass-list=<-loopback>to Chromium.
Check that the user data directory is writable:
ls -la ~/.browsercontrol/Override with BROWSER_USER_DATA_DIR=/some/writable/path.
pkill -f browsercontrol
browsercontrolRecordings are Playwright trace zips. Open them with:
npx playwright show-trace ~/.browsercontrol/recordings/session_20260228.zipElement IDs are only valid until the next screenshot. After any navigation, click, or state change, call the tool again and read the new IDs from the latest screenshot/map. Tools re-screenshot automatically — your agent just needs to look at the response.
📁 Project structure
browsercontrol/
├── __init__.py # Public exports (`mcp`)
├── __main__.py # `python -m browsercontrol` entry point
├── server.py # FastMCP server definition + tool registration
├── browser.py # BrowserManager — Playwright lifecycle + SoM renderer
├── config.py # Env-var configuration
└── tools/
├── navigation.py # navigate_to, go_back, go_forward, refresh, scroll
├── interaction.py # click, click_at, type_text, press_key, hover, scroll_to_element, wait
├── forms.py # select_option, check_checkbox, upload_file
├── content.py # get_page_content, get_text, get_page_info, run_javascript, screenshot
├── devtools.py # console, network, errors, inspect, perf, cookies, viewport
├── recording.py # start/stop_recording, take_snapshot, list_recordings
└── tabs.py # create_tab, switch_tab, close_tab, list_tabs
tests/ # 1,664 lines of pytest tests (mocked Playwright)
├── conftest.py # Shared fixtures: mock_page, mock_context, mock_browser_manager…
└── test_*.py # One file per module🛠️ Development
git clone https://github.com/adityasasidhar/browsercontrol
cd browsercontrol
# Install deps + Playwright + Chromium
uv sync
uv run playwright install chromium --with-deps
# Run the server in dev mode (FastMCP inspector)
uv run fastmcp dev browsercontrol/server.py
# Run tests
uv run pytest
# Lint, format, security checks
uv run ruff check .
uv run ruff format .
uv run pre-commit run --all-filesCI runs the full suite on Ubuntu, Windows, and macOS with ruff, mypy --strict, bandit, and pytest.
🤝 Contributing
Contributions welcome! See CONTRIBUTING.md for the workflow and CODE_QUALITY.md for the toolchain.
Good first contributions:
Firefox / WebKit support
DOM diffing (detect changes between snapshots)
Accessibility audit tools
Mobile emulation presets (iPhone, Pixel, iPad)
Cookie import/export (Netscape format)
--record-videoMP4 output (in addition to traces)Network request blocking / mocking
When adding a tool, please add a matching tests/test_<your_tool>.py covering the happy path and error path.
📄 License
MIT — do whatever you want, just keep the copyright.
🙏 Acknowledgments
Built on FastMCP and Playwright.
The Set of Marks idea comes from the Set-of-Marks prompting literature for visual tool use — adapted here as the agent's primary interface rather than a fallback.
Thanks to the MCP community for making AI-tool integration this pleasant.
This server cannot be installed
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Latest Blog Posts
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/adityasasidhar/browsercontrol'
If you have feedback or need assistance with the MCP directory API, please join our Discord server