surf-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@surf-mcp navigate to example.com and click the login button"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.

Surf MCP
MCP server for visual browser automation via Fara.
Overview
Surf provides browser automation through visual grounding - you describe what you see, and it clicks, types, and navigates based on that description. No CSS selectors, no DOM traversal, just natural language.
The core insight: an AI that can see the page doesn't need to parse HTML.
Related MCP server: LAM-mcp Browser Automation Server
Features
Visual grounding: Click/type by natural language description ("the blue Submit button")
Direct Fara execution: Fara decides the action, we execute it
Autonomous mode: Multi-step goal completion with progress tracking
Multi-server LM Studio: Auto-discovery and failover across GPU servers
Session persistence: Storage state (cookies, localStorage) round-trips through tool calls
Security controls: Domain allowlists, rate limiting, audit logging
Security
surf-mcp is designed for LOCAL use only via stdio transport.
Not Suitable For
Multi-tenant environments - trust boundary is the machine
Untrusted networks without SSH tunneling
Compliance-sensitive contexts - no formal security audit
Untrusted MCP clients - surf-mcp trusts its client completely
Residual Risks
No encryption at MCP protocol level
LLM responses (Fara/Gemini) executed without verification
Browser automation can click/type anything visible
Remote Execution
Use SSH as the transport - surf-mcp sees normal stdio:
{
"mcpServers": {
"surf-remote": {
"command": "ssh",
"args": ["-i", "~/.ssh/key", "user@gpu-box", "surf-mcp"]
}
}
}See SECURITY.md for the full threat model and security controls.
How It Works
Surf uses Fara-7B (Microsoft's agentic vision model) to understand web pages:
sequenceDiagram
participant Client as MCP Client
participant Surf as surf-mcp
participant PW as Playwright
participant Fara as Fara-7B
Client->>Surf: act("click the search button")
activate Surf
Surf->>PW: screenshot()
PW-->>Surf: PNG image
Surf->>Fara: analyze(image, goal)
Note right of Fara: Visual grounding
Fara-->>Surf: FaraToolCall{left_click, [624,280]}
Surf->>PW: click(624, 280)
PW-->>Surf: done
deactivate Surf
Surf-->>Client: Result + new screenshotSupported Actions
Action | Description |
| Click at coordinates |
| Double-click at coordinates |
| Type text (optionally at coordinates) |
| Scroll page up/down |
| Press keyboard keys |
| Navigate to URL |
| Task complete signal (agent mode) |
| Wait for page to load |
Installation
# Install from source
pip install -e .
# Install Playwright browsers
playwright install chromium
# Optional: Install harness dependencies
pip install -e ".[harness]"Quick Start
As MCP Server
Add to your MCP client configuration:
{
"mcpServers": {
"surf": {
"command": "surf-mcp"
}
}
}Docker
# Recommended: use docker compose (reads .env automatically)
cp .env.example .env
# Edit .env with your settings
docker compose up
# Or build and run directly (note: --env-file doesn't strip quotes)
docker build -t surf-mcp .
docker run -it --rm \
--add-host=host.docker.internal:host-gateway \
-e LMSTUDIO_SERVERS=default=http://host.docker.internal:1234/v1 \
surf-mcpFara Test Harness
Interactive UI for testing visual grounding:
cd tools/fara-harness
./run.sh # Linux/Mac
run.bat # WindowsSee tools/fara-harness/CHEATSHEET.md for command reference.
Usage Examples
Browser Navigation with Visual Grounding
# Create session
session = await mcp.call("session_create", {
"drivers": {
"web": {
"type": "browser",
"headless": False,
"storage_state": saved_state # Optional: restore cookies
}
}
})
# Navigate to page
await mcp.call("goto", {
"session_id": session["session_id"],
"driver": "web",
"location": "https://example.com"
})
# Click element by description
await mcp.call("click", {
"session_id": session["session_id"],
"driver": "web",
"description": "the blue Submit button"
})
# Direct Fara execution (recommended)
await mcp.call("act", {
"session_id": session["session_id"],
"driver": "web",
"goal": "type 'hello world' into the search box"
})
# Autonomous multi-step execution
await mcp.call("act_autonomous", {
"session_id": session["session_id"],
"driver": "web",
"goal": "log in with username 'demo' and password 'demo123'"
})
# Destroy session and capture storage_state
result = await mcp.call("session_destroy", {"session_id": session["session_id"]})
saved_state = result["summary"]["web"]["storage_state"]Configuration
Environment Variables
# Multi-server LM Studio (visual grounding)
LMSTUDIO_SERVERS="rtx3090=http://localhost:1234/v1,rtx8000=http://192.168.1.100:1234/v1"
FARA_MODEL_IDS="microsoft_fara-7b,fara-7b-gguf,gao-zijian/fara-7b"
FARA_MAX_FAILURES=2
FARA_PROBE_TIMEOUT=2.0
# Confidence and Agent Mode
FARA_MIN_CONFIDENCE=0.7
FARA_CONFIDENCE_RETRIES=2
FARA_MAX_AGENT_STEPS=20
# Alternative: Single OpenAI-compatible endpoint
OPENAI_API_KEY=lm-studio
OPENAI_BASE_URL=http://localhost:1234/v1
SURF_LLM_MODEL=microsoft_fara-7b
# Alternative: Gemini
GOOGLE_API_KEY=...
SURF_LLM_PROVIDER=gemini
SURF_LLM_MODEL=gemini-2.0-flash
# Browser defaults
SURF_BROWSER_HEADLESS=true
SURF_BROWSER_VIEWPORT_WIDTH=1920
SURF_BROWSER_VIEWPORT_HEIGHT=1080
# Session management
SURF_MAX_SESSIONS=10
SURF_SESSION_TIMEOUT_SECONDS=3600Multi-Server LM Studio
Surf supports multiple LM Studio instances for redundancy:
LMSTUDIO_SERVERS="gpu1=http://localhost:1234/v1,gpu2=http://192.168.1.50:1234/v1"Behavior:
Auto-discovery: Probes each server's
/v1/modelsto find loaded Fara modelPrefer loaded: Prioritizes servers with Fara already in VRAM
Failover: Automatically retries on another server if one fails
MCP Tools
Session Lifecycle
Tool | Description |
| Create browser session |
| Cleanup session, returns storage_state |
| List active sessions |
Navigation
Tool | Description |
| Navigate to URL |
| Get current URL |
| Navigate history |
| Get navigation history |
Content
Tool | Description |
| Extract page links |
| Read page content |
| Capture screenshot |
Visual Grounding
Tool | Description |
| Find element by description, return coordinates |
| Click element by description |
| Type into element by description |
| Scroll page up/down |
| Wait for element or delay |
| Direct Fara execution - Fara decides the action |
| Multi-step autonomous execution until task complete |
Architecture
See docs/ARCHITECTURE.md for detailed architecture documentation.
Design decisions are recorded in docs/adr/.
Development
# Install dev dependencies
pip install -e ".[dev]"
# Run tests
pytest # All tests
pytest -m "not live" # Skip LLM tests (for CI)
pytest -m live # Only live LLM tests
# Type checking
mypy src/
# Linting
ruff check src/License
MIT
© 2025 Shane V Cantwell | reflectiveattention.ai
Available Tools
18 toolsactA
Execute a goal using direct Fara visual grounding. Fara decides what action to take (click, type, scroll, etc.) based on the goal and screenshot.
Inputs: goal (required), driver (required), session_id (required).
act_autonomousB
Execute a goal autonomously with multiple steps. Fara loops until task completion (terminate action) or max steps reached.
Inputs: goal (required), driver (required), session_id (required).
backC
Navigate back in history
Inputs: driver (required), session_id (required).
clickC
Click UI element by description (browser driver only)
Inputs: driver (required), session_id (required), description (required).
currentC
Get current location for a driver
Inputs: driver (required), session_id (required).
forwardC
Navigate forward in history
Inputs: driver (required), session_id (required).
gotoC
Navigate to a location (path for filesystem, URL for browser)
Inputs: driver (required), location (required), session_id (required).
historyC
Get navigation history for a driver
Inputs: driver (required), session_id (required).
listA
List contents at current location (directory entries for filesystem, links for browser)
Inputs: driver (required), session_id (required).
locateB
Locate UI element by natural language description (browser driver only)
Inputs: driver (required), session_id (required), description (required).
readA
Read content (file for filesystem, page text for browser)
Inputs: driver (required), session_id (required), target.
scrollB
Scroll page (browser driver only)
Inputs: driver (required), session_id (required), amount, direction.
session_createB
Create a new navigator session with one or more drivers
Inputs: drivers (required).
session_destroyA
Destroy a session and cleanup all its drivers
Inputs: session_id (required).
session_listB
List all active sessions
Inputs: no named parameters.
snapshotC
Capture current state (JSON for filesystem, PNG screenshot for browser)
Inputs: driver (required), session_id (required).
typeB
Type text into UI element by description (browser driver only)
Inputs: text (required), driver (required), session_id (required), description (required), +1 more.
waitA
Wait for element or delay (browser driver only)
Inputs: driver (required), session_id (required), seconds, description.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 18 tools
Most tools have distinct purposes (navigation, UI actions, session management), though 'act' and 'act_autonomous' could be confused without reading descriptions. Overall well-differentiated.
Naming is inconsistent: some tools use single verbs ('back', 'click'), others use verb_noun ('session_create'), and some are adjectives ('current') or adverbs ('forward'). No clear pattern.
18 tools is well-scoped for a navigation/UI automation server covering both browser and filesystem. Each tool serves a clear role without being excessive.
Covers core navigation, UI interaction, session management, and state reading. Minor gaps like missing 'refresh' or 'stop' actions, but the tool surface feels solid for its domain.
Maintenance
Related MCP Connectors
AI-powered browser automation — navigate, click, fill forms, and extract data from any website.
AI-powered web automation. Navigate websites using AI agents for one page or a thousand
AI-powered web automation. Navigate websites using AI agents for one page or a thousand
Control a real browser from AI agents: navigate, click, type, screenshot, read the page.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to interact with web browsers using natural language, featuring automated browsing, form filling, vision-based element detection, and structured JSON responses for systematic browser control.63MIT
- AlicenseNot gradedqualityDmaintenanceEnables AI assistants to perform intelligent browser automation with session-based context analysis, including navigation, form filling, and content extraction through natural language.MIT
- AlicenseNot gradedqualityNot gradedmaintenanceEnables AI assistants to control a browser for web automation tasks such as navigation, typing, clicking, and taking screenshots.Apache 2.0
- AlicenseNot gradedqualityNot gradedmaintenanceEnables AI assistants to control a browser through a set of tools, allowing them to perform web automation tasks like navigation, typing, clicking, and taking screenshots.Apache 2.0