Skip to main content
Glama

Surf MCP

Python 3.11+ License: MIT Code style: ruff

Surf MCP

MCP server for visual browser automation via Fara.

Overview

Surf provides browser automation through visual grounding - you describe what you see, and it clicks, types, and navigates based on that description. No CSS selectors, no DOM traversal, just natural language.

The core insight: an AI that can see the page doesn't need to parse HTML.

Related MCP server: LAM-mcp Browser Automation Server

Features

  • Visual grounding: Click/type by natural language description ("the blue Submit button")

  • Direct Fara execution: Fara decides the action, we execute it

  • Autonomous mode: Multi-step goal completion with progress tracking

  • Multi-server LM Studio: Auto-discovery and failover across GPU servers

  • Session persistence: Storage state (cookies, localStorage) round-trips through tool calls

  • Security controls: Domain allowlists, rate limiting, audit logging

Security

surf-mcp is designed for LOCAL use only via stdio transport.

Not Suitable For

  • Multi-tenant environments - trust boundary is the machine

  • Untrusted networks without SSH tunneling

  • Compliance-sensitive contexts - no formal security audit

  • Untrusted MCP clients - surf-mcp trusts its client completely

Residual Risks

  • No encryption at MCP protocol level

  • LLM responses (Fara/Gemini) executed without verification

  • Browser automation can click/type anything visible

Remote Execution

Use SSH as the transport - surf-mcp sees normal stdio:

{
  "mcpServers": {
    "surf-remote": {
      "command": "ssh",
      "args": ["-i", "~/.ssh/key", "user@gpu-box", "surf-mcp"]
    }
  }
}

See SECURITY.md for the full threat model and security controls.

How It Works

Surf uses Fara-7B (Microsoft's agentic vision model) to understand web pages:

sequenceDiagram
    participant Client as MCP Client
    participant Surf as surf-mcp
    participant PW as Playwright
    participant Fara as Fara-7B

    Client->>Surf: act("click the search button")
    activate Surf
    Surf->>PW: screenshot()
    PW-->>Surf: PNG image
    Surf->>Fara: analyze(image, goal)
    Note right of Fara: Visual grounding
    Fara-->>Surf: FaraToolCall{left_click, [624,280]}
    Surf->>PW: click(624, 280)
    PW-->>Surf: done
    deactivate Surf
    Surf-->>Client: Result + new screenshot

Supported Actions

Action

Description

left_click

Click at coordinates

double_click

Double-click at coordinates

type

Type text (optionally at coordinates)

scroll

Scroll page up/down

key

Press keyboard keys

visit_url

Navigate to URL

terminate

Task complete signal (agent mode)

wait

Wait for page to load

Installation

# Install from source
pip install -e .

# Install Playwright browsers
playwright install chromium

# Optional: Install harness dependencies
pip install -e ".[harness]"

Quick Start

As MCP Server

Add to your MCP client configuration:

{
  "mcpServers": {
    "surf": {
      "command": "surf-mcp"
    }
  }
}

Docker

# Recommended: use docker compose (reads .env automatically)
cp .env.example .env
# Edit .env with your settings
docker compose up

# Or build and run directly (note: --env-file doesn't strip quotes)
docker build -t surf-mcp .
docker run -it --rm \
  --add-host=host.docker.internal:host-gateway \
  -e LMSTUDIO_SERVERS=default=http://host.docker.internal:1234/v1 \
  surf-mcp

Fara Test Harness

Interactive UI for testing visual grounding:

cd tools/fara-harness
./run.sh    # Linux/Mac
run.bat     # Windows

See tools/fara-harness/CHEATSHEET.md for command reference.

Usage Examples

Browser Navigation with Visual Grounding

# Create session
session = await mcp.call("session_create", {
    "drivers": {
        "web": {
            "type": "browser",
            "headless": False,
            "storage_state": saved_state  # Optional: restore cookies
        }
    }
})

# Navigate to page
await mcp.call("goto", {
    "session_id": session["session_id"],
    "driver": "web",
    "location": "https://example.com"
})

# Click element by description
await mcp.call("click", {
    "session_id": session["session_id"],
    "driver": "web",
    "description": "the blue Submit button"
})

# Direct Fara execution (recommended)
await mcp.call("act", {
    "session_id": session["session_id"],
    "driver": "web",
    "goal": "type 'hello world' into the search box"
})

# Autonomous multi-step execution
await mcp.call("act_autonomous", {
    "session_id": session["session_id"],
    "driver": "web",
    "goal": "log in with username 'demo' and password 'demo123'"
})

# Destroy session and capture storage_state
result = await mcp.call("session_destroy", {"session_id": session["session_id"]})
saved_state = result["summary"]["web"]["storage_state"]

Configuration

Environment Variables

# Multi-server LM Studio (visual grounding)
LMSTUDIO_SERVERS="rtx3090=http://localhost:1234/v1,rtx8000=http://192.168.1.100:1234/v1"
FARA_MODEL_IDS="microsoft_fara-7b,fara-7b-gguf,gao-zijian/fara-7b"
FARA_MAX_FAILURES=2
FARA_PROBE_TIMEOUT=2.0

# Confidence and Agent Mode
FARA_MIN_CONFIDENCE=0.7
FARA_CONFIDENCE_RETRIES=2
FARA_MAX_AGENT_STEPS=20

# Alternative: Single OpenAI-compatible endpoint
OPENAI_API_KEY=lm-studio
OPENAI_BASE_URL=http://localhost:1234/v1
SURF_LLM_MODEL=microsoft_fara-7b

# Alternative: Gemini
GOOGLE_API_KEY=...
SURF_LLM_PROVIDER=gemini
SURF_LLM_MODEL=gemini-2.0-flash

# Browser defaults
SURF_BROWSER_HEADLESS=true
SURF_BROWSER_VIEWPORT_WIDTH=1920
SURF_BROWSER_VIEWPORT_HEIGHT=1080

# Session management
SURF_MAX_SESSIONS=10
SURF_SESSION_TIMEOUT_SECONDS=3600

Multi-Server LM Studio

Surf supports multiple LM Studio instances for redundancy:

LMSTUDIO_SERVERS="gpu1=http://localhost:1234/v1,gpu2=http://192.168.1.50:1234/v1"

Behavior:

  • Auto-discovery: Probes each server's /v1/models to find loaded Fara model

  • Prefer loaded: Prioritizes servers with Fara already in VRAM

  • Failover: Automatically retries on another server if one fails

MCP Tools

Session Lifecycle

Tool

Description

session_create

Create browser session

session_destroy

Cleanup session, returns storage_state

session_list

List active sessions

Navigation

Tool

Description

goto

Navigate to URL

current

Get current URL

back / forward

Navigate history

history

Get navigation history

Content

Tool

Description

list

Extract page links

read

Read page content

snapshot

Capture screenshot

Visual Grounding

Tool

Description

locate

Find element by description, return coordinates

click

Click element by description

type

Type into element by description

scroll

Scroll page up/down

wait

Wait for element or delay

act

Direct Fara execution - Fara decides the action

act_autonomous

Multi-step autonomous execution until task complete

Architecture

See docs/ARCHITECTURE.md for detailed architecture documentation.

Design decisions are recorded in docs/adr/.

Development

# Install dev dependencies
pip install -e ".[dev]"

# Run tests
pytest                    # All tests
pytest -m "not live"      # Skip LLM tests (for CI)
pytest -m live            # Only live LLM tests

# Type checking
mypy src/

# Linting
ruff check src/

License

MIT


© 2025 Shane V Cantwell | reflectiveattention.ai

Available Tools

18 tools
actA

Execute a goal using direct Fara visual grounding. Fara decides what action to take (click, type, scroll, etc.) based on the goal and screenshot.

Inputs: goal (required), driver (required), session_id (required).

act_autonomousB

Execute a goal autonomously with multiple steps. Fara loops until task completion (terminate action) or max steps reached.

Inputs: goal (required), driver (required), session_id (required).

backC

Navigate back in history

Inputs: driver (required), session_id (required).

clickC

Click UI element by description (browser driver only)

Inputs: driver (required), session_id (required), description (required).

currentC

Get current location for a driver

Inputs: driver (required), session_id (required).

forwardC

Navigate forward in history

Inputs: driver (required), session_id (required).

gotoC

Navigate to a location (path for filesystem, URL for browser)

Inputs: driver (required), location (required), session_id (required).

historyC

Get navigation history for a driver

Inputs: driver (required), session_id (required).

listA

List contents at current location (directory entries for filesystem, links for browser)

Inputs: driver (required), session_id (required).

locateB

Locate UI element by natural language description (browser driver only)

Inputs: driver (required), session_id (required), description (required).

readA

Read content (file for filesystem, page text for browser)

Inputs: driver (required), session_id (required), target.

scrollB

Scroll page (browser driver only)

Inputs: driver (required), session_id (required), amount, direction.

session_createB

Create a new navigator session with one or more drivers

Inputs: drivers (required).

session_destroyA

Destroy a session and cleanup all its drivers

Inputs: session_id (required).

session_listB

List all active sessions

Inputs: no named parameters.

snapshotC

Capture current state (JSON for filesystem, PNG screenshot for browser)

Inputs: driver (required), session_id (required).

typeB

Type text into UI element by description (browser driver only)

Inputs: text (required), driver (required), session_id (required), description (required), +1 more.

waitA

Wait for element or delay (browser driver only)

Inputs: driver (required), session_id (required), seconds, description.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    B3.2/5.0

    Scored across 18 tools

    Disambiguation4/5

    Most tools have distinct purposes (navigation, UI actions, session management), though 'act' and 'act_autonomous' could be confused without reading descriptions. Overall well-differentiated.

    Naming Consistency2/5

    Naming is inconsistent: some tools use single verbs ('back', 'click'), others use verb_noun ('session_create'), and some are adjectives ('current') or adverbs ('forward'). No clear pattern.

    Tool Count5/5

    18 tools is well-scoped for a navigation/UI automation server covering both browser and filesystem. Each tool serves a clear role without being excessive.

    Completeness4/5

    Covers core navigation, UI interaction, session management, and state reading. Minor gaps like missing 'refresh' or 'stop' actions, but the tool surface feels solid for its domain.

    Maintenance

    ActivityInactive
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers