Skip to main content
Glama
camofox-browser MCP server (stage2)

camofox-browser MCP server (stage2)

by jfjensen

Local LLM agent that reads AND acts on web pages (camofox-browser + MCP)

Code for Part 6 of the Build Your Own Claude Code series: Acting on Web Pages with a Small Local LLM — Click, Type, Submit.

The series so far:

  • Part 1: the agent (CLI, tools, skills, history, compaction)

  • Part 2: the browser UI (FastAPI + WebSockets)

  • Part 3: web search (SearXNG via MCP)

  • Part 4: web browsing (camofox-browser via MCP) + composing with Part 3's search

  • Part 5: reading whole pages without truncation — chunked extract and summarize, plus a family of small single-purpose reader tools (fetch_snippet, fetch_urls, fetch_structure)

  • Part 6 (this repo): interaction — a persistent-tab lifecycle (open_tab / read_tab / close_tab) plus action tools (click, type_into, press_key, select_option, or a single interact tool, depending on config) so the agent can fill in a search box, click Next on a paginated list, or submit a form, not just read what is already on the page.

The problem this part solves: Part 5's reader tools are URL-first — each one opens a fresh tab, snapshots it, and closes it, so there is nothing to act on. Real tasks (search a site, page through results, fill out and submit a form) need a tab that stays open across calls and element references that survive long enough to click or type into. camofox keeps a tab alive until told to close it; this part adds the MCP plumbing on top — open a persistent tab, read its current state to find an element's [eN] ref, act on that ref, then re-read before the next action because every action renumbers the refs.

Install

git clone https://github.com/jfjensen/local-LLM-agent-mcp-interaction.git
cd local-LLM-agent-mcp-interaction
python -m venv .venv
# Linux / macOS:
source .venv/bin/activate
# Windows PowerShell:
.\.venv\Scripts\Activate.ps1
pip install -e .

You will also need:

  • Docker to run camofox-browser and SearXNG (Stage 1 brings up both).

  • Ollama with a tool-capable model. The default is qwen3.5:9b:

    ollama pull qwen3.5:9b

SearXNG comes pre-configured (the settings file is mounted into the container), so the JSON API works without editing anything.

Related MCP server: browserpilot

Configuration

All tunable settings live in config.toml at the repo root:

  • model name, temperature, thinking;

  • SearXNG and camofox URLs;

  • chunking and reading budgets (the page snapshot size is capped by camofox itself, not here — see the note in config.toml's [browser]);

  • [acting] — the new section for Part 6: action_tool_style picks between four intent-named action tools (click, type_into, press_key, select_option) and a single interact(action=...) tool; wait_timeout_ms / wait_for_network control how long the server waits for a page to settle after an action; action_result_snippet optionally attaches a head-snippet of the post-action page to the action's result;

  • log verbosity (logging.level = "DEBUG" shows each tool call, its arguments, and a preview of what came back).

To see what is currently active:

mcp-config-show

The loader looks for config.toml in the current working directory first, then falls back to the one bundled with the repo.

How to run

Stage

What it is

How to run

1

Standing up camofox-browser and SearXNG via Docker

cd stage1 && docker compose up -d (see stage1/README.md for the camofox image build step)

2

The full agent: search plus the browser reader and action tools

mcp-agent-stage2

The browser MCP server (mcp-browser-stage2) exposes three groups of tools.

URL-first readers (from Part 5; each opens a one-shot tab, reads it, closes it):

  • fetch_snippet — the head of a page, for a quick look.

  • fetch_urls — the page's links as {text, url} pairs, absolute.

  • fetch_structure — the heading outline.

  • extract — named fields as JSON, via chunk-and-merge over the whole page (no truncation).

  • summarize — a prose summary, built by combining per-chunk summaries (map-reduce by default, or refine).

Persistent-tab lifecycle and readers (new in Part 6; for when you need to act on a page, not just read it):

  • open_tab — opens a page in a tab that stays open and returns a tab_id.

  • read_tab — reads the CURRENT state of an open tab: mode="snippet" (default, shows the [eN] element refs you need for an action), "urls", or "structure".

  • summarize_tab / extract_tab — the same whole-page, no-truncation summarize / extract logic, applied to the live tab instead of a fresh URL fetch (use these after navigating or acting your way to a page that may not be re-fetchable by URL, such as a form's POST target).

  • close_tab — closes one tab; close_all_tabs — tears down every tab in the session (the agent calls this automatically on shutdown).

Action tools (new in Part 6; act on an [eN] ref from the latest read_tab). The surface shown to the model is picked by config.toml's [acting].action_tool_style:

  • "separate" (default) — four intent-named tools: click, type_into (optionally submit=True to press Enter afterwards), press_key, select_option (for native <select> dropdowns).

  • "interact" — one interact(tab_id, action, ...) tool with an action enum (click / type / press / select), mirroring camofox's own /act dispatcher.

Every action re-snapshots the tab and reports the resulting url and a fresh refsCount: refs are renumbered after every action, so the rule is always read_tab → act → read_tab again before the next action.

Plus the search server (mcp-search-part3), a copy of Part 3's SearXNG MCP server, so the repo is self-contained.

Probing MCP servers with inspect_any.py

inspect_any.py makes one tool call per process, which is fine for the URL-first readers but not for interaction (each process would get its own tab). For the lifecycle/action tools, use test_act_flow.py instead (see below).

# List a server's tools:
python inspect_any.py mcp_browser_02.main

# Call a reader tool:
python inspect_any.py mcp_browser_02.main fetch_snippet --kv url=https://example.com
python inspect_any.py mcp_browser_02.main fetch_urls --kv url=https://example.com

# Call extract with a JSON Schema (use --args or --args-file for nested args):
python inspect_any.py mcp_browser_02.main extract --args-file extract_args.json

Where extract_args.json might look like:

{
  "url": "https://en.wikipedia.org/wiki/Vleteren",
  "schema": {
    "type": "object",
    "properties": {
      "mayor": {"type": "string", "description": "The current mayor, from the infobox"},
      "postal_code": {"type": "string", "description": "The postal code"},
      "population": {"type": "string", "description": "The total population"}
    }
  }
}

Exercising the action tools with test_act_flow.py

Unlike inspect_any.py, this opens a single MCP stdio session and runs the whole interaction loop in one process, so the persistent tab stays warm across calls:

python test_act_flow.py
# or against a different target:
python test_act_flow.py https://en.wikipedia.org/wiki/Main_Page

It runs open_tab → read_tab → type_into(..., submit=True) → read_tab → a deliberate reuse of the now-stale ref (to demonstrate why refs must be re-read after every action) → close_tab. The default target is DuckDuckGo Lite, a tiny HTML page whose search box makes a clean before/after-submit comparison.

Probing camofox directly with probe_camofox.py

A lower-level probe that talks to camofox's HTTP API directly (no MCP layer), useful when something in the action tools misbehaves and you need to rule out the MCP server as the cause. It first checks which "open a tab" endpoint your camofox build actually answers to, then drives open → snapshot → type → submit → snapshot → a stale-ref check → close against a real public form:

python probe_camofox.py
# or against a different form:
python probe_camofox.py https://html.duckduckgo.com/html/

Notes

  • The agent creates a history/ folder in the current working directory on first run.

  • The repo ships a copy of Part 3's SearXNG MCP server as mcp-search-part3, byte-for-byte the same, so you do not need to install Part 3.

  • The agent calls close_all_tabs automatically on shutdown, since the model does not reliably close tabs itself; persistent tabs would otherwise accumulate in camofox across sessions.

License

MIT © 2026 Jes Fink-Jensen. See LICENSE for details.

Troubleshooting

  • The Docker build fails with "dist not found". Use Dockerfile.ci instead of the default Dockerfile. See stage1/README.md.

  • FileNotFoundError: [WinError 2] when the agent spawns an MCP server. Your venv is not activated. Activate it so console scripts are on PATH.

  • Script exits silently on Windows. The default asyncio event loop on Windows cannot spawn subprocesses. The agent sets WindowsProactorEventLoopPolicy at startup; do the same if you copy the code elsewhere.

  • extract returns nulls on a page you know has the data. With the chunked extract this should be rare, but a very large page makes many model calls. Lower chunking.chunk_chars for smaller, more numerous chunks, or raise it for fewer, larger ones.

  • fetch_structure returns few or no headings. Some pages (short stubs, pages whose content lives in tables or infoboxes) have a thin heading outline. Use summarize or extract for those.

  • camofox returns a small or empty snapshot. Some pages need more than the default 1.5-second settle. Bump browser.settle_seconds.

  • An action fails with a stale-ref or "page may have changed" error. Every action renumbers the tab's [eN] refs. Call read_tab again to get fresh refs before retrying; never reuse a ref from before an action.

  • An action on a result page seems to silently target the wrong element, or the page doesn't look like what you expected. Don't re-fetch the result by URL — use read_tab / summarize_tab / extract_tab on the tab instead. Some result pages (a form's POST target, for example) only exist inside the tab and are not independently fetchable.

  • select_option fails even though the dropdown is visible. camofox only exposes a ref for <select>/combobox elements that its accessibility snapshot assigns one to; if read_tab shows no [eN] ref on the dropdown, it cannot be selected on that build. Use click on the visible options instead if the site renders them as a custom (non-<select>) widget.

  • probe_camofox.py can't find an open endpoint. It tries the canonical POST /tabs route first, falling back to the older POST /tabs/open shim. If neither returns a tabId, your camofox build's API has likely changed; paste the script's output to see which step failed.

Available Tools

15 tools
clickA

Click an interactive element in an open tab, by its ref.

The ref is an [eN] marker (like e42) from the tab's latest read_tab snapshot. Use this for links, buttons, checkboxes, and to submit a form that has a submit button. To submit a form that has no button, type into the field with submit=True, or press_key(tab_id, "Enter").

Args: tab_id: The handle from open_tab. ref: The element ref to click, e.g. "e42".

Returns: JSON with the resulting url and fresh ref count. Refs renumber after the click, so call read_tab again before your next action.

Inputs: tab_id (required), ref. Returns: result

close_all_tabsA

Close ALL persistent browser tabs at once (tears down the browsing session). You normally do not need this; the agent calls it automatically on shutdown so tabs do not leak between sessions. Opening a tab afterwards starts a fresh session.

Returns: A short confirmation.

Inputs: no named parameters. Returns: result

close_tabA

Close a persistent tab opened with open_tab. Call this when you have finished interacting with a page, to free the browser tab.

Args: tab_id: The handle returned by open_tab.

Returns: A short confirmation.

Inputs: tab_id (required). Returns: result

extractA

Fetch a webpage and extract structured data from it according to a JSON Schema. The MCP server fetches the page, then asks a local Ollama model to populate the schema from the page contents. So the caller gets clean JSON back, without having to read or parse the snapshot itself.

Use this tool when the user asks for specific fields that you can name in advance, especially on pages with structured content (WHOIS lookups, product pages, GitHub repos, recipes, tables of data). Arrays and nested objects are supported, since the work is done by an LLM and not by a constrained server-side extractor.

Prefer fetch when the user asks an open-ended question or wants a free-form summary.

Args: url: The full URL to extract from. schema: A JSON Schema describing the fields to extract. Property descriptions guide the extraction model in finding the right page content. Example: { "type": "object", "properties": { "registrar": {"type": "string", "description": "Domain registrar name"}, "expiration_date": {"type": "string", "description": "Registrar Registration Expiration Date"}, "nameservers": {"type": "array", "items": {"type": "string"}, "description": "Name Server entries"} } } user_id: Optional, same semantics as for fetch.

Returns: A JSON string with the extracted fields. If a field cannot be found, it is set to null.

Inputs: url (required), schema (required), user_id. Returns: result

extract_tabA

Extract structured data from the CURRENT page in an open tab according to a JSON Schema, using the same whole-page chunk-and-merge extractor as extract, but reading the live tab instead of opening a URL. Use this for named fields on a page you have navigated or interacted your way to.

Args: tab_id: The handle from open_tab. schema: A JSON Schema describing the fields to extract (same shape as for extract).

Returns: A JSON string with the extracted fields; missing fields are null.

Inputs: tab_id (required), schema (required). Returns: result

fetch_snippetA

Fetch a webpage and return a short snippet from the top of it.

This is the quick-look tool. It returns the head of the page's accessibility-tree snapshot, which is usually enough to tell what the page is and whether it is the right one. If the page is longer than the snippet, the result ends with a marker telling you to use summarize for the full content or extract for specific fields.

Use this when you want a fast look at a page, or to confirm a URL is what you expect before doing more with it. For a full understanding of a long page, prefer summarize; for named fields, prefer extract.

Args: url: The full URL to fetch (must include http:// or https://). user_id: Optional. If set, camofox reuses a browser context across calls (faster). Default opens a one-shot tab.

Returns: The head of the page snapshot, with a marker if it was longer.

Inputs: url (required), user_id. Returns: result

fetch_structureA

Fetch a webpage and return its heading outline: the page's headings with their levels, in order, like a table of contents.

Use this to see how a page is organized and whether the section you want is on it, before deciding what to read in full. Note that some pages (short stubs, pages whose content sits in tables or infoboxes rather than under headings) have a thin outline; in that case prefer summarize or extract.

Args: url: The full URL to outline. user_id: Optional, same semantics as for fetch_snippet.

Returns: A plain-text outline, one heading per line, indented by level.

Inputs: url (required), user_id. Returns: result

fetch_urlsA

Fetch a webpage and return the links on it as a list of {text, url} pairs, where text is the link's visible label and url is the absolute target.

Use this when you need to navigate from a page: to find which link to follow next, or to see what a page links out to. The list is deduplicated and the URLs are made absolute, so you can pass any of them straight to another tool.

Args: url: The full URL to read links from. user_id: Optional, same semantics as for fetch_snippet.

Returns: A JSON array of {"text": ..., "url": ...} objects.

Inputs: url (required), user_id. Returns: result

open_tabA

Open a webpage in a persistent browser tab and return a tab_id handle.

Use this ONLY when you need to interact with the page (click a link or button, type into a field, submit a form). For plain reading, prefer the URL-first tools (fetch_snippet, summarize, extract, fetch_urls, fetch_structure), which do not need a tab handle.

The tab stays open across calls so you can act on it. After opening, call read_tab(tab_id) to see the page and its interactive element refs. Close the tab with close_tab(tab_id) when you are finished.

Args: url: The full URL to open (must include http:// or https://).

Returns: JSON with the tab_id, the settled url, and the element ref count.

Inputs: url (required). Returns: result

press_keyA

Press a single key in an open tab (e.g. "Enter" to submit a focused form, "Tab" to move between fields, "Escape" to dismiss a dialog).

Args: tab_id: The handle from open_tab. key: The key name, e.g. "Enter".

Returns: JSON with the resulting url and fresh ref count.

Inputs: tab_id (required), key (required). Returns: result

read_tabA

Read the CURRENT state of an open tab (one you opened with open_tab).

Always read_tab right before an action, because the element refs you need (the [eN] markers) come from the latest snapshot and are renumbered after every action and navigation. A ref from an earlier read is stale.

Args: tab_id: The handle returned by open_tab. mode: What to return: "snippet" - the head of the page snapshot, including the [eN] refs on interactive elements (the default; use this to find the ref you want to click or type into). "urls" - the page's links as {text, url} pairs. "structure" - the page's heading outline.

Returns: The requested view of the current tab.

Inputs: tab_id (required), mode. Returns: result

select_optionA

Choose an option in a dropdown (a native / combobox) in an open tab, by the dropdown's ref and the option's label or value.

This is ONLY for dropdowns. For radio buttons, checkboxes, links and ordinary buttons, use click instead. A radio "Large" or a checkbox "Onion" is clicked, not selected.

Args: tab_id: The handle from open_tab. ref: The ref of the dropdown element (the combobox/select), from the latest read_tab. Note: if the dropdown has no [eN] ref in the snapshot, it cannot be selected. value: The option to choose, by its visible label or value (e.g. "Belgium (nl)").

Returns: JSON with the resulting url and fresh ref count.

Inputs: tab_id (required), value (required), ref. Returns: result

summarizeA

Fetch a webpage and return a concise summary of it. The MCP server fetches the page, and if it is large, splits it into overlapping chunks and combines per-chunk work into one summary (the strategy, map-reduce or refine, is set in config), so the whole page is summarized rather than a truncated slice.

Use this tool when the user wants a free-form summary of a page, or an answer to an open question about a long page, rather than a fixed set of named fields (use extract for named fields).

Args: url: The full URL to summarize. question: Optional. If set, the summary is focused on answering this question rather than being a general overview. user_id: Optional, same semantics as for fetch.

Returns: A plain-text summary of the page.

Inputs: url (required), question, user_id. Returns: result

summarize_tabA

Summarize the CURRENT page in an open tab, using the same whole-page chunk-and-combine summarizer as summarize, but reading the live tab instead of opening a URL. Use this after you have navigated or interacted your way to a page and want a full summary of it (read_tab only returns the head of the page; this reads all of it without truncation).

Args: tab_id: The handle from open_tab. question: Optional focus question, same meaning as in summarize.

Returns: A plain-text summary of the whole current page.

Inputs: tab_id (required), question. Returns: result

type_intoA

Type text into a form field in an open tab, by its ref.

Args: tab_id: The handle from open_tab. ref: The field's element ref, e.g. "e7", from the latest read_tab. text: The text to type. submit: If true, press Enter after typing, which submits most search boxes and simple forms in one step. If the form needs a button instead, leave this false and click the submit button's ref.

Returns: JSON with the resulting url and fresh ref count.

Inputs: tab_id (required), text (required), ref, submit. Returns: result

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4.5/5.0

    Scored across 15 tools

    Disambiguation5/5

    Each tool has a distinct purpose: interactions (click, type_into, press_key, select_option), tab management (open, close, read), and content retrieval (fetch_snippet, summarize, extract, fetch_urls, fetch_structure, plus tab variants). No ambiguity between tools.

    Naming Consistency5/5

    Names follow a predictable verb_noun pattern (e.g., open_tab, close_tab, read_tab, fetch_snippet, summarize_tab). Single-word verbs like click and extract are acceptable as they are common actions. The convention is consistent throughout.

    Tool Count5/5

    15 tools cover the full scope of browser automation: tab lifecycle, element interaction, multiple reading methods (snippet, summary, extract, structure, links). The count is well-balanced, not excessive or sparse.

    Completeness4/5

    Covers core browsing operations thoroughly: open/close tabs, interact with elements, read pages in various ways, extract structured data. Missing explicit navigation history or tab listing, but these can be managed externally. Minor gap, overall solid.

    Maintenance

    ActivityInactive
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      D
      maintenance
      Enables AI agents to control a persistent Chromium browser or attach to an existing Chrome with sandboxed JavaScript and structured tools for web interaction and automation.
      16
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables agents to control an existing Chromium browser or isolated Playwright profile through 50 tools for reading, interacting with, navigating, capturing, transferring files, and coordinating tabs, with per-site permissions and one-use approvals.
      5
      MIT
    • F
      license
      Not graded
      quality
      A
      maintenance
      Enables AI agents to browse and interact with websites through a persistent headless Chromium session, returning compact structured page representations instead of full accessibility-tree dumps. Agents can navigate, query specific elements and forms, click, type, manage tabs and cookies, and monitor console or network activity while paying a fraction of the tokens.
      4 npm
      -
    • F
      license
      Not graded
      quality
      A
      maintenance
      Enables AI agents to browse and interact with web pages through a token-efficient, structured page representation — orientation summaries, targeted element queries, and full detail only on request — instead of raw accessibility-tree dumps. Supports navigation, interaction, forms, screenshots, console and network monitoring, session/tab management, and browser-based development auditing.
      4 npm
      -