computer
Control a Firefox tab by simulating pointer, keyboard, and wheel input, then capture screenshots. Perform one action per call using coordinates or element references.
Instructions
Drive one Firefox tab with synthetic pointer, wheel and keyboard input, and capture what it looks like. Each call performs exactly one action from the enum, against the tab named by tabId; ask tabs_context_mcp for a valid id before the first call.
Aim with
coordinate(viewport pixels) or, preferably, withref(an element id from read_page or find). Coordinates must come from your latest screenshot: pages scroll and re-render, and stale numbers land on whatever occupies that spot now.A ref belongs to the snapshot that produced it. After a navigation or a re-render, ask read_page or find for new ids rather than reusing old ones.
Put the pointer in the middle of a control, not on its border. When a click appears to have done nothing, take a fresh screenshot and re-aim before repeating it.
Firefox specifics: this drives a profile the server owns, separate from your everyday browser, and no per-site approval step stands in the way. A native alert, confirm, prompt or beforeunload freezes the tab, and every action here fails until firefox_dialog accepts or dismisses it.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| ref | No | Element handle from an earlier read_page or find call, written like "ref_1". `scroll_to` requires one and accepts nothing else. For clicks, hover, drag and scroll it stands in for `coordinate`, and it is the sturdier of the two: the element is looked up and scrolled into view at the moment the call runs. With `type` it means "click this field first, then type into it". Handles expire with the snapshot that produced them - navigate, or let the page re-render, and you need fresh ones. A handle that resolves to something hidden (a closed menu, an inactive tab panel) comes back as an error instead of a click at the page origin. | |
| text | No | Payload for two of the actions. Under `type` it is the literal string to enter, newlines included - each newline goes out as an Enter press. Under `key` it names a key or a chord, or several of them separated by spaces to be pressed in order: "Enter", "cmd+a", "ArrowDown ArrowDown Enter". The modifier names understood are ctrl, shift, alt, cmd (meta) and win (windows), attached to the key with "+"; macOS spells with cmd what Windows and Linux spell with ctrl. Chords that rescale the page - "ctrl+-", "cmd+0" and the like - are refused, since the new scale would be invisible to you; crop with the `zoom` action instead. | |
| tabId | Yes | The tab this action is performed on, from the tab group this session owns. The schema marks it required: tabs_context_mcp lists the ids that exist, tabs_create_mcp returns the id of a tab it opens, and a page opened from a tab already in the group joins it. An id from outside the group is rejected. | |
| action | Yes | Which operation to run. Every other field is read according to this choice. * `left_click`: press and release the primary button on the target. * `right_click`: secondary-button click, the usual way to raise a context menu. * `double_click`: two primary clicks in quick succession - opens an item, selects a word. * `triple_click`: three in a row, which selects the whole line or paragraph under the pointer. * `hover`: move the pointer onto the target and press nothing, to raise a tooltip, drop a menu open or trigger a :hover style. * `left_click_drag`: hold the primary button down from `start_coordinate` (or from wherever the pointer already sits) and release it on the target. * `scroll`: turn the wheel `scroll_amount` notches in `scroll_direction` over the point you give, so the scrollable box under that point is what moves. * `type`: send `text` to whatever holds keyboard focus; pass `ref` to click that field first. * `key`: press the keys or chords named in `text`, `repeat` times over. * `wait`: pause for `duration` seconds while something loads or animates. * `screenshot`: capture the visible area of the tab. * `zoom`: capture `region` alone, for reading small icons or fine print. * `scroll_to`: bring the element named by `ref` into view; it needs no coordinate. | |
| region | No | Crop box for `zoom`, required by it and read by nothing else: [x0, y0, x1, y1] in viewport pixels, top-left corner first and bottom-right second. x1 has to exceed x0 and y1 exceed y0, and the whole box must fit inside the viewport. What comes back is that rectangle at capture resolution, which is what makes a 16-pixel icon or a line of fine print legible. | |
| repeat | No | How many times to replay the whole sequence in `text`: a whole number from 1 to 100, and 1 when you leave it out. Only the `key` action looks at it. One call carrying repeat 20 beats twenty calls when you are walking a list with ArrowDown or emptying a field with Backspace. | |
| duration | No | Seconds the `wait` action sits idle, anywhere from 0 up to 10. Required by `wait`, read by nothing else. Fractions count (0.5 is half a second), and one call will never sit longer than 10 seconds, so wait a second time - with a screenshot in between - when a page needs more. | |
| modifiers | No | Keys held down for the length of a pointer action - the four clicks, hover and drag all honour it. Name one of ctrl, shift, alt, cmd (or meta) or win (or windows), and join several with "+" when you need more than one, as in "ctrl+shift". This is how you shift-click a range or cmd-click a link into its own tab. Optional, and unrelated to the chords the `key` action parses out of `text`. | |
| coordinate | No | Target point as [x, y] in CSS pixels measured from the top-left corner of the viewport, not of the screen. Give this or `ref` for left_click, right_click, double_click, triple_click, hover, scroll and left_click_drag; on a drag it marks where the button comes up, the end of the movement. Take the numbers from your most recent screenshot - an older one may name a spot the page has since moved. A tab here renders at devicePixelRatio 1, so one pixel of that image is one unit here; no scaling to do. `type` ignores this field, because typing follows keyboard focus. | |
| save_to_disk | No | Only meaningful for `screenshot` and `zoom`: set it true to also write the captured image into the firefox-use downloads folder, and the result line will carry the file path so you can attach it to a message. The picture is returned in the response either way, so leave this off while you are merely looking at the page yourself. | |
| scroll_amount | No | Wheel notches per `scroll` call, from 1 to 10; omit it and you get 3. A notch is worth roughly 100 pixels of travel here, so the default moves about 300. | |
| scroll_direction | No | Which way the wheel turns during `scroll`, and mandatory there: up, down, left or right. `down` walks further down the document, the way a real wheel would. | |
| start_coordinate | No | Where `left_click_drag` puts the button down, as [x, y] in viewport pixels; `coordinate` or `ref` supplies the point where it lifts again. Optional - leave it out and the drag begins wherever the previous action left the pointer, which is only predictable when that action was one of yours. |