Tato Runtime
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Tato Runtimeopen Chrome, look up the price of a coffee maker and write it in Notepad"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Tato Runtime
The hands of an AI agent: a computer and a browser for any agent that speaks MCP.
Claude Code · Codex · Cursor · Gemini CLI
English · Português (BR)
Tato Runtime is an MCP server that gives the agent three tools: computador
(computer), browser and gravacao (recording). The agent decides what to do;
Tato reads the interface, performs the gesture, checks that it took effect and
returns the result. Whenever it can, it reads the screen's structure as text
before falling back to an image, which makes each step cheaper and more
accurate.
While the agent uses the machine, a frame shows what it is doing, and a shortcut stops everything instantly.
Tool names, actions, parameters and environment variables are in Portuguese, exactly as the agent sees them. This README keeps them as-is and explains them in English.
What it does
Computer
Sees through structure: reads the window's elements through the system's accessibility layer (UI Automation on Windows, AT-SPI on Linux) and the text of the foreground window, with term search. It only falls back to a screenshot when the structure isn't enough, and can crop and zoom into a small region of the screen.
Uses mouse and keyboard: clicks by element number or by coordinate, types, presses key combinations, holds keys, sends key sequences in a single call, drags with the middle button and modifiers, scrolls.
Checks every gesture: after acting, it compares the screen with the one before and says whether the gesture worked, whether something changed or whether nothing happened, so the agent doesn't repeat a click that already worked.
Finds programs: lists what is open, in the tray and installed, brings a window to the front and opens programs through the system menu.
Waits for the screen: for a fixed time or until the screen changes, ignoring things that animate on their own.
It also draws: in Paint, the Claude Code mascot comes out of shapes, the bucket fill and the mouse.
Browser
The Chrome extension opens a dedicated tab per conversation, in a group named after the agent. The agent can't reach the person's other tabs.
Reads the page through the accessibility tree, the DOM, structured data and the console, and only takes a screenshot when text isn't enough.
Clicks, fills, scrolls and runs scripts (
roteiro): a sequence of steps with checks in between, which stops at the first step that fails.Through the screen, it can browse in the person's Chrome or in a separate profile just for the agent (see Configuration).
Local recording
When the person asks to record a demo,
gravacaostarts capturing after specific approval. An always-visible notice shows that the screen is being recorded and has a Stop button. If the notice doesn't open, there is no capture.parar(stop) finishes it;estado(status) says whether capture is still running.ver(view) returns a contact sheet with up to 12 frames. Inmudancas(changes) mode, it shows the moments before and after the biggest changes within a window of up to 30 seconds. You can zoom into anareaand narrowde/ate(from/to) in new calls — useful for investigating something that appears and disappears without sending hundreds of images to the agent.montar(assemble) builds a GIF with cuts, speed, captions, title cards, cropping, shortened waits and blur by interval. A segment with a differentidjoins finished local recordings in the given order.Frames and the GIF stay only in
TATO_HOME/gravacoes/<id>. Tato doesn't send them anywhere, doesn't publish them and stops capture if the MCP connection closes. Capture stops automatically within 15 minutes. Recording through MCP is available on Windows in this first version.
For example, after iniciar (start) and parar, the agent can look for a
quick change and assemble only the chosen moments:
{"acao":"ver","id":"<id returned by iniciar>","modo":"mudancas",
"de":12,"ate":22,"area":[200,100,700,500],"amostras":8}{"acao":"montar","id":"<id returned by iniciar>","trechos":[
{"titulo":"Price research"},
{"de":12,"ate":35,"velocidade":3,"legenda":"Comparing the offers",
"borrar":[
{"de":15,"ate":19,"caixa":[10,20,150,80]}]}
]}TATO_GRAVAR=1 mode, set before starting the server, includes the
computer frame in the video; without it, the frame stays out of the capture.
The recording notice stays visible in both modes. The GIF is for local review:
check for personal data before sharing it.
Related MCP server: desktop-touch-mcp
You're in control
Frame on screen: a border, its own pointer in place of the cursor, a bubble with the gesture and a small box with the agent's messages, since the chat sits behind the window. The frame stays out of the screenshots the agent sees.
Stop shortcut:
Ctrl + Alt + Shift + Sinterrupts the gesture in progress, and no gesture runs after it. The agent can't trigger this shortcut.Approval: the first gesture of each turn asks for approval through the MCP client. Reads don't.
Doesn't type in the wrong place: if another window comes to the front, the keyboard stops, even in the middle of a text.
One agent at a time: two clients don't fight over the same mouse and keyboard.
What it never does: type a password (the agent clicks the field and uses the browser's autofill) or press combinations that kill the session or delete irreversibly. Login and verification codes are handed to the person with
pedir_a_pessoa(ask the person).In the browser: it opens web pages (http or https), local ones included, so the agent can check the site you are building. Other schemes, like
file://, are refused, and actions that submit data or have consequences ask for approval.
Turn-based games
Turn-based games work well: the game waits while the agent looks and decides. The agent sends several moves in one sequence and reads a zoomed-in crop of the screen to see what is drawn.
Games in the demo, free and distributed by their own authors, running on the mGBA emulator:
Porklike, by Krystian Majewski and Lazy Devs, Game Boy version by Ben Smith.
Tobu Tobu Girl Deluxe, by Tangram Games, open source.
Libbet and the Magic Floor, by Damian Yerrick, based on Martin Korth's Magic Floor, free software (zlib license).
Real-time games aren't its strength: each look-and-decide loop takes a few seconds. In the Tobu Tobu Girl clip the agent pauses the emulator to think.
Install
Requirements: Python 3.11 or newer; Windows 10 or 11, or Linux with X11; Google Chrome for the browser.
git clone https://github.com/Ryanleoncoder/tato-runtime.git
cd tato-runtime
pip install -e .Tato runs with python -m tato. In the examples below, python is the Python
you installed it into; if you used a venv, put the path to its Python.
Register with your agent
Claude Code
claude mcp add tato -- python -m tato --aprovacao-no-clienteCodex, in ~/.codex/config.toml:
[mcp_servers.tato]
command = "python"
args = ["-m", "tato"]Cursor (~/.cursor/mcp.json) and Gemini CLI (~/.gemini/settings.json):
{
"mcpServers": {
"tato": { "command": "python", "args": ["-m", "tato"] }
}
}--aprovacao-no-cliente (approval in the client) is for clients that already
ask for approval before each tool call, like Claude Code. Without it, Tato asks
through the MCP protocol itself.
Browser extension
Once only: in chrome://extensions, turn on developer mode, choose "Load
unpacked" and select the tato/extensao folder. It connects on its own when
the agent starts Tato. Details in the
extension guide (in Portuguese).
Configuration
Everything through environment variables, with defaults that work without any setup.
Variable | Default | What it does |
|
| Stop shortcut. |
|
| With |
|
| The mouse glides to the target; |
|
| With |
|
| Which Chrome to browse through the screen: |
|
| Name of the separate profile: it becomes "Chrome do ". Create the shortcut with |
|
| Closes the browser session after this many idle minutes; |
|
| Local port where the extension finds Tato. |
|
| Where Tato keeps what it needs between runs. |
Actions
computador
Action | What it does |
| Screenshot; with |
| Elements of the focused window through accessibility, no image. |
| Text of the foreground window; with |
| Open windows and which one has focus. |
| Programs that are open, in the tray and installed. |
| Opens a program through the system menu / brings a window to the front / closes a window like its X button (the program still asks whether to save). |
| Click, double-click, right-click, move — by element number or screenshot coordinate. A coordinate click after other gestures checks that the screen around the target still matches the screenshot; |
| Drag / scroll, with button and modifiers. |
| Types text into the focused field, with |
| A key combination, held for a while, or a |
| Triggers or fills an element through accessibility, without the mouse. |
| Waits a fixed time or |
| Hands over to the person / releases the session. |
Every gesture accepts dizer (say): a short sentence that appears in the
frame's message box.
browser
Action | What it does |
| Opens an address and returns the page tree with references. |
| Reads the page again. |
| Click / type / scroll, by the element's reference or text. |
| Several steps at once, with |
| Screenshot of the page, when text isn't enough. |
| The page's console messages. |
| Ends the browser session. |
gravacao
Action | What it does |
| Starts recording the screen, with approval and a visible notice; accepts |
| Finishes the recording / says whether it's still running. |
| A contact sheet with up to 12 frames, by |
| Builds the GIF from |
How it works
On connection, the server sends the agent, as MCP instructions, what applies to all three tools: which one to use, talking through the frame, handing login to the person and ending when done. Each tool describes only its own usage.
The server speaks MCP over stdio. Each call sends progress and accepts cancellation; a call that doesn't return in time is abandoned and the agent receives the reason.
The first Tato to start opens the extension's local port; the others relay their commands through it.
The frame stays on screen while the session is open;
encerrarremoves it.
Limitations
Real-time games: too slow for the look-and-decide loop.
Online games: don't. Automating matches is forbidden in most games, and anti-cheat systems detect simulated input.
Administrator windows: Windows doesn't let a regular program control an elevated window.
macOS and Linux with Wayland aren't supported yet.
Development
pip install -e ".[dev]"
python -m pytest
node --test tato/extensao/service-worker.test.cjsThe Linux tests (X11 and AT-SPI) only run where Xvfb is available; on Windows they are skipped.
License
MIT.
Available Tools
3 toolsbrowserA
Abre e opera uma aba própria no Chrome da pessoa: lê, clica, preenche e rola. Use para tarefa num site; programa do computador vai pelo computador.
Comece com navegar (url): a resposta já traz a árvore da página, com um ref em cada elemento (e8, e23). Mire clicar e digitar por um ref que você viu, nunca inventado; o ref serve só para mirar e não vai para a resposta. digitar sem ref escreve no campo em foco; enviar: true aperta Enter depois. snapshot lê a página de novo; imagens tira um print, só quando o texto não bastar; console traz as mensagens do console.
Em clicar e digitar, declare consequencia: nenhuma para filtro, menu, busca ou paginação; comunicar para enviar, publicar ou se candidatar. Sem declarar, a pessoa é perguntada.
Leia o estado depois de cada ação; se nada mudou, procure o bloqueio em vez de repetir. O que não aparece na leitura costuma estar a um clique (dentro do item, do card) ou só na tela: abra o item ou peça imagens.
Quando já conhece a página, use roteiro (passos): ações (navegar, clicar, digitar, rolar) e controles (esperar e conferir com texto, imagens, pensar para ler a página e voltar a decidir). O roteiro para no primeiro conferir que falhar e devolve a página. Logo depois de um navegar no roteiro, mire por nome, o texto do elemento. Exemplo: {"acao":"roteiro","argumentos":{"passos":[{"acao":"digitar","argumentos":{"ref":"e3","texto":"fones","enviar":true},"consequencia":"nenhuma"},{"acao":"esperar","argumentos":{"texto":"Resultados"}},{"acao":"pensar"}]}}.
A aba continua nos próximos turnos da conversa; encerrar só quando a pessoa pedir.
| Name | Required | Description | Default |
|---|---|---|---|
| acao | Yes | ||
| provider | No | ||
| argumentos | No | ||
| consequencia | No | Em clicar/digitar: o que a acao causa fora da pagina. Sem declarar, pergunta. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and delivers: the tab persists across conversation turns, 'encerrar' only on request, and 'consequencia' behavior including that omission triggers a user prompt. It exposes non-obvious mechanics (ref doesn't appear in the response, roteiro halts on first failed 'conferir'). This is exactly the extra context annotations would otherwise provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads purpose and the 'computador' distinction before drilling into actions. Density is high but nearly every sentence earns its place for a nine-action tool. Slightly heavy overall, which keeps it just shy of perfect.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex multi-action tool with nested objects and no output schema, the description supplies the missing pieces: that 'navegar' returns the page tree with refs (return shape), the roteiro result contract, and the persistence model. An agent has everything needed to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 25%, so the description must compensate and does: url, ref semantics (must come from a prior reading, never invented, aim-only), ref-less 'digitar' writing to focus, 'enviar' pressing Enter, 'nome' after navegar in a script, and the full 'passos' structure. This is far beyond what the sparse schema documents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource ('Abre e opera uma aba própria no Chrome') and enumerates the operations (lê, clica, preenche, rola). It explicitly distinguishes itself from the sibling 'computador' by routing desktop-program tasks there, so an agent can tell the tools apart without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives per-action when-to-use guidance ('Comece com navegar', 'snapshot lê a página de novo', 'imagens... só quando o texto não bastar'), plus when to escalate to 'roteiro'. It even states exclusion rules (use 'computador' for desktop programs) and the retry heuristic ('se nada mudou, procure o bloqueio em vez de repetir').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
computadorA
Vê a tela e usa o mouse e o teclado de verdade. Use para programas do computador e para o que o browser não alcança; tarefa num site vai melhor pelo browser.
Oriente-se com ver e marcar: true: o print vem com um número em cada elemento. Para conferir um passo pequeno, elementos lista os mesmos itens em texto, sem imagem. Mire pelo número (elemento: n); sem número, pela coordenada do print, nunca por palpite. ler traz o texto da janela da frente; com procurar, só as linhas com o termo.
Depois de cada gesto vem um veredito: confirmado (não repita), mudou (confira em mudou se foi o esperado), sem_efeito_aparente (não repita; siga o proximo) ou nao_da_para_confirmar (confira no print). na_janela diz onde o gesto caiu: se não é o programa pedido, foi no lugar errado.
Programas: programas com nome diz se está aberto, na bandeja ou instalado. Aberto, focar com parte do título; fechado, abrir com o nome do menu Iniciar. App que acabou de abrir pode ficar atrás: confira em janelas antes de abrir de novo. Programa com abas (Bloco de notas, navegador) pode abrir na janela da pessoa: para não mexer no que é dela, abra uma janela nova pelo atalho do programa. fechar com parte do título fecha a janela, como o X; o programa ainda pergunta se quer salvar.
Texto: clique no campo e use digitar. ritmo: natural (pausas de mão, para site), rapido ou instantaneo (tudo de uma vez, para app local), automatico (padrão: rápido nos editores, natural no resto). Na barra de endereço do navegador, aperte delete antes do enter, ou ele completa com outra página do histórico. tecla aperta uma combinação (ctrl+s), segura com segurar ou manda uma sequencia. invocar e definir_valor agem no controle pela acessibilidade, sem mouse.
Quando já sabe o caminho, mande os gestos seguidos na mesma resposta; se um falhar, os seguintes não rodam.
Jogo, editor de imagem e modelagem desenham a tela: a lista vem quase vazia e o print orienta. ver com regiao amplia um pedaço pequeno; esperar com ate_mudar espera a animação em vez de adivinhar o tempo; arrastar e rolar aceitam botao e com (modificadores).
Para navegar pela tela, use o Chrome que o campo navegador do primeiro resultado indica. Carrossel e menu animado andam um clique por vez.
Sempre: código no celular e login passam para a pessoa com pedir_a_pessoa, e aí tudo é recusado até ela devolver. O primeiro gesto de cada turno pede aprovação. Fale com a pessoa pelo dizer, que aparece na moldura, e chame encerrar quando parar de usar a tela.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | Coordenada no print (clicar, mover, arrastar, rolar). | |
| y | No | Coordenada no print. | |
| com | No | Em `arrastar` e `rolar`, modificadores segurados durante o gesto, como `shift` ou `ctrl+alt`. | |
| acao | Yes | ||
| nome | No | Em `programas`, o programa procurado (diz se roda sem janela e se está instalado); em `abrir`, o nome do programa como aparece no menu Iniciar. | |
| botao | No | Em `arrastar`, o botao do mouse (padrao esquerdo). | |
| dizer | No | Em qualquer acao, uma frase curta (ate 120 caracteres) que aparece na caixinha da moldura para a pessoa, que nao ve o chat enquanto voce usa a tela. | |
| pausa | No | Em `tecla` com `sequencia`, segundos entre uma tecla e a outra (padrao 0,15). | |
| ritmo | No | Em `digitar`: automatico (padrao) acelera editores locais conhecidos e usa pausas naturais nos demais programas; rapido envia teclas com pausa curta; instantaneo manda o texto inteiro de uma vez; natural digita caractere por caractere com pausas variaveis. | |
| tecla | No | Em `tecla`, a combinacao, como `ctrl+s` ou `enter`. | |
| texto | No | Em `digitar`. | |
| valor | No | Em `definir_valor`, o texto do campo ou o nome da opcao da lista. | |
| depois | No | Como conferir o gesto. `texto` (padrao) volta a lista de elementos, e o print so quando a janela nao se descreve. | |
| janela | No | Em `focar` e `fechar`, parte do titulo da janela, como aparece em `janelas`. | |
| marcar | No | Em `ver`, numera os elementos da janela no print; vale para os prints seguintes do turno. | |
| motivo | No | Em `pedir_a_pessoa`, o passo que e da pessoa (ex. fazer login no portal). | |
| para_x | No | Em `arrastar`, o destino no print. | |
| para_y | No | Em `arrastar`, o destino no print. | |
| regiao | No | Em `ver`, so este pedaco, ampliado para ler (menu de jogo, painel de editor); em `esperar` com `ate_mudar`, so este pedaco e vigiado. No espaco do print. | |
| direcao | No | ||
| segurar | No | Em `tecla`, quantos segundos a combinacao fica apertada (maximo 10), para andar ou correr num jogo; solta sozinha no fim. | |
| contexto | No | Em `ler` com `procurar`, quantas linhas antes e depois de cada achado (padrao 1, ate 6). | |
| elemento | No | O numero observado na ultima leitura (`elementos` ou `ver` com `marcar`), sempre a partir de 1; nunca use 0. Obrigatorio em `invocar` e `definir_valor`. | |
| procurar | No | Em `ler`, so as linhas com este termo (como "R$"), cada uma com a de antes e a de depois: le a janela inteira e devolve o que interessa. | |
| segundos | No | Em `esperar` (maximo 30). | |
| ate_mudar | No | Em `esperar`, volta assim que a tela (ou a `regiao`) mudar, ate `segundos`. | |
| sequencia | No | Em `tecla`, varias teclas em ordem numa chamada, como ["direita", "direita", "z"]; "cima:0.6" segura 0,6 s. Ate 40 teclas. | |
| quantidade | No | Em `rolar`, cliques de roda (padrao 3). | |
| ver_depois | No | Antigo; `depois` manda. Falso e o mesmo que `depois: nada`, verdadeiro o mesmo que `depois: print`. | |
| confiar_no_print | No | Em clique ou arrasto por coordenada depois de outros gestos: confirma que a tela em volta do alvo mudou porque você esperava (pintar sobre o que acabou de desenhar). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations present, the description carries the full behavioral burden and does so richly. It discloses post-gesture verdicts (`confirmado`, `mudou`, `sem_efeito_aparente`, `nao_da_para_confirmar`), the `na_janela` placement check, approval requirements for the first gesture each turn, refusal of all actions after `pedir_a_pessoa` until the human returns, and special behaviors like apps opening behind or programs opening in the person's window. It also explains that `fechar` behaves like the X and that programs may still ask to save.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but front-loaded: it opens with what the tool does and when to use it before moving into workflow details. Each paragraph has a clear theme (orientation, verdicts, programs, text, games, approval), and most sentences add distinct operational value. It is slightly dense and could use tighter grouping, but the length is defensible for a 30-parameter computer-control tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 30 parameters, no annotations, no output schema, and high schema coverage, the description supplies the missing behavioral and workflow context. It covers approval, human handoff with `pedir_a_pessoa`, communication via `dizer`, when to call `encerrar`, and special cases like games, carousels, and animated menus. Nothing essential for correct invocation appears to be missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 93%, so the baseline is 3, but the description adds practical usage semantics beyond the schema. It explains how to orient with `ver` and `marcar: true`, to target by `elemento: n` or by screenshot coordinates rather than guessing, and to use `ritmo` differently for websites versus local apps. It also notes browser-address-bar behavior (press `delete` before `enter`) and batching gestures in one turn with failure halting subsequent gestures, which enriches several parameters beyond their schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence gives a concrete verb and resource: it sees the screen and uses real mouse and keyboard. It immediately distinguishes this tool from the sibling `browser`, saying site tasks are better handled there and this tool covers what the browser cannot reach. An agent can tell it apart from `browser` and `gravacao` without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool versus the alternative: for computer programs and anything the browser does not reach, while website tasks should go through `browser`. It also gives detailed routing guidance for sub-actions like `ver` with `marcar: true`, `elementos`, `ler` with `procurar`, `focar` versus `abrir`, and `pedir_a_pessoa` for login or phone-code steps. When-not conditions are covered through the handoff to the human and the browser alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
gravacaoA
Grava, inspeciona e edita uma captura local da tela. iniciar requer pedido da pessoa e aprovação específica; parar finaliza; estado consulta; ver retorna uma única prancha com até 12 quadros, por intervalo ou pelas maiores mudanças visuais (útil para algo que aparece e some); montar cria um GIF com cortes, legendas e trechos de outras gravações locais. Use ver com de/ate e area para investigar uma região, refinando o intervalo em novas chamadas, sem pedir centenas de imagens. Não publique ou compartilhe a captura sem revisar dados pessoais.
| Name | Required | Description | Default |
|---|---|---|---|
| de | No | Em ver, segundo inicial a investigar. | |
| id | No | Id devolvido por iniciar/parar; obrigatório em ver e montar. | |
| ate | No | Em ver, segundo final a investigar. | |
| fps | No | Em iniciar, quadros por segundo (padrão 10). | |
| acao | Yes | ||
| area | No | Em ver, região [x0,y0,x1,y1] da gravação para medir mudanças e ampliar na prancha. | |
| modo | No | Em ver: amostras espaçadas ou pares antes/depois das maiores mudanças visuais. Para mudancas, use no máximo 30 s por chamada. | |
| nome | No | Em montar, nome do arquivo do GIF, sem extensão (padrão resumo). | |
| caixa | No | Em iniciar, recorte opcional da tela [x0,y0,x1,y1]. | |
| largura | No | Em iniciar, largura máxima do quadro (padrão 960). | |
| minutos | No | Em iniciar, limite automático de tempo (padrão 15). | |
| trechos | No | Em montar, cortes em ordem; omitir usa a gravação inteira. | |
| amostras | No | Em ver, número máximo de miniaturas numa única prancha (padrão 8). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=false. The description adds real behavioral context beyond them: that `iniciar` requires a person's request and specific approval, the privacy caution against publishing/sharing without reviewing personal data, and the efficiency constraint of not pulling hundreds of frames. It stops short of describing the return format of `estado` or the reversibility of edits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the overall purpose, then organized by action using backticked names, followed by workflow and privacy notes. Dense but each sentence carries information, though several clauses re-state schema-level constraints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 13-parameter, five-action tool with no output schema, it covers each action, the approval prerequisite, the `ver` workflow, and a privacy caution. `estado` is only described as 'consulta' with no indication of what it returns, which is the main remaining gap without an output schema to rely on.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 92%, so the schema already documents essentially every parameter (de, ate, area, modo, trechos, etc.). The description only reinforces how de/ate and area combine for region investigation and restates the modo 'mudancas' 30 s limit, which is already in the schema. Baseline 3 is correct when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear purpose ('Grava, inspeciona e edita uma captura local da tela') and enumerates all five sub-actions with a specific meaning for each (iniciar/parar/estado/ver/montar). An agent can tell what the tool does and how each action differs. It does not, however, distinguish this tool from the sibling tools 'browser' and 'computador'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives concrete when-to-use guidance: 'iniciar' requires a person's request and specific approval, `ver` should be used with de/ate and area to investigate a region and refine the interval across calls rather than requesting hundreds of images, and 'mudancas' mode suits something that appears and disappears. No explicit exclusion against the sibling tools, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
3 tool updates
v0.1.0- First observed
browser - First observed
computador - First observed
gravacao
TDQS
Scored across 3 tools
The three tools target mostly distinct domains: browser for website tasks, computador for desktop programs, and gravacao for screen recording. However, the computador and browser tools overlap when the computer tool navigates Chrome or when both perform click/type/scroll, which is why descriptions must explicitly say 'site tasks go better through browser' to keep them apart.
Names are domain nouns rather than a verb_noun pattern, and the set mixes English (browser) with Portuguese (gravacao, computador). Still readable and each name maps to a clear domain, but the convention is inconsistent.
Three tools for a screen-control runtime (browser, desktop, recording) is well-scoped and each earns its place. The tradeoff is that each tool is heavily overloaded with many internal sub-actions, which is more of a design style than a count problem.
Coverage is broad: navigation, reading, clicking, typing, scrolling, scripting, program management, keyboard shortcuts, drag, recording, and human handoff/approval flows are all present. Minor gaps like explicit file upload/download handling exist but core lifecycle operations are covered.
Maintenance
Related MCP Connectors
Control a real browser from AI agents: navigate, click, type, screenshot, read the page.
OCR, transcription, file extraction, and image generation for AI agents via MCP.
MCP server for your apps' tools and custom tools, plus hosted AI agents and approval-gated workflows
Hosted MCP server and REST API that lets AI agents use apps with no API, signed in as you.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables AI agents to view and control the local desktop through browser-based viewport and MCP tools.MIT
- AlicenseAqualityCmaintenanceAllows AI clients to see and control Windows 10/11 desktops via MCP, with screenshots, UI Automation, Chrome CDP, keyboard/mouse, and terminal using semantic element targeting.30503 npmMIT
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to operate local desktops and Chromium browsers through MCP tools, unifying accessibility trees, physical input, screenshots, DOM/ARIA, visual grounding, and result verification.313 npmMIT
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to observe and control the user's live desktop by listing apps, reading accessibility trees, and performing clicks, typing, scrolling, dragging, and other input actions with per-action approvals and local audit archives.1MIT