Screen Vision MCP Server
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Screen Vision MCP Server find text 'Submit' on screen and return coordinates"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Screen Vision MCP Server
A Model Context Protocol (MCP) server that provides comprehensive screen capture, OCR, and visual understanding capabilities for macOS.
Features
capture_fullscreen: Capture the entire screen
capture_window: Capture specific application windows
capture_region: Capture defined screen regions
extract_text_from_screen: OCR text extraction from screenshots
find_text_on_screen: Locate text on screen and return coordinates
get_window_list: List all open windows with details
get_screen_info: Get display and screen information
click_at_position: Automated clicking at specific coordinates
monitor_screen_region: Monitor regions for changes over time
Screenshot resource management and retrieval
Related MCP server: cua-local
Installation
Quick Install
npm install -g screen-vision-mcpFrom Source
Clone the repository:
git clone https://github.com/TIMBOTGPT/screen-vision-mcp.git cd screen-vision-mcpInstall dependencies:
npm installTest the server:
npm start
Usage with Claude Desktop
Add this server to your Claude Desktop MCP configuration (claude_desktop_config.json):
{
"mcpServers": {
"screen-vision": {
"command": "npx",
"args": ["-y", "screen-vision-mcp"],
"description": "Screen capture and vision analysis"
}
}
}Or if installed locally:
{
"mcpServers": {
"screen-vision": {
"command": "node",
"args": ["/path/to/screen-vision-mcp/index.js"],
"description": "Screen capture and vision analysis"
}
}
}Available Tools
capture_fullscreen
Capture the entire screen.
Parameters:
save_path(optional): Custom save path for the screenshot
Example:
{
"name": "capture_fullscreen",
"arguments": {
"save_path": "/path/to/save/screenshot.png"
}
}capture_window
Capture a specific application window.
Parameters:
app_name(required): Name of the application (e.g., "Safari", "Terminal")save_path(optional): Custom save path
Example:
{
"name": "capture_window",
"arguments": {
"app_name": "Safari",
"save_path": "/path/to/save/window.png"
}
}capture_region
Capture a specific region of the screen.
Parameters:
x(required): X coordinatey(required): Y coordinatewidth(required): Width of regionheight(required): Height of regionsave_path(optional): Custom save path
extract_text_from_screen
Capture screen and extract text using OCR.
Parameters:
region(optional): Specific region to capturex,y,width,height: Region coordinates
find_text_on_screen
Find text on screen and return its location.
Parameters:
text(required): Text to search forcase_sensitive(optional): Whether search should be case sensitive (default: false)
get_window_list
Get list of all open windows with their positions.
get_screen_info
Get information about available screens/displays.
click_at_position
Click at a specific screen position.
Parameters:
x(required): X coordinatey(required): Y coordinatebutton(optional): Mouse button ('left', 'right', 'middle', default: 'left')double_click(optional): Whether to double-click (default: false)
monitor_screen_region
Monitor a screen region for changes over time.
Parameters:
x,y,width,height(required): Region to monitorduration_seconds(optional): How long to monitor (max 30 seconds, default: 5)interval_ms(optional): Check interval in milliseconds (default: 1000)
Requirements
macOS (uses native
screencapturecommand)Node.js 16+
Claude Desktop with MCP support
Screen recording permissions for automation features
Permissions
On first use, macOS may request permissions for:
Screen recording
Accessibility (for clicking automation)
File system access (for saving screenshots)
Grant these permissions in System Preferences > Security & Privacy.
Screenshots Storage
Screenshots are automatically saved to a screenshots/ directory within the server folder. You can:
Access screenshots via the resource URI system
Specify custom save paths for individual captures
View saved screenshots through Claude's resource system
Development
# Install dependencies
npm install
# Start development server
npm run dev
# Run tests
npm testAdvanced Features
OCR Integration
The server includes hooks for macOS Vision framework integration for advanced OCR capabilities. Full OCR requires additional setup with native macOS Vision APIs.
Automation
The clicking and monitoring features enable automation workflows when combined with other MCP servers.
Security
All screen captures require explicit permission
File system access is controlled by macOS permissions
No network access required for core functionality
License
MIT License - see LICENSE file for details
Contributing
Fork the repository
Create a feature branch
Make your changes
Add tests if applicable
Submit a pull request
Support
For issues and questions, please use the GitHub Issues page.
Available Tools
9 toolscapture_fullscreenC
Capture the entire screen
Inputs: save_path.
capture_regionC
Capture a specific region of the screen
Inputs: x (required), y (required), width (required), height (required), +1 more.
capture_windowB
Capture a specific window by app name
Inputs: app_name (required), save_path.
click_at_positionB
Click at a specific screen position
Inputs: x (required), y (required), button, double_click.
extract_text_from_screenC
Capture screen and extract text using OCR
Inputs: region.
find_text_on_screenA
Find text on screen and return its location
Inputs: text (required), case_sensitive.
get_screen_infoA
Get information about available screens/displays
Inputs: no named parameters.
get_window_listA
Get list of all open windows with their positions
Inputs: no named parameters.
monitor_screen_regionC
Monitor a screen region for changes
Inputs: x (required), y (required), width (required), height (required), +2 more.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 9 tools
Each tool targets a clearly distinct action: capturing a specific target, extracting/finding text, retrieving system info, clicking, or monitoring. Even the capture variants are disambiguated by their target (fullscreen, window, region).
All tool names follow a consistent snake_case verb_noun pattern, such as capture_window, get_screen_info, and find_text_on_screen. This makes the tool set predictable and easy to navigate.
Nine tools is well-scoped for a screen vision and automation server. Each tool covers a meaningful capability without unnecessary overlap or bloat.
The tool surface covers capture, OCR, text location, screen/window information, clicking, and region monitoring. Minor gaps exist around input actions like typing or scrolling, but core workflows are well supported.
Maintenance
Related MCP Connectors
Web scraping, browser automation, screenshots and AI data extraction for AI agents via MCP.
OCR, transcription, file extraction, and image generation for AI agents via MCP.
Clean PNG/JPEG screenshots via REST or MCP, with goal-driven multi-step navigation.
Screenshots, PDFs, image editing, QR codes, email validation and more through one hosted MCP server.
Related MCP Servers
- AlicenseAqualityDmaintenanceProvides screenshot and OCR capabilities for macOS.127 npm23MIT
- AlicenseBqualityDmaintenanceProvides a local MCP bridge for AI assistants to control a Mac by observing screen state and performing actions like mouse movement, clicking, typing, and opening URLs.12MIT
- AlicenseNot gradedqualityDmaintenanceEnables MCP clients to control macOS via accessibility and screen recording, providing tools to list apps, observe UI, click, type, press keys, and scroll.MIT
- AlicenseNot gradedqualityBmaintenanceAn ultra-fast native MCP server for macOS desktop automation, enabling visual OCR, text-based clicking, window management, and keyboard/mouse control without hijacking the physical cursor.28 npm1MIT