Computer Control MCP

MCP server that provides computer control capabilities, like mouse, keyboard, OCR, etc. using PyAutoGUI, RapidOCR, ONNXRuntime. Similar to 'computer-use' by Anthropic. With Zero External Dependencies.

Quick Usage (MCP Setup Using uvx)

Note: Running uvx computer-control-mcp@latest for the first time will download python dependencies (around 70MB) which may take some time. Recommended to run this in a terminal before using it as MCP. Subsequent runs will be instant.

{
  "mcpServers": {
    "computer-control-mcp": {
      "command": "uvx",
      "args": ["computer-control-mcp@latest"]
    }
  }
}

OR install globally with pip:

pip install computer-control-mcp

Then run the server with:

computer-control-mcp # instead of uvx computer-control-mcp, so you can use the latest version, also you can `uv cache clean` to clear the cache and `uvx` again to use latest version.

Related MCP server: vision-input

Features

• Control mouse movements and clicks

• Type text at the current cursor position

• Take screenshots of the entire screen or specific windows with optional saving to downloads directory

• Extract text from screenshots using OCR (Optical Character Recognition)

• List and activate windows

• Press keyboard keys

• Drag and drop operations

Available Tools

Mouse Control

• click_screen(x: int, y: int): Click at specified screen coordinates

• move_mouse(x: int, y: int): Move mouse cursor to specified coordinates

• drag_mouse(from_x: int, from_y: int, to_x: int, to_y: int, duration: float = 0.5): Drag mouse from one position to another

• mouse_down(button: str = "left"): Hold down a mouse button ('left', 'right', 'middle')

• mouse_up(button: str = "left"): Release a mouse button ('left', 'right', 'middle')

Keyboard Control

• type_text(text: str): Type the specified text at current cursor position

• press_key(key: str): Press a specified keyboard key

• key_down(key: str): Hold down a specific keyboard key until released

• key_up(key: str): Release a specific keyboard key

• press_keys(keys: Union[str, List[Union[str, List[str]]]]): Press keyboard keys (supports single keys, sequences, and combinations)

Screen and Window Management

• take_screenshot(title_pattern: str = None, use_regex: bool = False, threshold: int = 60, scale_percent_for_ocr: int = None, save_to_downloads: bool = False): Capture screen or window

• take_screenshot_with_ocr(title_pattern: str = None, use_regex: bool = False, threshold: int = 10, scale_percent_for_ocr: int = None, save_to_downloads: bool = False): Extract adn return text with coordinates using OCR from screen or window

• get_screen_size(): Get current screen resolution

• list_windows(): List all open windows

• activate_window(title_pattern: str, use_regex: bool = False, threshold: int = 60): Bring specified window to foreground

Development

Setting up the Development Environment

# Clone the repository
git clone https://github.com/AB498/computer-control-mcp.git
cd computer-control-mcp

# Install in development mode
pip install -e .

# Start server
python -m computer_control_mcp.core

# -- OR --

# Build
hatch build

# Non-windows
pip install dist/*.whl --upgrade

# Windows
$latest = Get-ChildItem .\dist\*.whl | Sort-Object LastWriteTime -Descending | Select-Object -First 1
pip install $latest.FullName --upgrade 

# Run
computer-control-mcp

Running Tests

python -m pytest

API Reference

See the API Reference for detailed information about the available functions and classes.

License

MIT

For more information or help

• Email (abcd49800@gmail.com)

• Discord (CodePlayground)