computer-use-mcp

What it does

computer-use-mcp gives an MCP client the same kind of mouse-and-keyboard control that Anthropic's computer use API offers, but as a local server you start with npx. It exposes a single tool, `computer`, whose `action` argument picks one of eleven operations: take a screenshot, read the cursor position, move the pointer, left/right/middle/double click, click-and-drag, scroll, press a key combination, or type text. Input is sent through the nut.js automation library (the `@nut-tree-fork/nut-js` package).

Screenshots are shrunk before they are returned so they stay within 1568 pixels on the long edge and about 1.15 megapixels in total, and the coordinates the model sends back are scaled up to real screen positions; coordinates outside the display are rejected with an error. Each screenshot has a red crosshair drawn at the cursor so the model can see where a click landed. Scrolling moves 300 pixels unless a count is given (for example `down:500`). These figures come from src/tools/computer.ts at v1.8.0, not the README.

The author's tips: run the display at a small common resolution (720p works well), use a model that is good at computer use, and install the Rango browser extension so web pages can be driven with keyboard hints instead of coordinate clicks. Besides stdio, the server has an HTTP mode the README does not describe: setting `MCP_TRANSPORT=http` serves streamable HTTP at `/mcp` on port 3000 (or `PORT`), with no authentication.

What it needs

Node.js with npx, on a graphical desktop session the server can see and control; the code has specific handling for macOS (a screencapture fallback) and Linux with X11. On Linux, typing uses `xdotool` when it is on PATH so non-US keyboard layouts come out right; without it, text is typed as if the layout were US QWERTY. Claude Desktop can instead install it from a .dxt build downloaded from the project's GitHub Actions.

Limits

One tool covers everything; there is no window list, app launcher, clipboard or file access, so the model works only from screenshots and input events. Coordinates outside the screen are refused. Screenshots are downscaled to at most 1568 px on the long edge and about 1.15 MP, so small text on large screens can be hard to read; the author recommends a 720p display. On macOS, a click on an unfocused window may only bring it forward, so the click must be repeated. The author warns that models make frequent mistakes and are open to prompt injection.

Who it suits

Suits someone who wants an assistant to operate ordinary desktop apps on their own computer and is prepared to watch it, ideally in a separate, sandboxed user account as the author advises. Because it acts on the real screen with your full user rights, it is a poor fit for unattended work or for machines holding sensitive data. For web-only tasks, a server that drives a browser through its own automation interface is usually more reliable than clicking at screen coordinates.