computer-use-linux

Name: computer-use-linux
Author: agent-sh

Linux desktop control over MCP: accessibility trees, screenshots, clicks and typing on Wayland and X11

By agent-sh Rust Local · stdio Browser automation Developer tools

GitHub npm Website

74 popularity score ★ 824 ↓ 4.4k/wk

Connect your client

Local (npm)

Add in one click

Add to Cursor Add to VS Code Add to LM Studio

Opens the app, which asks before it adds the server.

Or add it by command or config

FirstTerminal

npm install -g @agent-sh/computer-use-linux
computer-use-linux setup

Claude CodeTerminal

claude mcp add computer-use-linux -- computer-use-linux mcp

Claude Desktopclaude_desktop_config.json

{
  "mcpServers": {
    "computer-use-linux": {
      "command": "computer-use-linux",
      "args": [
        "mcp"
      ]
    }
  }
}

Cursor~/.cursor/mcp.json

{
  "mcpServers": {
    "computer-use-linux": {
      "command": "computer-use-linux",
      "args": [
        "mcp"
      ]
    }
  }
}

VS Code.vscode/mcp.json

{
  "servers": {
    "computer-use-linux": {
      "type": "stdio",
      "command": "computer-use-linux",
      "args": [
        "mcp"
      ]
    }
  }
}

Codex~/.codex/config.toml

[mcp_servers.computer-use-linux]
command = "computer-use-linux"
args = ["mcp"]

Gemini CLITerminal

gemini mcp add --scope user computer-use-linux computer-use-linux -- mcp

Local (from source)

Add in one click

Add to Cursor Add to VS Code Add to LM Studio

Opens the app, which asks before it adds the server. Then replace each <NAME> in its settings with your value.

Or add it by command or config

FirstTerminal

git clone https://github.com/agent-sh/computer-use-linux
cd computer-use-linux
./install.sh

Claude CodeTerminal

claude mcp add computer-use-linux -- '<ABSOLUTE_PATH_TO_HOME>/.local/bin/computer-use-linux' mcp

Claude Desktopclaude_desktop_config.json

{
  "mcpServers": {
    "computer-use-linux": {
      "command": "<ABSOLUTE_PATH_TO_HOME>/.local/bin/computer-use-linux",
      "args": [
        "mcp"
      ]
    }
  }
}

Cursor~/.cursor/mcp.json

{
  "mcpServers": {
    "computer-use-linux": {
      "command": "<ABSOLUTE_PATH_TO_HOME>/.local/bin/computer-use-linux",
      "args": [
        "mcp"
      ]
    }
  }
}

VS Code.vscode/mcp.json

{
  "servers": {
    "computer-use-linux": {
      "type": "stdio",
      "command": "<ABSOLUTE_PATH_TO_HOME>/.local/bin/computer-use-linux",
      "args": [
        "mcp"
      ]
    }
  }
}

Codex~/.codex/config.toml

[mcp_servers.computer-use-linux]
command = "<ABSOLUTE_PATH_TO_HOME>/.local/bin/computer-use-linux"
args = ["mcp"]

Gemini CLITerminal

gemini mcp add --scope user computer-use-linux '<ABSOLUTE_PATH_TO_HOME>/.local/bin/computer-use-linux' -- mcp

Settings 12

Set together: opt-in tools
`COMPUTER_USE_LINUX_ENABLE_SHELL`	Optional: Set exactly to 1 to register the run_shell tool. Default `unset`.
`COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE`	Optional: Set exactly to 1 to register complete_interaction, which needs notify-send. Default `unset`.
Set together: indicator
`COMPUTER_USE_LINUX_INDICATOR`	Optional: Set to 0, false, off or no to turn off the on-screen overlay. Default `on`.
`COMPUTER_USE_LINUX_INDICATOR_HIDE_TEXT`	Optional: Set exactly to 1 to mask typed text and keycaps in the overlay. Default `unset`.
`COMPUTER_USE_LINUX_AGENT_NAME`	Optional: Name shown in the overlay in place of the MCP client's name.
`COMPUTER_USE_LINUX_INDICATOR_BIN`	Optional: Path to computer-use-linux-indicator when it is not beside the server binary or on PATH.
Set together: input
`COMPUTER_USE_LINUX_PERSIST_REMOTE_DESKTOP`	Optional: Set exactly to 1 to keep RemoteDesktop portal grants across processes through a stored token. Default `unset`.
`COMPUTER_USE_LINUX_PORTAL_SCROLL_INVERT`	Optional: Set to 1 to reverse vertical scrolling sent through the RemoteDesktop portal. Default `unset`.
`COMPUTER_USE_LINUX_FORCE_PORTAL_POINTER`	Optional: Prefer the RemoteDesktop portal for pointer input on Wayland. Default `unset`.
`COMPUTER_USE_LINUX_FORCE_YDOTOOL_KEYBOARD`	Optional: Always send keyboard input through ydotool. Default `unset`.
`YDOTOOL_SOCKET`	Optional: Explicit ydotool socket path; when unset only the owner-only socket in the user runtime dir is used.
Set together: capture
`COMPUTER_USE_LINUX_SCREENSHOT_BACKEND`	Optional: Force one screenshot backend: gnome-shell, portal, x11 or gnome-screenshot. Default `auto`.

What it does

computer-use-linux is a Rust program that gives an agent control of the Linux desktop it runs on. Started with the `mcp` argument it is a stdio MCP server. It reads application accessibility trees through AT-SPI, lists and focuses windows, takes screenshots, and sends clicks, drags, scrolls, key chords and text. You can point at a control by its element index from a `get_app_state` snapshot or by a role/name/text selector. Pixel coordinates work for canvases, and each action can return a fresh screenshot or tree in the same call.

Wayland is the main target. Window lookup tries backends in turn and reports which answered: a bundled GNOME Shell extension or GNOME Introspect, KWin scripting on Plasma, hyprctl, niri IPC, i3 IPC (which also covers Sway), a COSMIC helper, the labwc socket from the Selkies patch, the wlr-foreign-toplevel-management protocol for stock labwc and other wlroots compositors, and finally wmctrl/xprop on X11. Input goes through the RemoteDesktop portal, uinput, ydotool, wtype or xdotool, depending on the session. A `doctor` tool returns a JSON readiness report that names what is missing.

Three binaries ship together: the server/CLI, the `computer-use-linux-cosmic` helper and `computer-use-linux-indicator`, an overlay that shows the agent's cursor and keys and hides itself during captures. Besides npm, crates.io and the source install there are Claude Code and Codex plugins that fetch pinned release binaries on first start. Pi and Hermes have their own setups.

In src/server.rs, 19 tools are registered by default. Two more appear only when set in the environment: `run_shell` with COMPUTER_USE_LINUX_ENABLE_SHELL=1 and `complete_interaction` with COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1. The same file has `get_app_state` read 1000 accessibility nodes and 32 levels by default. `max_nodes` and `max_depth` raise that to hard caps of 10000 and 64. Trees too large to return inline are written to a private file, and `get_tree_page` pages through them.

What it needs

Runtime	A prebuilt binaryNode.js 18+
Runs on	Linux
Also	A graphical desktop session (Wayland or X11) AT-SPI accessibility enabled An input backend: RemoteDesktop portal, ydotool 1.0.3+ with ydotoold, or xdotool on X11 curl or wget for the Claude Code and Codex plugin launcher

Node.js is only for the npm wrapper, which fetches the x86_64 or aarch64 binaries at install. Those prebuilt binaries need glibc 2.28 or newer (RHEL 8, Debian 10, Ubuntu 20.04 and later). On older systems build it with `cargo install computer-use-linux`, which needs Rust. `./install.sh` installs Rust and system packages itself. Run `computer-use-linux setup` (or the `setup_accessibility` tool) to turn on AT-SPI. Exact window focus on GNOME Wayland needs the bundled Shell extension: run `setup-window-targeting`, then log out and in. Pi's native-tools mode needs Pi 0.84.4+ on Node.js 22.19+. Plugin and Pi installs keep binaries in a private cache and don't put `computer-use-linux` on PATH.

Limits

Platform Linux only. The prebuilt x86_64 and aarch64 binaries need glibc 2.28 or newer, and older systems must build with cargo install.
Limit Tested by hand only on GNOME Shell 50.1 Wayland (Ubuntu 25.10) and niri window listing. Other desktops rest on parser and contract tests.
Not covered On stock labwc and other wlroots compositors windows come from wlr-foreign-toplevel-management, which gives no pids or bounds. Compositors without it, such as Cage, are unsupported.
Not covered move_window and resize_window need the GNOME Shell extension or the generic X11 backend.
Limit ydotool text takes printable ASCII, tab and newline only. Under non-US layouts ydotool and xdotool can type different characters than asked.
Limit On GNOME, text that any configured keyboard layout group cannot produce is rejected before typing.
Size cap Screenshots are cut to 1920 px and 2 MiB by default, with hard caps above that.
Size cap get_app_state reads 1000 nodes and 32 levels by default (caps 10000 and 64) and stops at 10 seconds, returning a truncated tree.
Limit Screenshot and RemoteDesktop portals may ask for consent, and each new process is prompted again unless grants are persisted.
Limit Codex Desktop's Linux build already bundles it. Its makers say not to register a second copy there.

Only GNOME Shell 50.1 on Ubuntu 25.10 Wayland (and niri window listing) was tested by hand. Other desktops rest on parser tests. Sway is reached through its i3 IPC. When the window backend gives no pid (i3, Sway, COSMIC), a targeted click or action needs a snapshot taken for that window. ydotool text accepts printable ASCII, tab and newline only and can mistype under non-US layouts. `move_window` and `resize_window` need the GNOME Shell extension or the generic X11 backend. Screenshots are cut to 1920 px and 2 MiB by default, and screenshot portals may ask for consent.

Who it suits

Suits someone who wants an agent to operate their own Linux desktop apps, especially on GNOME Wayland, and is willing to enable AT-SPI and set up a portal or ydotool. It drives the live session with the user's full desktop reach, so it fits supervised use better than unattended agents. For a desktop the agent does not share with you, its makers point to agent-workspace-linux as the sibling project for an isolated workspace. Claude Code, Codex and Pi users get the quickest setup through the plugins or `pi install`. On macOS or Windows it does not apply.

Tools 21

6 only read, 14 change things, 1 can delete or overwrite. A ? marks an optional parameter.

doctor: Return a JSON report on platform, portals, accessibility, windowing and input readiness
setup_accessibility changes: Turn on GNOME's toolkit accessibility setting so apps expose AT-SPI trees
setup_window_targeting changes: Install and enable the bundled GNOME Shell extension for exact window control
list_apps: List running desktop apps known to the AT-SPI registry
get_tree_page: Page through the last cached accessibility tree when it was too large to return inlineoffset?: integer
list_windows: List windows with title, app id, class, focus, client type, bounds and terminal details
focused_window: Report the window that has keyboard focus
activate_window changes: Focus a window by id, pid, app id, class, title or terminalwindow_id?: integerpid?: integerapp_id?: stringwm_class?: stringtitle?: stringtty?: stringterminal_pid?: integerterminal_command?: stringterminal_cwd?: stringscreenshot_after?: anyinclude_tree?: boolean
get_app_state: Return a screenshot and an indexed accessibility tree for an app or windowapp_name_or_bundle_identifier?: stringwindow_id?: integerpid?: integerapp_id?: stringwm_class?: stringtitle?: stringmax_nodes?: integermax_depth?: integerinclude_screenshot?: booleanmax_width?: integermax_height?: integermax_bytes?: integerformat?: enumverbose?: boolean
screenshot changes: Capture the screen, or raise and crop one window, as a size-limited PNG or JPEGwindow_id?: integerpid?: integerapp_id?: stringwm_class?: stringtitle?: stringraise_window?: booleanfull_screen?: booleanmax_width?: integermax_height?: integermax_bytes?: integerscale?: numberformat?: enumquality?: integer
click changes: Click an element by index or selector, or at screen or window coordinateselement_index?: integerrole?: stringname?: stringtext?: stringstates?: arrayx?: integery?: integercoordinate_space?: enumbutton?: stringclick_count?: integerwindow_id?: integerapp_id?: stringwindow_title?: stringrelative?: booleanscreenshot_after?: anyinclude_tree?: boolean
drag changes: Drag the pointer from one screen coordinate to anotherstart_x: integerstart_y: integerend_x: integerend_y: integerscreenshot_after?: anyinclude_tree?: boolean
scroll changes: Scroll by pages on an element, a window or a screen positiondirection: stringpages?: numberelement_index?: integerx?: integery?: integerwindow_id?: integerapp_id?: stringwindow_title?: stringrelative?: booleanscreenshot_after?: anyinclude_tree?: boolean
press_key changes: Send a key or key chord, optionally focusing a window or terminal firstkey: stringwindow_id?: integerpid?: integerapp_id?: stringwm_class?: stringtitle?: stringtty?: stringterminal_pid?: integerterminal_command?: stringterminal_cwd?: stringscreenshot_after?: anyinclude_tree?: boolean
type_text changes: Type literal text, optionally into a chosen window or terminaltext: stringwindow_id?: integerpid?: integerapp_id?: stringwm_class?: stringtitle?: stringtty?: stringterminal_pid?: integerterminal_command?: stringterminal_cwd?: stringscreenshot_after?: anyinclude_tree?: boolean
perform_action changes: Invoke an accessibility action such as Press or Toggle on an elementelement_index?: integerrole?: stringname?: stringtext?: stringstates?: arrayelement_identifier?: stringaction?: stringscreenshot_after?: anyinclude_tree?: boolean
set_value changes: Write a new value into an editable field, slider or spinner and read it backelement_index?: integerrole?: stringname?: stringtext?: stringstates?: arrayelement_identifier?: stringvalue: stringscreenshot_after?: anyinclude_tree?: boolean
move_window changes: Move a window's frame to a new desktop positionwindow_id?: integerpid?: integerapp_id?: stringwm_class?: stringtitle?: stringx: integery: integerscreenshot_after?: anyinclude_tree?: boolean
resize_window changes: Change a window's frame size, unmaximizing it first if neededwindow_id?: integerpid?: integerapp_id?: stringwm_class?: stringtitle?: stringwidth: integerheight: integerscreenshot_after?: anyinclude_tree?: boolean
run_shell deletes: Run one /bin/sh command as the server's user, registered only with COMPUTER_USE_LINUX_ENABLE_SHELL=1command: stringcwd?: stringenv?: objecttimeout_seconds?: integer
complete_interaction changes: Send a desktop notification that the agent is done, registered only with COMPUTER_USE_LINUX_NOTIFY_ON_COMPLETE=1

Popularity

74 popularity score, on our list (the bar is 36)

★ 824 ↓ 4.4k/wk

GitHub stars	824	35 points
Stars in 30 days	+158	10 points
npm downloads, last week	4,353	29.1 points

Measured 2026-11-07. How the score works.

Where it is published

Code	agent-sh/computer-use-linux ★ 824
npm	@agent-sh/computer-use-linux 0.7.47, 2026-11-05

What it can reach

Runs	On your machine
Reaches	Files on your machine, Runs shell commands
Changes things	Yes: it can write, send, create or delete
Sign-in	None
Licence	MIT (npm)
Usage data	Sends none
Last code change	2026-11-07

Runs locally as your user and drives your live desktop session. It can read any window's accessibility tree and screen pixels, then click, type, invoke actions and set field values in any app. The setup tools change a GNOME setting and install a Shell extension. run_shell is off unless COMPUTER_USE_LINUX_ENABLE_SHELL=1 and then runs /bin/sh with your file and network rights (30 s default, 120 s cap). The makers state it opens no listener, makes no outbound connections and has no telemetry. Large trees are written to private 0600 files. Any process that can reach the ydotoold socket can inject input. COMPUTER_USE_LINUX_PERSIST_REMOTE_DESKTOP=1 stores a portal restore token under ~/.local/state. The overlay shows typed text unless masked. No sign-in needed.

Over time

GitHub stars 824 +158 (+24 %)

824666

8 Oct 20267 Nov 2026

npm downloads a week 4.4k +3.8k (+630 %)

4.6k150

25 May 20267 Nov 2026

Before the site measured it, read from npm's daily download counts.

Popularity score 74 +13 (+22 %)

7456

8 Oct 20267 Nov 2026

What is rising across the directory

Sources

Page checked 2026-11-03.

Badge for your README

Shows this server's rank in its category, and stars for a server that runs on your machine. It updates every hour.

MarkdownREADME.md

[![computer-use-linux on EveryMCPServer](https://everymcpserver.com/badge/computer-use-linux.svg)](https://everymcpserver.com/servers/computer-use-linux)

Connect your client

Add in one click

Or add it by command or config

Add in one click

Or add it by command or config

Settings 12

What it does

What it needs

Limits

Who it suits

Tools 21

Similar servers

open-computer-use

Open Computer Use

computer-use-mcp

Browser Use

BrowserTools MCP

jshookmcp