# niri-computer-use

> An MCP server that lets AI agents see and act on a niri Wayland desktop, with a lease, a stop key and an audit log.

Version 0.1.1: [release notes](https://github.com/ayagmar/niri-computer-use/releases/tag/v0.1.1)

Your agent can use your niri desktop. One key takes it back. An MCP server for Claude Code, Codex, Pi and other MCP clients. The agent reads your windows and screenshots; while it holds the lease, it clicks, types and runs niri actions.

[How the stop key and the lease work](https://ayagmar.github.io/niri-computer-use/concepts/safety/)

## Recorded sessions

Recorded on a real niri 26.04 desktop with Claude Code (Opus 5.5) and the installed server; nothing was scripted. Thinking time between calls is cut to 1.2 s; the calls run in real time.

- Stop key: You press the stop key between two calls. The lease goes back to you, and the agent’s next click comes back stopped.
- Focus moved: The agent types a paragraph into nvim, then you move focus to Files. Its next key was meant for nvim, so it is refused.
- Errand: The agent selects two screenshots in Files, makes a folder for them from the right-click menu, names it release-0.1 and gives focus back.
- Layout: The agent sizes Files and the browser to a third and two thirds, then floats its own terminal small in the corner, all with niri’s own actions.
- Noctalia: The agent opens Noctalia’s control center, turns on Do Not Disturb and closes the panel.

## What an action answers

Each action says whether niri, Noctalia or the app took the request, what happened afterwards, and what niri saw change on the desktop. What launch {"preset": "notes"} answered in the README’s example run, in a headless nested niri.

```json
{
  "accepted": true,
  "observed": "one",
  "windows": [4],
  "focused_window": 4,
  "changes": {
    "focus": {"from": 3, "to": 4},
    "opened": [{"app_id": "notes", "id": 4,
                "title": "harness window", "workspace_id": 1}]
  }
}
```

- `accepted`: Whether niri, Noctalia or, for an element action, the app took the request. False when nothing was sent, null when the request went out but the reply was lost.
- `observed`: What happened afterwards, such as focused, opened or closed in niri’s event stream; for input, sent once niri has handled it; for set_element_text, matched or differs by the app’s own count.
- `changes`: What niri saw change: windows opened and closed, focus and workspace moving, a title, size or floating state. Left out when niri saw nothing.
- `error`: When an action is refused or fails, in place of the above: a stable name such as focus_mismatch, and a detail that keeps niri’s, Noctalia’s or the program’s own message. An app’s own messages from element actions are left out, since they can hold a field’s text. Wrong arguments get a plain-text error instead.

## Before anything reaches niri

Every action passes the first three checks, in this order; the last two belong to the tools. The first check that fails answers the agent, and nothing goes out.

1. Stop flag: You pressed the stop key. Applies to every action. Answer: `stopped`, the stop flag is set; the user clears it with niri-computer-use resume
2. Lease: This agent didn’t take the lease, or another one holds it. Applies to every action. Answer: `lease_required`, this session doesn't hold the lease; call acquire_desktop first
3. Lock state: The screen is locked, or neither logind nor Noctalia says whether it is. Applies to every action. Answer: `screen_locked`, the screen is locked
4. Focus: The window the agent named in expect, or an element action’s window, doesn’t have focus. Typing checks again before each key. Applies to typing, keys and pastes, checked first; element actions, checked after the deny list. Answer: `focus_mismatch`, expected window_id 19; window 11 with app_id Some("org.gnome.Nautilus")
5. Deny list: An app on your deny list has focus, or owns the window an element action or a close aims at. Applies to pointer, keyboard and element actions, and closing a window. Answer: `app_denied`, the focused window's app_id "org.keepassxc.KeePassXC" is on the policy's deny list

The details are the server’s own messages; the app_id and window ids are examples. Two more checks sit between these, for stuck input and for niri’s version; Safety lists them all.

## Guardrails, not a sandbox

What it enforces:

- One agent at a time holds the lease, and only that agent acts. Looking needs no lease, except reading the clipboard.
- The stop key cancels the running action and takes the lease back. An agent can’t press it.
- With confirm_lease on, you allow each lease from a desktop notification. It is off by default.
- Element actions refuse fields the toolkit marks as password fields.
- The launch tool starts only presets you write, and niri actions that run programs need unrestricted on.
- An audit log of every call that reaches a tool, without typed text, the clipboard, window titles or images. A failed write shows in status.

What it doesn’t stop:

- Any connected agent sees your screen and window titles.
- A click can land on a denied window that doesn’t have focus.
- Typing and clicking don’t check for password fields.
- A lock that comes after the last read isn’t caught.
- An agent that can run shell commands can answer the confirm_lease notification itself.
- With the lease, a click or a key can do anything you could do.

[Read Safety before letting an agent act](https://ayagmar.github.io/niri-computer-use/concepts/safety/)

## What it needs

- niri: 26.04. A 26.04 patch release works with a warning; any other release is read-only.
- System: Linux on x86_64 or aarch64. Release builds need glibc 2.39 or newer; cargo needs Rust 1.99.
- At run time: grim, loginctl and libxkbcommon; notify-send if you turn on confirm_lease.
- Agents: Claude Code, Codex, Pi or any MCP client that starts a server over stdio.
- Pointer tools: One monitor at transform Normal. The keyboard, screenshots and the rest work with any.
- Elements: GTK 4 works. GTK 3 and Qt apps that draw their own title bar give no element boxes.
- Noctalia: Optional. A session without Noctalia or an accessibility bus has 17 of the 22 tools.

## Install

Three steps, from a shell inside your niri session.

1. Download the install script, read it, run it. It installs the binary and the skill that tells your agents how to use the server, then offers to register the server in each of Claude Code, Pi and Codex it finds. [What the script does](https://ayagmar.github.io/niri-computer-use/start/install-script/)

   ```sh
   curl -fsSLO https://github.com/ayagmar/niri-computer-use/releases/latest/download/install.sh
   less install.sh
   sh install.sh
   ```

   Or build the crate with cargo; then add the skill. [The skill](https://ayagmar.github.io/niri-computer-use/start/install/#the-skill)

   ```sh
   cargo install --locked niri-computer-use
   ```

2. If you said no, or built with cargo, register it yourself, here in Claude Code with the binary’s path (~/.cargo/bin after cargo). [Codex, Pi and other clients](https://ayagmar.github.io/niri-computer-use/start/clients/)

   ```sh
   claude mcp add --scope user niri-computer-use -- ~/.local/bin/niri-computer-use serve
   ```

3. Bind the stop key in niri’s config.kdl.

   ```kdl
   Mod+Shift+Escape allow-inhibiting=false allow-when-locked=true { spawn "/home/you/.local/bin/niri-computer-use" "stop"; }
   ```

[Full install guide](https://ayagmar.github.io/niri-computer-use/start/install/)