Skip to content

OpenCode Desktop MCP

Status: Working (built 2026-08-09). Custom Hyprland desktop-control MCP server, built instead of installing agent-sh/computer-use-linux.

What it is

  • /home/dini/.local/bin/desktop-mcp — Python, stdlib only, raw JSON-RPC over stdio, zero deps
  • Registered in opencode.json as desktop (type local)
  • 12 tools: list_windows, focused_window, focus_window, window_info, screenshot, ocr_screen, ocr_window, move, click, scroll, type_text, key
  • Vision = grim + tesseract OCR (model has no image input; OCR returns text it can read)
  • Screenshots → /tmp/opencode/desktop/
  • Input injection via ydotool → ydotoold user service (systemctl --user start ydotool), socket /run/user/1000/.ydotool_socket
  • User is already in input group; /dev/uinput accessible

Hyprland 0.56.1 gotchas (all fixed in the server)

Gotcha Fix
Legacy hyprctl dispatch focuswindow address:0x.. broken: ')' expected near 'address' Lua form hl.dsp.focus({ window = "title:..." }); raw addresses get window not found, use title: / class: prefixes
hyprctl clients -j has no focused key Use focusHistoryID == 0
Window geometry clamp used wrong monitor Clamp per client's own monitor index
ydotool mousemove --absolute scales coords ~2× on dual-monitor (500→1000) move tool uses native hl.dsp.cursor.move({ x = .., y = .. }) — pixel exact
ydotool scroll: this version has no scroll command Wheel via ydotool mousemove --wheel
follow_mouse = 1 in ~/.config/hypr/modules/input.lua — focus follows cursor over windows Park cursor inside target window before typing (move tool)

Known issues / future fixes

  • Chrome swallows injected key events intermittently. First ctrl+t worked, later ctrl+l / typing / enter / ctrl+t produced nothing visible, despite correct focus (chrome = fh 0, cursor inside window). ydotool alive, typing into kitty terminals works fine. Suspects: Chrome-specific delivery quirk, uinput device state after cursor moves, or fast chord timing. Workaround: use google-chrome-stable --new-tab URL / CLI for browser navigation. Debug next session: test key events into a fresh scratch kitty vs chrome, add --next-delay to chords, check hyprctl deviceinfo.
  • click / scroll implemented but never live-tested on a real target.
  • AT-SPI accessibility-tree reading (semantic app state) not implemented — that's the main feature gap vs agent-sh/computer-use-linux. dasbus is available if needed.

Future ideas

  • Voice control of the desktop MCP — combine [[OpenCode Voice Input]] (Whisper STT) with desktop-mcp tools: "open abc news" → transcribe → list_windows + CLI launch. Natural next step once STT is in.
  • Move server under ~/.config/opencode/ for consistency.
  • Optional --with translate proxy for YouTube transcripts (translate endpoint currently IP-blocked).