dsh-pve-vm

dsh plugin --profile web add dsh-pve-vm

dsh-pve-vm

Proxmox VE VM control for DeepSeek Harness: see a VM's screen and click in its console over VNC, plus commands, files and typed text through the QEMU guest agent.

Almost every computer-use plugin drives the desktop the agent itself runs on — and therefore needs

X11 or Wayland, a screenshot tool and an input injector on that host. This plugin does something

different: it drives the console of a virtual machine on a Proxmox VE node, over one ssh

connection. The host can be a headless server, a Wayland desktop or a laptop in another country;

the guest can be Windows, Linux with any GUI, or a machine that has not booted yet.

agent ──► plugin ──► ssh ──► node: qm vncproxy ──► unix socket /var/run/qemu-server/<vmid>.vnc
                      └────► node: qm guest exec ──► qemu-guest-agent inside the VM

What you get

• Pixels — console_shot returns the VM screen as an inline image (with region and scale

to keep the token cost sane), and mouse/keyboard tools operate in VM screen coordinates: move,

click, double click, drag, wheel, type, key combinations, and a batch runner that performs a

chain of actions and returns a single frame at the end.

• Commands — vm_exec, vm_file_read, vm_file_write, vm_status through the QEMU guest

agent. Deterministic and cheap: use pixels only where a GUI really is in the way.

• Reliable input — uppercase goes through Caps Lock (qm monitor → HMP sendkey), shifted

symbols through RFB keysyms with a settle delay. This is what makes typing a password on the

Windows login screen work instead of producing xxXX → xxxX.

• Unicode text — a Windows guest gets a scheduled task in the logged-in user's session

(SendKeys), a Linux guest gets xdotool/wtype through the guest agent. Layout and

modifiers stop mattering.

• Clipboard — console_paste puts UTF-8 text into the guest clipboard and presses Ctrl+V;

console_clipboard reads the guest clipboard back, so text can leave the VM without OCR. The guest

agent is the primary channel (a scheduled task in the logged-in Windows session, xsel/xclip/

wl-copy/wl-paste on Linux), so it also works on SPICE VMs (vga: qxl) where the VNC clipboard

cannot reach the guest. The VNC clipboard (clipboard=vnc) remains as a fallback.

• A settings card in the web UI (English, Russian and Chinese labels), backed by the profile patch

layer.

• A live screen panel — the “VM screen” tab in the right sidebar shows the frames the agent

sees, taken from the very same VNC session, while the agent works. It follows the agent's VM

automatically or stays on the VM you pick by hand. Watching only: the panel never clicks or types.

• Zero runtime dependencies — RFB, DES authentication, keysym tables and PNG encoding are

implemented in this package on top of node:crypto and node:zlib. There is nothing to install

on the host, and nothing to install on the node either.

Verified against pve-manager 8.4.21 with Windows 11 / Windows Server 2025 and Linux guests, on

DeepSeek Harness 0.2.0-rc.2.

Requirements

What · Why

DeepSeek Harness ≥ 0.2.0-rc.2 · The plugin uses the 0.2.x plugin contract (defineTool, ctx.tools, ctx.subprocess, ctx.attachments, the computerUse slot, client-side configForms). It does not load on 0.1.x.

Node.js ≥ 22 · The harness runtime.

A Proxmox VE node with the qm CLI · /usr/sbin/qm vncproxy, qm guest exec, qm monitor. Containers (LXC) are not supported.

SSH access to that node as root (or an account that may run qm) · qm needs root on the node. Key-based access is expected (BatchMode, no password prompts).

**AcceptEnv LC_* in the node's sshd_config** · The VNC password is handed to qm as the LC_PVE_TICKET environment variable via SendEnv, so it never appears in ps. Without AcceptEnv, ssh silently drops it and the console cannot authenticate.

Host key already accepted · Connections run with BatchMode=yes; an unknown host key fails. Run ssh <node> once by hand, or add StrictHostKeyChecking=accept-new to sshExtraOptions.

qemu-guest-agent inside the guest (optional) · Only the vm_* tools need it. Console tools work without any agent at all.

Install

# 1. Add the bundle
dsh plugin --profile web add github:taxah92/dsh-pve-vm

# 2. Make sure the bundle is enabled in the profile: dsh.profile.bundles must list "dsh-pve-vm"
#    (the plugin manager / market toggle does the same thing)

# 3. Restart the harness:  the host half of a plugin is loaded once per process

The plugin adds one row to the profile patch layer:

- id: pve-vm
  name: dsh-pve-vm

The row id pve-vm is the settings namespace — do not rename it, or the settings card and the

patch layer will stop finding each other.

Configure

Settings ▸ Plugins ▸ Proxmox VE in the web UI. Everything is written into the profile patch

layer, and the same values can be edited there by hand:

- id: pve-vm
  name: dsh-pve-vm
  config:
    sshHost: proxmox            # alias from ~/.ssh/config, or an address
    sshUser: root
    sshPort: 22
    sshKeyPath: ''              # empty — default keys
    sshConfigFile: ''           # like `ssh -F`
    sshExtraOptions: []         # extra -o options, one per item
    allowVmids: [100, 101]      # VMs the plugin may touch; empty list forbids everything
    enableConsole: true
    enableGuest: true
    inputChannel: auto          # auto | hmp | vnc
    sendkeyDelayMs: 80
    consoleTimeoutMs: 20000
    sessionIdleMs: 300000
    guestTimeoutMs: 60000
    guestTypeCommand: ''        # script for typing non-ASCII text into a Linux guest
    viewAutoOpen: false         # open the "VM screen" tab by itself when the agent touches a console
    viewFollowAgent: true       # the panel follows the VM the agent 

…(truncated)