refactor: SKILL audit + os-image cocoon-agent/sshd unification

KNOWN_ISSUES.md

@@ -228,3 +228,21 @@ Attaches via `cocoon vm fs attach` and `cocoon vm device attach` are runtime-onl
 ## virtiofsd is a single-shot daemon
 
 Upstream virtiofsd serves exactly one vhost-user client and exits when that client disconnects. Consequence: after `cocoon vm fs detach`, the daemon is gone — a follow-up `cocoon vm fs attach` against the same socket path will hang or time out until a fresh `virtiofsd` instance is launched. The same applies after `cocoon vm stop` (CH closes the socket on shutdown). Scripts that cycle attach/detach should respawn virtiofsd between calls. This is a virtiofsd behavior, not a cocoon limitation.
+
+## Official OS images ship with default `root:cocoon` and `PermitRootLogin yes`
+
+Every Ubuntu image under `os-image/` enables `openssh-server` with `PermitRootLogin yes` and the default `root:cocoon` credentials baked in. This is convenient for development and matches the existing OCI-image behavior, but it is **not** safe for production exposure.
+
+Mitigations for production users:
+
+- Rotate the root password (`passwd root`) and/or disable password auth (`PasswordAuthentication no`) inside the guest before exposing it.
+- Add a non-root sudo user, then flip `PermitRootLogin` back to `no`.
+- Or fork the Dockerfile and adjust the `install-agent.sh` invocation to skip the SSH config step.
+
+Control-plane traffic from cocoon-managed hosts (vk-cocoon, `cocoon vm exec`) goes through cocoon-agent over vsock and never depends on SSH credentials.
+
+## Android cocoon-agent service may be blocked by SELinux
+
+`os-image/android/{14.0,15.0}` install the cocoon-agent binary at `/system/bin/cocoon-agent` and register it via `/system/etc/init/cocoon-agent.rc`. Android's SELinux policies don't ship with a domain for cocoon-agent, so the service may run in `init`'s domain or be denied outright depending on the redroid build.
+
+If `cocoon vm exec` against an Android VM returns `dial agent: ...`, check `logcat | grep -i avc` inside the guest. The fix is build-time — adjust the Android sepolicy to grant the new binary network/socket permissions — and is out of scope for the Dockerfile.

README.md

@@ -329,7 +329,7 @@ $ cocoon vm exec -e FOO=bar myvm -- sh -c 'echo $FOO'
 bar
 ```
 
-Requires cocoon-agent to be running inside the guest (already baked into the official `ghcr.io/cocoonstack/cocoon/ubuntu:24.04` image and started via systemd). Windows guests are not yet supported.
+Requires cocoon-agent to be running inside the guest. All official `ghcr.io/cocoonstack/cocoon/ubuntu:*` and `ghcr.io/cocoonstack/cocoon/android:*` images now bake the binary and enable it on boot (systemd unit on Ubuntu, init.rc service on Android). Windows guests are not yet supported.
 
 ### Logs Flags
 
@@ -418,7 +418,7 @@ Cloudimg VMs receive a NoCloud cidata disk (FAT12 with `CIDATA` volume label) co
 
 The cidata disk is **automatically excluded on subsequent boots** — after the first successful start, the VM record is marked as `first_booted` and the cidata disk is no longer attached, preventing cloud-init from re-running.
 
-Note: `--user`/`--password` only apply to **cloudimg** VMs (cloud-init). OCI VM images bake credentials at build time. Host-to-guest control plane operations (kubectl exec, kubectl logs) go through cocoon-agent over vsock, not SSH. Only `os-image/ubuntu/24.04-picoclaw` ships sshd; its credentials are documented via `LABEL cocoon.ssh.username` / `cocoon.ssh.password` in the Dockerfile so glance and other SSH-aware tooling can populate their own credential stores.
+Note: `--user`/`--password` only apply to **cloudimg** VMs (cloud-init). OCI VM images bake credentials at build time — every official `os-image/ubuntu/*` image ships `openssh-server` enabled with `PermitRootLogin yes` and the default `root:cocoon` credentials. Host-to-guest control plane operations (kubectl exec, kubectl logs) prefer cocoon-agent over vsock; SSH stays available as the human-on-keyboard path.
 
 ## Data Disks
 
@@ -807,7 +807,7 @@ cocoon image pull ghcr.io/cocoonstack/cocoon/ubuntu:24.04
 cocoon image pull ghcr.io/cocoonstack/cocoon/ubuntu:22.04
 ```
 
-These images include kernel, initramfs, and a systemd-based rootfs with an overlayfs boot script.
+These images include kernel, initramfs, and a systemd-based rootfs with an overlayfs boot script. Every official OS image (Ubuntu + Android) bakes `cocoon-agent` (vsock exec) with auto-start; Ubuntu images additionally enable `sshd` with `PermitRootLogin yes` so `ssh root@<vm>` works out of the box (default `root:cocoon`).
 
 ## Shell Completion
 

cmd/images/handler.go

@@ -2,6 +2,11 @@ package images
 
 import cmdcore "github.com/cocoonstack/cocoon/cmd/core"
 
+// imageType identifies the content type detected from a stream.
+type imageType int
+
+type importSourceKind int
+
 const (
 	// digestDisplayLen = len("sha256:") + 12 hex digits for compact display.
 	digestDisplayLen = 19
@@ -18,11 +23,6 @@ type Handler struct {
 	cmdcore.BaseHandler
 }
 
-// imageType identifies the content type detected from a stream.
-type imageType int
-
-type importSourceKind int
-
 type importLocalPlan struct {
 	kind  importSourceKind
 	files []string

hypervisor/cloudhypervisor/config.go

@@ -19,7 +19,7 @@ func NewConfig(conf *config.Config) *Config {
 
 func (c *Config) BinaryName() string { return filepath.Base(c.CHBinary) }
 
-func (c *Config) PIDFileName() string { return "ch.pid" }
+func (c *Config) PIDFileName() string { return pidFileName }
 
 func (c *Config) COWRawPath(vmID string) string {
 	return filepath.Join(c.VMRunDir(vmID), "cow.raw")

hypervisor/cloudhypervisor/helper.go

@@ -16,18 +16,19 @@ import (
 	"github.com/cocoonstack/cocoon/utils"
 )
 
+// chMemoryRestoreMode controls how CH restores guest memory from a snapshot.
+type chMemoryRestoreMode string
+
 const (
+	pidFileName     = "ch.pid"
 	cmdlineFileName = "cmdline"
 
 	// chMemoryRestoreOnDemand uses userfaultfd (UFFD) to lazily page in
 	// guest memory from the snapshot file, avoiding a full upfront copy.
 	chMemoryRestoreOnDemand chMemoryRestoreMode = "OnDemand"
 )
 
-var runtimeFiles = []string{hypervisor.APISocketName, "ch.pid", hypervisor.ConsoleSockName, cmdlineFileName, hypervisor.VsockSockName}
-
-// chMemoryRestoreMode controls how CH restores guest memory from a snapshot.
-type chMemoryRestoreMode string
+var runtimeFiles = []string{hypervisor.APISocketName, pidFileName, hypervisor.ConsoleSockName, cmdlineFileName, hypervisor.VsockSockName}
 
 type chRestoreConfig struct {
 	SourceURL         string              `json:"source_url"`

hypervisor/utils.go

@@ -22,6 +22,9 @@ import (
 	"github.com/cocoonstack/cocoon/utils"
 )
 
+// SnapshotFileKind classifies a snapshot file for CloneSnapshotFiles.
+type SnapshotFileKind int
+
 const (
 	// SnapshotFileMemory is a read-only memory/state file (hard link or symlink).
 	SnapshotFileMemory SnapshotFileKind = iota
@@ -41,9 +44,6 @@ const (
 	socketReadyPollInterval = 1 * time.Millisecond
 )
 
-// SnapshotFileKind classifies a snapshot file for CloneSnapshotFiles.
-type SnapshotFileKind int
-
 func RemoveVMDirs(runDir, logDir string) error {
 	return errors.Join(
 		os.RemoveAll(runDir),

images/oci/commit.go

@@ -94,16 +94,16 @@ func commitAndRecord(conf *Config, idx *imageIndex, ref string, manifestDigest i
 		}
 		totalSize += size
 	}
-	if size, err := validFileSize(conf.KernelPath(kernelLayer.Hex())); err != nil {
+	size, err := validFileSize(conf.KernelPath(kernelLayer.Hex()))
+	if err != nil {
 		return fmt.Errorf("kernel missing for %s (concurrent GC?)", kernelLayer)
-	} else {
-		totalSize += size
 	}
-	if size, err := validFileSize(conf.InitrdPath(initrdLayer.Hex())); err != nil {
+	totalSize += size
+	size, err = validFileSize(conf.InitrdPath(initrdLayer.Hex()))
+	if err != nil {
 		return fmt.Errorf("initrd missing for %s (concurrent GC?)", initrdLayer)
-	} else {
-		totalSize += size
 	}
+	totalSize += size
 
 	idx.Images[ref] = &imageEntry{
 		Ref:            ref,

network/bridge/bridge_other.go

@@ -19,7 +19,7 @@ type Bridge struct{}
 
 // New returns an error on non-Linux.
 func New(_ *config.Config, _ string) (*Bridge, error) {
-	return nil, fmt.Errorf("bridge TAP networking requires Linux (running on %s)", runtime.GOOS)
+	return nil, errUnsupported
 }
 
 // Type returns the provider identifier.

os-image/README.md

@@ -68,6 +68,15 @@ IMAGE_NAME="ghcr.io/cocoonstack/cocoon/ubuntu:24.04" bash start.sh
 IMAGE_NAME="ghcr.io/cocoonstack/cocoon/android:14.0" bash start.sh
 ```
 
+## In-VM Services
+
+Every official OS image bakes the following on top of its base distro:
+
+- **cocoon-agent** (vsock exec) — pinned binary from [cocoonstack/cocoon-agent](https://github.com/cocoonstack/cocoon-agent), auto-started on boot. Backs `cocoon vm exec` (kubectl-style stdin/stdout/stderr/exit, no SSH/network dependency). Ubuntu uses a systemd unit; Android uses `/system/etc/init/cocoon-agent.rc`.
+- **sshd** *(Ubuntu only)* — `openssh-server` enabled with `PermitRootLogin yes`. Default credentials are `root:cocoon`. SSH covers the human-on-keyboard case while cocoon-agent handles control-plane traffic.
+
+Default credentials apply to fresh VMs. If you fork an image you should rotate the root password and (if you keep sshd) flip `PermitRootLogin` back to `no` once you have a non-root sudoer.
+
 ## DHCP and VM Cloning
 
 All Ubuntu images configure systemd-networkd with `ClientIdentifier=mac` in their DHCP settings. This ensures that when a VM is cloned from a snapshot, each clone uses its unique MAC address as the DHCP client identifier instead of the machine-id-derived DUID. Without this, clones from the same snapshot share an identical DUID and dnsmasq treats them as a single client, causing IP conflicts.

os-image/android/14.0/Dockerfile

@@ -16,17 +16,21 @@ FROM ubuntu:24.04 AS builder
 
 ENV DEBIAN_FRONTEND=noninteractive
 
+ARG COCOON_AGENT_VERSION=0.1.0
+
 RUN --mount=type=secret,id=cocoon_overlay \
     --mount=type=secret,id=cocoon_network \
     --mount=type=secret,id=cocoon_init_wrapper \
     --mount=type=secret,id=cocoon_network_rc \
     --mount=type=secret,id=cocoon_omx_fix_rc \
     --mount=type=secret,id=cocoon_disable_bt_rc \
+    --mount=type=secret,id=cocoon_agent_rc \
     # --- Install kernel + initramfs tooling ---
     apt-get update && apt-get install -y --no-install-recommends \
     linux-image-generic \
     initramfs-tools \
     busybox-static \
+    ca-certificates curl \
     && \
     apt-get install -y --no-install-recommends \
         linux-modules-extra-$(ls /lib/modules/ | head -1) \
@@ -38,6 +42,7 @@ RUN --mount=type=secret,id=cocoon_overlay \
     { \
       echo erofs; echo overlay; echo ext4; \
       echo virtio_blk; echo virtio_pci; echo virtio_ring; echo virtio_net; \
+      echo vsock; echo vmw_vsock_virtio_transport; \
       echo binder_linux; echo loop; \
     } >> /etc/initramfs-tools/modules && \
     # netfilter: Android netd requires iptables tables.
@@ -68,6 +73,11 @@ RUN --mount=type=secret,id=cocoon_overlay \
     cp /run/secrets/cocoon_network_rc /output/system/etc/init/cocoon-network.rc && \
     cp /run/secrets/cocoon_omx_fix_rc /output/system/etc/init/cocoon-omx-fix.rc && \
     cp /run/secrets/cocoon_disable_bt_rc /output/system/etc/init/cocoon-disable-bt.rc && \
+    # --- cocoon-agent (vsock exec) — Android amd64 only ---
+    cp /run/secrets/cocoon_agent_rc /output/system/etc/init/cocoon-agent.rc && \
+    curl -fsSL "https://github.com/cocoonstack/cocoon-agent/releases/download/v${COCOON_AGENT_VERSION}/cocoon-agent_${COCOON_AGENT_VERSION}_Linux_x86_64.tar.gz" \
+        | tar -xz -C /output/system/bin/ cocoon-agent && \
+    chmod 0755 /output/system/bin/cocoon-agent && \
     rm -rf /var/lib/apt/lists/*
 
 # ---- Stage 2: Android rootfs + single cocoon layer ----

os-image/android/15.0/Dockerfile

@@ -16,17 +16,21 @@ FROM ubuntu:24.04 AS builder
 
 ENV DEBIAN_FRONTEND=noninteractive
 
+ARG COCOON_AGENT_VERSION=0.1.0
+
 RUN --mount=type=secret,id=cocoon_overlay \
     --mount=type=secret,id=cocoon_network \
     --mount=type=secret,id=cocoon_init_wrapper \
     --mount=type=secret,id=cocoon_network_rc \
     --mount=type=secret,id=cocoon_omx_fix_rc \
     --mount=type=secret,id=cocoon_disable_bt_rc \
+    --mount=type=secret,id=cocoon_agent_rc \
     # --- Install kernel + initramfs tooling ---
     apt-get update && apt-get install -y --no-install-recommends \
     linux-image-generic \
     initramfs-tools \
     busybox-static \
+    ca-certificates curl \
     && \
     apt-get install -y --no-install-recommends \
         linux-modules-extra-$(ls /lib/modules/ | head -1) \
@@ -38,6 +42,7 @@ RUN --mount=type=secret,id=cocoon_overlay \
     { \
       echo erofs; echo overlay; echo ext4; \
       echo virtio_blk; echo virtio_pci; echo virtio_ring; echo virtio_net; \
+      echo vsock; echo vmw_vsock_virtio_transport; \
       echo binder_linux; echo loop; \
     } >> /etc/initramfs-tools/modules && \
     # netfilter: Android netd requires iptables tables.
@@ -68,6 +73,11 @@ RUN --mount=type=secret,id=cocoon_overlay \
     cp /run/secrets/cocoon_network_rc /output/system/etc/init/cocoon-network.rc && \
     cp /run/secrets/cocoon_omx_fix_rc /output/system/etc/init/cocoon-omx-fix.rc && \
     cp /run/secrets/cocoon_disable_bt_rc /output/system/etc/init/cocoon-disable-bt.rc && \
+    # --- cocoon-agent (vsock exec) — Android amd64 only ---
+    cp /run/secrets/cocoon_agent_rc /output/system/etc/init/cocoon-agent.rc && \
+    curl -fsSL "https://github.com/cocoonstack/cocoon-agent/releases/download/v${COCOON_AGENT_VERSION}/cocoon-agent_${COCOON_AGENT_VERSION}_Linux_x86_64.tar.gz" \
+        | tar -xz -C /output/system/bin/ cocoon-agent && \
+    chmod 0755 /output/system/bin/cocoon-agent && \
     rm -rf /var/lib/apt/lists/*
 
 # ---- Stage 2: Android rootfs + single cocoon layer ----

os-image/android/agent.rc

@@ -0,0 +1,12 @@
+# /system/etc/init/cocoon-agent.rc
+#
+# Cocoon vsock exec agent (host-side `cocoon vm exec` listens on AF_VSOCK).
+# Persistent service: init restarts the binary if it exits.
+
+service cocoon-agent /system/bin/cocoon-agent serve
+    class core
+    user root
+    group root
+
+on boot
+    start cocoon-agent

os-image/ubuntu/22.04/Dockerfile

@@ -1,12 +1,14 @@
 # Use the latest Ubuntu 22.04 (Jammy) LTS
 FROM ubuntu:22.04
 
+ARG TARGETARCH
 ENV DEBIAN_FRONTEND=noninteractive
 
 # Combined System Setup (Single Layer)
 # Install packages first so /etc/initramfs-tools/scripts/ exists, then inject the hook.
 RUN --mount=type=secret,id=cocoon_overlay \
     --mount=type=secret,id=cocoon_network \
+    --mount=type=secret,id=cocoon_install_agent \
     apt-get update && apt-get install -y --no-install-recommends \
         linux-image-virtual \
         initramfs-tools \
@@ -16,13 +18,15 @@ RUN --mount=type=secret,id=cocoon_overlay \
         udev \
         kmod \
         iproute2 \
+        openssh-server \
+        ca-certificates curl \
     && \
     cp /run/secrets/cocoon_overlay /etc/initramfs-tools/scripts/cocoon-overlay && \
     chmod 0755 /etc/initramfs-tools/scripts/cocoon-overlay && \
     cp /run/secrets/cocoon_network /etc/initramfs-tools/scripts/init-bottom/cocoon-network && \
     chmod 0755 /etc/initramfs-tools/scripts/init-bottom/cocoon-network && \
     # [Kernel Config]
-    printf "erofs\noverlay\next4\nvirtio_blk\nvirtio_pci\nvirtio_ring\nvirtio_net\n" >> /etc/initramfs-tools/modules && \
+    printf "erofs\noverlay\next4\nvirtio_blk\nvirtio_pci\nvirtio_ring\nvirtio_net\nvsock\nvmw_vsock_virtio_transport\n" >> /etc/initramfs-tools/modules && \
     sed -i 's/^COMPRESS=.*/COMPRESS=gzip/' /etc/initramfs-tools/initramfs.conf && \
     # [Networking] IP=off prevents initramfs from running DHCP during boot.
     # Kernel ip= parameters (when present) override this and still trigger ipconfig.
@@ -36,6 +40,8 @@ RUN --mount=type=secret,id=cocoon_overlay \
     systemctl enable systemd-networkd systemd-resolved systemd-timesyncd && \
     mkdir -p /etc/systemd/network && \
     printf "[Match]\nName=e* v*\n[Network]\nDHCP=yes\n\n[DHCPv4]\nClientIdentifier=mac\n" > /etc/systemd/network/20-wired.network && \
+    # [Cocoon agent + sshd] vsock exec daemon and SSH access.
+    sh /run/secrets/cocoon_install_agent && \
     # [Final Touches]
     echo 'root:cocoon' | chpasswd && \
     rm -rf /var/lib/apt/lists/*

os-image/ubuntu/24.04-chrome/Dockerfile

@@ -5,6 +5,7 @@ ENV DEBIAN_FRONTEND=noninteractive
 
 RUN --mount=type=secret,id=cocoon_overlay \
     --mount=type=secret,id=cocoon_network \
+    --mount=type=secret,id=cocoon_install_agent \
     apt-get update && apt-get install -y --no-install-recommends \
     linux-image-virtual \
     initramfs-tools \
@@ -13,6 +14,7 @@ RUN --mount=type=secret,id=cocoon_overlay \
     software-properties-common htop ncdu nload net-tools \
     openbox \
     xserver-xorg-core dbus-x11 xrdp xorgxrdp \
+    openssh-server ca-certificates \
     && \
     ARCH=${TARGETARCH:-$(dpkg --print-architecture)}; \
     if [ "$ARCH" = "amd64" ]; then \
@@ -32,7 +34,7 @@ RUN --mount=type=secret,id=cocoon_overlay \
     chmod 0755 /etc/initramfs-tools/scripts/cocoon-overlay && \
     cp /run/secrets/cocoon_network /etc/initramfs-tools/scripts/init-bottom/cocoon-network && \
     chmod 0755 /etc/initramfs-tools/scripts/init-bottom/cocoon-network && \
-    printf "erofs\nlz4\nlz4_compress\nzstd\nzstd_compress\noverlay\next4\nvirtio_blk\nvirtio_pci\nvirtio_ring\nvirtio_net\nvirtio_gpu\n" >> /etc/initramfs-tools/modules && \
+    printf "erofs\nlz4\nlz4_compress\nzstd\nzstd_compress\noverlay\next4\nvirtio_blk\nvirtio_pci\nvirtio_ring\nvirtio_net\nvirtio_gpu\nvsock\nvmw_vsock_virtio_transport\n" >> /etc/initramfs-tools/modules && \
     sed -i 's/^COMPRESS=.*/COMPRESS=gzip/' /etc/initramfs-tools/initramfs.conf && \
     sed -i '/^IP=/d' /etc/initramfs-tools/initramfs.conf && \
     echo 'IP=off' >> /etc/initramfs-tools/initramfs.conf && \
@@ -49,6 +51,7 @@ RUN --mount=type=secret,id=cocoon_overlay \
     sed -i 's/^tcp_nodelay=.*/tcp_nodelay=true/' /etc/xrdp/xrdp.ini && \
     adduser xrdp ssl-cert && \
     systemctl enable xrdp && \
+    sh /run/secrets/cocoon_install_agent && \
     echo 'root:cocoon' | chpasswd && \
     rm -rf /var/lib/apt/lists/* && \
     mkdir -p /var/lib/apt/lists/partial /var/cache/apt/archives/partial

os-image/ubuntu/24.04-picoclaw/Dockerfile

@@ -1,11 +1,11 @@
 FROM ubuntu:24.04
-LABEL cocoon.ssh.username="root" cocoon.ssh.password="cocoon"
 
 ARG TARGETARCH
 ENV DEBIAN_FRONTEND=noninteractive
 
 RUN --mount=type=secret,id=cocoon_overlay \
     --mount=type=secret,id=cocoon_network \
+    --mount=type=secret,id=cocoon_install_agent \
     --mount=type=secret,id=picoclaw_service \
     --mount=type=secret,id=config_json \
     --mount=type=secret,id=picoclaw_init \
@@ -15,7 +15,7 @@ RUN --mount=type=secret,id=cocoon_overlay \
     systemd systemd-sysv systemd-timesyncd systemd-resolved \
     udev kmod iproute2 iputils-ping curl wget arping tcpdump \
     software-properties-common htop ncdu nload net-tools \
-    openssh-server \
+    openssh-server ca-certificates \
     openbox \
     xserver-xorg-core dbus-x11 xrdp xorgxrdp \
     # Chinese language support
@@ -58,7 +58,7 @@ RUN --mount=type=secret,id=cocoon_overlay \
     chmod 0755 /etc/initramfs-tools/scripts/cocoon-overlay && \
     cp /run/secrets/cocoon_network /etc/initramfs-tools/scripts/init-bottom/cocoon-network && \
     chmod 0755 /etc/initramfs-tools/scripts/init-bottom/cocoon-network && \
-    printf "erofs\nlz4\nlz4_compress\nzstd\nzstd_compress\noverlay\next4\nvirtio_blk\nvirtio_pci\nvirtio_ring\nvirtio_net\nvirtio_gpu\n" >> /etc/initramfs-tools/modules && \
+    printf "erofs\nlz4\nlz4_compress\nzstd\nzstd_compress\noverlay\next4\nvirtio_blk\nvirtio_pci\nvirtio_ring\nvirtio_net\nvirtio_gpu\nvsock\nvmw_vsock_virtio_transport\n" >> /etc/initramfs-tools/modules && \
     sed -i 's/^COMPRESS=.*/COMPRESS=gzip/' /etc/initramfs-tools/initramfs.conf && \
     sed -i '/^IP=/d' /etc/initramfs-tools/initramfs.conf && \
     echo 'IP=off' >> /etc/initramfs-tools/initramfs.conf && \
@@ -77,10 +77,8 @@ RUN --mount=type=secret,id=cocoon_overlay \
     sed -i 's/^tcp_nodelay=.*/tcp_nodelay=true/' /etc/xrdp/xrdp.ini && \
     adduser xrdp ssl-cert && \
     systemctl enable xrdp && \
-    # SSH: enable root login
-    mkdir -p /run/sshd && \
-    sed -i 's/^#*PermitRootLogin.*/PermitRootLogin yes/' /etc/ssh/sshd_config && \
-    systemctl enable ssh && \
+    # SSH + cocoon-agent (vsock exec): unified install script.
+    sh /run/secrets/cocoon_install_agent && \
     # Root password
     echo 'root:cocoon' | chpasswd && \
     # PicoClaw: config + service + init script (NOT enabled until picoclaw-init runs)