Skip to content

dpkg upgrade does not restart the service — users keep running the old binary silently #305

Description

@MichaelTaylor3d

Measured on all three standing fleet boxes, 0.135.0 -> 0.138.0

After dpkg -i dig-node_0.138.0_arm64.deb:

  • dig-node --version reported 0.138.0
  • systemctl reported active/running
  • MainPID was unchanged (160064), ActiveEnterTimestamp still the previous day
  • the running process was still executing 0.135.0 code, and 0.138.0's wallet schema migration
    had not run

An explicit systemctl restart net.dignetwork.dig-node was required. Only then did MainPID move and
PRAGMA user_version advance 1 -> 3.

Why this is user-visible and not a fleet-ops detail

Every user who upgrades through the .deb — or through apt once apt.dig.net serves it — keeps running
the old binary until something else restarts it.
A reboot, eventually. Possibly never on a server.

The failure is silent and actively misleading in both directions:

  • --version reads the on-disk binary, so it reports the new version immediately
  • systemctl is-active reports active, because the old process is genuinely healthy

So the two checks a person would naturally run both say the upgrade worked. Nothing surfaces the truth
except MainPID or ActiveEnterTimestamp, which nobody checks casually.

This also means a security fix shipped via the .deb does not take effect on upgrade. That is the part
that makes this more than cosmetic.

Likely cause

The package's postinst is not invoking the service restart. Standard Debian practice is
deb-systemd-invoke restart (usually via dh_installsystemd) on configure when the unit is enabled and
running.

Fix

Restart the unit in postinst on upgrade, gated on the unit being enabled and previously active, so a
fresh install that has not yet been configured is unaffected.

Worth checking the same path on Windows and macOS — the same "new binary on disk, old process running"
shape is possible wherever the installer replaces a file without cycling the service.

Verification that would prove the fix

An upgrade where MainPID changes without a manual restart. A --version check cannot prove it, and
neither can ActiveState=active — both report success today while the old code runs.

Found during the dig_ecosystem#1927 fleet upgrade; also relevant to the auto-update path (dig-updater /
beacon), which should be checked for the same assumption.

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:businessa person can DO something new, or money moves, or a shipped surface stops lying to them

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions