feat(system): add nerdctl system df - #5130
Conversation
38b701c to
5f44c03
Compare
5f44c03 to
9f63ef5
Compare
|
Please rebase |
9f63ef5 to
472c30e
Compare
There was a problem hiding this comment.
Pull request overview
Adds Docker-compatible nerdctl system df reporting for images, containers, volumes, and BuildKit cache.
Changes:
- Implements summary, verbose, JSON, and templated table output.
- Adds per-resource disk-usage accounting and shared-size calculations.
- Adds API types, documentation, unit tests, and integration tests.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
pkg/formatter/table_test.go |
Tests table-template formatting. |
pkg/formatter/common.go |
Adds Docker-style table templates. |
pkg/containerdutil/content.go |
Walks locally present image content. |
pkg/cmd/volume/rm.go |
Counts volume-container references. |
pkg/cmd/volume/df.go |
Calculates volume usage. |
pkg/cmd/volume/df_test.go |
Tests mounted-volume parsing. |
pkg/cmd/system/df.go |
Orchestrates and renders disk usage. |
pkg/cmd/system/df_test.go |
Tests formatting and rendering. |
pkg/cmd/image/list.go |
Counts containers per image. |
pkg/cmd/image/df.go |
Calculates deduplicated image usage. |
pkg/cmd/image/df_test.go |
Tests image accounting. |
pkg/cmd/container/df.go |
Calculates container writable-layer usage. |
pkg/cmd/builder/df.go |
Calculates BuildKit cache usage. |
pkg/cmd/builder/df_test.go |
Tests BuildKit parsing and aggregation. |
pkg/api/types/system_types.go |
Adds system-df options. |
pkg/api/types/diskusage_types.go |
Defines disk-usage result types. |
pkg/api/types/builder_types.go |
Adds builder usage options. |
docs/command-reference.md |
Documents the command. |
cmd/nerdctl/system/system.go |
Registers system df. |
cmd/nerdctl/system/system_df.go |
Implements the CLI command. |
cmd/nerdctl/system/system_df_test.go |
Adds cross-platform integration tests. |
cmd/nerdctl/system/system_df_linux_test.go |
Tests Linux volume accounting. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| du := types.VolumeDiskUsage{} | ||
|
|
||
| // The size is what we are after here, so it is always requested. | ||
| vols, err := Volumes(gOptions.Namespace, gOptions.DataRoot, gOptions.Address, true, nil) |
There was a problem hiding this comment.
Confirmed. volumeStore.List skips any volume whose rawGet fails and still returns a nil error (pkg/mountutil/volumestore/volumestore.go:219-223), so system df can under-report the volume total.
I would rather not do this one here. List is part of the VolumeStore interface, and its only caller is Volumes() in pkg/cmd/volume/list.go, shared by nerdctl volume ls, system df and shell completion (cmd/nerdctl/completion/completion.go:181). Strictness cannot be a blanket change there - completion has to stay best-effort - so it needs a new opt-in listing mode on the store, plus a decision on what volume ls should report when a volume is unreadable. That is a change to a shared subsystem with its own compatibility surface, and the best-effort behaviour predates this PR.
Proposal: a follow-up PR adding a strict listing mode to the volume store and switching the accounting path to it, keeping this PR scoped to system df. The other two comments in this review are valid and I am fixing them here.
Happy to fold it into this PR instead if maintainers prefer.
|
Needs rebasing |
472c30e to
9c583c6
Compare
Add `nerdctl system df`, the equivalent of `docker system df`, reporting how much disk
space the images, containers and local volumes of the current namespace use, plus the
BuildKit build cache.
The sizes follow the Docker v29 definitions for the containerd image store:
- the size of an image is the content present in the content store plus its unpacked
snapshots, which is the same value `nerdctl images` shows as DISK USAGE,
- the SIZE of the Images row counts every snapshot and every blob once, so it is the
space really taken on disk rather than the sum of the image sizes,
- an image is active when a container references it, and only the part of an unused
image that no other image shares is reclaimable,
- containers contribute their read-write layer only, and everything not running is
reclaimable,
- a volume is active when a container mounts it, counted once however many paths it is
mounted at, as the LINKS column of Docker is a count of containers,
- a build cache record is reclaimable when it is neither in use nor shared.
An image reports when it was built, read from the config of the manifest the platform
matcher of this host selects: the platforms of an index are not necessarily built
together, and the creation time of the local record only says when the image was pulled
or tagged. The record time stays as the fallback for the images that state nothing.
Both `--format` and `-v/--verbose` are supported, including the Docker `table TEMPLATE`
format, e.g. `table {{.Type}}\t{{.Size}}`: the literal \t and \n are expanded, the chosen
columns get a header that names them whatever the template does to the values below, and
the rows stay aligned. That helper lives in pkg/formatter so that the other commands,
which all share this gap, can adopt it. Identifiers are shortened for the table output
only, so that a custom format stays usable to look a resource up.
The work is split the way `system prune` already is: `pkg/cmd/system` orchestrates,
and each kind of resource is measured by its own package.
Closes containerd#3942
Signed-off-by: Eugene Kalinin <e.v.kalinin@gmail.com>
9c583c6 to
16033f4
Compare
Add
nerdctl system df, the equivalent ofdocker system df: the disk space taken by the images,containers and local volumes of the current namespace, plus the BuildKit build cache.
The size definitions
The issue was blocked on a question: Docker v29 changed how an image size is computed (#5027), so
what should
system dfreport? The formulas below were derived from moby/moby v29 with thecontainerd image store (
daemon/disk_usage.go,daemon/containerd/service.go) and docker/cli v29(
cli/command/formatter/disk_usage.go).Σ (Size - SharedSize)of the unreferencedΣ SizeRw(read-write layer only)Σ SizeRwof the non-activeΣ SizeΣ Sizeof the unreferencedΣ SizeΣ Sizeof those neither in use nor sharedSIZEis not the sum of the image sizes: what images share is counted once, sothe row reports the space really taken on disk.
nerdctl imagesalready shows asDISK USAGE. The two commands agree. Docker sizes an image by walkingits manifests, so the index listing them is counted in the row total while no single image is
charged for it: an unused image is never quite 100% reclaimable.
names (
repo:tag,repo@digest, plus the config digest underk8s.io), which would otherwisemultiply every count and every size.
createdof its config, not the local record time, which only says when itwas pulled or tagged and would show an old image as brand new. For a multi-platform image it is
the config of the manifest the host platform matcher selects, since the platforms of an index are
not necessarily built together; the record time stays as the fallback.
Output
-v/--verboseprints the four detail sections with Docker'sSHARED SIZE/UNIQUE SIZEcolumns.--formatalso accepts the Dockertable TEMPLATEform, e.g.table {{.Type}}\t{{.Size}}: theliteral
\tand\nare expanded, the columns get an aligned header, and the labels go throughdocker/cli's
HeaderFunctions, sotable {{lower .Type}}still names the columnTYPE. Nonerdctl command supported
table TEMPLATEso far, so the helper lives inpkg/formatterfor theothers to adopt. Identifiers are shortened for table output only, matching
Format.IsTable(), sothat a custom format stays usable to look a resource up.
Notes
host of that namespace, and shows as zeros when BuildKit is down, so the shape of the output does
not depend on which daemons happen to be up.
system prunealready is:pkg/cmd/systemorchestrates, each kind of resource ismeasured by its own package.
gets one), the summary and verbose rendering,
table TEMPLATE, the platform selection of thebuild time, and the parsing of
buildctl du. The integration tests run in a private namespaceand check the counts after a pull, a run and a stop, plus the verbose sections and the three
output formats; the counts are asserted as a difference from a baseline taken before the setup,
since docker has no namespaces and its daemon holds whatever the other tests left behind. Only
the Local Volumes row needs a
_linuxfile, a mount target being written differently on eachplatform, so the rest runs on Windows too.
Closes #3942