DAOS-18690 vos: fix engine abort in DTX commit blob fallback on ENOSPC

[hgichon]

Summary

Regression introduced by PR #18141 (DAOS-18690 "vos: handle DTX commit under space pressure", commit b36dd7b, 2026-05-14) — master only. Release branches (2.6.x/2.8) are not affected: their vos_dtx_commit_internal() returns -DER_NOSPACE directly on blob allocation failure without touching the aborted TX.

When the pool is physically full (ENOSPC), extending the DTX committed table during a punch/update commit crashes the whole engine with SIGABRT.

vos_dtx_extend_cmt_table() allocates a new commit blob with umem_zalloc(). On allocation failure it falls back to vos_dtx_reuse_cmt_blob() inside the same transaction. However, all three allocator backends (pmemobj, dav, dav_v2) abort the transaction automatically when a tx alloc fails (no *_NO_ABORT flag is passed by the umem wrappers). The fallback then calls umem_tx_add_ptr() / dbtree_delete() on the aborted TX, which is a fatal usage error in the allocator:

ERR  pmdk pmdk/src/common/set.c:2260 util_pool_extend() extending the pool by appending parts with headers is not supported!
ERR  pmdk pmdk/tx.c:618 tx_alloc_common() out of memory
CRIT pmdk pmdk/tx.c:1403 pmemobj_tx_add_range_direct() pmemobj_tx_add_range_direct called in invalid stage 3
ERR  *** Process 1551736 (daos_engine) received signal 6 (Aborted) ***

Backtrace (master, commit ~f6b827e3e):

pmemobj_tx_add_range_direct
libdaos_common_pmem.so(+0x2979f)            # umem_tx_add_ptr
libvos_srv.so(+0x9ca53)                     # vos_dtx_* (reuse/commit path)
vos_dtx_commit_internal+0xf91
vos_dtx_prepared+0xb7d
vos_tx_end+0x5fe
vos_obj_punch+0x263
ds_obj_punch_handler+0x840

Why CI doesn't catch it

The reuse-blob fallback is exercised in tests via the DAOS_DTX_NOSPACE_NOREFRESH fail-check, which skips the allocation entirely — the TX stays alive, so the fallback works. A real ENOSPC allocation failure aborts the TX first, which the fail-check path never reproduces.

Reproduction

1. Small pool (tested: 400 MB tmpfs SCM, 2 targets, master build).
2. Fill the pool to ENOSPC via POSIX container (dd through dfuse).
3. Run a create/unlink storm (a few hundred small files) so punch DTX commits force vos_dtx_extend_cmt_table() while the pool is full.
4. Engine aborts within seconds (invalid stage 3 + SIGABRT). A pool-full production system can be taken down by any client issuing unlinks.

Suggested fix (verified)

Add a UMEM_FLAG_NO_ABORT alloc flag, map it to POBJ_XALLOC_NO_ABORT / DAV_XALLOC_NO_ABORT in the pmem/dav/dav_v2 backends, and use it for the commit-blob allocation in vos_dtx_extend_cmt_table() so the TX survives the failure and the reuse fallback can run as designed:

• src/include/daos/mem.h: #define UMEM_FLAG_NO_ABORT (((uint64_t)1) << 2)
• src/common/mem.c: pmem_tx_alloc() → POBJ_XALLOC_NO_ABORT, bmem_tx_alloc()/bmem_tx_alloc_v2() → DAV_XALLOC_NO_ABORT
• src/vos/vos_dtx.c vos_dtx_extend_cmt_table():
  umem_zalloc(umm, DTX_CMT_BLOB_SIZE) → umem_alloc_verb(umm, UMEM_FLAG_ZERO | UMEM_FLAG_NO_ABORT, DTX_CMT_BLOB_SIZE, UMEM_DEFAULT_MBKT_ID)

With this patch the same reproduction survives: unlink storms on a full pool return -DER_NOSPACE cleanly and the engine stays up (verified on tmpfs SCM/pmemobj backend; dav paths compile-verified only).

---

Note: filed as a PR because GitHub issues are disabled on this repo; happy to file a Jira ticket as well if preferred.

---

Update (2026-07-04):

• UMEM_FLAG_NO_ABORT moved to 1 << 3 to avoid sharing a bit with UMEM_XADD_NO_SNAPSHOT (1 << 2) in the same flags namespace.
• Added a umem regression test (UMEM010): a failed UMEM_FLAG_NO_ABORT allocation must leave the TX alive so a subsequent alloc/umem_tx_add_ptr()/commit succeeds — this covers the TX-abort path that the DAOS_DTX_NOSPACE_NOREFRESH fail-check cannot reproduce. Passes 10/10 locally (pmem backend on tmpfs needs PMEMOBJ_CONF=sds.at_create=0).
• Clarified in the commit message that the failing allocation is a plain (non-NO_ABORT) alloc inside the regular vos_tx_begin() object transaction, whose failure behavior is left at the default (abort) rather than TX_FAILURE_RETURN.

[github-actions]

Ticket title is 'Aurora daos_user: SCM single target ran out of space (min:0 B) and not able to finish GC. '
Status is 'Awaiting backport'
Labels: 'request_for_2.8.1'
https://daosio.atlassian.net/browse/DAOS-18690

[hgichon]

Note on backport scope: the Jira bot above shows DAOS-18690 is Awaiting backport with request_for_2.8.1. Since #18141 is what introduces the in-TX reuse fallback, any release branch that picks up #18141 will also pick up this crash — before #18141 the allocation-failure path simply returned -DER_NOSPACE without touching the aborted TX.

So this fix should ride along with the #18141 backport to 2.8.1 (and any other target branches), not just land on master. Happy to prepare the 2.8-based patch as well if useful.