ENH: Out-of-core architecture rewrite and filter optimizations - #1568
Open
joeykleingers wants to merge 2 commits into
Open
ENH: Out-of-core architecture rewrite and filter optimizations#1568joeykleingers wants to merge 2 commits into
joeykleingers wants to merge 2 commits into
Conversation
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
from
March 24, 2026 18:13
b4ef97f to
99b49ed
Compare
joeykleingers
requested review from
JDuffeyBQ,
imikejackson,
jmarquisbq,
mmarineBlueQuartz and
nyoungbq
March 24, 2026 18:17
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
2 times, most recently
from
March 24, 2026 18:51
b4ef97f to
bb09048
Compare
This was referenced Mar 24, 2026
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
4 times, most recently
from
April 2, 2026 00:55
102c436 to
b4c1358
Compare
joeykleingers
changed the title
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
6 times, most recently
from
April 8, 2026 17:41
2bd614a to
110c054
Compare
joeykleingers
changed the title
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
4 times, most recently
from
April 16, 2026 13:03
35aecd0 to
3a88bbf
Compare
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
from
April 27, 2026 18:18
bdfed87 to
6fbfc8d
Compare
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
from
June 25, 2026 17:25
ed64934 to
a56f168
Compare
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
3 times, most recently
from
July 13, 2026 18:29
c6bb7fe to
ee6b370
Compare
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
2 times, most recently
from
July 15, 2026 17:22
5a8c281 to
b61d3fd
Compare
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
5 times, most recently
from
July 30, 2026 14:22
9e11033 to
03ac304
Compare
joeykleingers
marked this pull request as ready for review
July 30, 2026 14:27
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
9 times, most recently
from
August 13, 2026 18:45
ec2e32c to
09d552a
Compare
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
from
August 17, 2026 13:55
1d31c6b to
a4cd160
Compare
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
from
August 24, 2026 14:32
4baad02 to
36d674d
Compare
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
2 times, most recently
from
September 1, 2026 17:49
2aaa5ff to
66a06fa
Compare
Add storage-neutral array creation, chunk-aware bulk I/O, and runtime dispatch between direct and scanline algorithm implementations. Extend simplnx core, SimplnxCore, OrientationAnalysis, ITK integration, nxrunner, documentation, and tests for bounded-memory workflows. 741 files changed, +105089 / -32743 lines. Storage selection and data structures ===================================== Files: IDataStoreFormatResolver.*, DataIOCollection.*, DataArray.*, DataStoreFormatParameter.*, Create*Action.*, EmptyStringStore.*, and geometry creation actions Resolve array, string, neighbor-list, and geometry support-array storage when callers do not specify a format. Preserve explicit formats, apply application preferences during creation and import, and let node geometry coordinates and connectivity use registered out-of-core stores. Bulk datastore and HDF5 I/O =========================== Files: AbstractDataStore.*, DataStoreUtilities.*, DataStoreIO.*, DatasetIO.*, ParallelChunkCodec.*, ChunkShapePolicy.*, DeflateEligibility.*, ChunkIndex.*, and PositionalFileIO.* Add caller-owned bulk transfer APIs and chunk-aware read and write paths. Parallelize eligible deflate work outside the serialized HDF5 API, retain serial fallbacks for unsupported filters and byte orders, preserve raw chunk filter masks on Windows, and use shared chunk-shape policies. Algorithm dispatch and bounded utilities ======================================== Files: AlgorithmDispatch.*, Extent.hpp, IExternalSort.*, ITemporaryRecordStore.*, BoundedRecordPageCache.*, CacheMemoryBudgetManager.*, SliceBufferedTransfer.*, ExternalEquivalence.*, and UnionFind.hpp Select direct or scanline implementations from target-array residency and record test witnesses for the selected path and store type. Provide bounded scratch storage, stable external sorting, paged records, slice transfers, shared cache budgeting, and storage-neutral equivalence and union-find tools. Storage-aware filter implementations ==================================== Files: SimplnxCore/Filters/Algorithms/*Direct.*, SimplnxCore/Filters/Algorithms/*Scanline.*, segmentation BFS and CCL files, and OrientationAnalysis/Filters/Algorithms/*Direct.* and *Scanline.* Add storage-specific paths for morphology, segmentation, clustering, statistics, feature transfer, meshing, array transforms, and orientation calculations. Keep dispatch wrappers shared while scanline, worklist, and bounded-record implementations avoid unbounded per-value out-of-core access. Import, persistence, and command-line integration ================================================= Files: Dream3dIO.*, Dream3dPreflightCache.*, HDF5 DataStructureWriter.*, ImportH5ObjectPathsAction.*, ITKImageWriterFilter.*, Application.*, Preferences.*, and nxrunner.* Apply the storage resolver to imported and preflight arrays, preserve storage metadata through DREAM3D I/O, and make writers consume storage-neutral arrays. Expose data-storage mode and cache-memory budget settings through application preferences and nxrunner configuration. Tests, build integration, and documentation =========================================== Files: test/, SimplnxCore/test/, OrientationAnalysis/test/, UnitTest/AlgorithmTestScope.*, plugin CMake files, docs/, and vcpkg.json Add in-core and out-of-core dispatch scenarios, execution witnesses, HDF5 codec and chunk-policy tests, memory-safety checks, storage migration tests, filter regressions, and V&V cases. Register the new sources and update filter and algorithm documentation for storage-neutral parameter and execution behavior. Verification ============ Clang-format 19.1.1 passes for all 682 changed C/C++ files. Git diff --check passes. No build or test suite is run for this history-only squash. Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
joeykleingers
force-pushed
the
worktree-ooc-architecture-rewrite
branch
from
September 1, 2026 19:25
66a06fa to
fe36633
Compare
Contributor
Remove/Extract Flagged Features: the Direct/Scanline rewrite reintroduces the fill-loop hangs fixed on
|
5 tasks
Extend the shared cache budget manager with explicit pin residency and bounded temporary reservations. Preserve storage-neutral filter behavior while keeping visible cache entries out of normal eviction and avoiding Windows stack overflow in M3C. 4 files changed, +602 / -13 lines. Pinned cache accounting ======================= Files: CacheMemoryBudgetManager.hpp and CacheMemoryBudgetManager.cpp Add initially pinned registrations, nested pin and unpin operations, budget-aware pin admission, temporary pinned reservations, and unique pinned byte reporting. Exclude pinned entries from direct LRU eviction, preserve recency when the final pin is released, and keep delegated-subsystem behavior and over-budget accounting explicit. Use subtraction-based budget checks to avoid overflow at size limits. M3C bounded-memory safety ========================= File: SimplnxCore/Filters/Algorithms/M3CSurfaceMeshing.cpp Move six fixed OOC batch buffers from the thread stack to equal-size heap-backed vectors. Keep the existing types, batch sizes, initialization, processing order, and output behavior while remaining below the default Windows stack limit. Tests ===== File: CacheMemoryBudgetManagerTest.cpp Cover nested pin balance, pinned-byte accounting, LRU selection, recency, budget reductions, temporary reservations, delegated subsystems, cleared handles, concurrent pin use, and pin-operation overhead. Exercise all-pinned over-budget behavior without allowing pinned entries to become a persistent unbounded policy. Verification ============ Windows/MSVC Release builds pass for in-core and out-of-core configurations. The sequential in-core CTest suite passes 2251 of 2251 tests. The sequential out-of-core-compatible suite passes 2357 of 2357 tests with ITK excluded. Focused and complete M3C tests pass in both configurations. Signed-off-by: Jessica Marquis <jessica.marquis@bluequartz.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch does two things at once:
Reworks the core out-of-core (OOC) data-storage architecture in
simplnxso that disk-backed storage is supplied by a runtime-registered IO manager through clean interfaces, with the core library holding no reference to any concrete OOC format or symbol. Core gains a bulk-I/O data-store API (copyIntoBuffer/copyFromBuffer), an injectable store-format resolver, IO-manager lifecycle hooks, a tri-state storage-mode preference, a memory-budget manager, and memory-safety guards. The OOC implementation itself (chunked HDF5 stores, the chunk cache, deflate decompression) lives in a separate privateSimplnxOocsource set and is not part of this diff — it plugs into core at runtime through these interfaces.Adds chunk-aware (OOC-optimized) algorithm variants for ~50 filters across the SimplnxCore and OrientationAnalysis plugins, plus shared core utilities that benefit many more. Random-access algorithms that thrash the chunk cache on disk-backed arrays now either stream data with bulk I/O or dispatch to a sequential "scanline / CCL" variant, eliminating per-element virtual dispatch and HDF5 chunk churn. In-core behavior and performance are preserved via a runtime storage-type check, so optimized filters keep two code paths: one for in-memory data, one for disk-backed data.
Scope of this diff
develop…HEAD: 378 files, +42,853 / −13,112.src/simplnx(core library)Plugins/SimplnxCorePlugins/OrientationAnalysisHow to review
The diff is large but highly patterned, and the two layers are independent:
src/simplnx/DataStructure/IO/Generic/*,Core/Preferences.*,Utilities/{MemoryBudgetManager,AlgorithmDispatch,SegmentFeatures,UnionFind,SliceBufferedTransfer}.*, andFilter/IFilter.cpp(Part 1).Part 1 — Core out-of-core architecture (
src/simplnx)1.1 Bulk-I/O data-store API
copyIntoBuffer/copyFromBufferonAbstractDataStore<T>(a flat range form and a tuple-addressed form) are the single mechanism for bulk reads/writes, implemented byDataStore<T>andEmptyDataStore<T>; the OOC implementation supplies its own override.IDataStore::StoreType { InMemory, OutOfCore, Empty }, queried viagetStoreType(). The chunk-traversal API (loadChunk,getNumberOfChunks,getChunkLowerBounds,getChunkUpperBounds,getChunkShape) and the chunk-shape-based OOC detection are removed — bulk I/O plusgetStoreType()replace them entirely.EmptyStringStore,AbstractStringStore/StringStoreplaceholder support, andStringArray::isPlaceholder().1.2 Runtime store-format resolution and IO-manager hooks
OOC capability is supplied entirely at runtime; core never names the concrete format.
IDataStoreFormatResolver: a const, thread-safe policy interface deciding which registered format a soon-to-be-created array uses (""= in-memory).InMemoryFormatResolveris the default policy.ArrayCreationUtilities::ResolveStorageFormatis the single decision point for every array-creation call site, with a fixed precedence: the unstructured/poly-geometry gate (ParentGeometrySupportsOoc) forces in-core, then an explicit per-filter format override, then theDataStructure's resolver.DataStructurecarries a per-instance resolver plus a lazily-seeded process-wide default (neither serialized).IDataIOManagerlifecycle hooks (default no-ops):finalizesImport,onImportFinalize,onRecoveryWrite,onFinalizeStores,setBaseDirectory,shutdownManager.DataIOCollectionfans these out to every registered manager, so an OOC manager participates in import finalization, recovery writes, store read-only transition, and shutdown without core knowing the specifics.CoreDataIOManager: the always-present default manager that registers the in-memory data-store/list-store factories; an OOC manager registers on top at runtime.CreateDataStore/CreateListStoresingle entry points;CreateNeighborListAction/CreateNeighborsthread an explicitdataFormatoverride through to the store.1.3 Storage-mode preference (+ legacy migration)
DataStorageMode { Adaptive, ForceInCore, ForceOutOfCore }(persisted asdata_storage_mode) is the single source of truth for storage placement, replacing thelarge_data_format/force_ooc_datapreferences. The enum is deliberately OOC-vocabulary-free — core states user intent; the registered manager maps it to a concrete format.dataStorageMode()migrates older preference files from the retained legacy keys;useOocData()is a convenience view (true unlessForceInCore).Preferencesseeds the OOC base directory and default format on startup via the IO-collection hooks, and gains aremoveValuehelper.1.4 Memory budget + memory safety
MemoryBudgetManager: tracks the budget governing in-core vs OOC placement.defaultBudgetBytes()(50% of RAM, ≥1 GiB, clamped to the cap),maxBudgetBytes()(max(min(total−6 GiB, 0.95·total), 1 GiB)), andsetBudgetBytes()which clamps to the cap and reports whether it clamped. Total-RAM detection is centralized inMemory::GetTotalMemory(). The default is clamped to the cap so it never exceeds it on <12 GiB machines (e.g. CI runners).nxrunner --memory-budget <GB>: CLI flag wired through to the budget manager so the override takes effect in headless runs, with parsing hardened against NaN/inf/trailing garbage.-264total-RAM hard block):-271— non-blocking preflight warning when an in-core array would exceed currently-available RAM. OOC arrays are excluded (EmptyDataStore::memoryUsage()reports 0 for OOC placeholders; the format is resolved in preflight via the sharedResolveStorageFormathelper).-272— astd::bad_allocsafety net at the singleIFilter::execute → executeImplboundary, turning an out-of-memory condition into a clean pipeline error instead of a crash.1.5 HDF5 streaming write + compression
DatasetIOprovides the streaming OOC write path —createEmptyDataset+writeSpanHyperslabfor arrays too large to be resident — alongside the single-shotwriteSpan; reads usereadChunk/readChunkIntoSpan.createEmptyDatasetbuilds the chunked-deflate creation property list (BuildChunkedDeflateDcpl), so a large OOC array taking the two-step streaming write is compressed identically to the single-shot path. Contiguous storage is still used for compression level 0 and for arrays below the small-array threshold.1.6
.dream3dloader APIDream3dIOexposes aLoadDataStructurefamily:LoadDataStructure,LoadDataStructureMetadata(metadata-only for preflight),LoadDataStructureArrays(array subset), plus resolver-aware overloads that stamp a per-DataStructureresolver before import finalization (so a read-only visualization load can direct arrays to disk for fast first-show). Import is eager or deferred based onanyManagerFinalizesImport(); writes run under a recovery-write guard supplied by the registered manager.ImportH5ObjectPathsActionuses this API: metadata-only on preflight, full load on execute, with a selective shortest-path-first merge of only the requested paths.1.7 Core algorithm infrastructure
AlgorithmDispatch.hpp—DispatchAlgorithm<InCoreAlgo, OocAlgo>(arrays, args…), a free function template selecting between an in-core and an OOC algorithm class at runtime. Priority:ForceInCoreAlgorithm()> any array OOC >ForceOocAlgorithm()> in-core default. RAII test guardsForceOocAlgorithmGuard(bool)andForceInCoreAlgorithmGuardlet a single build exercise both paths.SegmentFeaturesOOC path —executeCCL(): Z-slice connected-component labeling with a 2-slice rolling buffer andUnionFindequivalence tracking (Face and FaceEdgeVertex connectivity, optional periodic BCs), replacing random-access BFS/DFS flood-fill on disk-backed data.UnionFind— vector-based disjoint set with union-by-rank and path-halving.SliceBufferedTransfer— type-dispatched Z-slice buffered tuple copy for morphological / neighbor-replacement transfer phases.Extent— region/range math helper (with unit tests).AlignSectionsOOC path — bulk slice read/write transfer for the align-sections family.1.8 Core utility bulk-I/O conversions
These live in core utilities and benefit every caller; each is guarded by a runtime storage-type check that preserves the original in-core code path:
DataArrayUtilities—ImportFromBinaryFile,AppendData,CopyData, and the mirrorswap_rangesops route through chunked bulk I/O when OOC. (PowersReadRawBinaryandAppendImageGeometry's mirror.)DataGroupUtilities::RemoveInactiveObjects— chunked featureIds renumbering viacopyIntoBuffer/copyFromBuffer.ClusteringUtilities::RandomizeFeatureIds— chunked bulk I/O (both overloads; benefits segmentation filters, SharedFeatureFace, MergeTwins).GeometryHelpers—FindElementsContainingVert/FindElementNeighborsuse 65K-element chunked passes with a current-chunk cache-hit check before falling back to per-element reads.ImageRotationUtilities— Z-slab source cache for nearest-neighbor and a ±2-slice trilinear margin, sliding-window slab updates (memmove + delta reads), and intra-slice parallelism. This is howApplyTransformationToGeometryandRotateSampleRefFrameget their OOC speedups (no plugin algorithm file changes for ApplyTransformation).TriangleUtilities— bulk-load triangles/labels for winding repair.H5DataStore— streaming row-batchFillOocDataStorereplacing full-dataset allocation on import.RectGridGeom/ImageGeomfindElementSizes— route through the resolver-awareCreateDataStore(voxel-sizes array can go OOC); the RectGrid inner loop refactored to per-axis precompute + Z-slicecopyFromBuffer.Part 2 — Filter optimizations
Optimization patterns
Every filter optimization is one of these four shapes:
DispatchAlgorithm<…Direct, …Scanline>: an in-coreDirectclass (unchanged or parallel) and an OOCScanlineclass that streams Z-slices / chunks. The originalFoo.cppbecomes a thin dispatcher.DispatchAlgorithm<…BFS, …CCL>(or an in-file branch): random-access flood-fill in-core, sequential Z-slice connected-component labeling for OOC.std::vectors, with a runtime storage-type check so in-core stays optimal.SliceBufferedTransfer, or an OOC-correctness guard (e.g. disabling threading for OOC stores) / progress-and-cancel additions.Algorithm structure: in-core + out-of-core variants
For dispatch-split filters the original algorithm file is renamed to the in-core variant and the OOC variant is added beside it; the original filename remains as a thin dispatcher:
FillBadDataBFSFillBadDataCCLIdentifySampleBFSIdentifySampleCCL…Direct…Scanline…Direct…Scanline…Direct…Scanline…Direct…Scanline…Direct…Scanline…Direct…Scanline…Direct…Scanline…Direct…Scanline…Direct…Scanline…Direct…Scanline…Worklist…Scanline…Direct…ScanlineNew shared header
IdentifySampleCommon.hpp(SimplnxCore) provides theVectorUnionFindand per-slice functor shared by the BFS/CCL variants;TupleTransfer.hppgainsquickSurfaceTransferBatch/surfaceNetsTransferBatchbulk APIs used by the mesh Scanline variants.SimplnxCore inventory
DispatchAlgorithm<FillBadDataBFS, FillBadDataCCL>; CCL streams Z-slices with coreUnionFindDispatchAlgorithm<IdentifySampleBFS, IdentifySampleCCL>; scanline labeling viaVectorUnionFindSegmentFeatures::executeCCL()(Z-slice CCL)copyIntoBuffer(256K-tuple)canMergefindClusters; bounded per-cluster peak memorynodeIdsfor rolling 2-plane node buffers;quickSurfaceTransferBatchsurfaceNetsTransferBatchstd::vectoraccumulators (no DataStore); 64K-tuple chunked featureIdsassignBadVoxelsvoting; sparse changed-voxel trackingtriMaskbitset + sparsetriPrefixSumpopcount (~6.4× less memory); 65K-element streamed passesZSliceWorkerparallel Z-slice rasterize → mutex-guarded bulkcopyFromBufferSliceBufferedTransferper-Z commit + Z-slice neighbor readsSliceBufferedTransferper-Z commitSliceBufferedTransferper-Z best-neighbor commitfindShiftsOoc()with per-Z-slice mask readscopyIntoBufferfor coordinate/data outputOrientationAnalysis inventory
DispatchAlgorithm<…Direct, …Scanline>; Direct keeps parallel in-core, Scanline 65K-tuple chunks + cached crystal structures; color key forwarded to the OOC pathDispatchAlgorithm<…Worklist, …Scanline>; bool-mask reads routed through bulk I/OSegmentFeatures::executeCCL()slice-by-slice, replacing DFS flood-fillfindShiftsOoc(), 2-slice buffered quats/phases/mask + cached crystal structuresSliceBufferedTransfer; cached crystal structures[plane−kZ, plane+kZ]slab reads; cached crystal structurescopyFromBuffertriIncluded); feature-level caching; raw-pointer parallel selectorfmt::memory_buffercopyFromBufferfor all cell arrays; chunked Euler interleave; in-place phase validationcopyFromBufferin CopyData template, phase copy, Euler interleavecopyFromBufferfrom raw HDF5 reader bufferscopyFromBufferfor intensity/image outputsCancel + progress
In-core and OOC variants gained
m_ShouldCancelchecks at the top of major loops andThrottledMessenger-based progress reporting with per-phase messages and percent-complete.Part 3 — Performance results
All benchmarks use arm64 Release builds. OOC measurements force the disk-backed path through
DataStorageMode::ForceOutOfCoreorForceOocAlgorithmGuard.CRITICAL filter optimizations (existing unit-test data, filter.execute() only)
Small existing datasets do not expose every asymptotic win: mesh bucketing/pruning and bounded-memory streaming can add fixed overhead at this scale. All 17 filters passed their affected suites in both configurations; the table reports measured values rather than hiding those small-test regressions.
Large synthetic filter benchmarks (filter.execute() only)
HIGH and MEDIUM benchmarks use 8,000,000 cells (200³) except the 2D-only tests (2048²), the generated Channel5 reader dataset (8,000 × 1,000), and CSV's 8,000,000 rows. Figures are medians of five execute-only runs for the MEDIUM batches and repeated runs for HIGH where practical;
>values are conservative timeout bounds. Every listed filter passed its normal affected suite and hidden large benchmark in both configurations.Additional 200³ execute-only OOC benchmarks
Full-test wall-clock OOC benchmarks
Geometry / Mesh / Phase Filters
ImageRotationUtilitiesslab cachePart 4 — Test infrastructure
UnitTestCommon.hpp):CompareDataArrays,CompareDataArraysByComponent,CompareArrays,CompareFloatArraysWithNansstream both arrays in 40K-element chunks viacopyIntoBufferinstead of per-elementoperator[](per-element access on OOC arrays is pathologically slow).ExpectedStoreType()(derives the expectedStoreTypefrom the activeDataStorageMode+ whether an OOC manager is registered) andRequireExpectedStoreType().PreferencesSentineltakes aDataStorageModeand restores the original preferences on destruction.TestFileSentinelreference-counts archive extraction via per-process holder files, so parallel test runs don't delete shared decompressed data prematurely.LoadDataStructuretest helper callsDREAM3D::LoadDataStructure(path)directly.SegmentFeaturesTestUtils.hpp(~638 lines): shared builders/verifiers for the SegmentFeatures family.ForceOocAlgorithmGuard+GENERATE(from_range(k_ForceOocTestValues))runs each case in both in-core and forced-OOC modes — adopted by 23 plugin test files (16 SimplnxCore, 7 OrientationAnalysis).test/:DataStoreFormatResolverTest,DataStorageModeMigrationTest,Dream3dLoadingApiTest,EmptyStringStoreTest,ExtentTest,MemoryBudgetManagerTest,MemorySafetyTest,IParallelAlgorithmTest.RotateSampleRefFrame/RotateEulerRefFrametest paths use slab/chunked bulk I/O.Part 5 — Build system
cmake/SimplnxConfig.hpp.inis a generated PUBLIC, ODR-safe config header.SIMPLNX_TEST_ALGORITHM_PATHcache option (default0):0=Both,1=OOC-only,2=InCore-only; plumbed into every plugin test as a compile definition (cmake/Plugin.cmake). An in-core build can validate both algorithm paths (forcing the Scanline path against in-core data still runs correctly and fast).SIMPLNX_UNIT_TEST_TARGETSglobal property —create_simplnx_plugin_unit_testrecords each test target so consumer (add_subdirectory) builds can attach extra sources/settings.zlibas a direct vcpkg dependency — the OOC layer compiled into consumers uses it directly for parallel deflate-chunk decompression off the global HDF5 mutex (andUnitTestCommonuses it for the in-house tar.gz extractor)./bigobjfor the MSVC build ofDream3dIO.cpp(exceeds the COMDAT limit / C1128 in Debug).Extent,EmptyStringStore,IDataStoreFormatResolver,InMemoryFormatResolver,MemoryBudgetManager,AlgorithmDispatch,SliceBufferedTransfer,UnionFind,IdentifySampleCommon).Part 6 — Documentation
src/Plugins/*/docs/(27 SimplnxCore, 21 OrientationAnalysis; ~+1,177 lines), each adding an## Algorithmsection with### Performanceand paired In-Core / Out-of-Core subsections that explain the dual implementation, memory footprint, and chunk/slab streaming strategy. No docs deleted.Part 7 — Test data archives
Three
download_test_data()entries added:fill_bad_data_exemplars.tar.gz(SimplnxCore)identify_sample_exemplars.tar.gz(SimplnxCore)segment_features_exemplars.tar.gz(referenced by both SimplnxCore and OrientationAnalysis)No existing archive entries were removed.
Related PR
Test Plan
SIMPLNX_TEST_ALGORITHM_PATH=2) passesSIMPLNX_TEST_ALGORITHM_PATH=1) passesSIMPLNX_TEST_ALGORITHM_PATH=0) pass