diff --git a/src/Plugins/SimplnxCore/docs/ErodeDilateBadDataFilter.md b/src/Plugins/SimplnxCore/docs/ErodeDilateBadDataFilter.md index 4e1c68abef..42dea7c9a9 100644 --- a/src/Plugins/SimplnxCore/docs/ErodeDilateBadDataFilter.md +++ b/src/Plugins/SimplnxCore/docs/ErodeDilateBadDataFilter.md @@ -23,16 +23,17 @@ Cell neighboring a *bad* **Cell** will be changed to *0*. If the *bad* data is *eroded*, the Filter shrinks the bad data by one **Cell** in an iterative sequence for a user defined number of iterations. During the *erode* process -the *Feature Id* of the *bad* **Cell** is changed from *0* to the *Feature Id* of the majority of its neighbors. If -there is a tie between two *Feature Ids*, then one of the *Feature Ids*, chosen randomly, will be assigned to the *bad* -**Cell**. +the *Feature Id* of the *bad* **Cell** is changed from *0* to the *Feature Id* of the majority of its neighbors. + +Ties are broken deterministically, not randomly. The Filter visits the six face neighbors in the fixed order +*-Z, -Y, -X, +X, +Y, +Z*, and a later neighbor must have a strictly greater count than the current leader to replace it. +When two or more *Feature Ids* are tied for the majority, the one belonging to the earliest neighbor in that scan order +is assigned. The same input therefore always produces the same output. | Before Erosion | After Erosion | |--------------------------------------|--------------------------------------| | ![](Images/ErodeDilateBadData_1.png) | ![](Images/ErodeDilateBadData_3.png) | -` - Goals a user might be trying to accomplish with this Filter include: - Remove small or thin regions of bad data by running a single (or two) iteration *erode* operation. @@ -54,6 +55,21 @@ The *Operation* parameter selects which morphological operation to apply: - **Dilate [0]**: Grows bad data regions by one **Cell** per iteration. Any **Cell** neighboring a bad **Cell** has its *Feature Id* changed to 0. - **Erode [1]**: Shrinks bad data regions by one **Cell** per iteration. Each bad **Cell** is assigned the *Feature Id* of the majority of its neighbors. +### Direction Restrictions + +The *X Direction*, *Y Direction*, and *Z Direction* parameters control which of the six face neighbors participate. With +all three enabled the Filter uses all six face neighbors (*-Z, -Y, -X, +X, +Y, +Z*); disabling *Z Direction*, for +example, restricts the operation to the four in-plane neighbors so that bad data grows or shrinks only within each XY +slice. + +### Preflight Errors + +The Filter refuses to run in two cases: + +- **-14601**: all three of *X Direction*, *Y Direction*, and *Z Direction* are disabled. At least one direction is + required, otherwise there are no neighbors to erode or dilate across. +- **-14602**: the selected **Image Geometry** has a dimension of *0* **Cells**. All three dimensions must be non-zero. + ## WARNING: Feature Data Will Become Invalid By modifying the cell level data, any feature data that was previously computed will most likely be invalid at this point. Filters that compute feature level data should be rerun to ensure accurate final results from your pipeline. diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ErodeDilateBadData.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ErodeDilateBadData.cpp index 0cc0ed0cd0..716f4bb2fd 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ErodeDilateBadData.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ErodeDilateBadData.cpp @@ -16,10 +16,9 @@ class ErodeDilateBadDataTransferDataImpl ErodeDilateBadDataTransferDataImpl() = delete; ErodeDilateBadDataTransferDataImpl(const ErodeDilateBadDataTransferDataImpl&) = default; - ErodeDilateBadDataTransferDataImpl(ErodeDilateBadData* filterAlg, usize totalPoints, ChoicesParameter::ValueType operation, const Int32AbstractDataStore& featureIds, - const std::vector& neighbors, const std::shared_ptr& dataArrayPtr, MessageHelper& messageHelper) - : m_FilterAlg(filterAlg) - , m_TotalPoints(totalPoints) + ErodeDilateBadDataTransferDataImpl(usize totalPoints, ChoicesParameter::ValueType operation, const Int32AbstractDataStore& featureIds, const std::vector& neighbors, + const std::shared_ptr& dataArrayPtr, MessageHelper& messageHelper) + : m_TotalPoints(totalPoints) , m_Operation(operation) , m_Neighbors(neighbors) , m_DataArrayPtr(dataArrayPtr) @@ -54,14 +53,31 @@ class ErodeDilateBadDataTransferDataImpl } private: - ErodeDilateBadData* m_FilterAlg = nullptr; usize m_TotalPoints = 0; ChoicesParameter::ValueType m_Operation = 0; - std::vector m_Neighbors; + const std::vector& m_Neighbors; const std::shared_ptr m_DataArrayPtr; const Int32AbstractDataStore& m_FeatureIds; MessageHelper& m_MessageHelper; }; + +/** + * @brief Masks out face neighbors whose axis has been disabled via the X/Y/Z Direction parameters. + * Indices follow the VoxelNeighbors ordering: [-Z,-Y,-X,+X,+Y,+Z]. + * @param isValidFaceNeighbor Per-voxel face-neighbor validity, already computed from geometry boundary. + * @param xDir Whether the X direction is enabled. + * @param yDir Whether the Y direction is enabled. + * @param zDir Whether the Z direction is enabled. + */ +void adjustValidNeighbors(std::array::k_FaceNeighborCount>& isValidFaceNeighbor, bool xDir, bool yDir, bool zDir) +{ + isValidFaceNeighbor[VoxelNeighbors::k_NegativeZNeighbor] = isValidFaceNeighbor[VoxelNeighbors::k_NegativeZNeighbor] && zDir; + isValidFaceNeighbor[VoxelNeighbors::k_NegativeYNeighbor] = isValidFaceNeighbor[VoxelNeighbors::k_NegativeYNeighbor] && yDir; + isValidFaceNeighbor[VoxelNeighbors::k_NegativeXNeighbor] = isValidFaceNeighbor[VoxelNeighbors::k_NegativeXNeighbor] && xDir; + isValidFaceNeighbor[VoxelNeighbors::k_PositiveXNeighbor] = isValidFaceNeighbor[VoxelNeighbors::k_PositiveXNeighbor] && xDir; + isValidFaceNeighbor[VoxelNeighbors::k_PositiveYNeighbor] = isValidFaceNeighbor[VoxelNeighbors::k_PositiveYNeighbor] && yDir; + isValidFaceNeighbor[VoxelNeighbors::k_PositiveZNeighbor] = isValidFaceNeighbor[VoxelNeighbors::k_PositiveZNeighbor] && zDir; +} } // namespace // ----------------------------------------------------------------------------- @@ -110,6 +126,11 @@ Result<> ErodeDilateBadData::operator()() } } + MessageHelper messageHelper(m_MessageHandler); + + // Build up a list of the DataArrays that we are going to operate on. + const std::vector> voxelArrays = nx::core::GenerateDataArrayList(m_DataStructure, m_InputValues->FeatureIdsArrayPath, m_InputValues->IgnoredDataArrayPaths); + constexpr FaceNeighborType k_NumFaceNeighbors = VoxelNeighbors::k_FaceNeighborCount; const std::array neighborVoxelIndexOffsets = initializeFaceNeighborOffsets(dims); constexpr std::array faceNeighborInternalIdx = initializeFaceNeighborInternalIdx(); @@ -120,6 +141,12 @@ Result<> ErodeDilateBadData::operator()() { for(int64 zIdx = 0; zIdx < dims[2]; zIdx++) { + // Check if the algorithm should cancel + if(m_ShouldCancel) + { + return {}; + } + const int64 zStride = dims[0] * dims[1] * zIdx; for(int64 yIdx = 0; yIdx < dims[1]; yIdx++) { @@ -132,7 +159,8 @@ Result<> ErodeDilateBadData::operator()() { int32 most = 0; // Loop over the 6 face neighbors of the voxel - const std::array isValidFaceNeighbor = computeValidFaceNeighbors(xIdx, yIdx, zIdx, dims); + std::array isValidFaceNeighbor = computeValidFaceNeighbors(xIdx, yIdx, zIdx, dims); + adjustValidNeighbors(isValidFaceNeighbor, m_InputValues->XDirOn, m_InputValues->YDirOn, m_InputValues->ZDirOn); for(const auto& faceIndex : faceNeighborInternalIdx) { if(!isValidFaceNeighbor[faceIndex]) @@ -177,11 +205,6 @@ Result<> ErodeDilateBadData::operator()() } } - // Build up a list of the DataArrays that we are going to operate on. - const std::vector> voxelArrays = nx::core::GenerateDataArrayList(m_DataStructure, m_InputValues->FeatureIdsArrayPath, m_InputValues->IgnoredDataArrayPaths); - - MessageHelper messageHelper(m_MessageHandler); - ParallelTaskAlgorithm taskRunner; taskRunner.setParallelizationEnabled(true); for(const auto& voxelArray : voxelArrays) @@ -193,14 +216,15 @@ Result<> ErodeDilateBadData::operator()() continue; } - taskRunner.execute(ErodeDilateBadDataTransferDataImpl(this, totalPoints, m_InputValues->Operation, featureIds, neighbors, voxelArray, messageHelper)); + taskRunner.execute(ErodeDilateBadDataTransferDataImpl(totalPoints, m_InputValues->Operation, featureIds, neighbors, voxelArray, messageHelper)); } taskRunner.wait(); // This will spill over if the number of DataArrays to process does not divide evenly by the number of threads. // Now update the feature Ids auto featureIDataArray = m_DataStructure.getSharedDataAs(m_InputValues->FeatureIdsArrayPath); taskRunner.setParallelizationEnabled(false); // Do this to make the next call synchronous - taskRunner.execute(ErodeDilateBadDataTransferDataImpl(this, totalPoints, m_InputValues->Operation, featureIds, neighbors, featureIDataArray, messageHelper)); + taskRunner.execute(ErodeDilateBadDataTransferDataImpl(totalPoints, m_InputValues->Operation, featureIds, neighbors, featureIDataArray, messageHelper)); + taskRunner.wait(); // Redundant while parallelization is disabled, but keeps the "transfer is complete" invariant local. } return {}; diff --git a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateBadDataFilter.cpp b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateBadDataFilter.cpp index 30ab525c7f..af5c7a4302 100644 --- a/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateBadDataFilter.cpp +++ b/src/Plugins/SimplnxCore/src/SimplnxCore/Filters/ErodeDilateBadDataFilter.cpp @@ -3,6 +3,7 @@ #include "SimplnxCore/Filters/Algorithms/ErodeDilateBadData.hpp" #include "simplnx/DataStructure/DataPath.hpp" +#include "simplnx/DataStructure/Geometry/ImageGeom.hpp" #include "simplnx/Parameters/ArraySelectionParameter.hpp" #include "simplnx/Parameters/AttributeMatrixSelectionParameter.hpp" #include "simplnx/Parameters/BoolParameter.hpp" @@ -18,6 +19,12 @@ using namespace nx::core; +namespace +{ +constexpr int32 k_NoDirectionsError = -14601; +constexpr int32 k_NoGeometryDimensionsError = -14602; +} // namespace + namespace nx::core { @@ -97,6 +104,10 @@ IFilter::PreflightResult ErodeDilateBadDataFilter::preflightImpl(const DataStruc auto pOperationValue = filterArgs.value(k_Operation_Key); auto pFeatureIdsArrayPathValue = filterArgs.value(k_CellFeatureIdsArrayPath_Key); auto pIgnoredDataArrayPathsValue = filterArgs.value(k_IgnoredDataArrayPaths_Key); + auto xDirOn = filterArgs.value(k_XDirOn_Key); + auto yDirOn = filterArgs.value(k_YDirOn_Key); + auto zDirOn = filterArgs.value(k_ZDirOn_Key); + auto imageGeometryPath = filterArgs.value(k_SelectedImageGeometryPath_Key); PreflightResult preflightResult; @@ -104,6 +115,18 @@ IFilter::PreflightResult ErodeDilateBadDataFilter::preflightImpl(const DataStruc std::vector preflightUpdatedValues; + if(!xDirOn && !yDirOn && !zDirOn) + { + return {MakeErrorResult(k_NoDirectionsError, "ErodeDilateBadData requires at least one direction to operate over")}; + } + + const auto& imageGeom = dataStructure.getDataRefAs(imageGeometryPath); + auto dims = imageGeom.getDimensions(); + if(dims[0] == 0 || dims[1] == 0 || dims[2] == 0) + { + return {MakeErrorResult(k_NoGeometryDimensionsError, "ErodeDilateBadData requires that the ImageGeom have its dimensions set. No dimension may be 0.")}; + } + std::string featureModificationWarning = "By modifying the cell level data, any feature data that was previously computed will most likely be invalid at this point. Filters that compute feature " "level data should be rerun to ensure accurate final results from your pipeline."; preflightUpdatedValues.emplace_back(PreflightValue{"Feature Data Modification Warning", featureModificationWarning}); diff --git a/src/Plugins/SimplnxCore/test/ErodeDilateBadDataTest.cpp b/src/Plugins/SimplnxCore/test/ErodeDilateBadDataTest.cpp index 5fc653917c..7cf97ef344 100644 --- a/src/Plugins/SimplnxCore/test/ErodeDilateBadDataTest.cpp +++ b/src/Plugins/SimplnxCore/test/ErodeDilateBadDataTest.cpp @@ -1,6 +1,7 @@ #include "SimplnxCore/SimplnxCore_test_dirs.hpp" #include +#include "SimplnxCore/Filters/Algorithms/ErodeDilateBadData.hpp" #include "SimplnxCore/Filters/ErodeDilateBadDataFilter.hpp" #include "simplnx/Core/Application.hpp" @@ -12,6 +13,7 @@ #include "simplnx/UnitTest/UnitTestCommon.hpp" #include "simplnx/Utilities/Parsing/HDF5/IO/FileIO.hpp" +#include #include #include @@ -30,7 +32,322 @@ const std::string k_EbsdScanDataName("EBSD Scan Data"); const DataPath k_InputData({"Input Data"}); const DataPath k_EbsdScanDataDataPath = k_InputData.createChildPath(k_EbsdScanDataName); const DataPath k_FeatureIdsDataPath = k_EbsdScanDataDataPath.createChildPath("FeatureIds"); +const StringLiteral k_MiscData = "Misc"; +constexpr usize k_NumTuples = 32; +// ImageGeom dimensions are ordered X, Y, Z. AttributeMatrix and DataArray tuple shapes are ordered +// slowest-to-fastest, i.e. Z, Y, X. Both describe the same 32 cells, so keep them as separate constants +// rather than reusing one that happens to multiply out to the same tuple count. +const SizeVec3 k_GeometryDimensions{4, 4, 2}; +const ShapeType k_TupleShape{k_GeometryDimensions[2], k_GeometryDimensions[1], k_GeometryDimensions[0]}; +const DataPath k_DataPath({::k_ImageGeometry, ::k_CellData, k_MiscData}); +const DataPath k_ImageFeatureIdsPath({::k_ImageGeometry, ::k_CellData, k_FeatureIds}); + +using DirectionType = std::array; +constexpr DirectionType k_XDir{true, false, false}; +constexpr DirectionType k_XYDir{true, true, false}; +constexpr DirectionType k_XYZDir{true, true, true}; +constexpr DirectionType k_XZDir{true, false, true}; +constexpr DirectionType k_YDir{false, true, false}; +constexpr DirectionType k_YZDir{false, true, true}; +constexpr DirectionType k_ZDir{false, false, true}; + +using ExemplarDataType = std::array; +// Exemplar Dilate data for A/B testing +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateX1{0, 0, 1, 2, 2, 1, 2, 2, 1, 0, 0, 0, 0, 0, 0, 0, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateX1{0, 0, 2, 3, 4, 5, 6, 7, 8, 10, 10, 10, 13, 13, 14, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 31, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateX2{0, 0, 0, 2, 2, 1, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 0, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateX2{0, 0, 0, 3, 4, 5, 6, 7, 10, 10, 10, 10, 13, 13, 14, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 31, 31, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateXY1{0, 0, 1, 2, 0, 1, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 0, 5, 6, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateXY1{0, 0, 2, 3, 0, 5, 10, 7, 8, 13, 10, 10, 13, 13, 14, 14, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 31, 28, 29, 31, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateXY2{0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 4, 4, 3, 3, 3, 0, 5, 5, 0, 0, 5, 0, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateXY2{0, 0, 10, 3, 0, 13, 10, 10, 13, 13, 10, 10, 13, 13, 14, 14, 16, 17, 18, 19, 20, 21, 22, 31, 24, 25, 31, 31, 28, 31, 31, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateXYZ1{0, 0, 1, 2, 0, 1, 0, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 4, 3, 3, 3, 3, 5, 5, 0, 0, 5, 0, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateXYZ1{0, 0, 2, 3, 0, 5, 10, 7, 8, 13, 10, 10, 13, 13, 14, 31, 0, 17, 18, 19, 20, 21, 22, 23, 24, 25, 10, 31, 28, 13, 31, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateXYZ2{0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 0, 3, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateXYZ2{0, 0, 10, 3, 0, 13, 10, 10, 13, 13, 10, 10, 13, 13, 14, 31, 0, 0, 18, 19, 0, 21, 10, 31, 24, 13, 10, 31, 13, 13, 31, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateXZ1{0, 0, 1, 2, 2, 1, 2, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 4, 3, 3, 3, 3, 5, 5, 0, 5, 5, 0, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateXZ1{0, 0, 2, 3, 4, 5, 6, 7, 8, 10, 10, 10, 13, 13, 14, 31, 0, 17, 18, 19, 20, 21, 22, 23, 24, 25, 10, 27, 28, 13, 31, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateXZ2{0, 0, 0, 2, 2, 1, 2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 3, 3, 3, 3, 5, 0, 0, 0, 0, 0, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateXZ2{0, 0, 0, 3, 4, 5, 6, 7, 10, 10, 10, 10, 13, 13, 14, 31, 0, 0, 18, 19, 20, 21, 22, 23, 24, 10, 10, 10, 13, 13, 31, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateY1{0, 1, 1, 2, 0, 1, 0, 2, 1, 0, 0, 2, 2, 0, 0, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 0, 5, 6, 6, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateY1{0, 1, 2, 3, 0, 5, 10, 7, 8, 13, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 31, 28, 29, 30, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateY2{0, 1, 0, 2, 0, 0, 0, 2, 0, 0, 0, 2, 2, 0, 0, 3, 4, 4, 4, 4, 3, 3, 3, 0, 5, 5, 5, 0, 5, 6, 6, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateY2{0, 1, 10, 3, 0, 13, 10, 7, 0, 13, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 31, 24, 25, 26, 31, 28, 29, 30, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateYZ1{0, 1, 1, 2, 0, 1, 0, 2, 1, 0, 0, 2, 2, 0, 0, 0, 0, 4, 4, 4, 3, 3, 3, 3, 5, 5, 0, 0, 5, 0, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateYZ1{0, 1, 2, 3, 0, 5, 10, 7, 8, 13, 10, 11, 12, 13, 14, 31, 0, 17, 18, 19, 20, 21, 22, 23, 24, 25, 10, 31, 28, 13, 14, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateYZ2{0, 1, 0, 2, 0, 0, 0, 2, 0, 0, 0, 0, 2, 0, 0, 0, 0, 4, 4, 4, 0, 3, 0, 0, 5, 0, 0, 0, 5, 0, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateYZ2{0, 1, 10, 3, 0, 13, 10, 7, 0, 13, 10, 31, 12, 13, 14, 31, 0, 17, 18, 19, 0, 21, 10, 31, 24, 13, 10, 31, 28, 13, 14, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateZ1{0, 1, 1, 2, 2, 1, 2, 2, 1, 1, 0, 2, 2, 0, 0, 0, 0, 4, 4, 4, 3, 3, 3, 3, 5, 5, 0, 5, 5, 0, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateZ1{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 31, 0, 17, 18, 19, 20, 21, 22, 23, 24, 25, 10, 27, 28, 13, 14, 31}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsDilateZ2{0, 1, 1, 2, 2, 1, 2, 2, 1, 1, 0, 2, 2, 0, 0, 0, 0, 4, 4, 4, 3, 3, 3, 3, 5, 5, 0, 5, 5, 0, 0, 0}; +constexpr ExemplarDataType k_ExemplarDataDilateZ2{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 31, 0, 17, 18, 19, 20, 21, 22, 23, 24, 25, 10, 27, 28, 13, 14, 31}; + +// Exemplar Erode data for A/B testing +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeX1{1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2, 2, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 6}; +constexpr ExemplarDataType k_ExemplarDataErodeX1{1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 9, 11, 12, 12, 15, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 30}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeX2{1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2, 2, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 6}; +constexpr ExemplarDataType k_ExemplarDataErodeX2{1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 9, 11, 12, 12, 15, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 30}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeXY1{1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 2, 2, 2, 1, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 5}; +constexpr ExemplarDataType k_ExemplarDataErodeXY1{1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 11, 12, 9, 15, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 27}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeXY2{1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 2, 2, 2, 1, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 5}; +constexpr ExemplarDataType k_ExemplarDataErodeXY2{1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 11, 12, 9, 15, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 27}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeXYZ1{1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 2, 2, 2, 1, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 3}; +constexpr ExemplarDataType k_ExemplarDataErodeXYZ1{1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 11, 12, 9, 15, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeXYZ2{1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 2, 2, 2, 1, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 3}; +constexpr ExemplarDataType k_ExemplarDataErodeXYZ2{1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 11, 12, 9, 15, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeXZ1{1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2, 2, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 3}; +constexpr ExemplarDataType k_ExemplarDataErodeXZ1{1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 9, 11, 12, 12, 15, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeXZ2{1, 1, 1, 2, 2, 1, 2, 2, 1, 1, 1, 2, 2, 2, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 3}; +constexpr ExemplarDataType k_ExemplarDataErodeXZ2{1, 1, 2, 3, 4, 5, 6, 7, 8, 9, 9, 11, 12, 12, 15, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeY1{2, 1, 1, 2, 2, 1, 2, 2, 1, 1, 2, 2, 2, 1, 0, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 5}; +constexpr ExemplarDataType k_ExemplarDataErodeY1{4, 1, 2, 3, 4, 5, 6, 7, 8, 9, 6, 11, 12, 9, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 27}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeY2{2, 1, 1, 2, 2, 1, 2, 2, 1, 1, 2, 2, 2, 1, 2, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 5}; +constexpr ExemplarDataType k_ExemplarDataErodeY2{4, 1, 2, 3, 4, 5, 6, 7, 8, 9, 6, 11, 12, 9, 6, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 27}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeYZ1{2, 1, 1, 2, 2, 1, 2, 2, 1, 1, 2, 2, 2, 1, 6, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 3}; +constexpr ExemplarDataType k_ExemplarDataErodeYZ1{4, 1, 2, 3, 4, 5, 6, 7, 8, 9, 6, 11, 12, 9, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeYZ2{2, 1, 1, 2, 2, 1, 2, 2, 1, 1, 2, 2, 2, 1, 6, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 3}; +constexpr ExemplarDataType k_ExemplarDataErodeYZ2{4, 1, 2, 3, 4, 5, 6, 7, 8, 9, 6, 11, 12, 9, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeZ1{4, 1, 1, 2, 2, 1, 2, 2, 1, 1, 5, 2, 2, 6, 6, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 3}; +constexpr ExemplarDataType k_ExemplarDataErodeZ1{16, 1, 2, 3, 4, 5, 6, 7, 8, 9, 26, 11, 12, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15}; + +constexpr ExemplarDataType k_ExemplarFeatureIdsErodeZ2{4, 1, 1, 2, 2, 1, 2, 2, 1, 1, 5, 2, 2, 6, 6, 3, 4, 4, 4, 4, 3, 3, 3, 3, 5, 5, 5, 5, 5, 6, 6, 3}; +constexpr ExemplarDataType k_ExemplarDataErodeZ2{16, 1, 2, 3, 4, 5, 6, 7, 8, 9, 26, 11, 12, 29, 30, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 15}; + +/** + * @brief The expected FeatureIds and Misc output for a single (operation, directions, iterations) combination. + */ +struct ExemplarRecord +{ + ChoicesParameter::ValueType operation; + DirectionType directions; + int32 iterations; + const ExemplarDataType& expectedFeatureIds; + const ExemplarDataType& expectedData; +}; + +// One row per valid parameter combination: 2 operations x 7 direction combinations x 2 iteration counts. +// The all-directions-off combination is rejected by preflight and therefore has no row. +// clang-format off +const std::array k_Exemplars{{ + {k_Dilate, k_XDir, 1, k_ExemplarFeatureIdsDilateX1, k_ExemplarDataDilateX1}, + {k_Dilate, k_XDir, 2, k_ExemplarFeatureIdsDilateX2, k_ExemplarDataDilateX2}, + {k_Dilate, k_XYDir, 1, k_ExemplarFeatureIdsDilateXY1, k_ExemplarDataDilateXY1}, + {k_Dilate, k_XYDir, 2, k_ExemplarFeatureIdsDilateXY2, k_ExemplarDataDilateXY2}, + {k_Dilate, k_XYZDir, 1, k_ExemplarFeatureIdsDilateXYZ1, k_ExemplarDataDilateXYZ1}, + {k_Dilate, k_XYZDir, 2, k_ExemplarFeatureIdsDilateXYZ2, k_ExemplarDataDilateXYZ2}, + {k_Dilate, k_XZDir, 1, k_ExemplarFeatureIdsDilateXZ1, k_ExemplarDataDilateXZ1}, + {k_Dilate, k_XZDir, 2, k_ExemplarFeatureIdsDilateXZ2, k_ExemplarDataDilateXZ2}, + {k_Dilate, k_YDir, 1, k_ExemplarFeatureIdsDilateY1, k_ExemplarDataDilateY1}, + {k_Dilate, k_YDir, 2, k_ExemplarFeatureIdsDilateY2, k_ExemplarDataDilateY2}, + {k_Dilate, k_YZDir, 1, k_ExemplarFeatureIdsDilateYZ1, k_ExemplarDataDilateYZ1}, + {k_Dilate, k_YZDir, 2, k_ExemplarFeatureIdsDilateYZ2, k_ExemplarDataDilateYZ2}, + {k_Dilate, k_ZDir, 1, k_ExemplarFeatureIdsDilateZ1, k_ExemplarDataDilateZ1}, + {k_Dilate, k_ZDir, 2, k_ExemplarFeatureIdsDilateZ2, k_ExemplarDataDilateZ2}, + {k_Erode, k_XDir, 1, k_ExemplarFeatureIdsErodeX1, k_ExemplarDataErodeX1}, + {k_Erode, k_XDir, 2, k_ExemplarFeatureIdsErodeX2, k_ExemplarDataErodeX2}, + {k_Erode, k_XYDir, 1, k_ExemplarFeatureIdsErodeXY1, k_ExemplarDataErodeXY1}, + {k_Erode, k_XYDir, 2, k_ExemplarFeatureIdsErodeXY2, k_ExemplarDataErodeXY2}, + {k_Erode, k_XYZDir, 1, k_ExemplarFeatureIdsErodeXYZ1, k_ExemplarDataErodeXYZ1}, + {k_Erode, k_XYZDir, 2, k_ExemplarFeatureIdsErodeXYZ2, k_ExemplarDataErodeXYZ2}, + {k_Erode, k_XZDir, 1, k_ExemplarFeatureIdsErodeXZ1, k_ExemplarDataErodeXZ1}, + {k_Erode, k_XZDir, 2, k_ExemplarFeatureIdsErodeXZ2, k_ExemplarDataErodeXZ2}, + {k_Erode, k_YDir, 1, k_ExemplarFeatureIdsErodeY1, k_ExemplarDataErodeY1}, + {k_Erode, k_YDir, 2, k_ExemplarFeatureIdsErodeY2, k_ExemplarDataErodeY2}, + {k_Erode, k_YZDir, 1, k_ExemplarFeatureIdsErodeYZ1, k_ExemplarDataErodeYZ1}, + {k_Erode, k_YZDir, 2, k_ExemplarFeatureIdsErodeYZ2, k_ExemplarDataErodeYZ2}, + {k_Erode, k_ZDir, 1, k_ExemplarFeatureIdsErodeZ1, k_ExemplarDataErodeZ1}, + {k_Erode, k_ZDir, 2, k_ExemplarFeatureIdsErodeZ2, k_ExemplarDataErodeZ2}, +}}; +// clang-format on + +DataStructure CreateTestData() +{ + DataStructure dataStructure; + auto* geom = ImageGeom::Create(dataStructure, ::k_ImageGeometry); + geom->setDimensions(k_GeometryDimensions); + + auto* cellData = AttributeMatrix::Create(dataStructure, ::k_CellData, k_TupleShape, geom->getId()); + + // Feature IDs + auto featureIdsPtr = std::make_shared(k_TupleShape, ShapeType{1}, 0); + auto* featureIdsArray = Int32Array::Create(dataStructure, ::k_FeatureIds, featureIdsPtr, cellData->getId()); + + // Index 0, 14, 31 + auto& featureIds = featureIdsArray->getDataStoreRef(); + featureIds[0] = 0; + featureIds[1] = 1; + featureIds[2] = 1; + featureIds[3] = 2; + + featureIds[4] = 2; + featureIds[5] = 1; + featureIds[6] = 2; + featureIds[7] = 2; + + featureIds[8] = 1; + featureIds[9] = 1; + featureIds[10] = 0; + featureIds[11] = 2; + + featureIds[12] = 2; + featureIds[13] = 0; + featureIds[14] = 0; + featureIds[15] = 3; + // Z + featureIds[16] = 4; + featureIds[17] = 4; + featureIds[18] = 4; + featureIds[19] = 4; + + featureIds[20] = 3; + featureIds[21] = 3; + featureIds[22] = 3; + featureIds[23] = 3; + + featureIds[24] = 5; + featureIds[25] = 5; + featureIds[26] = 5; + featureIds[27] = 5; + + featureIds[28] = 5; + featureIds[29] = 6; + featureIds[30] = 6; + featureIds[31] = 0; + + // Misc DataArray + auto dataStorePtr = std::make_shared(k_TupleShape, ShapeType{1}, 0); + auto* miscArray = Int32Array::Create(dataStructure, k_MiscData, dataStorePtr, cellData->getId()); + + auto& dataStore = miscArray->getDataStoreRef(); + for(usize i = 0; i < dataStore.size(); i++) + { + dataStore[i] = i; + } + + return dataStructure; +} + +/** + * @brief Verifies that an array listed in IgnoredDataArrayPaths still holds its original values. + */ +void CheckPathIgnored(const DataStructure& dataStructure) +{ + const DataStructure exemplarStructure = CreateTestData(); + + REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_DataPath)); + const auto& dataStore = dataStructure.getDataRefAs(k_DataPath).getDataStoreRef(); + REQUIRE_NOTHROW(exemplarStructure.getDataRefAs(k_DataPath)); + const auto& exemplarStore = exemplarStructure.getDataRefAs(k_DataPath).getDataStoreRef(); + + REQUIRE(dataStore.size() == exemplarStore.size()); + + for(usize i = 0; i < dataStore.size(); i++) + { + REQUIRE(dataStore[i] == exemplarStore[i]); + } +} + +/** + * @brief Verifies that the filter actually modified FeatureIds. Without this, a "the ignored array was left + * alone" assertion would also pass for a filter that never ran at all. + */ +void CheckFeatureIdsModified(const DataStructure& dataStructure) +{ + const DataStructure exemplarStructure = CreateTestData(); + + REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_ImageFeatureIdsPath)); + const auto& dataStore = dataStructure.getDataRefAs(k_ImageFeatureIdsPath).getDataStoreRef(); + REQUIRE_NOTHROW(exemplarStructure.getDataRefAs(k_ImageFeatureIdsPath)); + const auto& exemplarStore = exemplarStructure.getDataRefAs(k_ImageFeatureIdsPath).getDataStoreRef(); + + REQUIRE(dataStore.size() == exemplarStore.size()); + + bool anyValueChanged = false; + for(usize i = 0; i < dataStore.size() && !anyValueChanged; i++) + { + anyValueChanged = dataStore[i] != exemplarStore[i]; + } + REQUIRE(anyValueChanged); +} + +/** + * @brief Compares the generated FeatureIds and Misc arrays against the expected values for this combination + * of operation, enabled directions, and iteration count. + */ +void CheckOutput(const DataStructure& dataStructure, ChoicesParameter::ValueType operation, const DirectionType& directions, int32 iterations) +{ + const auto exemplarIter = std::find_if(k_Exemplars.cbegin(), k_Exemplars.cend(), + [&](const ExemplarRecord& record) { return record.operation == operation && record.directions == directions && record.iterations == iterations; }); + if(exemplarIter == k_Exemplars.cend()) + { + FAIL(fmt::format("No expected output is tabulated for operation {} with directions X={} Y={} Z={} and {} iteration(s)", operation, directions[0], directions[1], directions[2], iterations)); + return; + } + + REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_ImageFeatureIdsPath)); + const Int32AbstractDataStore& featureIds = dataStructure.getDataRefAs(k_ImageFeatureIdsPath).getDataStoreRef(); + REQUIRE_NOTHROW(dataStructure.getDataRefAs(k_DataPath)); + const Int32AbstractDataStore& dataStore = dataStructure.getDataRefAs(k_DataPath).getDataStoreRef(); + + REQUIRE(featureIds.size() == exemplarIter->expectedFeatureIds.size()); + REQUIRE(dataStore.size() == exemplarIter->expectedData.size()); + + for(usize i = 0; i < dataStore.size(); i++) + { + REQUIRE(featureIds[i] == exemplarIter->expectedFeatureIds[i]); + REQUIRE(dataStore[i] == exemplarIter->expectedData[i]); + } +} + +void RunFilter(DataStructure& dataStructure, ChoicesParameter::ValueType operation, int32 numIterations, const DirectionType& directions, const DataPath& geometryPath, const DataPath& featureIdsPath) +{ + const ErodeDilateBadDataFilter filter; + Arguments args; + + // Create default Parameters for the filter. + args.insertOrAssign(ErodeDilateBadDataFilter::k_Operation_Key, std::make_any(operation)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_NumIterations_Key, std::make_any(numIterations)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_XDirOn_Key, std::make_any(directions[0])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_YDirOn_Key, std::make_any(directions[1])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_ZDirOn_Key, std::make_any(directions[2])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(featureIdsPath)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any(MultiArraySelectionParameter::ValueType{})); + args.insertOrAssign(ErodeDilateBadDataFilter::k_SelectedImageGeometryPath_Key, std::make_any(geometryPath)); + + // Preflight the filter and check result + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions) + + // Execute the filter and check the result + auto executeResult = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result) +} } // namespace TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter(Erode)", "[SimplnxCore][ErodeDilateBadDataFilter]") @@ -39,8 +356,6 @@ TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter(Erode)", "[SimplnxCore][ErodeDi const nx::core::UnitTest::TestFileSentinel testDataSentinel(nx::core::unit_test::k_TestFilesDir, "6_6_erode_dilate_test.tar.gz", "6_6_erode_dilate_test"); - UnitTest::LoadPlugins(); - // Read Exemplar DREAM3D File Filter auto exemplarFilePath = fs::path(fmt::format("{}/6_6_erode_dilate_test/6_6_erode_dilate_bad_data.dream3d", unit_test::k_TestFilesDir)); DataStructure dataStructure = LoadDataStructure(exemplarFilePath); @@ -81,46 +396,162 @@ TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter(Erode)", "[SimplnxCore][ErodeDi UnitTest::CheckArraysInheritTupleDims(dataStructure); } -TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter(Dilate)", "[SimplnxCore][ErodeDilateBadDataFilter]") +TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter(Erode) Expanded", "[SimplnxCore][ErodeDilateBadDataFilter]") { UnitTest::LoadPlugins(); - const nx::core::UnitTest::TestFileSentinel testDataSentinel(nx::core::unit_test::k_TestFilesDir, "6_6_erode_dilate_test.tar.gz", "6_6_erode_dilate_test"); + bool dirX = GENERATE(true, false); + bool dirY = GENERATE(true, false); + bool dirZ = GENERATE(true, false); + int32 numIterations = GENERATE(1, 2); + + const DirectionType directions = {dirX, dirY, dirZ}; + const ChoicesParameter::ValueType operation = nx::core::detail::k_ErodeIndex; + + // At least one direction is required; preflight rejects the all-off combination. See the No Direction test. + if(!dirX && !dirY && !dirZ) + { + SUCCEED("at least one direction is required"); + return; + } - const std::string k_ExemplarDataContainerName("Exemplar Bad Data Dilate"); - const DataPath k_DilateCellAttributeMatrixDataPath = DataPath({k_ExemplarDataContainerName, "EBSD Scan Data"}); + DataStructure dataStructure = CreateTestData(); + + RunFilter(dataStructure, operation, numIterations, directions, DataPath({k_ImageGeometry}), k_ImageFeatureIdsPath); + CheckOutput(dataStructure, operation, directions, numIterations); + + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} +TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter(Dilate) Expanded", "[SimplnxCore][ErodeDilateBadDataFilter]") +{ UnitTest::LoadPlugins(); - // Read Exemplar DREAM3D File Filter - auto exemplarFilePath = fs::path(fmt::format("{}/6_6_erode_dilate_test/6_6_erode_dilate_bad_data.dream3d", unit_test::k_TestFilesDir)); - DataStructure dataStructure = LoadDataStructure(exemplarFilePath); + bool dirX = GENERATE(true, false); + bool dirY = GENERATE(true, false); + bool dirZ = GENERATE(true, false); + int32 numIterations = GENERATE(1, 2); + const DirectionType directions = {dirX, dirY, dirZ}; + const ChoicesParameter::ValueType operation = nx::core::detail::k_DilateIndex; + + // At least one direction is required; preflight rejects the all-off combination. See the No Direction test. + if(!dirX && !dirY && !dirZ) { - const ErodeDilateBadDataFilter filter; + SUCCEED("at least one direction is required"); + return; + } - Arguments args; + DataStructure dataStructure = CreateTestData(); - // Create default Parameters for the filter. - args.insertOrAssign(ErodeDilateBadDataFilter::k_Operation_Key, std::make_any(k_Dilate)); - args.insertOrAssign(ErodeDilateBadDataFilter::k_NumIterations_Key, std::make_any(2)); - args.insertOrAssign(ErodeDilateBadDataFilter::k_XDirOn_Key, std::make_any(true)); - args.insertOrAssign(ErodeDilateBadDataFilter::k_YDirOn_Key, std::make_any(true)); - args.insertOrAssign(ErodeDilateBadDataFilter::k_ZDirOn_Key, std::make_any(true)); - args.insertOrAssign(ErodeDilateBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(k_FeatureIdsDataPath)); - args.insertOrAssign(ErodeDilateBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any(MultiArraySelectionParameter::ValueType{})); - args.insertOrAssign(ErodeDilateBadDataFilter::k_SelectedImageGeometryPath_Key, std::make_any(k_InputData)); + RunFilter(dataStructure, operation, numIterations, directions, DataPath({k_ImageGeometry}), k_ImageFeatureIdsPath); + CheckOutput(dataStructure, operation, directions, numIterations); - // Preflight the filter and check result - auto preflightResult = filter.preflight(dataStructure, args); - SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions) + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} - // Execute the filter and check the result - auto executeResult = filter.execute(dataStructure, args); - SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result) - } +TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter Ignored Path", "[SimplnxCore][ErodeDilateBadDataFilter]") +{ + UnitTest::LoadPlugins(); - UnitTest::CompareExemplarToGeneratedData(dataStructure, dataStructure, k_EbsdScanDataDataPath, k_ExemplarDataContainerName); + DataStructure dataStructure = CreateTestData(); + const DirectionType directions = {true, true, true}; + const ChoicesParameter::ValueType operation = GENERATE(k_Dilate, k_Erode); + const int32 numIterations = 1; + + const DataPath ignoredPath = k_DataPath; + + const ErodeDilateBadDataFilter filter; + Arguments args; + + // Create default Parameters for the filter. + args.insertOrAssign(ErodeDilateBadDataFilter::k_Operation_Key, std::make_any(operation)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_NumIterations_Key, std::make_any(numIterations)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_XDirOn_Key, std::make_any(directions[0])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_YDirOn_Key, std::make_any(directions[1])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_ZDirOn_Key, std::make_any(directions[2])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(k_ImageFeatureIdsPath)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any(MultiArraySelectionParameter::ValueType{ignoredPath})); + args.insertOrAssign(ErodeDilateBadDataFilter::k_SelectedImageGeometryPath_Key, std::make_any(DataPath({k_ImageGeometry}))); + + // Preflight the filter and check result + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(preflightResult.outputActions); + + // Execute the filter and check the result + auto executeResult = filter.execute(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_VALID(executeResult.result); + + // The ignored array must come through untouched, and the filter must have actually done work -- without + // the second check the first one would also pass for a filter that never ran. + CheckPathIgnored(dataStructure); + CheckFeatureIdsModified(dataStructure); + + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter No Direction", "[SimplnxCore][ErodeDilateBadDataFilter]") +{ + UnitTest::LoadPlugins(); + + DataStructure dataStructure = CreateTestData(); + const DirectionType directions = {false, false, false}; + const ChoicesParameter::ValueType operation = GENERATE(k_Dilate, k_Erode); + const int32 numIterations = GENERATE(1, 2); + + const ErodeDilateBadDataFilter filter; + Arguments args; + + // Create default Parameters for the filter. + args.insertOrAssign(ErodeDilateBadDataFilter::k_Operation_Key, std::make_any(operation)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_NumIterations_Key, std::make_any(numIterations)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_XDirOn_Key, std::make_any(directions[0])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_YDirOn_Key, std::make_any(directions[1])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_ZDirOn_Key, std::make_any(directions[2])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(k_ImageFeatureIdsPath)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any(MultiArraySelectionParameter::ValueType{})); + args.insertOrAssign(ErodeDilateBadDataFilter::k_SelectedImageGeometryPath_Key, std::make_any(DataPath({k_ImageGeometry}))); + + // Preflight the filter and check result + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); + + REQUIRE(preflightResult.outputActions.errors()[0].code == -14601); + + UnitTest::CheckArraysInheritTupleDims(dataStructure); +} + +TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter No Dimensions", "[SimplnxCore][ErodeDilateBadDataFilter]") +{ + UnitTest::LoadPlugins(); + + DataStructure dataStructure = CreateTestData(); + const DirectionType directions = {true, true, true}; + const ChoicesParameter::ValueType operation = k_Dilate; + const int32 numIterations = 1; + const DataPath geomPath({k_ImageGeometry}); + + auto* imageGeom = dataStructure.getDataAs(geomPath); + imageGeom->setDimensions(SizeVec3()); + + const ErodeDilateBadDataFilter filter; + Arguments args; + + // Create default Parameters for the filter. + args.insertOrAssign(ErodeDilateBadDataFilter::k_Operation_Key, std::make_any(operation)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_NumIterations_Key, std::make_any(numIterations)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_XDirOn_Key, std::make_any(directions[0])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_YDirOn_Key, std::make_any(directions[1])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_ZDirOn_Key, std::make_any(directions[2])); + args.insertOrAssign(ErodeDilateBadDataFilter::k_CellFeatureIdsArrayPath_Key, std::make_any(k_ImageFeatureIdsPath)); + args.insertOrAssign(ErodeDilateBadDataFilter::k_IgnoredDataArrayPaths_Key, std::make_any(MultiArraySelectionParameter::ValueType{})); + args.insertOrAssign(ErodeDilateBadDataFilter::k_SelectedImageGeometryPath_Key, std::make_any(geomPath)); + + // Preflight the filter and check result + auto preflightResult = filter.preflight(dataStructure, args); + SIMPLNX_RESULT_REQUIRE_INVALID(preflightResult.outputActions); + + REQUIRE(preflightResult.outputActions.errors()[0].code == -14602); UnitTest::CheckArraysInheritTupleDims(dataStructure); } @@ -131,7 +562,7 @@ TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter: SIMPL Backwards Compatibility" UnitTest::LoadPlugins(); auto filterList = app->getFilterList(); - const fs::path conversionDir = fs::path(nx::core::unit_test::k_SourceDir.view()) / "test" / "simpl_conversion"; + const fs::path conversionDir = fs::path(unit_test::k_SourceDir.view()) / "test" / "simpl_conversion"; const std::vector> fixtures = { {"SIMPL 6.5 (UUID)", conversionDir / "6_5" / "ErodeDilateBadDataFilter.json"}, @@ -158,7 +589,7 @@ TEST_CASE("SimplnxCore::ErodeDilateBadDataFilter: SIMPL Backwards Compatibility" CHECK(pipelineFilter->getComments().empty()); const Arguments args = pipelineFilter->getArguments(); - CHECK(args.value(ErodeDilateBadDataFilter::k_Operation_Key) == 0); + CHECK(args.value(ErodeDilateBadDataFilter::k_Operation_Key) == k_Dilate); CHECK(args.value(ErodeDilateBadDataFilter::k_NumIterations_Key) == 5); CHECK(args.value(ErodeDilateBadDataFilter::k_XDirOn_Key) == true); CHECK(args.value(ErodeDilateBadDataFilter::k_YDirOn_Key) == true); diff --git a/src/Plugins/SimplnxCore/vv/ErodeDilateBadDataFilter.md b/src/Plugins/SimplnxCore/vv/ErodeDilateBadDataFilter.md new file mode 100644 index 0000000000..8a4fbb00bb --- /dev/null +++ b/src/Plugins/SimplnxCore/vv/ErodeDilateBadDataFilter.md @@ -0,0 +1,142 @@ +# V&V Report: ErodeDilateBadDataFilter + +| | | +|-----------------------------|--------------------------------------------------------------------------| +| Plugin | SimplnxCore | +| SIMPLNX UUID | `7f2f7378-580e-4337-8c04-a29e7883db0b` | +| SIMPLNX Human Name | Erode/Dilate Bad Data | +| DREAM3D 6.5.171 equivalent | `ErodeDilateBadData` — SIMPL UUID `3adfe077-c3c9-5cd0-ad74-cf5f8ff3d254` | +| Verified commit | ** | +| Status | READY FOR REVIEW | +| Sign-off | *pending* | + +## At a glance + +| Aspect | Current state | +|------------------------|----------------| +| Algorithm Relationship | **Port** — legacy `ErodeDilateBadData.{h,cpp}` was diffed line-by-line against `Algorithms/ErodeDilateBadData.cpp` this pass. Neighbor offsets, boundary-validity checks, vote/tie-break order, and transfer conditions are structurally identical. One divergence found and fixed — see Bug Fixes. | +| Oracle (confirmed) | **Class 2 (Reference implementation).** The 28 expected `FeatureIds`/`Misc` arrays (7 direction combinations × 2 operations × 2 iteration counts) are genuine DREAM3D 6.5.171 output, matched element-wise against SIMPLNX and compiled into `ErodeDilateBadDataTest.cpp` as constants so the comparison re-runs in CI without the legacy binary. Confirmed — `(Erode) Expanded` and `(Dilate) Expanded` pass, 28/28 combinations. | +| Code paths enumerated | **10 of 11 exercised.** All 6 face directions (-Z/-Y/-X/+X/+Y/+Z) confirmed hit by instrumentation. Only gap: the `m_ShouldCancel` early-exit (Path 11) — no test injects a cancel signal. | +| Tests today | **7 TEST_CASEs, all pass in both in-core and OOC builds** (2283 assertions, identical in each): 1 production-scale exemplar-archive comparison + 2 `GENERATE` parameter sweeps (14 valid runs each over direction × iteration count) + 1 ignored-path test + 2 preflight-error tests + 1 SIMPL backwards-compat. | +| Exemplar archive | `6_6_erode_dilate_test.tar.gz` — provides `Input Data` plus legacy-generated `Exemplar Bad Data Erode` / `Exemplar Bad Data Dilate` containers on a 189×201×20 Small IN100 slice. Consumed by the `(Erode)` test here and shared with `ErodeDilateMaskTest` and `ErodeDilateCoordinationNumberTest`. SHA512 verified against `test/CMakeLists.txt`. | +| Legacy comparison | **Run.** Two independent comparisons: (1) all 28 parameter combinations run through DREAM3D 6.5.171 `PipelineRunner` against an HDF5 twin of the inline fixture and diffed element-wise — **28/28 exact matches** on both `FeatureIds` and `Misc`; (2) the `(Erode)` test compares SIMPLNX against a legacy-generated exemplar at production scale (759,780 cells, 6 cell arrays). | +| Bug flags | `ErodeDilateBadDataFilter-D1` (X/Y/Z direction parameters had no effect) — **confirmed and fixed this pass.** One additional hypothesis (Dilate tie-break order) was investigated, found to be a false lead, and reverted — see deviations doc. | +| V&V phase | Oracle chosen and confirmed; legacy comparison run; direction-masking bug fixed; zero-dimensions preflight path now covered. Outstanding before promotion to COMPLETE: second-engineer sign-off, the uncovered cancel path (Path 11), and formalizing the manual 28-combination A/B run as an automated archive-based test (see deviations doc). | + +## Summary + +`ErodeDilateBadDataFilter` erodes or dilates voxels with `FeatureId == 0` ("bad data") in an `ImageGeometry`, optionally restricted to any non-empty combination of X, Y, and Z face directions. Verification is **Class 2**: the 28 expected output arrays compiled into `(Erode) Expanded` / `(Dilate) Expanded` are genuine DREAM3D 6.5.171 output for the same fixture, matched element-wise across every operation × direction × iteration combination, and the `(Erode)` test additionally compares against a legacy-generated exemplar archive at production scale. One SIMPLNX-side bug was found and fixed this pass (`ErodeDilateBadDataFilter-D1` — the direction parameters had no effect at all); all 7 tests pass in both in-core and OOC builds with 2283 assertions. + +## Algorithm Relationship + +*Classification:* **Port** ~~| Minor changes | Rewrite | New filter~~ + +*Evidence:* `SimplnxCoreLegacyUUIDMapping.hpp` maps legacy SIMPL UUID `3adfe077-c3c9-5cd0-ad74-cf5f8ff3d254` directly to `FilterTraits`, and `test/simpl_conversion/{6_4,6_5}/ErodeDilateBadDataFilter.json` carry the legacy `Direction`/`NumIterations`/`XDirOn`/`YDirOn`/`ZDirOn`/`FeatureIdsArrayPath`/`IgnoredDataArrayPaths` parameter set unchanged — the same filter with the same parameter model, not a reimplementation. Legacy source (`Source/Plugins/Processing/ProcessingFilters/ErodeDilateBadData.{h,cpp}`, from a sibling `DREAM3D` checkout on the authoring engineer's machine, not committed to this repository) was diffed line-by-line against `Algorithms/ErodeDilateBadData.cpp` this pass rather than inferred from documentation. + +*Port-time deltas:* + +1. **Face-neighbor offsets** — legacy computed `neighpoints[]` inline; SIMPLNX calls `initializeFaceNeighborOffsets(dims)` from `NeighborUtilities`. Same six offsets in the same `[-Z,-Y,-X,+X,+Y,+Z]` order; no output change. +2. **Boundary-validity checks** — legacy tested each face boundary with inline conditionals; SIMPLNX calls `computeValidFaceNeighbors(x, y, z, dims)`. Reproduces the same six conditions; no output change. +3. **Direction gating** — legacy ORs the direction flag into each per-face boundary check (`|| !m_ZDirOn`); SIMPLNX masks the per-voxel `isValidFaceNeighbor` array in `adjustValidNeighbors`. Equivalent once wired in — but it was *not* wired in, which is `ErodeDilateBadDataFilter-D1`. Now fixed and legacy-verified. +4. **Parallel array transfer** — legacy transferred all cell arrays in one serial pass interleaved with `FeatureIds`; SIMPLNX uses `ParallelTaskAlgorithm` for the non-`FeatureIds` arrays and transfers `FeatureIds` afterward, serially. Proven equivalent: erode only maps 0→>0 and dilate only >0→0, `neighbors[]` always points at a voxel whose relevant polarity is preserved, and each index is written at most once per pass, so no transfer predicate can observe a changed value. No output change. +5. **Cancel check** — SIMPLNX reads `m_ShouldCancel` once per Z-slice; legacy has no cancel check at all. Additive; no output change on a run to completion. +6. **Progress reporting** — SIMPLNX reports per-array progress through `MessageHelper`/`ThrottledMessenger`; legacy used `notifyStatusMessage`. No output change. + +*Material PRs since baseline:* four PRs have touched `Algorithms/ErodeDilateBadData.cpp` and account for the port-time deltas above: + +- **#1523** — factored the 6-face-neighbor code out into `NeighborUtilities` (deltas 1–2, and the `isValidFaceNeighbor` array that delta 3 masks). +- **#1590** — standardized 2D image handling (`VoxelNeighbors` specialization). +- **#1340** — thread-safe messaging rework (delta 6). +- **#1687** — this branch: fixes D1, adds the `-14601`/`-14602` preflight guards, and rebuilds the test suite. + +Earlier commits (#1249, #1017, #1013, #801) are compiler-warning, store-API, and rename churn with no behavioral content. + +*SIMPLNX implementation:* `Algorithms/ErodeDilateBadData.cpp` (231 lines) uses `NeighborUtilities::VoxelNeighbors` for face-neighbor offsets and boundary validity, and `ParallelTaskAlgorithm` to transfer non-`FeatureIds` arrays in parallel. + +## Bug Fixes (this pass) + +### ErodeDilateBadDataFilter-D1: Direction parameters had no effect — fixed + +See deviations doc for full detail. Summary: `adjustValidNeighbors` was dead code (defined, never called); `XDirOn`/`YDirOn`/`ZDirOn` had zero effect on which face neighbors participated. Fixed by retyping the helper to mask the actual per-voxel validity array (`isValidFaceNeighbor`) with correct axis mapping, and calling it at `Algorithms/ErodeDilateBadData.cpp:162-163` for every bad-data voxel. Verified by regenerating all 28 exemplar constants from real DREAM3D 6.5.171 output (they were previously byte-identical across all 7 direction combinations for a given operation/iteration count) and matching them element-wise against SIMPLNX for every combination — see Oracle section. + +### Investigated, disproven, reverted: Dilate tie-break "fix" + +A plausible-looking bug hypothesis (last-bad-neighbor-wins vs. first-bad-neighbor-wins, for a good voxel with multiple bad neighbors) was implemented as a fix and then falsified by an actual legacy binary run. Reverted in full. Recorded as a confirmed non-deviation in the deviations doc so it isn't relitigated. This is why the Oracle section emphasizes binary-verified results over source-only reasoning: source comparison alone did not catch it, since legacy's own source has the identical "unconditional overwrite" line. Only running both binaries against the same input and diffing a value that is not blind to the tie-break (`Misc`, not `FeatureIds`) surfaced the truth. + +## Oracle + +*Class:* **2 (Reference implementation)** — expected values are genuine DREAM3D 6.5.171 output, at two different scales. + +*Applied:* Two oracles, both sourced from the legacy filter and neither derived from SIMPLNX output. + +- **Small-scale (28 combinations).** `CreateTestData()` builds an in-memory 4×4×2 (32-voxel) `ImageGeom` with a hand-authored `FeatureIds` array (features 1–6; bad voxels at flat indices 0, 10, 13, 14, 31) and a `Misc` `int32` array initialized so `Misc[i] == i`, making every copied tuple traceable to its source voxel by value alone. Pipeline JSONs (`DataContainerReader` → `ErodeDilateBadData` → `DataContainerWriter`) were run through the DREAM3D 6.5.171 `PipelineRunner` against an HDF5 twin of that fixture — verified byte-for-byte identical in dims, `FeatureIds`, and `Misc` before use — for all **28 combinations** of {Dilate, Erode} × {X, XY, XYZ, XZ, Y, YZ, Z} × {1, 2 iterations}. Outputs were diffed with `h5py` against SIMPLNX: **28/28 exact matches** on both arrays. Those legacy arrays are what the `k_ExemplarFeatureIds*` / `k_ExemplarData*` constants now hold — they were regenerated from the legacy binary, not hand-traced, which removes any question of their having been fitted to SIMPLNX's own behavior. +- **Production-scale (1 combination).** `6_6_erode_dilate_test.tar.gz` carries `Exemplar Bad Data Erode` and `Exemplar Bad Data Dilate` containers generated by the legacy SIMPL `ErodeDilateBadData` filter (UUID `3adfe077-…`, Erode and Dilate, `NumIterations=2`, all three directions on) on a 189×201×20 Small IN100 slice. The archive's embedded pipeline records that legacy build as `FilterVersion 6.6.338` — later than the 6.5.171 baseline — so this exemplar is genuine legacy output but is **not itself a 6.5.171 comparison**; the 6.5.171 comparison of record is the 28-combination run above. See `## Exemplar archive` and its provenance sidecar for both caveats. + +*Coverage split:* the archive exemplar was generated with all three directions enabled, so it cannot discriminate direction gating — a build with D1 still present passes it. That is exactly the gap the 28-combination sweep closes, and why both oracles are needed. Conversely the 32-voxel fixture is single-typed and single-component, so only the archive test exercises `copyTuple` across mixed types and component counts (`EulerAngles` float32×3, `IPFColor` uint8×3, `Mask` uint8, `Phases` int32, `Confidence Index`/`Image Quality` float32). + +*Encoded:* + +- `SimplnxCore::ErodeDilateBadDataFilter(Erode) Expanded` and `(Dilate) Expanded` — each `GENERATE`s `dirX,dirY,dirZ ∈ {true,false}` and `numIterations ∈ {1,2}`, reports the invalid all-directions-off combination with `SUCCEED`, and looks the expected arrays up in the 28-row `k_Exemplars` table. 14 valid parameterized runs each, both `FeatureIds` and `Misc` asserted, 1039 assertions each — all pass. +- `SimplnxCore::ErodeDilateBadDataFilter(Erode)` — `UnitTest::CompareExemplarToGeneratedData` against `6_6_erode_dilate_bad_data.dream3d`, 55 assertions. Passes. + +*Second-engineer review:* **pending.** The prior pass's open items — erode/dilate tie-break order, and whether direction combinations produce genuinely different output — were both resolved this pass by the legacy binary comparison above rather than by review alone. A named second-engineer sign-off on the oracle design is still required before Status can move to COMPLETE. + +## Code path coverage + +10 of 11 paths exercised. The algorithm has three logical phases: (a) preflight validation, (b) a per-voxel vote/mark scan over face neighbors, and (c) a per-array transfer pass. + +Source: `src/Plugins/SimplnxCore/src/SimplnxCore/Filters/Algorithms/ErodeDilateBadData.cpp` (231 lines). + +| # | Phase | Path | Test case | +|---|-------|------|-----------| +| 1 | (a) Setup | `numFeatures` scan, face-offset/validity initialization, `adjustValidNeighbors` direction masking | All tests. Functional as of this pass — in the prior revision `adjustValidNeighbors` was dead code; it is now wired in and confirmed exercised for all 6 face directions (see below). | +| 2 | (b) Per-voxel | `featureName != 0` (good voxel) → skip | All tests (most of the 32 voxels are good) | +| 3 | (b) Per-voxel | `featureName == 0` + Dilate + neighbor `feature > 0` → `neighbors[neighborPoint] = voxelIndex` | `(Dilate) Expanded`, all 6 face directions confirmed hit | +| 4 | (b) Per-voxel | `featureName == 0` + Erode + neighbor `feature > 0` → vote accumulation, `neighbors[voxelIndex] = neighborPoint` on new max (ties keep the first-processed neighbor) | `(Erode) Expanded`, all 6 face directions confirmed hit | +| 5 | (b) Per-voxel | Erode post-vote cleanup — `featureCount[feature] = 0` for each valid neighbor of the bad voxel | `(Erode) Expanded`, all 6 face directions confirmed hit | +| 6 | (c) Transfer | `neighbor >= 0` + Erode condition (`featureName == 0 && featureIds[neighbor] > 0`) → `copyTuple` | `(Erode) Expanded`, `(Erode)` | +| 7 | (c) Transfer | `neighbor >= 0` + Dilate condition (`featureName > 0 && featureIds[neighbor] == 0`) → `copyTuple` | `(Dilate) Expanded` | +| 8 | (c) Transfer | `neighbor == -1` → skip (voxel untouched this iteration) | Both `Expanded` tests, implicitly (voxels far from bad data are unchanged in every expected array) | +| 9 | (a) Preflight | `!xDirOn && !yDirOn && !zDirOn` → error `-14601` (`k_NoDirectionsError`) | `No Direction` — asserts `invalid()` and `errors()[0].code == -14601` for both operations | +| 10 | (a) Preflight | `dims[0] == 0 \|\| dims[1] == 0 \|\| dims[2] == 0` → error `-14602` (`k_NoGeometryDimensionsError`) | `No Dimensions` — directions all **on** so the `-14601` check does not mask this path (the prior revision zeroed them and never reached here); asserts error code is exactly `-14602` | +| 11 | (b) Cancel | `m_ShouldCancel` read once per Z-slice inside the iteration loop → early return | *Not directly tested. Requires cancel-signal injection; no test sets `m_ShouldCancel` and asserts early termination.* Legacy has no cancel check at all, so SIMPLNX is ahead of legacy here — not a deviation. | + +**Per-direction coverage, confirmed by instrumentation this pass:** `Algorithms/ErodeDilateBadData.cpp` was temporarily instrumented with per-face-direction hit counters at (a) the point immediately after the `isValidFaceNeighbor` gate in the vote/mark loop, (b) the point where the Dilate mark / Erode vote condition (`feature > 0`) fires, and (c) the equivalent point in the Erode cleanup loop. Running the full `(Erode) Expanded` + `(Dilate) Expanded` sweep produced non-zero counts for **every one of the 6 directions at all 3 measurement points** — vote/mark loop reached counts `-Z=38 -Y=111 -X=108 +X=106 +Y=64 +Z=97`; Dilate marking fired at `-Z=9 -Y=46 -X=37 +X=35 +Y=16 +Z=44`; Erode voting at `-Z=8 -Y=25 -X=24 +X=24 +Y=8 +Z=32`. The instrumentation was removed afterward; this records the empirical result, not a standing code artifact. + +Confirmed correct and deliberately not counted as deviations: + +- **Direction masking of the Erode `featureCount` reset loop.** SIMPLNX reuses the direction-masked `isValidFaceNeighbor` in the reset loop; legacy resets over *boundary-valid* neighbors and ignores the direction flags there. Simulating both variants across all 28 combinations gives identical output, and the equivalence is general: the reset set is a superset of the increment set in both variants, so `featureCount` returns to all-zeros after every bad voxel either way. +- **`neighbors` is intentionally not reset between iterations** — matches legacy. The deviations doc records the investigation that established this. +- **`MessageHelper` shared across parallel tasks.** Each task gets its own `ThrottledMessenger` (independent timing state) over a shared `std::shared_ptr`; `trySendMessage` is the documented cross-thread path. + +## Test inventory + +| Test case | Status | Notes | +|-----------|--------|-------| +| `SimplnxCore::ErodeDilateBadDataFilter(Erode)` | kept | Class 2 production-scale oracle. `6_6_erode_dilate_test.tar.gz`, Erode, all directions on, 2 iterations, 189×201×20. `CompareExemplarToGeneratedData` over the 6 cell arrays present in both containers (`Confidence Index`, `EulerAngles`, `FeatureIds`, `Image Quality`, `Mask`, `Phases`; the exemplar's `IPFColor` has no counterpart in `Input Data` and is skipped); 55 assertions. The only test that exercises `copyTuple` across mixed types and component counts. Restored this pass after an earlier revision dropped it. | +| `SimplnxCore::ErodeDilateBadDataFilter(Erode) Expanded` | new-for-V&V | Class 2 small-scale oracle. `GENERATE` over 7 valid direction combinations × 2 iteration counts (14 runs); all-off reported via `SUCCEED`. Both `FeatureIds` and `Misc` asserted against the `k_Exemplars` table; 1039 assertions. Modified this pass: expected arrays regenerated from legacy output (previously byte-identical across direction combinations, per D1), the `Misc` assertion enabled for the first time, and the two duplicated dispatch helpers collapsed into one table lookup. | +| `SimplnxCore::ErodeDilateBadDataFilter(Dilate) Expanded` | new-for-V&V | Same sweep and same table, Dilate operation; 1039 assertions. Same modifications as the Erode sweep. | +| `SimplnxCore::ErodeDilateBadDataFilter Ignored Path` | new-for-V&V | Confirms an array listed in `IgnoredDataArrayPaths` (`Misc`) is left untouched, for both operations via `GENERATE(k_Dilate, k_Erode)`; 91 assertions. Modified this pass: the prior revision only preflighted, so comparing the DataStructure against a fresh fixture could not fail. It now executes the filter and additionally asserts that `FeatureIds` *did* change, which is what keeps the ignored-path check from passing vacuously. Verified by mutation — emptying the ignore list, or dropping the `execute()` call, each now fails the test. | +| `SimplnxCore::ErodeDilateBadDataFilter No Direction` | new-for-V&V | Preflight-error test: all directions off, geometry otherwise valid, both operations × 2 iteration counts. Asserts `invalid()` and `errors()[0].code == -14601`; 25 assertions. Covers Path 9. | +| `SimplnxCore::ErodeDilateBadDataFilter No Dimensions` | new-for-V&V | Preflight-error test: `ImageGeom` dimensions forced to `{0,0,0}`, directions all on, Dilate. Asserts `invalid()` and `errors()[0].code == -14602`; 7 assertions. Covers Path 10. Modified this pass: previously it also zeroed all direction flags, which tripped `-14601` first and left Path 10 unreached. | +| `SimplnxCore::ErodeDilateBadDataFilter: SIMPL Backwards Compatibility` | kept | `DYNAMIC_SECTION` over `simpl_conversion/6_5/ErodeDilateBadDataFilter.json` (matched by `Filter_Uuid`) and `simpl_conversion/6_4/ErodeDilateBadDataFilter.json` (matched by `Filter_Name`; that fixture has no UUID field). Loads each legacy pipeline via `Pipeline::FromSIMPLFile`, confirms a single `PipelineFilter` with the right UUID, and checks the converted arguments: `Operation == k_Dilate`, `NumIterations == 5`, `XDirOn/YDirOn/ZDirOn == true`, geometry path `DataPath({"DataContainer"})`, feature-ids path `DataPath({"DataContainer","CellData","TestArray"})`; 27 assertions. `IgnoredDataArrayPaths` verified only by successful pipeline load, matching the pattern in `FillBadDataTest.cpp`. Not an oracle test. | + +All 7 tests pass in both the in-core build (`NX-Com-Qt69-Vtk96-Rel`) and the out-of-core build (`simplnx-ooc-Rel`), with identical assertion counts in each — 2283 total (55 + 1039 + 1039 + 91 + 25 + 7 + 27). + +## Exemplar archive + +- **Archive:** `6_6_erode_dilate_test.tar.gz` +- **SHA512:** `5f0773e5d296936effbb2239965f5847e7c18533b0a2c3ec6a1d6a83b03417e5b459cce29808c8e0273613b3b6fa032c675e84926eb35d8da8a6ddc0641a0ef5` +- **Provenance:** `src/Plugins/SimplnxCore/vv/provenance/6_6_erode_dilate_test.md` + +The archive is shared: `6_6_erode_dilate_bad_data.dream3d` serves this filter, `6_6_erode_dilate_mask.dream3d` serves `ErodeDilateMaskTest`, and `6_6_erode_dilate_coordination_number.dream3d` serves `ErodeDilateCoordinationNumberTest`. Any regeneration must account for all three consumers. + +## Deviations from DREAM3D 6.5.171 + +No confirmed legacy deviations. The comparison was run at both scales described in the Oracle section — 28/28 exact matches on the 32-voxel fixture, and an element-wise match against the legacy-generated exemplar on the 189×201×20 Small IN100 slice. + +One SIMPLNX-side bug was found and fixed: + +- `ErodeDilateBadDataFilter-D1` — the `XDirOn`/`YDirOn`/`ZDirOn` parameters had no effect on which face neighbors participated — see [`deviations/ErodeDilateBadDataFilter.md`](deviations/ErodeDilateBadDataFilter.md) + +One hypothesis was investigated and disproven (Dilate tie-break order: legacy matches SIMPLNX's original last-write-wins behavior), and is recorded in the same file as a confirmed non-deviation so it is not relitigated. diff --git a/src/Plugins/SimplnxCore/vv/deviations/ErodeDilateBadDataFilter.md b/src/Plugins/SimplnxCore/vv/deviations/ErodeDilateBadDataFilter.md new file mode 100644 index 0000000000..2322b4e534 --- /dev/null +++ b/src/Plugins/SimplnxCore/vv/deviations/ErodeDilateBadDataFilter.md @@ -0,0 +1,91 @@ +# Deviations from DREAM3D 6.5.171: ErodeDilateBadDataFilter + +This file lists every documented behavioral difference between this SIMPLNX filter and its DREAM3D 6.5.171 equivalent. + +Entries are referenced by stable ID (`ErodeDilateBadDataFilter-D`) from the V&V report and from public migration guidance. The ID is stable across renames; the Filter UUID field is the permanent cross-reference anchor. + +> **ID note:** `ErodeDilateBadDataFilter-D1` was cited as `ErodeDilateBadDataFilter-B1` in earlier revisions of this file and in the PR #1687 review thread, under a local `-B` convention for SIMPLNX-side bugs. That convention is used nowhere else in the repository, so the entry has been renumbered to the project-standard `-D` form (matching `CAxisSegmentFeaturesFilter-D1`, which documents an equivalent SIMPLNX-side bug found during its own V&V cycle). No external references to the old ID exist. + +## Headline: legacy A/B comparison performed — one SIMPLNX-side bug, fixed + +The gap recorded in the previous revision of this file ("no legacy comparison has been performed") is closed. A genuine DREAM3D 6.5.171 `PipelineRunner` was run against a hand-verified input twin of the C++ test fixture across **all 28 combinations** of operation (Erode/Dilate) × direction combination (7 valid combos) × iteration count (1, 2), and the legacy `ErodeDilateBadData.{h,cpp}` source was diffed line-by-line against the SIMPLNX algorithm. Both the binary and the source live in a sibling checkout on the authoring engineer's machine and are not committed to this repository. See `ErodeDilateBadDataFilter-D1` below and the V&V report's Oracle and Bug Fixes sections. + +--- + +## ErodeDilateBadDataFilter-D1 + +| Field | Value | +|---|---| +| **Deviation ID** | `ErodeDilateBadDataFilter-D1` (formerly cited as `-B1`) | +| **Filter UUID** | `7f2f7378-580e-4337-8c04-a29e7883db0b` | +| **Status** | active (SIMPLNX bug **fixed during this V&V cycle** in PR #1687; documented for users of prior SIMPLNX releases) | + +**Symptom:** In SIMPLNX releases prior to PR #1687, the *X Direction*, *Y Direction*, and *Z Direction* parameters had **no effect on the output**. They were parsed correctly from filter args into `ErodeDilateBadDataInputValues` (`ErodeDilateBadDataFilter.cpp:151-153`), but every face neighbor remained eligible — subject only to the geometry boundary — regardless of the flags. Disabling a direction silently produced the all-directions-on result. DREAM3D 6.5.171 honors the flags correctly, so any run with fewer than all three directions enabled diverges from legacy. This is also what produced the previous V&V pass's observation that "all 7 direction-combination fixtures encode byte-identical expected output": the fixture was not under-discriminating, the algorithm was ignoring direction entirely. + +**Root cause:** Bug (SIMPLNX). `adjustValidNeighbors` — the helper intended to mask face-neighbor validity by direction — was defined in `Algorithms/ErodeDilateBadData.cpp` but **never called** from `operator()()`. Confirmed by grepping the translation unit for `XDirOn` / `YDirOn` / `ZDirOn` / `adjustValidNeighbors(`: only the function definition matched, with no call site. Legacy achieves the same gating by ORing the direction flag into each per-face boundary check (`|| !m_ZDirOn` and siblings), so the legacy behavior was never in question. + +*Branch-history note:* an earlier commit on the fix branch, `7e543f701` "Fixed XYZ direction off bug", *had* added a call to `adjustValidNeighbors`, but passed it the face-index-order array and bitwise-ANDed the index constants `0..5` against the direction booleans — which corrupts the iteration order rather than gating validity — and additionally gated `+X` by `zDir` and `+Z` by `xDir` (swapped axes). Since `2&1=0` and `3&1=1`, the iterated face list collapsed to `{-Z,-Y}` for *every* flag combination, including all-on. That call was later removed, leaving direction fully inert — the state this V&V pass found and fixed from scratch. + +**Fix:** `Algorithms/ErodeDilateBadData.cpp:64-80` — `adjustValidNeighbors` now takes the per-voxel `isValidFaceNeighbor` boolean array (the actual validity gate consumed by the vote/mark loop) and ANDs each of the six entries against the correct axis flag, using the named `VoxelNeighbors` constants rather than raw indices. It is called at `:162-163`, immediately after `computeValidFaceNeighbors`, for every bad-data voxel. + +**Verification:** + +- All 28 `k_ExemplarFeatureIds*` / `k_ExemplarData*` constants in `ErodeDilateBadDataTest.cpp` were regenerated from genuine DREAM3D 6.5.171 binary output. They are legacy output, not a hand derivation — a Class 2 oracle. (They were previously byte-identical across all 7 direction combinations for a given operation and iteration count, which is what masked the bug.) +- 28/28 combinations (7 directions × 2 operations × 2 iteration counts) match legacy exactly, on both `FeatureIds` and the `Misc` tracer array — see the V&V report's Oracle section for the run details. +- `(Erode) Expanded` / `(Dilate) Expanded` (28 parameterized runs total, 2078 assertions combined) pass in both in-core and OOC builds. +- Per-direction structural coverage confirmed by temporary hit-count instrumentation — see "Per-direction code-path coverage" below. + +**Affected users:** Anyone who ran `ErodeDilateBadData` on a SIMPLNX build predating PR #1687 with fewer than all three directions enabled. Their output silently matched the all-directions-on result, eroding or dilating across axes they had explicitly disabled. Users who left all three directions on (the default) are **unaffected** — that path was always correct, and the archive-based `(Erode)` regression test passes on pre-fix builds for exactly that reason. + +**Recommendation:** Trust SIMPLNX at or after PR #1687, which agrees with 6.5.171 across all 28 parameter combinations. Results from affected pre-fix builds that used a restricted direction set should be regenerated. + +--- + +## Non-deviations (confirmed correct — do not "fix") + +### Dilate tie-break: last-bad-neighbor-wins is correct, not a bug + +**Investigated and ruled out this pass.** When a good voxel has two or more bad face-neighbors, `neighbors[neighborPoint] = voxelIndex` unconditionally overwrites on each bad neighbor visited, so whichever bad voxel is scanned *last* (highest flat index, in z/y/x order) wins. Since every bad voxel shares `FeatureId == 0`, this choice is invisible to a `FeatureIds`-only comparison — it shows up only in the `Misc` tracer array, which is why it was flagged as unverified in the prior pass and initially suspected as a bug in this one. + +A "first bad neighbor wins" fix (skip the overwrite if `neighbors[neighborPoint]` is already set, plus resetting `neighbors` to `-1` at the top of each iteration) was implemented and *appeared* correct until checked against real DREAM3D 6.5.171 output: `PipelineRunner` running Dilate / XYZ / 1 iteration against the matching legacy input produced `Misc` values matching the **original, unmodified** last-write-wins SIMPLNX behavior, not the "first-wins" rewrite (diverging at 3 of 32 indices: 9, 15, 30). The change was reverted in full — both the per-iteration reset and the overwrite guard. Unconditional last-write-wins, with `neighbors` initialized once before the iteration loop rather than per-iteration, is confirmed legacy-faithful. + +This resolves the prior V&V pass's "second-engineer review pending: erode/dilate tie-break order" item — verified against actual legacy binary output, not source reading alone. + +### Erode tie-break: first-processed-neighbor-wins is correct + +Vote-count-based, using `[-Z,-Y,-X,+X,+Y,+Z]` scan order; a later neighbor's vote must strictly exceed the current maximum to replace the leader. Matches legacy source line-for-line (identical vote and comparison logic) and matches legacy binary output for all 28 tested combinations. Not a deviation. + +### Direction masking of the Erode `featureCount` reset loop + +SIMPLNX reuses the direction-masked `isValidFaceNeighbor` array in the post-vote reset loop; legacy resets over *boundary-valid* neighbors and ignores the direction flags there. Simulating both variants across all 28 combinations gives identical output, and the equivalence is general rather than incidental: the reset set is a superset of the increment set in both variants, so `featureCount` returns to all-zeros after every bad voxel either way. Not a deviation. + +### Deferring the FeatureIds transfer to a second pass + +Legacy interleaves the `FeatureIds` transfer with the other cell arrays in a single pass, mutating `m_FeatureIds` as it goes. SIMPLNX transfers the other arrays first (in parallel) and `FeatureIds` afterward, serially. Equivalent: erode only maps 0→>0 and dilate only >0→0, `neighbors[]` always points at a voxel whose relevant polarity is preserved by the operation, and each index is written at most once per pass — so neither transfer predicate can ever observe a changed value. Not a deviation. + +### Legacy tie-break language says "chosen randomly"; SIMPLNX is deterministic + +The SIMPLNX filter markdown (`docs/ErodeDilateBadDataFilter.md`), carried over from legacy documentation, stated that erode ties are broken "randomly." Both the legacy *source* (`ErodeDilateBadData.cpp`, `Source/Plugins/Processing/ProcessingFilters/`) and the legacy *binary* output are fully deterministic — the same first-processed-wins scan order as SIMPLNX, with no RNG anywhere in the algorithm. "Randomly" is inaccurate documentation language, not a behavioral characteristic; SIMPLNX's determinism is not a deviation. + +**The user-facing doc has been corrected** to describe the actual behavior: the six face neighbors are visited in the fixed order `[-Z,-Y,-X,+X,+Y,+Z]` and a later neighbor must have a strictly greater count to displace the leader, so the earliest tied neighbor in that scan order wins. The same edit documents the two preflight errors (`-14601` no directions enabled, `-14602` zero-length geometry dimension) and the effect of the direction restrictions. + +### Cancel check present in SIMPLNX, absent in legacy + +`operator()` reads `m_ShouldCancel` once per Z-slice inside the iteration loop and returns immediately if set. Legacy's equivalent loop has no cancel check at all. Additive capability with no effect on a run to completion — SIMPLNX is ahead of legacy here, not behind. Not a deviation. (This path is Path 11 in the V&V report's coverage table and is the one enumerated path no test exercises.) + +## Per-direction code-path coverage + +Previously an open question ("could not distinguish correct per-direction gating from a no-op gate" — see prior V&V pass). Resolved this pass by two independent means: + +1. **Behavioral:** expected data now differs by direction combination (see D1 above) and matches legacy per-combination, 28/28. +2. **Structural:** `Algorithms/ErodeDilateBadData.cpp` was temporarily instrumented with per-face-direction (`-Z/-Y/-X/+X/+Y/+Z`) hit counters and run through the full `(Erode) Expanded` + `(Dilate) Expanded` sweep. All six directions were hit, both at the "loop reached" level and at the "vote/mark condition fired" level, in both the vote/mark loop and the Erode-only cleanup loop. Instrumentation was removed afterward and is not shipped. + +## What would need to happen to extend this further + +Everything listed in the prior revision of this file has been done: legacy binary obtained and run, SIMPLNX output compared element-wise against legacy output, and the neighbor-selection and tie-break logic diffed directly against legacy source. The zero-dimensions preflight path (`-14602`), also previously listed here as uncovered, is now reached and asserted by the `No Dimensions` test. + +Remaining follow-up (not gating — see the V&V report's V&V phase row): + +1. **Automate the 28-combination A/B run.** It was a manual, one-time verification (pipeline JSONs through `PipelineRunner`, output diffed with `h5py`), not wired into CI. The expected values are now compiled into the test as constants, so the *comparison* does re-run in CI — but a future change to the fixture would require redoing the legacy run by hand. Consider checking in the legacy `.dream3d` input/output pairs as an exemplar archive with a provenance sidecar, matching the pattern used by `FillBadDataFilter`'s `FillBadData_SmallIN100` test. +2. **Cover the cancel path (Path 11).** Requires cancel-signal injection; no test currently sets `m_ShouldCancel` and asserts early termination. +3. **Add a production-scale Dilate test.** `6_6_erode_dilate_test.tar.gz` already contains an unused `Exemplar Bad Data Dilate` container, so this costs no new test data. See `../provenance/6_6_erode_dilate_test.md`. diff --git a/src/Plugins/SimplnxCore/vv/provenance/6_6_erode_dilate_test.md b/src/Plugins/SimplnxCore/vv/provenance/6_6_erode_dilate_test.md new file mode 100644 index 0000000000..66e9971716 --- /dev/null +++ b/src/Plugins/SimplnxCore/vv/provenance/6_6_erode_dilate_test.md @@ -0,0 +1,97 @@ +# Exemplar Archive Provenance: 6_6_erode_dilate_test.tar.gz + +This sidecar records the origin of the exemplar archive used by the three Erode/Dilate filter unit tests. The archive name and SHA512 must match `download_test_data()` in `src/Plugins/SimplnxCore/test/CMakeLists.txt`. + +--- + +## Archive identity + +| Field | Value | +|---|---| +| **Archive** | `6_6_erode_dilate_test.tar.gz` (68,857,947 bytes) | +| **SHA512** | `5f0773e5d296936effbb2239965f5847e7c18533b0a2c3ec6a1d6a83b03417e5b459cce29808c8e0273613b3b6fa032c675e84926eb35d8da8a6ddc0641a0ef5` | +| **Used by tests** | `SimplnxCore::ErodeDilateBadDataFilter(Erode)`; `SimplnxCore::ErodeDilateMaskFilter(Dilate)` and `(Erode)`; `SimplnxCore::ErodeDilateCoordinationNumberFilter` | +| **Generated by** | ** — the embedded pipeline references `/Users/mjackson/Applications/Data/Output/Reconstruction/SmallIN100.h5ebsd`, so it was authored by Michael Jackson | +| **Generated on** | **; archive mtime on the shared data mirror is 2026-02-19 | +| **Generated at commit** | ** — see the embedded pipeline's `FilterVersion` strings below | + +SHA512 verified 2026-08-14 against both `test/CMakeLists.txt` and the on-disk archive in the shared `DREAM3D_Data/TestFiles` mirror. They match. + +## Contents + +| File | Consumer | +|---|---| +| `6_6_erode_dilate_bad_data.dream3d` + `.xdmf` | `ErodeDilateBadDataFilter` | +| `6_6_erode_dilate_mask.dream3d` + `.xdmf` | `ErodeDilateMaskFilter` | +| `6_6_erode_dilate_coordination_number.dream3d` + `.xdmf` | `ErodeDilateCoordinationNumberFilter` | + +Any regeneration must account for all three consumers, not just `ErodeDilateBadData`. + +`6_6_erode_dilate_bad_data.dream3d` is in the legacy (`DataContainers`) layout, `FileVersion` 7.0, and holds three data containers on the same 189×201×20 `ImageGeom` (759,780 cells): + +| Container | Role | +|---|---| +| `Input Data` | Filter input. `EBSD Scan Data`: `Confidence Index` float32, `EulerAngles` float32×3, `FeatureIds` int32, `Image Quality` float32, `Mask` uint8, `Phases` int32. Plus `Grain Data/Active` uint8 (1104 tuples) and `Phase Data` (`CrystalStructures`, `LatticeConstants`, `MaterialName`). | +| `Exemplar Bad Data Erode` | Expected output for Erode, all directions on, 2 iterations. Same arrays plus `IPFColor` uint8×3. | +| `Exemplar Bad Data Dilate` | Expected output for Dilate, all directions on, 2 iterations. Same arrays plus `IPFColor` uint8×3. | + +## How it was generated + +The archive was produced by a legacy DREAM3D pipeline that is **embedded in the `.dream3d` file itself** (`/Pipeline/Pipeline`, `Pipeline Version` 2) — so the provenance is self-documenting even though no external `.d3dpipeline` was preserved. The 18-step pipeline reads the first slice of Small IN100 from `SmallIN100.h5ebsd`, performs a standard reconstruction, then branches twice to produce the two exemplars: + +| Step | Filter | +|---|---| +| 00 | `PipelineAnnotation` | +| 01 | `ReadH5Ebsd` → `Input Data` | +| 02–09 | `MultiThresholdObjects`, `ConvertOrientations`, `AlignSectionsMisorientation`, `IdentifySample`, `AlignSectionsFeatureCentroid`, `BadDataNeighborOrientationCheck`, `NeighborOrientationCorrelation`, `EBSDSegmentFeatures` | +| 10 | `RemoveArrays` | +| 11–13 | `CopyDataContainer` → `Exemplar Bad Data Erode`, `ErodeDilateBadData`, `GenerateIPFColors` | +| 14–16 | `CopyDataContainer` → `Exemplar Bad Data Dilate`, `ErodeDilateBadData`, `GenerateIPFColors` | +| 17 | `DataContainerWriter` | + +The `PipelineAnnotation` step records that the pipeline was run against a build where `EBSDSegmentFeatures` had `RandomizeFeatureIds` forced to `false` internally, so that `FeatureIds` are reproducible for unit testing, and that only the first slice of Small IN100 was used with minimal processing. + +The two `ErodeDilateBadData` steps are the **legacy SIMPL filter**, `Filter_Uuid` `{3adfe077-c3c9-5cd0-ad74-cf5f8ff3d254}`, with these parameters: + +| Step | `Direction` | `NumIterations` | `XDirOn` / `YDirOn` / `ZDirOn` | `IgnoredDataArrayPaths` | +|---|---|---|---|---| +| 12 (Erode container) | 1 (Erode) | 2 | 1 / 1 / 1 | `[]` | +| 15 (Dilate container) | 0 (Dilate) | 2 | 1 / 1 / 1 | `[]` | + +These match the arguments the `(Erode)` unit test passes to the SIMPLNX filter, so the comparison is apples-to-apples. + +## Canonical oracle output + +| DataPath | Source of expected values | +|---|---| +| `Exemplar Bad Data Erode/EBSD Scan Data/*` | **Class 2** — legacy SIMPL `ErodeDilateBadData`, Erode, 2 iterations, all directions on (pipeline step 12) | +| `Exemplar Bad Data Dilate/EBSD Scan Data/*` | **Class 2** — legacy SIMPL `ErodeDilateBadData`, Dilate, 2 iterations, all directions on (pipeline step 15) | +| `Input Data/EBSD Scan Data/*` | Not an oracle — filter input | + +`ErodeDilateBadDataTest.cpp` currently consumes only the Erode exemplar. The Dilate exemplar is present in the archive and unused; wiring up a `(Dilate)` production-scale test would close that gap at no data cost. + +## Oracle provenance detail + +### Class 2 — Reference implementation + +- **Reference:** legacy SIMPL `ErodeDilateBadData` filter, UUID `{3adfe077-c3c9-5cd0-ad74-cf5f8ff3d254}` +- **Exact version:** the embedded pipeline records `FilterVersion` `6.6.338` for the two `ErodeDilateBadData` steps and for the surrounding EBSD reconstruction filters. This is **later than the 6.5.171 V&V baseline** — see the caveat below. +- **Runner:** legacy DREAM3D `PipelineRunner` (`DataContainerWriter` at step 17) +- **Pipeline file:** not preserved externally; embedded at `/Pipeline/Pipeline` inside `6_6_erode_dilate_bad_data.dream3d` +- **Input dataset:** first slice of Small IN100 (`SmallIN100.h5ebsd`), not synthetic + +**Baseline caveat.** The archive's `6_6_` prefix and the `6.6.338` filter version mean this exemplar comes from a legacy build *later* than the 6.5.171 baseline this V&V cycle compares against. That does not weaken it as an oracle — it is still genuine legacy output from an independent implementation, and the algorithm did not change between those legacy versions (confirmed by the line-by-line source diff recorded in the V&V report's Algorithm Relationship section). It does mean the archive alone is not a 6.5.171 comparison. The 6.5.171 comparison of record is the separate 28-combination `PipelineRunner` A/B run documented in the V&V report's Oracle section. + +**Direction-gating caveat.** Both exemplars were generated with all three directions enabled, so this archive cannot discriminate direction gating — a SIMPLNX build with `ErodeDilateBadDataFilter-D1` still present passes the `(Erode)` test. That is precisely the gap the 28-combination inline sweep exists to close. Do not treat this archive as sufficient coverage on its own. + +## Second-engineer oracle review + +- **Reviewer:** *pending* — required before the V&V report Status moves to COMPLETE +- **Date:** *pending* +- **Review focus when performed:** the two caveats above (later-than-baseline legacy version; all-directions-on coverage limit), and whether a `(Dilate)` production-scale test should be added from the unused exemplar already in the archive. + +## Regenerated to fix a circular-oracle situation? + +No. This archive was **not** regenerated from SIMPLNX output at any point — the embedded pipeline proves the exemplars were written by the legacy SIMPL filter, not by SIMPLNX. The archive is retained unchanged. + +The circular-oracle concern raised during V&V applied to a different artifact: the 28 inline expected-value constants in `ErodeDilateBadDataTest.cpp`, which an earlier revision of the report described as hand-traced while they were in fact byte-identical across all 7 direction combinations (the signature of `ErodeDilateBadDataFilter-D1`). Those constants were regenerated from the 6.5.171 binary and are now a genuine Class 2 oracle. See the V&V report's Oracle section.