add: unbinned detector calibration, unit normalization, and tilt (3/4) - #138
Draft
pecomyint wants to merge 1 commit into
Draft
add: unbinned detector calibration, unit normalization, and tilt (3/4)#138pecomyint wants to merge 1 commit into
pecomyint wants to merge 1 commit into
Conversation
pecomyint
force-pushed
the
rsm/03-detector-physics
branch
2 times, most recently
from
August 12, 2026 22:11
6c8c969 to
5cf7012
Compare
Collaborator
Author
|
@Osayi-ANL when you get a chance — this one is still unreviewed and it's the beam-physics gate for the trip. It's the PR whose correctness can't be settled from CI: the detector axis-0 mapping and the tilt/detrot sign conventions need a known reflection. A wrong axis-0 produces a transposed volume that still looks like a plausible diffraction pattern, so it needs a second pair of eyes on the reasoning before we rely on it at the beamline. Everything else in it is checked against a hand-built Rebased onto the fixed |
Replaces two silent assumptions in the angle-to-Q path with declared, validated calibration, and adds the detector degrees of freedom rsMap3D has and DashPVA did not. Units (new dashpva.utils.units): - One conversion table for length, energy and angle; internal geometry is always eV, mm and degrees. Unknown units raise instead of defaulting, because a mis-declared unit is a calibration error and guessing would reintroduce the bug this removes. - DETECTOR_SETUP.UNITS was previously present in every profile and never read, so millimetres were assumed regardless. It is now honored, with per-field DISTANCE_UNITS / SIZE_UNITS / PIXEL_SIZE_UNITS overrides. - The blind `energy * 1000` in hpc_rsm_consumer and area_det_viewer is gone; both read ENERGY_UNITS and default to keV so existing profiles keep scale. - Units are validated at parse time but converted only when the DetectorModel is built. Converting during normalization would leave the stored value and stored unit disagreeing, so a second pass would convert again. Detector calibration: - center_channel, shape and pixel_width are now explicitly the UNBINNED FULL-FRAME calibration. ROI and BINNING are handed to xrayutilities as roi/Nav, which applies them itself in _get_detparam_area. Pre-shifting the centre would double-apply that correction. - The legacy `SIZE / frame_shape` pixel width did exactly that: it absorbed binning into the pixel size, so a 2x-binned scan produced a Q scale wrong by a factor of two with no error. It is now permitted only for full-frame, unbinned data and rejected with a named error otherwise. - ROI is half-open in unbinned channels and its span must divide exactly by the binning; xrayutilities would otherwise ceil it and report a frame larger than the detector produced. - DETROT / TILT / TILTAZIMUTH are passed through to init_area. - FRAME_AXIS_ORDER makes the detector axis-0 convention explicit and configurable. Q is transposed to the acquired layout so intensity, mask and Q share one indexing convention. - require_frame_shape() rejects a calibration that disagrees with the data instead of producing a plausible, wrongly-scaled volume. detector_model_from_setup() is the single boundary where declared units become canonical; the offline converter and both live paths now go through it, and a setup with none of the new keys reproduces the previous model exactly. Calibration literals are persisted to HDF5 with a type-preserving writer so PIXEL_SIZE/ROI/BINNING survive as numbers -- the plain string writer would have reintroduced the repr bug fixed in 687e943. The HKL Setup static-geometry fields are now editable, with JSON parse errors reported as such. Adds test_detector_physics.py: every Q assertion is checked against a hand-built xu.HXRD rather than a stored value. Covers unit equivalence (keV/eV, cm/mm/um), ROI+binning parity, exact-divisibility rejection, frame-axis transposition, tilt/detrot parity, sampleor='det' continuity as detrot approaches zero, numeric HDF5 round trip, and legacy preservation. Beam-gated: the axis-0 mapping and the tilt/detrot sign conventions still need confirming against a known reflection.
pecomyint
force-pushed
the
rsm/03-detector-physics
branch
from
August 14, 2026 20:12
5cf7012 to
d575f4c
Compare
Collaborator
Author
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack 3 of 4. Base
rsm/02-naxis-issue132— retarget tomainas the stack merges.Draft: this is the beam-physics gate. The axis-0 mapping and the tilt/detrot sign conventions cannot be settled without a known reflection. Everything else is verified against xrayutilities directly.
Two silent assumptions removed
DETECTOR_SETUP.UNITSwas in every profile and never read, so millimetres were assumed regardless of what the beamline declared. Energy was blindly× 1000, correct only for a keV source. Both produced a plausible-looking, wrong Q with no error.New
dashpva/utils/units.pyis one table for length, energy and angle; internal geometry is always eV, mm, degrees. Unknown units raise rather than defaulting — a mis-declared unit is a calibration error, and guessing would reintroduce exactly what this removes. Per-fieldDISTANCE_UNITS/SIZE_UNITS/PIXEL_SIZE_UNITSoverride the section default.ENERGY_UNITSdefaults to keV so existing profiles keep their scale.Units are validated at parse time but converted only when the
DetectorModelis built. Converting during normalization would leave the stored value and stored unit disagreeing, so a second pass would convert again — caught by the idempotence test.The binning bug
pixel_width = SIZE / frame_shapesilently absorbed binning into the pixel size: a 2×-binned scan produced a Q scale wrong by a factor of two, with no error raised.center_channel,shapeandpixel_widthare now explicitly the unbinned full-frame calibration. ROI and binning go to xrayutilities asroi/Nav, which applies them itself — verified inQConversion._get_detparam_area, which doescch/nav,pwidth*nav, and maps an unbinned ROI into binned channels. Pre-shifting the centre here would double-apply that. The legacySIZE/frame_shapederivation is now permitted only for full-frame unbinned data and rejected by name otherwise.ROI is half-open in unbinned channels and its span must divide exactly by the binning — xrayutilities would otherwise
ceilit and hand back a frame larger than the detector produced.require_frame_shape()rejects a calibration that disagrees with the data instead of yielding a plausible, wrongly-scaled volume.New degrees of freedom
DETROT/TILT/TILTAZIMUTH, andFRAME_AXIS_ORDERmaking the detector axis-0 convention explicit and configurable — a transposed volume becomes a profile edit rather than a source change. Q is transposed to the acquired layout so intensity, mask and Q index identically.detector_model_from_setup()is the single boundary where declared units become canonical; the offline converter and both live paths go through it, and a setup with none of the new keys reproduces the previous model exactly.Verification
test_detector_physics.py— every Q assertion checked against a hand-builtxu.HXRD, not a stored value, so it fails if we stop agreeing with xrayutilities rather than merely changing. Unit equivalence (keV/eV, cm/mm/µm give identical Q), ROI+binning parity atatol=0, exact-divisibility rejection, frame-axis transposition, tilt/detrot parity, andsampleor='det'continuity asdetrot → 0(detrot appends a beam-axis rotation as the innermost detector axis;'det'ignores beam-parallel rotations, so the limit must be smooth). I also mutation-checked that tilt and detrot genuinely move Q, so the parity tests are not vacuous.Calibration literals persist to HDF5 through a type-preserving writer — the plain string writer would have reintroduced the repr bug fixed in 687e943.
Blocked on beam