Skip to content

add: unbinned detector calibration, unit normalization, and tilt (3/4) - #138

Draft
pecomyint wants to merge 1 commit into
rsm/02-naxis-issue132from
rsm/03-detector-physics
Draft

add: unbinned detector calibration, unit normalization, and tilt (3/4)#138
pecomyint wants to merge 1 commit into
rsm/02-naxis-issue132from
rsm/03-detector-physics

Conversation

@pecomyint

@pecomyint pecomyint commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Stack 3 of 4. Base rsm/02-naxis-issue132 — retarget to main as the stack merges.

Draft: this is the beam-physics gate. The axis-0 mapping and the tilt/detrot sign conventions cannot be settled without a known reflection. Everything else is verified against xrayutilities directly.

Two silent assumptions removed

DETECTOR_SETUP.UNITS was in every profile and never read, so millimetres were assumed regardless of what the beamline declared. Energy was blindly × 1000, correct only for a keV source. Both produced a plausible-looking, wrong Q with no error.

New dashpva/utils/units.py is one table for length, energy and angle; internal geometry is always eV, mm, degrees. Unknown units raise rather than defaulting — a mis-declared unit is a calibration error, and guessing would reintroduce exactly what this removes. Per-field DISTANCE_UNITS / SIZE_UNITS / PIXEL_SIZE_UNITS override the section default. ENERGY_UNITS defaults to keV so existing profiles keep their scale.

Units are validated at parse time but converted only when the DetectorModel is built. Converting during normalization would leave the stored value and stored unit disagreeing, so a second pass would convert again — caught by the idempotence test.

The binning bug

pixel_width = SIZE / frame_shape silently absorbed binning into the pixel size: a 2×-binned scan produced a Q scale wrong by a factor of two, with no error raised.

center_channel, shape and pixel_width are now explicitly the unbinned full-frame calibration. ROI and binning go to xrayutilities as roi/Nav, which applies them itself — verified in QConversion._get_detparam_area, which does cch/nav, pwidth*nav, and maps an unbinned ROI into binned channels. Pre-shifting the centre here would double-apply that. The legacy SIZE/frame_shape derivation is now permitted only for full-frame unbinned data and rejected by name otherwise.

ROI is half-open in unbinned channels and its span must divide exactly by the binning — xrayutilities would otherwise ceil it and hand back a frame larger than the detector produced. require_frame_shape() rejects a calibration that disagrees with the data instead of yielding a plausible, wrongly-scaled volume.

New degrees of freedom

DETROT / TILT / TILTAZIMUTH, and FRAME_AXIS_ORDER making the detector axis-0 convention explicit and configurable — a transposed volume becomes a profile edit rather than a source change. Q is transposed to the acquired layout so intensity, mask and Q index identically.

detector_model_from_setup() is the single boundary where declared units become canonical; the offline converter and both live paths go through it, and a setup with none of the new keys reproduces the previous model exactly.

Verification

test_detector_physics.py — every Q assertion checked against a hand-built xu.HXRD, not a stored value, so it fails if we stop agreeing with xrayutilities rather than merely changing. Unit equivalence (keV/eV, cm/mm/µm give identical Q), ROI+binning parity at atol=0, exact-divisibility rejection, frame-axis transposition, tilt/detrot parity, and sampleor='det' continuity as detrot → 0 (detrot appends a beam-axis rotation as the innermost detector axis; 'det' ignores beam-parallel rotations, so the limit must be smooth). I also mutation-checked that tilt and detrot genuinely move Q, so the parity tests are not vacuous.

Calibration literals persist to HDF5 through a type-preserving writer — the plain string writer would have reintroduced the repr bug fixed in 687e943.

Blocked on beam

  • Detector axis-0 mapping against a known Bragg peak
  • Tilt / detrot sign conventions vs the beamline's own calibration

@pecomyint

Copy link
Copy Markdown
Collaborator Author

@Osayi-ANL when you get a chance — this one is still unreviewed and it's the beam-physics gate for the trip.

It's the PR whose correctness can't be settled from CI: the detector axis-0 mapping and the tilt/detrot sign conventions need a known reflection. A wrong axis-0 produces a transposed volume that still looks like a plausible diffraction pattern, so it needs a second pair of eyes on the reasoning before we rely on it at the beamline.

Everything else in it is checked against a hand-built xu.HXRD rather than stored values — unit equivalence, ROI+binning parity at atol=0, and sampleor='det' continuity as detrot → 0.

Rebased onto the fixed rsm/02, so git pull first.

Replaces two silent assumptions in the angle-to-Q path with declared,
validated calibration, and adds the detector degrees of freedom rsMap3D has
and DashPVA did not.

Units (new dashpva.utils.units):
- One conversion table for length, energy and angle; internal geometry is
  always eV, mm and degrees. Unknown units raise instead of defaulting,
  because a mis-declared unit is a calibration error and guessing would
  reintroduce the bug this removes.
- DETECTOR_SETUP.UNITS was previously present in every profile and never read,
  so millimetres were assumed regardless. It is now honored, with per-field
  DISTANCE_UNITS / SIZE_UNITS / PIXEL_SIZE_UNITS overrides.
- The blind `energy * 1000` in hpc_rsm_consumer and area_det_viewer is gone;
  both read ENERGY_UNITS and default to keV so existing profiles keep scale.
- Units are validated at parse time but converted only when the DetectorModel
  is built. Converting during normalization would leave the stored value and
  stored unit disagreeing, so a second pass would convert again.

Detector calibration:
- center_channel, shape and pixel_width are now explicitly the UNBINNED
  FULL-FRAME calibration. ROI and BINNING are handed to xrayutilities as
  roi/Nav, which applies them itself in _get_detparam_area. Pre-shifting the
  centre would double-apply that correction.
- The legacy `SIZE / frame_shape` pixel width did exactly that: it absorbed
  binning into the pixel size, so a 2x-binned scan produced a Q scale wrong by
  a factor of two with no error. It is now permitted only for full-frame,
  unbinned data and rejected with a named error otherwise.
- ROI is half-open in unbinned channels and its span must divide exactly by
  the binning; xrayutilities would otherwise ceil it and report a frame larger
  than the detector produced.
- DETROT / TILT / TILTAZIMUTH are passed through to init_area.
- FRAME_AXIS_ORDER makes the detector axis-0 convention explicit and
  configurable. Q is transposed to the acquired layout so intensity, mask and
  Q share one indexing convention.
- require_frame_shape() rejects a calibration that disagrees with the data
  instead of producing a plausible, wrongly-scaled volume.

detector_model_from_setup() is the single boundary where declared units become
canonical; the offline converter and both live paths now go through it, and a
setup with none of the new keys reproduces the previous model exactly.

Calibration literals are persisted to HDF5 with a type-preserving writer so
PIXEL_SIZE/ROI/BINNING survive as numbers -- the plain string writer would have
reintroduced the repr bug fixed in 687e943. The HKL Setup static-geometry
fields are now editable, with JSON parse errors reported as such.

Adds test_detector_physics.py: every Q assertion is checked against a
hand-built xu.HXRD rather than a stored value. Covers unit equivalence
(keV/eV, cm/mm/um), ROI+binning parity, exact-divisibility rejection,
frame-axis transposition, tilt/detrot parity, sampleor='det' continuity as
detrot approaches zero, numeric HDF5 round trip, and legacy preservation.

Beam-gated: the axis-0 mapping and the tilt/detrot sign conventions still need
confirming against a known reflection.
@pecomyint
pecomyint force-pushed the rsm/03-detector-physics branch from 5cf7012 to d575f4c Compare August 14, 2026 20:12
@pecomyint

Copy link
Copy Markdown
Collaborator Author

Nudging — this one's still unreviewed and it's the beam-physics gate for the whole stack (unbinned detector calibration, unit normalization, tilt/detrot). #136 is approved, #137 and #139 have fixes in for your latest comments and are re-requested. Would appreciate a look here when you get a chance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant