add: arbitrary-axis HKL chain and profile-driven IOC RSM editor (2/4, closes #132) - #137
add: arbitrary-axis HKL chain and profile-driven IOC RSM editor (2/4, closes #132)#137pecomyint wants to merge 5 commits into
Conversation
#132) Removes the fixed four-sample/two-detector circle limit from every stage of the RSM chain and replaces the hardcoded 6-ID-B axis table in the IOC RSM Parameter tool with a profile-backed, geometry-agnostic editor. Chain (no fixed limits, no PV-name sniffing): - hkl_axes.resolved_axis_groups discovers numbered SAMPLE/DETECTOR_CIRCLE_AXIS_N groups per role, sorted by integer suffix so _10 follows _9, with the legacy MU/ETA/CHI/PHI and NU/DELTA layout retained as a per-role fallback. - hdf5_writer, rsm_converter, hpc_rsm_consumer, hpc_metadata_consumer, area_det_viewer, hdf5_loader and metadata_converter now dispatch on semantic configuration keys (DIRECTION_AXIS, POSITION) instead of matching PV-name suffixes, so a circle whose PV ends in .RBV resolves correctly. - NeXus transformation links are generated per axis from the stable RECORD_NAME rather than the beamline-specific sample_phi/chi/eta/mu table, and detector circles now get links too. - Axis metadata is preflighted before the output file is created, so a missing direction or position fails by name instead of silently dropping a circle. Editor (#132): - IOC_RSM_PARAMETER becomes the canonical geometry section: ordered SAMPLE_AXES/DETECTOR_AXES arrays, LABEL split from a stable EPICS-safe RECORD_NAME, per-axis DIRECTION and ANGLE_UNITS, ENERGY_UNITS, SAMPLE_ORIENTATION and SCHEMA_VERSION. - Axis add, remove, rename and reorder; any circle count including zero for a role when the sample orientation makes the geometry valid. - Effective [HKL] is generated in memory from the canonical section; the raw hand-authored [HKL] on disk is never rewritten, and unknown groups survive. - One staged Apply & Save: validate, show the first-adoption diff, CAS-save, read back, then activate the IOC from the exact saved snapshot. A revision conflict applies nothing and sends no IOC commands. - Explicit, idempotent import of the legacy ~/.config JSON. - IOC_PREFIX is the single prefix source; generated channels are fully expanded. Adds test_naxis_data_chain, test_hkl_config_semantics and test_rsm_parameter_ioc covering 0/3/8-axis geometries, _10 ordering, fifth-axis HDF5 round trip, kappa vectors, cross-role RECORD_NAME uniqueness, CAS conflict behavior and legacy migration idempotence.
Ordered, role-split views of the effective HKL circle groups, rebuilt on reload. Call sites that only need "the sample circles, in order" no longer have to know the group-naming or numeric-sort rules. Carried over from @Osayi-ANL's independent #132 work on origin/issue/132, which exposed the same convenience. The implementation here derives the views through resolved_axis_groups -- the same resolution rsm_geometry uses to build the Q conversion -- so the convenience view can never disagree with the geometry the physics actually runs on.
There was a problem hiding this comment.
For issue 137 on a previously existing profile, I try to reseed, so add the IOC_RSM_PARAMETER onto the profile. It adds it to the profile, but when I attempt to open the HKL Setup I get this error below.
ValueError: IOC_RSM_PARAMETER.SAMPLE_AXES must be a list of axis tables
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/beams0/OODIASEIGIEHON/DashPVA/src/dashpva/consumers/ioc_rsm_parameter.py", line 26, in
import dashpva.settings as app_settings
File "/home/beams0/OODIASEIGIEHON/DashPVA/src/dashpva/settings.py", line 533, in
reload()
File "/home/beams0/OODIASEIGIEHON/DashPVA/src/dashpva/settings.py", line 325, in reload
cfg = resolve_profile_config(raw_cfg)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams0/OODIASEIGIEHON/DashPVA/src/dashpva/utils/config/resolver.py", line 115, in resolve_profile_config
for index, axis in enumerate(_canonical_axes(parameters, key), start=1):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/beams0/OODIASEIGIEHON/DashPVA/src/dashpva/utils/config/resolver.py", line 69, in _canonical_axes
raise ValueError(f"IOC_RSM_PARAMETER.{key} must be a list of axis tables")
ValueError: IOC_RSM_PARAMETER.SAMPLE_AXES must be a list of axis tables
…gh the config tree
… paths Follow-up to the review fixes on #137. Empty tables were still corrupted. The previous commit taught the config tree to round-trip lists, but a table with no children is indistinguishable from a leaf, so its blank display text was read back as ''. That turned `[HKL]` -- empty by design once IOC_RSM_PARAMETER generates it -- into a string, and because the same commit added pre-save validation the result was Apply & Save failing outright with "Refusing to save an unloadable profile: HKL must be a table" on the very layout this stack ships. Group items now carry a marker so an empty table extracts as {} rather than falling into the leaf path. The list round-trip also worked only by accident. PyQt stores a dict as a key-sorted QVariantMap, so a list of axis tables came back with its fields reordered and the "text still matches str(original)" guard never matched; recovery happened via the ast.literal_eval fallback instead, and every save silently rewrote the axis tables with alphabetised keys. Values now travel in an opaque wrapper Qt passes through untouched, and edits are detected by comparing against the text actually rendered. The round-trip is now an exact identity, key order included. Removes two redundant configuration paths. AGENTS.md is explicit that the database is the config source and TOML is the import/export format; both of these were a third way: - HKL Setup's "Import legacy JSON..." button, along with migrate_legacy_json and the ~/.config path constant. The Config tab's Import TOML Config and Export Config to File already round-trip the whole profile, including IOC_RSM_PARAMETER. - The --snapshot temp file. The IOC child was handed the entire profile as a serialized JSON file; it now reads the database through the ConfigSource path it already had as a fallback. Activation only happens after a successful compare-and-swap save, so the database already is the approved snapshot -- the file added a format and a cleanup path for nothing. The stdin/stdout JSON pipe to the IOC child stays: it predates this stack and carries live PV readings, not configuration. Adds test_workflow_config_tree.py (25 tests) driving the real QTreeWidget, since the mechanism under test is how PyQt stores item data. Asserts the shipped sample_config round-trips identically and still resolves -- Osayi's crash as a regression test -- plus the same for a pre-canonical profile with a populated [HKL] and no IOC_RSM_PARAMETER, empty tables staying tables, every list-valued key keeping type and value, axis field order surviving, and an edited list recovering structurally.
|
Thanks @Osayi-ANL — reproduced exactly, and it's ours, not something pre-existing you tripped over. It was broader than reseed. Fixed across two commits:
New 480 tests pass on the stack tip, The stack is also rebased so #138/#139 show clean diffs again — Worth a |
QPushButton("Apply & Save") left a single unescaped '&' before a space,
which Qt's mnemonic parser consumes and assigns to the space -- some
styles render that as a stray underscore glyph instead of an invisible
underline. Escape it as "Apply && Save".
The Reload/Retry/Apply & Save control row also lived inside the same
QScrollArea as the axis tables, so it scrolled out of view on longer
profiles. Move it into the window's own layout, below the scroll area.
Both reported by Osayi-ANL on #139.
|
Traced this to the exact commit: the traceback matches Both were fixed the same day, ~20–60 min after this review:
I reproduced your exact scenario against current tip (legacy profile → reseed → adds Current tip is |
2 participants
Stack 2 of 4. Base
rsm/01-config-geometry— retarget tomainafter PR 1 merges. Simulator only, no beam time.Closes #132.
The problem
Making the IOC editor generic is necessary but not sufficient. Five other places capped the geometry at 4 sample + 2 detector circles, and every one failed silently — wrong physics, no error:
hdf5_writer.py:202rsm_converter.py:200range(1,5)/range(1,3)on readhpc_rsm_consumer.py:197pv_name.endswith('Position')area_det_viewer.py:1889settings.py:78-81sample_phi/chi/eta/mu, no detector links at allA beamline whose PV ends in
.RBVlost that circle. A kappa goniometer was unrepresentable.What this changes
Chain.
resolved_axis_groupsdiscovers numbered groups per role, sorted by integer suffix so_10follows_9, with the legacyMU/ETA/CHI/PHIlayout kept as a per-role fallback. Dispatch is on semantic config keys (DIRECTION_AXIS,POSITION), never PV-name shape. NeXus transformation links are generated per axis from the stableRECORD_NAME, and detector circles finally get links too. Axis metadata is preflighted before the output file is created, so a missing direction fails by name instead of dropping a circle.Editor. Ordered
SAMPLE_AXES/DETECTOR_AXESarrays;LABELsplit from an EPICS-safeRECORD_NAMEso renaming an axis in the GUI doesn't rename its PV; per-axisDIRECTIONandANGLE_UNITS;ENERGY_UNITS;SAMPLE_ORIENTATION;SCHEMA_VERSION. Add/remove/rename/reorder, any circle count including zero for a role when the orientation makes that valid. Sample axes accept[xyzk][+-], so kappa goniometers work.One staged
Apply & Save: validate → show the first-adoption diff → CAS-save → read back → restart the IOC, which re-reads the profile from the database. A revision conflict applies nothing and sends no IOC commands.Per AGENTS.md the database is the config source and TOML is the import/export format, so this PR keeps exactly those two: the Config tab's Import TOML Config / Export Config to File already round-trip the whole profile including
IOC_RSM_PARAMETER. There is no separate JSON import, and the IOC child is no longer handed a serialized snapshot file.Verification
Simulator IOC; 0/3/8-axis geometries;
_10ordering; fifth-axis HDF5 round trip; kappa vectors; cross-roleRECORD_NAMEuniqueness;.RBV-suffixed sources; CAS conflict behavior; old profile and old file compatibility. Full suite green,ruffclean.