From e0e7e613306eebaaa6138cd933bf2b7d3412c0d2 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Thu, 23 Jul 2026 12:44:10 +0100 Subject: [PATCH 1/6] header_updates --- .../Datastreaming_udp_packet_formats.md | 97 ++++++++++--------- 1 file changed, 49 insertions(+), 48 deletions(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index 05774c9a0..65031f642 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -12,37 +12,28 @@ trailing zero-padding bytes, but these may be added in future and should be igno The header is 16 words (64 bytes). -### Word 0: marker byte +### Word 0: marker word Always `0xFFFFFFFF`. ### Word 1: header information -- **Bit 0**: End of run header marker (active low) -- **Bit 1**: Veto frame packet header marker (active low) -- **Bit 2**: Sample environment packet header marker (active low) -- **Bit 3**: Not used -- **Bits 4..=31**: Always `0xFFFFFFF` -If all bits are high, this packet is a neutron data packet. +- **Bits 0..7**: Length of header, in 32-bit words (max 255 *too many?*) +- **Bits 8..23**: Header Type (max 65535 *too many?*) +- **Bits 24..31**: Marker - Always `0xFF` *(think it will be useful to keep extra Fs to show start of header but not sure if necessary?)* -### Word 2: Information +### Word 2: Header Information -- **Bits 0..=3**: Reserved for header type -- **Bits 4..=8**: Length of header, in 32-bit words -- **Bits 9..=10**: Not used -- **Bit 11**: Bad frame (not currently used by streaming control board) -- **Bit 12**: Run will continue (not currently used by streaming control board) -- **Bit 13**: No frame sync -- **Bit 14**: Frame Memory Full veto -- **Bit 15**: Frame data overrun -- **Bits 16..=31**: Not used +- **Bit 0..=7**: Header Flags + - **Bit 0**: End of run header marker (active low) + - **Bit 1**: Veto frame packet header marker (active low) + - **Bit 2**: Pause frame packet header marker (active low) + - **Bit 3**: No frame sync (active low) (not implemented) + - **Bit 4..=15**: Reserved for future use +- **Bits 16..=31**: PCB Board Number -### Word 3: Frame number - -- **Bits 0..=31**: frame number as u32 - -### Word 4: GPS timestamp +### Word 3: GPS timestamp - **Bits 0..=3**: seconds (most significant bits; combine with least significant bits from word 5) - **Bits 4..=9**: minutes @@ -50,17 +41,21 @@ If all bits are high, this packet is a neutron data packet. - **Bits 15..=23**: days - **Bits 24..=31**: years (as offset from year 2000) -### Word 5: GPS timestamp +### Word 4: GPS timestamp - **Bits 0..=9**: nanoseconds - **Bits 10..=19**: microseconds - **Bits 20..=29**: milliseconds - **Bits 30..=31**: seconds (least significant bits; combine with most significant bits from word 4) +### Word 5: Frame number + +- **Bits 0..=31**: frame number as u32 + ### Word 6: period number - **Bits 0..=15**: Period number -- **Bits 16..=31**: Not used +- **Bits 16..=31**: Frame repeat number ### Word 7: events in frame @@ -70,51 +65,57 @@ If all bits are high, this packet is a neutron data packet. This is not necessarily the same as the number of events in this UDP message, as the events may be split between multiple UDP messages. This is the *total* number of events in the ISIS frame. -See header word 13 if looking for length of *this* message. +See header word 8 if looking for length of *this* message. ::: -### Word 8: protons-per-pulse +### Word 8: packet length & protons-per-pulse - **Bits 0..=7**: protons-per-pulse in this ISIS frame. -- **Bits 8..=31**: unused +- **Bits 16..27**: number of 32-bit words from the beginning of this header to the start of the next header. To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math}`1.738{\times}10^{-6}`. ### Word 9: vetoes - -- **Bit 0**: FIFO veto -- **Bit 1**: SMP veto -- **Bit 2**: TS2 pulse veto -- **Bit 3**: Wrong pulse veto -- **Bit 4**: Unused -- **Bit 5**: ISIS slow -- **Bits 6..=9**: External vetoes -- **Bits 10..=13**: Fast chopper vetoes -- **Bits 14..=15**: Reserved vetoes -- **Bits 16..=23**: Unused -- **Bits 24..=31**: Frame repeat number +- **Bits 0..=15**: Common vetoes + - **Bit 0**: Data Overflow veto + - **Bit 1**: SMP veto + - **Bit 2**: TS2 pulse veto + - **Bit 3**: Wrong pulse veto *-awaiting clarity what this is* + - **Bit 4**: ISIS slow *-awaiting clarity what this is* + - **Bit 5**: period overflow veto *-awaiting clarity what this is* + - **Bit 6**: run sig veto *-not sure if this will be needed when not using DAE3* + - **Bit 7**: pause veto + - **Bit 8**: Status Packet CRC fault veto + - **Bits 9..=12**: Fast chopper vetoes + - **Bits 13..=15**: Reserved for future use +- **Bits 16..=31**: Instrument Specific Vetoes + - **Bits 16..=19**: External vetoes + - **Bits 20..=31**: Reserved for future use ### Word 10: next frame address - **Bits 0..=31**: Address of the next frame, in bytes. -### Word 11: next frame address +### Word 11: streamed frame number -- **Bits 0..=31**: Address of the next frame, in 64-bit words. +- **Bits 0..=31**: streamed frame number. -### Word 12: streamed frame number +### Word 12: checksum -- **Bits 0..=31**: streamed frame number. +- **Bits 0..=31**: Pre-DDR checksum + +### Word 13: unused -### Word 13: packet length +- **Bits 0..=31**: Reserved for future use -- **Bits 0..=11**: number of 32-bit words from the beginning of this header to the start of the next header. -- **Bits 12..=31**: Unused +:::{note} +Did consider adding information for how long the event data is and how long the time data is to make it easier to slice but think that might come from the packet type anyway +::: ### Word 14: unused - **Bits 0..=31**: Unused -### Word 15: checksum +### Word 15: unused -- **Bits 0..=31**: Pre-DDR checksum +- **Bits 0..=31**: Unused From bf73fb103f86fb9bb21ae6abd4ddc065e4dd148d Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Fri, 24 Jul 2026 10:52:36 +0100 Subject: [PATCH 2/6] Update PCB Board Number description in documentation Clarified PCB Board Number encoding in UDP packet format documentation. --- .../datastreaming/Datastreaming_udp_packet_formats.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index 65031f642..dae116d3f 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -31,7 +31,7 @@ Always `0xFFFFFFFF`. - **Bit 2**: Pause frame packet header marker (active low) - **Bit 3**: No frame sync (active low) (not implemented) - **Bit 4..=15**: Reserved for future use -- **Bits 16..=31**: PCB Board Number +- **Bits 16..=31**: PCB Board Number, encoded as an integer. e.g. for `PC1234M1S`, the board identifier would be `1234`. Ignore any `PC` prefix and any suffix such as `M1S`. ### Word 3: GPS timestamp From 23bb6f3c2784999f7141124866c3192874f8f8df Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Mon, 27 Jul 2026 12:07:26 +0100 Subject: [PATCH 3/6] Update doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md Co-authored-by: Jack Harper --- .../datastreaming/Datastreaming_udp_packet_formats.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index dae116d3f..0e304a845 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -72,6 +72,7 @@ See header word 8 if looking for length of *this* message. - **Bits 0..=7**: protons-per-pulse in this ISIS frame. - **Bits 16..27**: number of 32-bit words from the beginning of this header to the start of the next header. +- **Bits 28..=31**: unused To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math}`1.738{\times}10^{-6}`. From da8d23a41ff345a0a6f948afe0499c4fc9b2dea3 Mon Sep 17 00:00:00 2001 From: DaveTempleman Date: Thu, 30 Jul 2026 12:14:07 +0100 Subject: [PATCH 4/6] Update UDP packet format veto descriptions Have tweaked the order of the vetoes and added a Dwell veto --- .../Datastreaming_udp_packet_formats.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index 0e304a845..f5b5dbc7d 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -83,15 +83,16 @@ To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math - **Bit 2**: TS2 pulse veto - **Bit 3**: Wrong pulse veto *-awaiting clarity what this is* - **Bit 4**: ISIS slow *-awaiting clarity what this is* - - **Bit 5**: period overflow veto *-awaiting clarity what this is* - - **Bit 6**: run sig veto *-not sure if this will be needed when not using DAE3* - - **Bit 7**: pause veto - - **Bit 8**: Status Packet CRC fault veto - - **Bits 9..=12**: Fast chopper vetoes - - **Bits 13..=15**: Reserved for future use + - **Bit 5**: Run signal veto *-not sure if this will be needed when not using DAE3* + - **Bit 6**: Pause veto + - **Bit 7**: Period overflow veto pause veto + - **Bit 8**: Dwell Period veto + - **Bit 9**: Status Packet CRC fault veto + - **Bits 10..=15**: Reserved for future use - **Bits 16..=31**: Instrument Specific Vetoes - **Bits 16..=19**: External vetoes - - **Bits 20..=31**: Reserved for future use + - **Bits 20..=23**: Fast chopper vetoes (Fermi) + - **Bits 24..=31**: Reserved for future use ### Word 10: next frame address From 8681b7980fc71e9c1111103f316f1f0edd300852 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Thu, 30 Jul 2026 13:12:40 +0100 Subject: [PATCH 5/6] Reference common veto bit definitions instead of duplicating --- .../datastreaming/Datastreaming-vetos.md | 16 +++------------- .../Datastreaming_udp_packet_formats.md | 1 + 2 files changed, 4 insertions(+), 13 deletions(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming-vetos.md b/doc/specific_iocs/datastreaming/Datastreaming-vetos.md index ab80f3ece..5a6fd1f1a 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming-vetos.md +++ b/doc/specific_iocs/datastreaming/Datastreaming-vetos.md @@ -20,16 +20,6 @@ As an example, `(1 << 14) & veto_configuration_flags` will be non-zero if veto ` ## Veto bit mask This is the same for: 1) the configured vetoes in the [`vc00`](https://github.com/ISISComputingGroup/streaming-data-types/blob/master/schemas/vc00_veto_configuration.fbs) schema, emitted by `kafka_dae_control` -2) the streamed UDP frame packet's header in word 9, and in turn the `pu00` [active vetoes](https://github.com/ISISComputingGroup/streaming-data-types/blob/master/schemas/pu00_pulse_metadata.fbs#L12) that are sent every frame regardless of whether or not events are being vetoed. - -Bits: -- **Bit 0**: FIFO veto -- **Bit 1**: SMP veto -- **Bit 2**: TS2 pulse veto -- **Bit 3**: Wrong pulse veto -- **Bit 4**: Unused -- **Bit 5**: ISIS slow -- **Bits 6..=9**: External vetoes -- **Bits 10..=13**: Fast chopper vetoes -- **Bits 14..=15**: Reserved vetoes -- **Bits 16..=31**: Unused +2) the streamed UDP frame packet's header, and in turn the `pu00` [active vetoes](https://github.com/ISISComputingGroup/streaming-data-types/blob/master/schemas/pu00_pulse_metadata.fbs#L12) that are sent every frame regardless of whether or not events are being vetoed. + +For definitions of each veto bit, see {ref}`UDP Packet Format veto bit definitions `. diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index f5b5dbc7d..e618c83ae 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -76,6 +76,7 @@ See header word 8 if looking for length of *this* message. To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math}`1.738{\times}10^{-6}`. +{#ds_veto_bit_definitions} ### Word 9: vetoes - **Bits 0..=15**: Common vetoes - **Bit 0**: Data Overflow veto From 03e52f5bd8874dceeb207620f8bc70d9852848ca Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Thu, 30 Jul 2026 13:15:17 +0100 Subject: [PATCH 6/6] remove question from docs --- .../datastreaming/Datastreaming_udp_packet_formats.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md index e618c83ae..11153c723 100644 --- a/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md +++ b/doc/specific_iocs/datastreaming/Datastreaming_udp_packet_formats.md @@ -111,10 +111,6 @@ To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math - **Bits 0..=31**: Reserved for future use -:::{note} -Did consider adding information for how long the event data is and how long the time data is to make it easier to slice but think that might come from the packet type anyway -::: - ### Word 14: unused - **Bits 0..=31**: Unused