Webpoda: Why packets are missing?

[laspsandoval]

Summary

Recently the following packets were not uploaded to s3 via webpoda. We need to investigate why that happened.

imap_glows_l0_raw_20260430-repoint00234_v001.pkts
imap_glows_l0_raw_20260501-repoint00235_v001.pkts

IDX – no gap in science products
COD, HIT, MAG, SWP, SWE – no L0 5/1
GLX – no L0 4/30-5/1
HIX, ILO, ULX – no L0 4/29-5/1 (did not check before 4/29 so it may be more)

[tmplummer]

I think that I can explain this. From Gina's email:

There was a large LASP network maintenance activity / outage that occurred on Wednesday, 04/29 that caused delays in data delivery.

Since we pull L0 data on a cron and look for data that was ingested since the last cron run, I suspect that after the network issue was resolved, the ingested data had an older GRT than our L0 downloader looked for. To confirm, we would need to look at the GRT of the packets that got missed.

[laspsandoval]

I think that I can explain this. From Gina's email:

There was a large LASP network maintenance activity / outage that occurred on Wednesday, 04/29 that caused delays in data delivery.

Since we pull L0 data on a cron and look for data that was ingested since the last cron run, I suspect that after the network issue was resolved, the ingested data had an older GRT than our L0 downloader looked for. To confirm, we would need to look at the GRT of the packets that got missed.

Yeah. That makes sense. This seems like it could cause issues in the future. For example, what if we deploy production code from sds-data-manager and miss the cron job in the process. I think I have a good fix, though. Working on it right now.