Skip to content

Overflow error when writing anonymized FIF with BIDS guideline #14236

Description

@raphbrd

Description of the problem

Here is an issue I encountered while anonymizing preprocessed data. BIDS specification recommends choosing a measurement date before 1925 when anonymizing data. However, the FIF format stores only 32-bit timestamps. It means that one cannot go back more than ~68 years.
It is not an issue for info['meas_date'] (apparently not written as a timestamp?), but it is for all timestamps in info['proc_history'].

Possible solution

We could set the timestamps for all block_id in the info['proc_history'] to DATE_NONE by default. But I am not sure if one ever needs this value. I did not find any place in the MNE codebase where the secs and usecs fields of block_id are used.

For the Maxwell filter, mne.preprocessing.maxwell._update_sss_info already sets this field to DATE_NONE. So there is no point in shifting the proc_history again with an identical number of days as the file_id and the meas_date (DATE_NOTE is set to 1970-1-1).

The current function _check_dates checks for overflow, but only on the meas_id and file_id, not on proc_history (which is incorrect most of the time).

So, maybe mne._fiff.meas_info.anonymize_info should set all proc_history timestamps to DATE_NONE ? This would avoid any overflow when writing the FIF file. What do you think?

Steps to reproduce

import os
import mne

sample_data_folder = mne.datasets.sample.data_path(download=False)

fine_cal_file = os.path.join(sample_data_folder, "SSS", "sss_cal_mgh.dat")
crosstalk_file = os.path.join(sample_data_folder, "SSS", "ct_sparse_mgh.fif")

sample_data_raw_file = os.path.join(
    sample_data_folder, "MEG", "sample", "sample_audvis_raw.fif"
)
raw = mne.io.read_raw_fif(sample_data_raw_file, verbose=False)
raw.crop(tmax=60)

# add stuff to processing history
raw_sss = mne.preprocessing.maxwell_filter(
    raw, cross_talk=crosstalk_file, calibration=fine_cal_file, verbose=True
)
# copying the info object to ensure it is writable 
# error occurs for daysback > 24820 or so
raw_sss.info = mne.io.anonymize_info(raw_sss.info.copy(), daysback=36000)
raw_sss.save("tmp_raw.fif", overwrite=True)

Link to data

No response

Expected results

Readable .fif file.

Actual results

Traceback (most recent call last):
  File "/Users/raphaelbordas/neurospin_local/2026_MNE_sprint/mne-python/mne/io/base.py", line 3022, in write
    _write_raw_metadata(
  File "/Users/raphaelbordas/neurospin_local/2026_MNE_sprint/mne-python/mne/io/base.py", line 3215, in _write_raw_metadata
    write_meas_info(fid, info, data_type=data_type, reset_range=reset_range)
  File "/Users/raphaelbordas/neurospin_local/2026_MNE_sprint/mne-python/mne/_fiff/meas_info.py", line 3252, in write_meas_info
    _write_proc_history(fid, info)
  File "/Users/raphaelbordas/neurospin_local/2026_MNE_sprint/mne-python/mne/_fiff/proc_history.py", line 142, in _write_proc_history
    writer(fid, id_, record[key])
  File "/Users/raphaelbordas/neurospin_local/2026_MNE_sprint/mne-python/mne/_fiff/write.py", line 260, in write_id
    arr = np.array(
          ^^^^^^^^^
OverflowError: Python integer -3110397853 out of bounds for int32

Additional information

MNE: 1.12.1 (latest release)

Could be related to #8128 (in that case, the file_id timestamp does not match the meas_date). For some reason, the overflow error does not occur for the timestamp fields of file_id.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions