Improve desktop mode performance#275
Open
grahamfindlay wants to merge 11 commits into
Open
Conversation
…lder argparse was stores --recording-base-folder as args.recording_base_folder, but run_mainwindow_cli() was trying to access the non-existent args.recording_base_path. That error was caught by the except and only printed a misleading 'check the path or the file format', so --recording silently failed.
…ixes (typo + exception handling)
For 83.5M spikes, reduces each call from ~730 ms to ~4 us aligned. This is important, because it is triggered by every trace view refresh.
It built `np.isin(arange(n_spikes_in_segment), selected)` on every refresh: ~1.3 s at 83.5M spikes, even with nothing selected. `selected_indices` are absolute, so filter them to the segment and index directly.
…sh (align spike dtype; direct-index selected spikes)
`recording.get_times()`` without frame bounds risks loading a whole zarr time vector into memory. We only need to read the visible window.
`visibilityChanged(True)` is already triggered for each front tab on the initial `win.show()`, is triggered for views behind tabs when they are brought to the front (fixed in previous commit), so the eager refresh-all loop was either redundant (for front tabs) or unnecessary (for hidden tabs).
When using e.g. the spikeamplitude view to split select spikes and then split a cluster, `Controller.make_manual_split_if_possible` was doing a slow O(N_selected x N_unit_spikes) loop to get the spike positions within the unit's train and to validate that the selected spikes belong to the unit. `np.searchsorted(spike_inds, indices)` can do both in one pass. On long recordings, this cuts the time it takes to split a cluster from minutes (or even hours) to <1 second.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Several parts:
(1) Align the spike vector
dtypefor fastnp.searchsortedon trace view refreshFor 83.5M spikes, reduces each
np.searchsortedcall (triggered by every trace view refresh) from ~730 ms to ~4 us (essentially free).(2) Avoid scanning the whole segment in
get_selected_spikes_dataIt built
np.isin(arange(n_spikes_in_segment), selected)on every refresh.For 83.5M spikes, was ~1.3s even with nothing selected, now essentially free.
(3) Avoid refreshing views hidden behind a tab (and do refresh them when brought forward)
(4) Avoid materializing a whole zarr time vector in
get_times_chunk, and just get visible times.(5) Remove eager refreshing of all views on startup -- this is now possible thanks to (3).
(6) Fast splitting of clusters from selected spikes (e.g. in spike amplitude view)
This cuts the time it takes to split clusters on long recordings from minutes (or even hours) to <1s.
(7) Fix some preexisting failing tests (from the pre-
SortingAnalyzerera).Not to actually have test coverage (yes, I read the README 😆):
Just to silence the warnings!