Skip to content

remove all bottlenecks in creating instrList with multiple sortIDs#44

Open
paciorek wants to merge 4 commits into
masterfrom
faster-multi-sortID
Open

remove all bottlenecks in creating instrList with multiple sortIDs#44
paciorek wants to merge 4 commits into
masterfrom
faster-multi-sortID

Conversation

@paciorek

Copy link
Copy Markdown
Contributor

This gives huge speedups when we have multiple sortID values in a calcRange and split into individual instructions. Formerly we were creating multiple calcRanges and then multiple instr_nClass objects.

This avoids initializing all those R6 class objects. Instead the calcRange methods splits the range into a list of multiple R lists where each R list is essentially the same form as an instr_nClass (i.e., the same named fields).

Before it took 40 ms to make an instr_nClass object and 0.5 ms to make a calcRangeClass object. The time to make a simple R list is negligible by comparison (I didn't microbenchmark that). The result is that for 10000 elements (y[i] <- y[i+1]), it used to take 500 seconds to make the instruction list and now takes 0.03. So a million such elements now takes 3 seconds.

For handling the resulting R list of "R instr", for the moment I made it an S3 class "Rlist_Rinstr". We could consider making an R6 class for the overall list and an R6 class for the inner lists. At the least if we do the former, we can give it an isCompiled method and that will fit into the existing checking in calc_op and simulate. Using S3 doesn't fit within our standard style.

This feels a bit like working around the system we devised but the working around is primarily within makeInstrList, and the speedup is huge, so seems ok.

@paciorek

Copy link
Copy Markdown
Contributor Author

Also we'll want to discuss whether anything in the setup code processing in nimble2 needs adjustment given that makeInstrList now produces an R list of lists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant