Skip to content

perf(zarr-transforms): partition sorted 1-D array maps by chunk#234

Open
selmanozleyen wants to merge 1 commit into
d-v-b:refactor/simplify-indexingfrom
selmanozleyen:perf/sorted-1d-lazy-resolution
Open

perf(zarr-transforms): partition sorted 1-D array maps by chunk#234
selmanozleyen wants to merge 1 commit into
d-v-b:refactor/simplify-indexingfrom
selmanozleyen:perf/sorted-1d-lazy-resolution

Conversation

@selmanozleyen

Copy link
Copy Markdown

See the conversation here: zarr-developers#4170 (comment)

Regression check:

case                    #3906 head   proposed   proposed/head
random, one shard          18.136      18.175       1.0053
descending, one shard      18.352      18.176       0.9898
random, all shards        274.906     275.353       0.9985

The speedup for this case:

runs    elements    transform build    proposed resolution
1024      1.48M          0.74 ms              2.28 ms
4096      5.94M          5.03 ms              9.24 ms
8192     11.9M           9.72 ms             18.00 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants