perf(take): avoid bounds checks and speed up take on List<T> by Rich-T-kid · Pull Request #10812 · apache/arrow-rs

Rich-T-kid · 2026-08-24T04:44:15Z

Which issue does this PR close?

works towards Reduce bound checks in take kernels #8879.

Rationale for this change

take on List<T> and LargeList<T> routes every selected row through MutableArrayData::try_extend, which pays virtual dispatch via stored function pointers & null-bitmap bookkeeping on every call, regardless of whether the child array actually needs any of that machinery. For primitive children (no nulls, no nesting), this overhead dominates the actual data movement cost.

What changes are included in this PR?

Instead of MutableArrayData::try_extend per row, the fast path:

Computes bytes_per_value from the buffer length once upfront
Copies each selected row's raw bytes with MutableBuffer::extend_from_slice (a direct memcpy)
Builds the output child ArrayData directly from the resulting buffer

Are these changes tested?

existing test test_take_list,test_take_list_with_value_nulls,test_take_list_with_nulls cover the changes in this PR

Are there any user-facing changes?

no

Rich-T-kid · 2026-08-24T04:46:10Z

run benchmark take_kernels

env:
BENCH_FILTER: take list

Rich-T-kid · 2026-08-24T04:46:26Z

run benchmark take_kernels

env:
BENCH_FILTER: take list

adriangbot · 2026-08-24T04:48:36Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5390853337-1897-qmc49 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing rich-T-kid/optimize-take-list (c6b1574) to cd7c6b8 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench take_kernels
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-24T04:49:23Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5390855052-1898-nmkmg 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing rich-T-kid/optimize-take-list (c6b1574) to cd7c6b8 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench take_kernels
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-24T04:52:54Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing rich-T-kid/optimize-take-list (c6b1574) to cd7c6b8 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   rich-T-kid_optimize-take-list
-----                                              ----                                   -----------------------------
take list i32 1024                                 1.84      9.3±0.03µs        ? ?/sec    1.00      5.1±0.06µs        ? ?/sec
take list i32 512                                  1.79      4.9±0.02µs        ? ?/sec    1.00      2.8±0.02µs        ? ?/sec
take list i32 null indices 1024                    1.23     10.3±0.11µs        ? ?/sec    1.00      8.4±0.04µs        ? ?/sec
take list i32 null values 1024                     1.43      6.4±0.01µs        ? ?/sec    1.00      4.5±0.02µs        ? ?/sec
take list i32 null values null indices 1024        1.00      7.0±0.07µs        ? ?/sec    1.02      7.1±0.05µs        ? ?/sec
take listview i32 1024                             1.01   1054.9±3.15ns        ? ?/sec    1.00   1048.4±4.35ns        ? ?/sec
take listview i32 512                              1.01    604.2±1.36ns        ? ?/sec    1.00    601.0±4.47ns        ? ?/sec
take listview i32 null indices 1024                1.00   1609.0±1.88ns        ? ?/sec    1.09   1751.9±4.16ns        ? ?/sec
take listview i32 null values 1024                 1.01   1972.6±1.28ns        ? ?/sec    1.00   1957.4±3.55ns        ? ?/sec
take listview i32 null values null indices 1024    1.00      2.4±0.00µs        ? ?/sec    1.03      2.5±0.00µs        ? ?/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	100.0s
Peak memory	15.3 MiB
Avg memory	13.3 MiB
CPU user	96.9s
CPU sys	0.0s
Peak spill	0 B

branch

Metric	Value
Wall time	100.0s
Peak memory	11.1 MiB
Avg memory	9.4 MiB
CPU user	98.1s
CPU sys	0.0s
Peak spill	0 B

File an issue against this benchmark runner

adriangbot · 2026-08-24T04:53:04Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing rich-T-kid/optimize-take-list (c6b1574) to cd7c6b8 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   rich-T-kid_optimize-take-list
-----                                              ----                                   -----------------------------
take list i32 1024                                 1.84      9.3±0.02µs        ? ?/sec    1.00      5.1±0.06µs        ? ?/sec
take list i32 512                                  1.79      4.9±0.02µs        ? ?/sec    1.00      2.8±0.02µs        ? ?/sec
take list i32 null indices 1024                    1.24     10.3±0.12µs        ? ?/sec    1.00      8.4±0.05µs        ? ?/sec
take list i32 null values 1024                     1.43      6.4±0.01µs        ? ?/sec    1.00      4.5±0.03µs        ? ?/sec
take list i32 null values null indices 1024        1.00      6.9±0.06µs        ? ?/sec    1.03      7.1±0.04µs        ? ?/sec
take listview i32 1024                             1.01   1059.3±1.83ns        ? ?/sec    1.00   1051.2±2.53ns        ? ?/sec
take listview i32 512                              1.01    605.4±1.51ns        ? ?/sec    1.00    602.2±2.01ns        ? ?/sec
take listview i32 null indices 1024                1.00   1617.5±5.49ns        ? ?/sec    1.08   1748.4±1.93ns        ? ?/sec
take listview i32 null values 1024                 1.01   1988.3±1.62ns        ? ?/sec    1.00   1959.5±1.86ns        ? ?/sec
take listview i32 null values null indices 1024    1.00      2.4±0.00µs        ? ?/sec    1.04      2.5±0.00µs        ? ?/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	100.0s
Peak memory	15.3 MiB
Avg memory	13.3 MiB
CPU user	96.9s
CPU sys	0.0s
Peak spill	0 B

branch

Metric	Value
Wall time	100.0s
Peak memory	12.9 MiB
Avg memory	9.4 MiB
CPU user	98.1s
CPU sys	0.0s
Peak spill	0 B

File an issue against this benchmark runner

Rich-T-kid · 2026-08-24T04:59:16Z

take listview i32 1024                             1.01   1059.3±1.83ns        ? ?/sec    1.00   1051.2±2.53ns        ? ?/sec
take listview i32 512                              1.01    605.4±1.51ns        ? ?/sec    1.00    602.2±2.01ns        ? ?/sec
take listview i32 null indices 1024                1.00   1617.5±5.49ns        ? ?/sec    1.08   1748.4±1.93ns        ? ?/sec
take listview i32 null values 1024                 1.01   1988.3±1.62ns        ? ?/sec    1.00   1959.5±1.86ns        ? ?/sec
take listview i32 null values null indices 1024    1.00      2.4±0.00µs        ? ?/sec    1.04      2.5±0.00µs        ? ?/sec

this should be noise since this PR doesn't touch take_fixed_size_list

Rich-T-kid · 2026-08-24T05:00:38Z

run benchmark take_kernels

env:
BENCH_FILTER: take list

Rich-T-kid · 2026-08-24T05:00:45Z

run benchmark take_kernels

env:
BENCH_FILTER: take list

adriangbot · 2026-08-24T05:03:51Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5390937203-1899-pbbpf 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing rich-T-kid/optimize-take-list (1f369de) to 46540d9 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench take_kernels
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-24T05:04:11Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5390937905-1900-7ltdl 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing rich-T-kid/optimize-take-list (1f369de) to 46540d9 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench take_kernels
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-24T05:08:20Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing rich-T-kid/optimize-take-list (1f369de) to 46540d9 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   rich-T-kid_optimize-take-list
-----                                              ----                                   -----------------------------
take list i32 1024                                 1.83      9.3±0.02µs        ? ?/sec    1.00      5.1±0.07µs        ? ?/sec
take list i32 512                                  1.80      4.9±0.08µs        ? ?/sec    1.00      2.7±0.03µs        ? ?/sec
take list i32 null indices 1024                    1.23     10.3±0.10µs        ? ?/sec    1.00      8.4±0.04µs        ? ?/sec
take list i32 null values 1024                     1.43      6.4±0.06µs        ? ?/sec    1.00      4.5±0.02µs        ? ?/sec
take list i32 null values null indices 1024        1.00      7.0±0.09µs        ? ?/sec    1.01      7.1±0.04µs        ? ?/sec
take listview i32 1024                             1.01   1051.9±2.24ns        ? ?/sec    1.00   1045.7±2.22ns        ? ?/sec
take listview i32 512                              1.01    605.0±1.76ns        ? ?/sec    1.00    596.8±1.28ns        ? ?/sec
take listview i32 null indices 1024                1.00   1609.3±1.99ns        ? ?/sec    1.08   1745.7±3.20ns        ? ?/sec
take listview i32 null values 1024                 1.01   1970.1±2.29ns        ? ?/sec    1.00   1948.2±1.30ns        ? ?/sec
take listview i32 null values null indices 1024    1.00      2.4±0.00µs        ? ?/sec    1.02      2.5±0.00µs        ? ?/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	100.0s
Peak memory	15.3 MiB
Avg memory	13.3 MiB
CPU user	96.9s
CPU sys	0.0s
Peak spill	0 B

branch

Metric	Value
Wall time	100.0s
Peak memory	11.0 MiB
Avg memory	9.3 MiB
CPU user	97.1s
CPU sys	0.0s
Peak spill	0 B

File an issue against this benchmark runner

adriangbot · 2026-08-24T05:08:22Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing rich-T-kid/optimize-take-list (1f369de) to 46540d9 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   rich-T-kid_optimize-take-list
-----                                              ----                                   -----------------------------
take list i32 1024                                 1.83      9.3±0.02µs        ? ?/sec    1.00      5.1±0.08µs        ? ?/sec
take list i32 512                                  1.78      4.9±0.02µs        ? ?/sec    1.00      2.8±0.03µs        ? ?/sec
take list i32 null indices 1024                    1.23     10.3±0.11µs        ? ?/sec    1.00      8.4±0.06µs        ? ?/sec
take list i32 null values 1024                     1.42      6.4±0.02µs        ? ?/sec    1.00      4.5±0.03µs        ? ?/sec
take list i32 null values null indices 1024        1.00      7.0±0.07µs        ? ?/sec    1.02      7.1±0.06µs        ? ?/sec
take listview i32 1024                             1.01   1053.1±1.52ns        ? ?/sec    1.00   1045.3±2.21ns        ? ?/sec
take listview i32 512                              1.01    605.0±1.65ns        ? ?/sec    1.00    597.5±1.86ns        ? ?/sec
take listview i32 null indices 1024                1.00   1611.3±6.37ns        ? ?/sec    1.08   1748.1±3.50ns        ? ?/sec
take listview i32 null values 1024                 1.01   1976.4±3.74ns        ? ?/sec    1.00   1952.8±2.35ns        ? ?/sec
take listview i32 null values null indices 1024    1.00      2.5±0.00µs        ? ?/sec    1.02      2.5±0.01µs        ? ?/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	100.0s
Peak memory	15.3 MiB
Avg memory	13.3 MiB
CPU user	96.9s
CPU sys	0.0s
Peak spill	0 B

branch

Metric	Value
Wall time	100.0s
Peak memory	13.1 MiB
Avg memory	9.4 MiB
CPU user	98.1s
CPU sys	0.0s
Peak spill	0 B

File an issue against this benchmark runner

Rich-T-kid · 2026-08-26T15:42:36Z

run benchmark take_kernels

env:
BENCH_FILTER: take list

adriangbot · 2026-08-26T15:44:01Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5427612544-1985-f864f 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing rich-T-kid/optimize-take-list (1f369de) to 46540d9 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench take_kernels
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-26T15:48:30Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing rich-T-kid/optimize-take-list (1f369de) to 46540d9 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   rich-T-kid_optimize-take-list
-----                                              ----                                   -----------------------------
take list i32 1024                                 1.84      9.3±0.03µs        ? ?/sec    1.00      5.1±0.06µs        ? ?/sec
take list i32 512                                  1.78      4.9±0.01µs        ? ?/sec    1.00      2.8±0.02µs        ? ?/sec
take list i32 null indices 1024                    1.24     10.4±0.19µs        ? ?/sec    1.00      8.4±0.04µs        ? ?/sec
take list i32 null values 1024                     1.43      6.4±0.01µs        ? ?/sec    1.00      4.5±0.02µs        ? ?/sec
take list i32 null values null indices 1024        1.00      7.0±0.03µs        ? ?/sec    1.02      7.1±0.05µs        ? ?/sec
take listview i32 1024                             1.01   1068.5±1.07ns        ? ?/sec    1.00   1059.8±4.47ns        ? ?/sec
take listview i32 512                              1.00    608.6±1.03ns        ? ?/sec    1.00    605.8±3.77ns        ? ?/sec
take listview i32 null indices 1024                1.00   1617.3±2.53ns        ? ?/sec    1.08   1751.6±5.98ns        ? ?/sec
take listview i32 null values 1024                 1.01   1993.5±1.89ns        ? ?/sec    1.00   1967.0±3.80ns        ? ?/sec
take listview i32 null values null indices 1024    1.00      2.4±0.00µs        ? ?/sec    1.08      2.5±0.01µs        ? ?/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	100.0s
Peak memory	15.2 MiB
Avg memory	13.2 MiB
CPU user	96.8s
CPU sys	0.0s
Peak spill	0 B

branch

Metric	Value
Wall time	100.0s
Peak memory	11.1 MiB
Avg memory	9.5 MiB
CPU user	98.1s
CPU sys	0.0s
Peak spill	0 B

File an issue against this benchmark runner

Rich-T-kid · 2026-08-26T18:01:14Z

run benchmark take_kernels

env:
BENCH_FILTER: take list

Rich-T-kid · 2026-08-26T18:02:42Z

-/// Copies the selected list entries' child slices into a new child array
-/// via `MutableArrayData`, then reconstructs a list array with new offsets


this isn't true for all cases but I don't think its worth making the comment larger, so i removed it.

adriangbot · 2026-08-26T18:04:21Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5429121885-1991-rrzlf 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing rich-T-kid/optimize-take-list (93e7938) to 0a8fdd5 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench take_kernels
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-26T18:08:38Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing rich-T-kid/optimize-take-list (93e7938) to 0a8fdd5 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   rich-T-kid_optimize-take-list
-----                                              ----                                   -----------------------------
take list i32 1024                                 2.16      9.3±0.02µs        ? ?/sec    1.00      4.3±0.06µs        ? ?/sec
take list i32 512                                  2.12      4.9±0.01µs        ? ?/sec    1.00      2.3±0.03µs        ? ?/sec
take list i32 null indices 1024                    1.24     10.3±0.09µs        ? ?/sec    1.00      8.3±0.10µs        ? ?/sec
take list i32 null values 1024                     1.61      6.4±0.01µs        ? ?/sec    1.00      4.0±0.03µs        ? ?/sec
take list i32 null values null indices 1024        1.00      6.9±0.04µs        ? ?/sec    1.02      7.1±0.03µs        ? ?/sec
take listview i32 1024                             1.00    970.0±3.63ns        ? ?/sec    1.00    972.0±0.97ns        ? ?/sec
take listview i32 512                              1.00    613.5±4.14ns        ? ?/sec    1.00    614.0±1.10ns        ? ?/sec
take listview i32 null indices 1024                1.01   1613.1±6.35ns        ? ?/sec    1.00   1603.4±1.31ns        ? ?/sec
take listview i32 null values 1024                 1.01   1985.8±5.35ns        ? ?/sec    1.00   1966.5±1.67ns        ? ?/sec
take listview i32 null values null indices 1024    1.00      2.5±0.01µs        ? ?/sec    1.00      2.4±0.00µs        ? ?/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	100.0s
Peak memory	15.4 MiB
Avg memory	13.2 MiB
CPU user	95.8s
CPU sys	0.0s
Peak spill	0 B

branch

Metric	Value
Wall time	100.0s
Peak memory	15.4 MiB
Avg memory	13.3 MiB
CPU user	95.1s
CPU sys	0.0s
Peak spill	0 B

File an issue against this benchmark runner

Rich-T-kid · 2026-08-26T18:13:28Z

run benchmark take_kernels

env:
BENCH_FILTER: take list

adriangbot · 2026-08-26T18:16:48Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5429275942-1992-tdbch 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing rich-T-kid/optimize-take-list (fb2c666) to 46540d9 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench take_kernels
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-26T18:21:06Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing rich-T-kid/optimize-take-list (fb2c666) to 46540d9 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   rich-T-kid_optimize-take-list
-----                                              ----                                   -----------------------------
take list i32 1024                                 1.86      9.3±0.03µs        ? ?/sec    1.00      5.0±0.05µs        ? ?/sec
take list i32 512                                  1.82      4.9±0.02µs        ? ?/sec    1.00      2.7±0.02µs        ? ?/sec
take list i32 null indices 1024                    1.28     10.7±0.10µs        ? ?/sec    1.00      8.4±0.03µs        ? ?/sec
take list i32 null values 1024                     1.44      6.4±0.02µs        ? ?/sec    1.00      4.5±0.03µs        ? ?/sec
take list i32 null values null indices 1024        1.00      7.0±0.03µs        ? ?/sec    1.01      7.1±0.06µs        ? ?/sec
take listview i32 1024                             1.01    974.5±2.73ns        ? ?/sec    1.00    966.3±3.52ns        ? ?/sec
take listview i32 512                              1.01    608.6±1.23ns        ? ?/sec    1.00    603.4±2.97ns        ? ?/sec
take listview i32 null indices 1024                1.04   1687.1±1.27ns        ? ?/sec    1.00   1620.0±3.99ns        ? ?/sec
take listview i32 null values 1024                 1.00   1975.6±1.90ns        ? ?/sec    1.02      2.0±0.00µs        ? ?/sec
take listview i32 null values null indices 1024    1.00      2.4±0.00µs        ? ?/sec    1.07      2.5±0.01µs        ? ?/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	100.0s
Peak memory	15.2 MiB
Avg memory	13.1 MiB
CPU user	96.3s
CPU sys	0.0s
Peak spill	0 B

branch

Metric	Value
Wall time	100.0s
Peak memory	11.0 MiB
Avg memory	9.0 MiB
CPU user	97.1s
CPU sys	0.0s
Peak spill	0 B

File an issue against this benchmark runner

Rich-T-kid · 2026-08-27T03:36:23Z

run benchmark take_kernels

adriangbot · 2026-08-27T03:38:30Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5434043797-2008-5lx59 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing rich-T-kid/optimize-take-list (1828c9b) to 7e65400 (merge-base) diff

Run configuration

run benchmark take_kernels

BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench take_kernels
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-27T03:58:55Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing rich-T-kid/optimize-take-list (1828c9b) to 7e65400 (merge-base) diff

Run configuration

run benchmark take_kernels

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                                                       main                                   rich-T-kid_optimize-take-list
-----                                                                       ----                                   -----------------------------
take bool 1024                                                              1.01   1043.2±6.31ns        ? ?/sec    1.00   1034.5±0.96ns        ? ?/sec
take bool 512                                                               1.00    576.6±1.06ns        ? ?/sec    1.00    576.9±0.60ns        ? ?/sec
take bool null indices 1024                                                 1.00   767.9±27.55ns        ? ?/sec    1.00    768.0±6.00ns        ? ?/sec
take bool null values 1024                                                  1.00      2.0±0.00µs        ? ?/sec    1.01      2.1±0.00µs        ? ?/sec
take bool null values null indices 1024                                     1.00  1448.9±10.30ns        ? ?/sec    1.01   1458.1±5.54ns        ? ?/sec
take check bounds i32 1024                                                  1.02    664.9±2.72ns        ? ?/sec    1.00    653.9±1.01ns        ? ?/sec
take check bounds i32 512                                                   1.00    393.6±7.46ns        ? ?/sec    1.15    453.1±1.51ns        ? ?/sec
take dense union 1024                                                       1.00      7.4±0.01µs        ? ?/sec    1.03      7.6±0.01µs        ? ?/sec
take dense union 512                                                        1.04      5.1±0.01µs        ? ?/sec    1.00      4.9±0.01µs        ? ?/sec
take fixed_size_list<i32>[22] 1024                                          1.00     14.7±0.01µs        ? ?/sec    1.00     14.7±0.01µs        ? ?/sec
take fixed_size_list<i32>[22] null indices 1024                             1.02     33.2±0.05µs        ? ?/sec    1.00     32.7±0.04µs        ? ?/sec
take fixed_size_list<i32>[22] null values 1024                              1.00     20.0±0.17µs        ? ?/sec    1.08     21.6±1.66µs        ? ?/sec
take fixed_size_list<i32>[8] 1024                                           1.00      7.2±0.22µs        ? ?/sec    1.00      7.2±0.19µs        ? ?/sec
take fixed_size_list<i32>[8] null indices 1024                              1.00     15.7±0.02µs        ? ?/sec    1.00     15.7±0.02µs        ? ?/sec
take fixed_size_list<i32>[8] null values 1024                               1.00      8.2±0.05µs        ? ?/sec    1.00      8.2±0.04µs        ? ?/sec
take fsb value len: 12, indices: 1024                                       1.00   1865.8±1.74ns        ? ?/sec    1.00   1870.4±2.23ns        ? ?/sec
take fsb value len: 12, null values, indices: 1024                          1.02      3.0±0.01µs        ? ?/sec    1.00      2.9±0.00µs        ? ?/sec
take fsb value optimized len: 16, indices: 1024                             1.14    678.9±8.53ns        ? ?/sec    1.00    594.9±2.07ns        ? ?/sec
take fsb value optimized len: 16, null values, indices: 1024                1.03   1665.7±1.84ns        ? ?/sec    1.00   1623.7±5.74ns        ? ?/sec
take i32 1024                                                               1.02    524.7±2.00ns        ? ?/sec    1.00    514.6±1.41ns        ? ?/sec
take i32 512                                                                1.00    352.6±1.46ns        ? ?/sec    1.00    352.0±1.51ns        ? ?/sec
take i32 null indices 1024                                                  1.00    855.4±1.86ns        ? ?/sec    1.01    860.3±1.40ns        ? ?/sec
take i32 null values 1024                                                   1.01   1558.2±3.23ns        ? ?/sec    1.00   1543.4±1.41ns        ? ?/sec
take i32 null values null indices 1024                                      1.00   1657.7±3.20ns        ? ?/sec    1.00   1662.4±3.68ns        ? ?/sec
take list i32 1024                                                          1.87      9.4±0.02µs        ? ?/sec    1.00      5.0±0.01µs        ? ?/sec
take list i32 512                                                           1.84      5.0±0.02µs        ? ?/sec    1.00      2.7±0.01µs        ? ?/sec
take list i32 null indices 1024                                             1.22     10.3±0.11µs        ? ?/sec    1.00      8.4±0.10µs        ? ?/sec
take list i32 null values 1024                                              1.45      6.4±0.01µs        ? ?/sec    1.00      4.4±0.03µs        ? ?/sec
take list i32 null values null indices 1024                                 1.01      7.0±0.03µs        ? ?/sec    1.00      6.9±0.04µs        ? ?/sec
take listview i32 1024                                                      1.16   1114.6±1.22ns        ? ?/sec    1.00    961.3±1.55ns        ? ?/sec
take listview i32 512                                                       1.04    620.5±1.07ns        ? ?/sec    1.00    598.3±1.15ns        ? ?/sec
take listview i32 null indices 1024                                         1.00   1640.1±2.38ns        ? ?/sec    1.02   1681.0±1.61ns        ? ?/sec
take listview i32 null values 1024                                          1.01   1993.7±1.30ns        ? ?/sec    1.00   1975.8±1.65ns        ? ?/sec
take listview i32 null values null indices 1024                             1.00      2.4±0.00µs        ? ?/sec    1.01      2.4±0.00µs        ? ?/sec
take map<str, i32> 1024                                                     1.00     34.3±0.05µs        ? ?/sec    1.00     34.2±0.05µs        ? ?/sec
take map<str, i32> 512                                                      1.01     19.6±0.02µs        ? ?/sec    1.00     19.3±0.02µs        ? ?/sec
take map<str, i32> null indices 1024                                        1.00     20.2±0.02µs        ? ?/sec    1.01     20.4±0.03µs        ? ?/sec
take map<str, i32> null values 1024                                         1.00     20.8±0.02µs        ? ?/sec    1.01     20.9±0.03µs        ? ?/sec
take primitive run logical len: 1024, physical len: 512, indices: 1024      1.01     15.5±0.02µs        ? ?/sec    1.00     15.3±0.01µs        ? ?/sec
take sparse union 1024                                                      1.02      7.8±0.02µs        ? ?/sec    1.00      7.6±0.01µs        ? ?/sec
take sparse union 512                                                       1.05      4.9±0.02µs        ? ?/sec    1.00      4.6±0.01µs        ? ?/sec
take str 1024                                                               1.03      8.3±0.10µs        ? ?/sec    1.00      8.0±0.01µs        ? ?/sec
take str 512                                                                1.00      3.7±0.01µs        ? ?/sec    1.05      3.9±0.01µs        ? ?/sec
take str null indices 1024                                                  1.00      4.4±0.07µs        ? ?/sec    1.01      4.4±0.06µs        ? ?/sec
take str null indices 512                                                   1.01      2.1±0.04µs        ? ?/sec    1.00      2.1±0.03µs        ? ?/sec
take str null values 1024                                                   1.02      5.0±0.06µs        ? ?/sec    1.00      4.9±0.05µs        ? ?/sec
take str null values null indices 1024                                      1.03      3.2±0.02µs        ? ?/sec    1.00      3.1±0.02µs        ? ?/sec
take string run logical len: 1024, physical len: 128, indices: 1024         1.00     21.0±0.01µs        ? ?/sec    1.00     21.0±0.01µs        ? ?/sec
take string run logical len: 1024, physical len: 512, indices: 1024         1.00     21.6±0.02µs        ? ?/sec    1.00     21.6±0.02µs        ? ?/sec
take string run logical len: 1024, physical len: 512, null indices: 1024    1.00     16.0±0.02µs        ? ?/sec    1.00     16.0±0.02µs        ? ?/sec
take stringview 1024                                                        1.15    845.5±1.49ns        ? ?/sec    1.00    734.8±1.58ns        ? ?/sec
take stringview 512                                                         1.04    491.3±1.67ns        ? ?/sec    1.00    471.8±0.83ns        ? ?/sec
take stringview null indices 1024                                           1.00    844.0±0.81ns        ? ?/sec    1.08    915.4±1.99ns        ? ?/sec
take stringview null indices 512                                            1.00    499.9±0.53ns        ? ?/sec    1.03    514.5±3.68ns        ? ?/sec
take stringview null values 1024                                            1.11   1838.5±1.63ns        ? ?/sec    1.00   1653.8±1.23ns        ? ?/sec
take stringview null values null indices 1024                               1.00   1633.9±3.33ns        ? ?/sec    1.00   1626.6±2.06ns        ? ?/sec
take_record_batch 5 primitive cols no nulls 1024                            1.00      3.5±0.01µs        ? ?/sec    1.00      3.5±0.01µs        ? ?/sec
take_record_batch 6 string/binary cols no nulls 1024                        1.00     27.1±0.03µs        ? ?/sec    1.01     27.4±0.04µs        ? ?/sec
take_record_batch 7 mixed cols null values 1024                             1.02     18.0±0.12µs        ? ?/sec    1.00     17.6±0.09µs        ? ?/sec
take_record_batch 7 mixed cols null values null indices 1024                1.00     14.1±0.05µs        ? ?/sec    1.01     14.2±0.07µs        ? ?/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	575.1s
Peak memory	18.1 MiB
Avg memory	14.2 MiB
CPU user	572.5s
CPU sys	0.0s
Peak spill	0 B

branch

Metric	Value
Wall time	575.1s
Peak memory	18.0 MiB
Avg memory	14.4 MiB
CPU user	570.7s
CPU sys	0.0s
Peak spill	0 B

File an issue against this benchmark runner

Rich-T-kid · 2026-08-28T00:45:44Z

run benchmark take_kernels

env: 
  BENCH_FILTER: take list

adriangbot · 2026-08-28T00:48:01Z

🤖 Arrow criterion benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5446962996-2030-nn2zt 6.12.85+ #1 SMP Sat Jun 27 09:31:30 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing rich-T-kid/optimize-take-list (51e792e) to 9500647 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench take_kernels
Results will be posted here when complete

File an issue against this benchmark runner

adriangbot · 2026-08-28T00:52:19Z

🤖 Arrow criterion benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing rich-T-kid/optimize-take-list (51e792e) to 9500647 (merge-base) diff

Run configuration

run benchmark take_kernels
env:
  BENCH_FILTER: "take list"

CPU Details (lscpu)

Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Details

group                                              main                                   rich-T-kid_optimize-take-list
-----                                              ----                                   -----------------------------
take list i32 1024                                 1.82      9.3±0.04µs        ? ?/sec    1.00      5.1±0.03µs        ? ?/sec
take list i32 512                                  1.80      4.9±0.02µs        ? ?/sec    1.00      2.7±0.01µs        ? ?/sec
take list i32 null indices 1024                    1.21     10.3±0.13µs        ? ?/sec    1.00      8.5±0.08µs        ? ?/sec
take list i32 null values 1024                     1.42      6.4±0.01µs        ? ?/sec    1.00      4.5±0.04µs        ? ?/sec
take list i32 null values null indices 1024        1.00      6.9±0.03µs        ? ?/sec    1.02      7.0±0.03µs        ? ?/sec
take listview i32 1024                             1.01   1051.9±1.35ns        ? ?/sec    1.00   1046.3±1.85ns        ? ?/sec
take listview i32 512                              1.00    605.4±1.27ns        ? ?/sec    1.01    609.9±2.03ns        ? ?/sec
take listview i32 null indices 1024                1.00   1613.6±4.86ns        ? ?/sec    1.09   1753.0±1.34ns        ? ?/sec
take listview i32 null values 1024                 1.01   1989.6±1.97ns        ? ?/sec    1.00   1962.0±1.20ns        ? ?/sec
take listview i32 null values null indices 1024    1.00      2.4±0.00µs        ? ?/sec    1.05      2.5±0.00µs        ? ?/sec

Resource Usage

base (merge-base)

Metric	Value
Wall time	100.0s
Peak memory	15.4 MiB
Avg memory	13.3 MiB
CPU user	95.9s
CPU sys	0.0s
Peak spill	0 B

branch

Metric	Value
Wall time	100.0s
Peak memory	15.3 MiB
Avg memory	13.6 MiB
CPU user	98.1s
CPU sys	0.0s
Peak spill	0 B

File an issue against this benchmark runner

Rich-T-kid · 2026-08-28T01:03:26Z

I added some extra test & addressed your comments @Jefffrey

- Eliminate per-index bounds checks in the hot paths of primitive take by splitting into checked (TakeOptions::check_bounds) and unchecked paths, then bulk-validating indices in a single pass before entering the copy loop. - Downcast and convert indices once in take_record_batch / take_arrays instead of re-doing it per column. - Fast path in take_list for primitive leaf children (single values buffer, no child nulls): copy raw bytes directly with MutableBuffer::extend_from_slice, skipping MutableArrayData per-row dynamic dispatch and null-bitmap bookkeeping overhead.

The fast path for take on List<T> estimated initial buffer capacity as avg_row_len * indices.len() * bytes_per_value. For the overflow fixture tests (value_len=1_000_000, n~=2148 indices), this computed ~8.6 GB, causing an OOM-kill before checked_add overflow detection could fire. Cap the estimate to OffsetType::Native::MAX_OFFSET * bytes_per_value, since child_len is bounded by that type's max before returning an error.

Jefffrey

should be good once CI is green

Jefffrey · 2026-08-28T07:32:26Z

+    let is_primitive_child = child_data.null_count() == 0 && child_data.data_type().is_primitive();
+
+    if is_primitive_child {
+        let values_buf = &child_data.buffers()[0];
+        let Some(bytes_per_value) = child_data.data_type().primitive_width() else {
+            unreachable!("is_primitive guarantees primitive_width is Some")
+        };


Suggested change

let is_primitive_child = child_data.null_count() == 0 && child_data.data_type().is_primitive();

if is_primitive_child {

let values_buf = &child_data.buffers()[0];

let Some(bytes_per_value) = child_data.data_type().primitive_width() else {

unreachable!("is_primitive guarantees primitive_width is Some")

};

if child_data.null_count() == 0 && let Some(bytes_per_value) = child_data.data_type().primitive_width() {

let values_buf = &child_data.buffers()[0];

Rich-T-kid · 2026-08-28T14:14:00Z

-    /// `value_len` int32 elements, plus the repetition count needed for
-    /// `child_len` (an `i32`) to overflow.
-    fn list_offset_overflow_fixture() -> (ListArray, usize) {
-        let value_len = 1_000_000usize;


this is what was causing the CI to fail. locally it runs fine but GitHub runners have only so much memory.

Jefffrey · 2026-08-29T07:48:48Z

thanks @Rich-T-kid

github-actions Bot added arrow Changes to the arrow crate arrow-select labels Aug 24, 2026

Rich-T-kid force-pushed the rich-T-kid/optimize-take-list branch from 0994752 to c6b1574 Compare August 24, 2026 04:44

Rich-T-kid force-pushed the rich-T-kid/optimize-take-list branch from c6b1574 to 7f566b1 Compare August 24, 2026 04:55

Rich-T-kid mentioned this pull request Aug 24, 2026

perf : optimize take(fixed_sized_list) #10441

Merged

Rich-T-kid commented Aug 26, 2026

View reviewed changes

Rich-T-kid force-pushed the rich-T-kid/optimize-take-list branch from 93e7938 to fb2c666 Compare August 26, 2026 18:12

Rich-T-kid marked this pull request as ready for review August 26, 2026 18:13

Jefffrey added the performance label Aug 27, 2026

Jefffrey reviewed Aug 27, 2026

View reviewed changes

Comment thread arrow-select/src/take.rs Outdated

Comment thread arrow-select/src/take.rs Outdated

Comment thread arrow-select/src/take.rs Outdated

Comment thread arrow-select/src/take.rs Outdated

Comment thread arrow-select/src/take.rs Outdated

Rich-T-kid force-pushed the rich-T-kid/optimize-take-list branch from 1828c9b to fb2c666 Compare August 27, 2026 16:06

Rich-T-kid requested a review from Jefffrey August 27, 2026 16:10

Rich-T-kid force-pushed the rich-T-kid/optimize-take-list branch 2 times, most recently from b13e7cf to 51e792e Compare August 28, 2026 00:44

Jefffrey reviewed Aug 28, 2026

View reviewed changes

Comment thread arrow-select/src/take.rs Outdated

Comment thread arrow-select/src/take.rs Outdated

Rich-T-kid added 5 commits August 28, 2026 02:01

clean up Pr and swap out mutableBuffer for vec<u8>

9e938d7

swap back to mutableBuffer for alignment

58a4bbc

add test to cover overflow case

723f5ef

respond to PR feedback

70bb83f

Rich-T-kid force-pushed the rich-T-kid/optimize-take-list branch from ca376d1 to 70bb83f Compare August 28, 2026 06:01

Rich-T-kid requested a review from Jefffrey August 28, 2026 06:02

Rich-T-kid force-pushed the rich-T-kid/optimize-take-list branch from 5368f8b to ea0ce35 Compare August 28, 2026 06:45

Jefffrey approved these changes Aug 28, 2026

View reviewed changes

avoid 8.5GB test allocation causing OOM, respond to PR reivew

cb0f3fd

Rich-T-kid force-pushed the rich-T-kid/optimize-take-list branch from f6b25d0 to cb0f3fd Compare August 28, 2026 14:09

revert useless pre-alloc cap; OOM fixed by removing the overflow tests

3981d6e

Rich-T-kid commented Aug 28, 2026

View reviewed changes

Jefffrey merged commit 25adb3b into apache:main Aug 29, 2026
34 checks passed

		/// Copies the selected list entries' child slices into a new child array
		/// via `MutableArrayData`, then reconstructs a list array with new offsets

Uh oh!

Conversation

Rich-T-kid commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

Rich-T-kid commented Aug 24, 2026

Uh oh!

Rich-T-kid commented Aug 24, 2026

Uh oh!

adriangbot commented Aug 24, 2026

Uh oh!

adriangbot commented Aug 24, 2026

Uh oh!

adriangbot commented Aug 24, 2026

Uh oh!

adriangbot commented Aug 24, 2026

Uh oh!

Rich-T-kid commented Aug 24, 2026

Uh oh!

Rich-T-kid commented Aug 24, 2026

Uh oh!

Rich-T-kid commented Aug 24, 2026

Uh oh!

adriangbot commented Aug 24, 2026

Uh oh!

adriangbot commented Aug 24, 2026

Uh oh!

adriangbot commented Aug 24, 2026

Uh oh!

adriangbot commented Aug 24, 2026

Uh oh!

Rich-T-kid commented Aug 26, 2026

Uh oh!

adriangbot commented Aug 26, 2026

Uh oh!

adriangbot commented Aug 26, 2026

Uh oh!

Rich-T-kid commented Aug 26, 2026

Uh oh!

Rich-T-kid Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

adriangbot commented Aug 26, 2026

Uh oh!

adriangbot commented Aug 26, 2026

Uh oh!

Rich-T-kid commented Aug 26, 2026

Uh oh!

adriangbot commented Aug 26, 2026

Uh oh!

adriangbot commented Aug 26, 2026

Uh oh!

Rich-T-kid commented Aug 27, 2026

Uh oh!

adriangbot commented Aug 27, 2026

Uh oh!

adriangbot commented Aug 27, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Rich-T-kid commented Aug 28, 2026

Uh oh!

adriangbot commented Aug 28, 2026

Uh oh!

adriangbot commented Aug 28, 2026

Uh oh!

Rich-T-kid commented Aug 28, 2026

Uh oh!

Uh oh!

Uh oh!

Jefffrey left a comment

Choose a reason for hiding this comment

Rich-T-kid commented Aug 24, 2026 •

edited

Loading