perf(arrow/array): compare list values by valid runs

[fallintoplace]

What does this PR do?

• compares List and LargeList child values once per contiguous valid parent run
• checks every parent list length before combining its child range
• compares FixedSizeList child values once per valid parent run
• avoids creating two child array slices for every valid parent value

ListView and LargeListView are not included because their child ranges can be non-contiguous.

Why?

List equality currently creates two temporary child arrays and recursively calls Equal for every valid parent value.

For 65,536 all-valid lists, that creates about 131,000 temporary child arrays. A valid-run comparison checks the parent lengths and compares one contiguous child range instead.

This follows the same approach as Arrow C++ list equality.

Benchmarks

Apple M1 Pro, 65,536 equal parent values:

case	before	after	change	allocations before	allocations after
List, size 1, all valid	15.6 ms	0.82 ms	-95%	262,145	5
List, size 4, all valid	16.5 ms	2.39 ms	-86%	262,145	5
List, size 16, all valid	22.3 ms	8.50 ms	-62%	262,145	5
List, size 16, all valid	23.0 ms	8.65 ms	-62%	262,145	5
LargeList, size 16, all valid	22.3 ms	8.47 ms	-62%	262,145	5
FixedSizeList, size 16, all valid	22.5 ms	8.50 ms	-62%	262,145	5
List, size 16, clustered 10% null	20.1 ms	7.90 ms	-61%	235,933	9

Alternating valid and null parents stays around 11.2 ms in both versions. This is the most fragmented case, where each valid run contains one parent value.

go test ./arrow/array -run '^$' -bench '^BenchmarkListEqual$' -benchmem -benchtime=200ms -count=3

Tests

• checks equal and different child values
• checks different parent list lengths with the same flattened child values
• checks that null parent payloads are ignored
• checks sliced parents with different left and right offsets
• covers List, LargeList, and FixedSizeList
• ran go test ./arrow/...
• ran go test -race ./arrow/array
• ran go vet ./arrow/array

[zeroshade]

The valid-run optimization looks good overall and CI is green. I found one behavior regression inline that should be fixed before merge.

---

This review was drafted by an AI-assisted tool and confirmed by an Apache Arrow Go maintainer. The finding below is an observation, not a blocking review; an Apache Arrow Go maintainer — a real person — will take the next look at the PR.

See the Apache Arrow Go contributing guide.

[zeroshade]

IsNull and IsValid treat both nil and a zero-length validity buffer as all-valid, but VisitSetBitRuns special-cases only nil. Consequently, a nonempty list created through NewData with a non-nil empty validity buffer now panics here when it is the left operand, while reversing the operands returns true. The previous implementation returned true in both directions.

Although Validate rejects this array shape, this is still a regression in Equal behavior for data constructible through the exported API. Please normalize len(validBits) == 0 to the all-valid path, apply the same handling in arrayEqualFixedSizeList, and add regression coverage for both operand orders.