Add portable HighwayHash to hwy/contrib/hash (#3246)

[tvost2]

Resolves #3246.

Re-implements HighwayHash (https://arxiv.org/abs/1612.06257) with Highway ops
so a single source returns the frozen 64/128/256-bit golden values on every
target, rather than the reference repo's separate hand-written SSE4.1 / AVX2 /
NEON / VSX versions.

Approach

The 256-bit state {v0, v1, mul0, mul1} (four u64 lanes) is mixed in 128-bit
halves, exactly like the reference HHStateSSE41. That keeps everything in
Vec128, so it also works on 128-bit-only targets (and EMU128). Op mapping:

reference	Highway
_mm_shuffle_epi32(v, 2,3,0,1) (swap u64 halves)	RotateRight<32>
_mm_mul_epu32	MulEven on the u32 view
_mm_shuffle_epi8 ZipperMerge	TableLookupBytes with a fixed 16-byte index
variable Rotate32By (length padding)	RotateLeftSame
128-bit <<1/<<2 in the x^128+x^2+x reduction	ShiftRight + ShiftLeftBytes<8>

Message bytes are little-endian; big-endian targets ReverseLaneBytes on load.
The 1..31-byte remainder builds the padded packet with the frozen length-padding
layout (the & 16 split and the "unordered" 3-byte pack) in scalar code, then
feeds it through the vector Update, so no masked loads / lane inserts are
needed.

Public API: HighwayHash64/128/256(key, bytes, size).

Not yet specialized for wider vectors — a 256-bit-native path (one Vec256 per
state vector on AVX2/AVX3/SVE256+) is a natural follow-up.

Tests

highwayhash_test.cc hashes every length in [0, 64] (message = bytes
0..n-1, key {0x0706..0100, ...}) against the frozen golden tables copied
from google/highwayhash for all three output widths, plus multi-packet inputs
and a one-bit-change check. Passes on AVX2 / SSE4 / SSSE3 / SSE2 / EMU128
locally.

Wired into CMake, Bazel BUILD and hwy_tests.bzl (new :highwayhash
target). clang-format clean.

[jan-wassenberg]

Nice work, thanks for adding. Some minor updates:

[jan-wassenberg]

Yes, a 256-bit specialization would be great. FYI we are planning on soon adding a 256-bit vector, at least if !HWY_HAVE_SCALABLE. That can already be relied upon if HWY_MIN_BYTES >= 32, and we can emulate this very similarly to the HWY_WASM_EMU256 target, but it seems difficult for SVE/RVV.

[tvost2]

Good to know — I'll leave the 128-bit-halves version as the baseline and can follow up with a HWY_MIN_BYTES >= 32 / !HWY_HAVE_SCALABLE path once the 256-bit vector lands.

[jan-wassenberg]

HWY_RCAST_ALIGNED(const uint64_t*, packet) is preferred to placate the compiler.

[tvost2]

Switched UpdatePacket to load bytes directly (BitCast(d, LoadU(d8, packet))), so there's no uint64_t* cast at all — the whole-packet path is fed arbitrarily-aligned bytes + i from the caller, so I avoided asserting alignment there. 32b1552.

[jan-wassenberg]

Oh right, that's a better fix.

[jan-wassenberg]

Just noticed we have two build failures. One is werror:
highwayhash_test.cc:301:61: error: implicit conversion changes signedness: 'size_t' (aka 'unsigned long') to 'difference_type' (aka 'long') [-Werror,-Wsign-conversion]
301 | std::vector<uint8_t> data2(data.begin(), data.begin() + n);

The other is that the implementation must be wrapped in #if HWY_TARGET != HWY_SCALAR, because that deprecated target does not support some of the required ops.

[tvost2]

I will make the code pass the tests so we can merge it, no comments at the code right?

[jan-wassenberg]

Yes, code LGTM, only remaining concern is the failing tests/actions.

[jan-wassenberg]

Updates LGTM. We still have issues with the RVV build and I've just realized why, sorry I didn't see this before.
With SVE and RVV, compilers do not allow putting vectors (which are sizeless) as class members.

What we can do is instead store the state as uint64_t[], and Load/Store during the update. (We'll also require HWY_ALIGN on the class then.) Often the compiler is able to optimize out most of the excess load/store.

Also, Vec128 is undocumented - an implementation detail that's often available, but not on RVV. What is always available is Full128, and we can use Vec<Full128<T>> or VFromD<Full128<T>>.

[tvost2]

CI is green now (23 checks, 0 failures) at 574eb69. The multi-target fallout from the first push is resolved: SCALAR + the scalable targets (RVV/SVE, whose vectors can't be struct members) fall back to a HWY_DASSERT stub with the same guard Vec2/Vec3 use; NEON needed State to be an aggregate so the target-specific Vec128 default ctor is never called from an unattributed implicit one; and a couple of -Wsign-conversion nits in the test. Ready for another look whenever you have a minute.

[tvost2]

At e8e764f every substantive check is green — x86 (all std versions), NEON/armv7, ppc64/ppc64le/s390x (incl. the big-endian runtime path), riscv64 (clang + gcc), loongarch, bazel, and Meson … on Noble AArch64 which builds SVE2_128/SVE_256/SVE2/SVE/NEON.

The two remaining red X's — Build and test Clang-22 on AArch64 and Build and test GCC-16 on AArch64 — are not the PR: both jobs log The runner has received a shutdown signal … The operation was canceled ~34 min in, and the Meson AArch64 job (same target set) passed on this SHA. Could you re-run those two when you get a chance? (I don't have re-run rights.)

Summary of what changed since your review: SCALAR + the scalable targets (RVV/SVE, whose vectors can't be struct fields) fall back to a HWY_DASSERT stub behind the same guard Vec2/Vec3 use; State is a plain aggregate so the target-specific Vec128 default ctor is never called from an unattributed implicit one; golden tables live inside that guard (clang -Wunused-const-variable on the stub build); plus the Reverse2 / LoadU bytes / ZeroBytes review nits.

[tvost2]

Update on the one red check — Build and test GCC-16 on AArch64 — at 0a0384f its Build step was cancelled after 79 min, Test skipped. That job is failing/cancelling on master too: it's red on 2d132e9 (the commit this branch is based on) and on fea4774, 4806e3c, e582e1d, 9c5f163 — it only occasionally passes, and when it does it takes 65-80 min. On this PR the equivalent AArch64 coverage is green: Build and test Clang-22 on AArch64 and Meson … on Noble AArch64 (both build SVE2_128/SVE_256/SVE2/SVE/NEON), plus armv7 clang+gcc. So I believe every check that reflects this PR's code is passing.

[jan-wassenberg]

Nice :) I agree the tests are now fine.

It seems like a pity to exclude SVE and RVV entirely - those could work, if we just refactor the state class to store uint64_t[] arrays instead of V64 vectors, and add Load/Store. Are you worried about perf implications? That would be understandable, seems like we could either disprove that via benchmark, or keep a second codepath using the uint64_t[] plus the original vector code.

[tvost2]

Done — SVE and RVV are enabled now.

detail::State stores HWY_ALIGN uint64_t half[8][2] (the eight 16-byte
halves) instead of eight V64 members; each round loads it into vectors,
mixes, and stores it back. The only remaining exclusion is HWY_SCALAR
(1 lane, no 16-byte block).

No perf regression — the compiler keeps the array in registers (SROA), so
the loads/stores fold away. A/B on the same buffer (1 MiB, hashed 2000× each
length), clang 22 -O2:

target	vector members	uint64_t[]
SSSE3	2.61 GB/s	2.62 GB/s
AVX2	2.93 GB/s	3.07 GB/s

(the spread between runs is ~3%, so these are equal.) objdump of the
non-inlined kernel shows zero stack vector moves in either version on both
targets — the state lives entirely in xmm/ymm.

highwayhash_test still passes on AVX3/AVX2/SSE4/SSSE3/SSE2/EMU128 locally;
the golden tables and TestGolden / TestLongInputs now also run on the
scalable targets (previously stubbed out). I don't have SVE/RVV hardware, so
those get their first real run in CI here.

The single unified codepath means no second implementation to keep in sync.

[jan-wassenberg]

Nice, glad to hear the compiler optimizes out the loads/stores successfully, thanks for updating.