On-device greedy sampling (-Dllama.deviceSample): GPU argmax, keep logits device-side (~500x less D2H)#134

mikepapadim wants to merge 1 commit into

beehive-lab:mainbeehive-lab/GPULlama3.java:mainfrom

mikepapadim:feat/on-device-samplingmikepapadim/GPULlama3.javaF:feat/on-device-samplingCopy head branch name to clipboard