fft-gpu-programming-models

Six implementations of the same FFT, benchmarked head to head on NVIDIA P100 and V100, to find out which version has what tradeoff and performance, and to decide when it's a good idea to use which implementation.

#	Implementation	Approach	Effort to write
1	CPU	Single-threaded C reference	—
2	CUDA V1	Global memory, one kernel launch per butterfly stage	High
3	CUDA V2	Shared memory, all stages in a single launch	High
4	CUDA V3	Shared memory + padding to avoid bank conflicts	High
5	OpenACC	parallel loop independent, collapse(2) on the same loops	Low
6	cuFFT	NVIDIA's library, cufftExecZ2Z	Lowest

What the runs showed

• cuFFT wins by 4.5× over the best hand-written kernel. If a vendor library covers what you need, use it.
• OpenACC is ~5× slower than hand-written CUDA — and slower than even the naive global-memory version.
• Shared memory is the whole optimisation, worth 4.6×. Directives won't do it for you.
• Bank-conflict padding and block size changed nothing. The kernel is bandwidth-bound, not occupancy-bound.

Stack: C · CUDA · OpenACC · cuFFT · NVIDIA HPC SDK · SLURM

Algorithm: radix-2 Cooley–Tukey FFT, forward transform, single GPU scaling

Where it ran: the COKA cluster at INFN — P100 and V100 nodes, skyvolta partition for the V100 runs, one GPU per job, exclusive.

Problem size	Value
N	2²³ = 8,388,608 points
Type	double-precision complex (double2), 16 bytes/point
Array	128 MiB
Operation count	5·N·log₂N = 0.96 GFLOP per transform
Timing	device-side; array resident on GPU, host transfer excluded for all six equally

All six are in one binary (src/fft_gpu_bench.cu) and run back to back on the same input, so the comparison is fair.

Conclusion

At N = 2²³ on a COKA V100 (7.8 TFLOP/s fp64 peak):

Implementation	Time	GFLOP/s	% of peak	vs best CUDA	vs CPU
cuFFT	~2 ms	482	6.2%	4.5× faster	~1000×
CUDA V2 / V3 (shared memory)	~9 ms	107	1.4%	baseline	~220×
CUDA V1 (global memory)	~25 ms	39	0.5%	2.8× slower	~80×
OpenACC	~45 ms	21	0.3%	5× slower	~45×
CPU, single-threaded	~2000 ms	0.5	—	—	1×

• Even cuFFT reaches only 6% of fp64 peak
• The transform moves the 128 MiB array once per stage, 23 stages
• So the difference between these six is how well each uses the memory hierarchy, not flop rate

→ This is a memory-hierarchy benchmark wearing an FFT costume.

What OpenACC actually costs you

• Took a fraction of the time to write — a few pragmas on loops that already existed
• The compiler won't move your data into shared memory for you
• That move is the entire optimisation here
• So directives get you a working GPU port and not much else

→ Use OpenACC to get onto the GPU quickly. Don't expect it to close the gap to hand-tuned CUDA.

Where the CUDA speedup comes from

On the COKA P100 at N = 2²⁴:

Version	Time	vs V1
V1 — global memory	33 ms	1×
V2 — shared memory	7.2 ms	4.6×
V3 — shared + padding	7.2 ms	4.6×

• V1 re-reads the whole array from VRAM 24 times, once per stage
• V2 loads a block's data into shared memory once and does every stage there
• V3's padding made no measurable difference at these sizes
• 256 vs 512 vs 1024 threads per block also barely moved the numbers

Across GPU generations

• All six on both cards, same ordering on each
• The V100 has more shared memory and more bandwidth
• So the gap between the shared-memory kernels and the global-memory one gets wider, not narrower

Caveats

Caveat	Detail
Timing methods differ	CUDA and cuFFT paths use cudaEventRecord; the OpenACC path uses clock() around its data region. Not equivalent measurements
Small-N results are an artefact	Below ~N = 2¹⁵ the OpenACC line comes out fastest in the plots, which can't be right for this kernel. Ignore that end
Large-N results hold	There the kernels run long enough that the measurement difference doesn't matter
Fix not done yet	Time all six with CUDA events and re-run
Times read off plots	The raw logs were not retained

Correctness

validate.py checks every implementation against NumPy's FFT. All six agree with the reference to ~1e-14.

Build and run

The benchmark has CUDA kernels, cuFFT and OpenACC in one file, so it needs the NVIDIA HPC SDK. nvcc on its own can't compile the OpenACC parts.

cd src
nvc++ -O3 -acc -gpu=cc70 -cuda -lcufft -lm fft_gpu_bench.cu -o fft_gpu_bench.x
./fft_gpu_bench.x 8388608          # N must be a power of two

CPU reference alone, no GPU toolchain needed:

gcc -O3 -fopenmp -o fft_cpu.x src/fft_cpu.c -lm && ./fft_cpu.x 1048576

On a cluster:

sbatch slurm/bench.sh

• slurm/bench.sh is written for COKA — skyvolta partition, one V100, nvhpc/24.5
• Change the partition and module for anywhere else
• Set -gpu=cc60 for P100, cc70 for V100, cc80 for A100

Layout

src/
  fft_gpu_bench.cu        all six implementations + benchmark harness
  fft_cpu.c               standalone CPU reference
  snapshots/              earlier development stages
    stage1_global_shared.cu
    stage2_padded.cu
benchmarks/               plots
slurm/bench.sh            size sweep, 2^11 to 2^23
validate.py               NumPy check
presentation.pdf          course presentation

Where this came from

Course	P2.2 — GPU Programming, MHPC, ICTP / SISSA Trieste
Written	February–June 2026
Public since	21 June 2026, in prabhkodes/low_level_optimisations
This repo	Same code, better filenames, result written down properly. Source unchanged
Course repository	Belongs to SISSA, private — can't be linked