Set gpu tpb by otbrown · Pull Request #736 · QuEST-Kit/QuEST

otbrown · 2026-04-24T17:16:49Z

Creating a facility for users to runtime set threads per block for tuning the GPU implementation. NOTE: only applies to kernels that are not handled by Thrust, which does its own thing. Resolves #735.

I considered and rejected the idea of creating a symmetric interface for the CPU for users who don't know OMP_NUM_THREADS or omp_set_num_threads() exist, but that's much riskier as the point of truth is external (in the OpenMP runtime).

TODO:

~~Should gpu_getNumThreadsPerBlock return a qindex? Probably~~.
Create a new home for user facing API, as environment doesn't really make sense.
Add a compile time default value -- that way expert maintainers can compile a tuned default into a library which is used on a system.
Query seemingly unused branch at

QuEST/quest/src/gpu/gpu_subroutines.cpp

Line 453 in b7d4a29

if constexpr (NumTargs != -1) {
Add TPB to QuEST GPU environment reporting.
Add tests for new interface.
@JPRichings To check if this is really worthwhile, but please wait a week to tell me if it isn't.
- It is!

…. Yes, it's confusing. Yes, the OpenMP ARB know.

…they would be easy.

otbrown · 2026-04-24T17:20:30Z

Rudimentary testing done with:

#include <cstdio>
#include "quest.h"

int main (void)
{
  const int NQUBITS = 24;
  const int TPB = 32;


  initQuESTEnv();
  reportQuESTEnv();

  std::printf("Initial number of threads per block: %d\n", getQuESTGpuThreadsPerBlock());

  setQuESTGpuThreadsPerBlock(TPB);
  std::printf("New number of threads per block: %d\n", getQuESTGpuThreadsPerBlock());

  Qureg qureg = createForcedQureg(NQUBITS);

  std::printf("Initialising Qureg.\n");
  initPlusState(qureg);
  reportQureg(qureg);

  std::printf("Applying Quantum Fourier Transform.\n");
  applyFullQuantumFourierTransform(qureg, false);
  reportQureg(qureg);

  destroyQureg(qureg);
  finalizeQuESTEnv();

  return 0;
}

JPRichings · 2026-04-24T17:31:22Z

Why would gpu_getNumThreadsPerBlock be a qindex this is not a quantum quantity. uint should be fine ( I am sure there is a recommendation from the cuda api we can match.

TysonRayJones · 2026-04-25T19:26:54Z

Is there an advantage to users having to set this as a runtime hyperparameter? My (mostly undeveloped) belief is we can use occupancy tools (alluded to here) to automate this. I definitely shy from giving users a greater onus to optimise for their settings (like other prolific softwares), which the v4 overhaul was supposed to avoid (via e.g. the autodeployer).

Note too that the kernels so far are very primitive - each thread handles the updating of the minimum possible number of amplitudes (often just one!). I quite like that because it's very readable and simple (great for an open-source scientific project) but is an obvious site for optimisation.

Why would gpu_getNumThreadsPerBlock be a qindex this is not a quantum quantity. uint should be fine ( I am sure there is a recommendation from the cuda api we can match.

It's true that it will never be anywhere as big as the quantities qindex is expected to store (like the number of basis states), but I have already used it in places where I thought an int might be insufficient. Inoffensive either way as uint or qindex imo

JPRichings · 2026-04-25T20:56:31Z

Hi Tyson,

I just noticed the fixed value to 128 and have a feeling that it was large.

I just wanted a handle so I could write a benchmark so we can easily automate performance tuning ourselves.

I have not played with the occupancy tools but I should take a proper look as this might solve this automatically.

My other concern is that there are differences between Nvidia and AMD on optimal sizes due to hardware differences so we might not be able to reply on the occupancy tuning in all cases unless this becomes available on all platforms.

TysonRayJones · 2026-04-25T20:58:12Z


    qindex numThreads = qureg.numAmpsPerNode / powerOf2(qubits.size());
-    qindex numBlocks = getNumBlocks(numThreads);
+    const int NUM_THREADS_PER_BLOCK = gpu_getNumThreadsPerBlock();


If we opt for this, why is NUM_THREADS_PER_BLOCK capitalised like a constant? It's runtime

I agree capitalisation here bad.

It's const in scope 😉 apologies, accidentally following my own style guide there rather than the QuEST one. I'll %s/NUM_THREADS_PER_BLOCK/numThreadsPerBlock/g it.

TysonRayJones · 2026-04-25T21:18:22Z

I just noticed the fixed value to 128 and have a feeling that it was large.

I guess it's very GPU specific! I think 128 was motivated by thinking of CC=3, which has a max active blocks per SM of 16, and a max active threads per SM of 2048. So using 128 threads per block perfectly maximally occupies the SMs (when there are enough amplitudes to admit more than 16 blocks per SM, of course!)

For illustration, the next smallest size is 96 (it must be a multiple of 32, else threads within a warp will be idle), which yields a number of active threads of 16 * 96 = 1536, which wastes 2048 - 1536 = 512 threads per SM!

Of course, newer GPUs support more active blocks per SM (even when the max active threads per SM is unchanged). E.g. CC=8 supports up to 32 active blocks per SM, so we could shrink to 64 threads per block while achieving the same occupancy - but I don't have a great intuition for the effect when we're memory-bandwidth bound.

Certainly seems prudent to consult a CUDA runtime API, if that doesn't hurt our AMD compatibility!

otbrown · 2026-04-27T11:14:08Z

Apologies, probably won't get to look at this again this week, but very happy to set this value programmatically if it can be done!

As it's architecture dependent, we definitely do need a way to adjust it, and ideally both at runtime and compile time. At compile time, so kindly HPC support teams can compile and maintain a tuned version, and at runtime, so they can scan through values without having to recompile in between. I'll have a chat with James abour approaches later this week!

I 100% agree that we don't really want unknowing users messing around with this. I think something like an architecture.h or perftune.h or similar might be the best solution. A set of functionality that we explicitly document is for users who know what they are doing to tune the performance of the library for a specific architecture. It might be this is the only value in there for the time being, but for slingshot-11 reasons we need to add a parameter capping the total in-flight data and this would be a good spot for that too.

TysonRayJones · 2026-04-29T21:54:01Z

Fair enough - you've convinced me! Being able to runtime adjust is of course extremely helping during development of a user-friendlier adaptive system anyhow.

I like the sound of perftune.h - it could also go into debug.h in the interim to there being more performance-tuning specific functionality.

TysonRayJones · 2026-05-03T16:39:50Z

+}
+
+void setQuESTGpuThreadsPerBlock(const int NEW_TPB) {
+    // just rely on the internal function to throw an error if there's no GPU support compiled


TODO: validate this is a factor of 32 (and is positive, etc etc)

Doc to user: HIP warpsize is 64!

Maybe a better alternative: add gpu_isHipCompiled() in gpu_config.cpp, right under gpu_isCuQuantumCompiled(), as:

bool gpu_isHipCompiled() { return (bool) (COMPILE_CUDA && defined(__HIP__)); }

Then we can validate explicitly that when GPU-accelerated and we're on HIP, arg must be a multiple of 64, else of 32. This means 32 is required even when not GPU-accelerated; so we make that error message:

The number of threads per block must be a multiple of 32 (or on AMD GPUs, 64)

otbrown · 2026-05-03T16:40:49Z

Should validate TPB is multiple of 32!

TysonRayJones · 2026-05-19T01:17:45Z



+int getQuESTGpuThreadsPerBlock() {
+    QuESTEnv env = getQuESTEnv();


Note getQuESTEnv() is an API function with its own validation, and shouldn't be called internally like this since if its validation throws, it will claim the user called getQuESTEnv(), when they actually called getQuESTGpuThreadsPerBlock().

Should therefore first call

validate_envIsInit(__func__);

and subsequently use globalEnvPtr directly.

(I see getEnvironmentString() calls getQuESTEnv() for some reason, when it too should just use globalEnvPtr)

TysonRayJones · 2026-05-19T01:21:16Z

+ * This is somehow probably the best pre-existing place for this. It only really applies to GPU, because for
+ * OpenMP the user can just export OMP_NUM_THREADS or call omp_set_num_threads.
+ */
+int getQuESTGpuThreadsPerBlock();


Should we include Num somewhere, e.g.

getNumQuESTGpuThreadsPerBlock

getQuESTNumGpuThreadsPerBlock

I've so far tried to avoid abbreviating where feasible.

TysonRayJones · 2026-05-19T01:22:28Z

+ * OpenMP the user can just export OMP_NUM_THREADS or call omp_set_num_threads.
+ */
+int getQuESTGpuThreadsPerBlock();
+void setQuESTGpuThreadsPerBlock(const int NEW_TPB);


(Same Num) consideration as for getQuESTGpuThreadsPerBlock)

TysonRayJones · 2026-05-19T01:36:08Z

+}
+
+void setQuESTGpuThreadsPerBlock(const int NEW_TPB) {
+    // just rely on the internal function to throw an error if there's no GPU support compiled


Should also call

validate_envIsInit(__func__);

TysonRayJones · 2026-05-19T01:40:08Z


+int getQuESTGpuThreadsPerBlock() {
+    QuESTEnv env = getQuESTEnv();
+    return env.isGpuAccelerated? gpu_getNumThreadsPerBlock() : 0;


Hmm I think this is a pitfall. If setQuESTGpuThreadsPerBlock() is permitted in non-GPU mode (and I really believe it should for healthy platform agnosticism), then the user would always get back 0 in lieu of what they had just passed to set. Maybe we should just always return gpu_getNumThreadsPerBlock().

The situation is slightly different to the GPU cache (fetchable by getGpuCacheSize() and clearable via clearGpuCache()), because that offers no setter. Users can always safely call both in non-GPU accelerated mode, and the former will return 0 (which is always correct).

TysonRayJones · 2026-05-19T01:46:13Z

+    return env.isGpuAccelerated? gpu_getNumThreadsPerBlock() : 0;
+}
+
+void setQuESTGpuThreadsPerBlock(const int NEW_TPB) {


NEW_TPB -> numThreadsPerBlock or numTPB, etc

TysonRayJones · 2026-05-19T01:55:04Z

    #include "quest/src/gpu/cuda_to_hip.hpp"
 #endif

+int numThreadsPerBlock = 128;


Should give this a global_ prefix, like here (like I failed to do for hasGpuBeenBound, oops!)

Further, given it's not accessed anywhere outside gpu_(g|s)etNumThreadsPerBlock(), I would move this definition to the ENVIRONMENT MANAGEMENT section, just before gpu_getNumThreadsPerBlock().

TysonRayJones · 2026-05-19T02:01:39Z

+    error_gpuQueriedButGpuNotCompiled();
+#endif
+    return;
+}


If we permit users to call the corresponding API functions when GPU acceleration is not enabled, then these guards can be removed entirely. I think that's fair/natural, because we certainly shouldn't introduce an API difference between compiling but not running with GPU acceleration.

I would also comment this exception. So this could become:

int gpu_getNumThreadsPerBlock() { // permitted even when GPU backend not compiled return globlal_numThreadsPerBlock; } void gpu_setNumThreadsPerBlock(const int newThreadsPerBlock) { // permitted even when GPU backend not compiled global_numThreadsPerBlock = newThreadsPerBlock; }

TysonRayJones · 2026-05-19T02:12:30Z



-__host__ qindex getNumBlocks(qindex numThreads) {
+__host__ qindex getNumBlocks(qindex numThreads, const int numThreadsPerBlock) {


I would remove the const qualifier since it's asymmetric and a bit counterproductive, because it makes the reader why why isn't numThreads const

TysonRayJones · 2026-05-19T02:13:37Z


    qindex numThreads = qureg.numAmpsPerNode / powerOf2(qubits.size());
-    qindex numBlocks = getNumBlocks(numThreads);
+    const int numThreadsPerBlock = gpu_getNumThreadsPerBlock();


I think the const before numThreadsPerBlock in gpu_subroutines.cpp are superfluous and only cause reader confusion (they might incorrectly infer that only the 2nd arg of <<< needs to be const, or something). Context already make the constness obvious

otbrown · 2026-05-19T17:59:19Z

Rather than attempt to post a thumbs up on each comment while on train WiFi, I'll just comment thanks @TysonRayJones here! I'm hoping to give this branch some proper attention on Thursday/Friday.

otbrown added 2 commits April 24, 2026 14:00

cpu_config.cpp: replaced omp_get_num_threads with omp_get_max_threads…

c150e0b

…. Yes, it's confusing. Yes, the OpenMP ARB know.

We do these things not because they are easy, but because we thought …

9b8ddd1

…they would be easy.

otbrown self-assigned this Apr 24, 2026

TysonRayJones reviewed Apr 25, 2026

View reviewed changes

Comment thread quest/src/cpu/cpu_config.cpp

TysonRayJones reviewed Apr 25, 2026

View reviewed changes

TysonRayJones reviewed May 3, 2026

View reviewed changes

otbrown added 2 commits May 4, 2026 14:06

updated var names to match QuEST style

680fdda

Merge branch 'devel' into set_gpu_tpb

af1e5cb

TysonRayJones mentioned this pull request May 11, 2026

Find an optimisation #749

Open

Merge remote-tracking branch 'origin/devel' into set_gpu_tpb

aaf5681

TysonRayJones reviewed May 19, 2026

View reviewed changes



		int getQuESTGpuThreadsPerBlock() {
		QuESTEnv env = getQuESTEnv();



		__host__ qindex getNumBlocks(qindex numThreads) {
		__host__ qindex getNumBlocks(qindex numThreads, const int numThreadsPerBlock) {

Conversation

otbrown commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

otbrown commented Apr 24, 2026

Uh oh!

JPRichings commented Apr 24, 2026

Uh oh!

TysonRayJones commented Apr 25, 2026

Uh oh!

JPRichings commented Apr 25, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TysonRayJones commented Apr 25, 2026

Uh oh!

otbrown commented Apr 27, 2026

Uh oh!

TysonRayJones commented Apr 29, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

otbrown commented May 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TysonRayJones May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

otbrown commented May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

otbrown commented Apr 24, 2026 •

edited

Loading

TysonRayJones May 19, 2026 •

edited

Loading