Skip to content

CUDA update policy - #104

Merged
atalman merged 4 commits into
masterfrom
cuda_update_policy
Aug 20, 2026
Merged

CUDA update policy#104
atalman merged 4 commits into
masterfrom
cuda_update_policy

Conversation

@atalman

@atalman atalman commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Supersedes #74.

Same goal as #74 — write down the CUDA update policy the PyTorch team follows — with two changes based on review feedback.

1. Policy only; the upgrade runbook is dropped from the RFC

#74 also inlined the CUDA/cuDNN Upgrade Runbook consolidated from pytorch/builder. Those steps reference specific PRs, scripts and versions and change on every upgrade, while the RFC should be stable policy. They also duplicated a CUDA/cuDNN support table that is already maintained (and is current) in RELEASE.md. The runbook content is unchanged and preserved in #74; it should land as a runbook next to the CI/CD scripts it describes.

A new Relationship to RELEASE.md and the upgrade runbook section states the split:

  • this RFC — policy (introduction/deprecation criteria, Legacy/Stable/Experimental categories, release gates);
  • pytorch/pytorch RELEASE.md — the per-release record and release mechanics (Release Compatibility Matrix, PyTorch CUDA Support Matrix, Accelerator Software, Drafting RCs, Modify release matrix);
  • runbook — the mechanical upgrade steps.

This mirrors CPython support, where RELEASE.md's Python section links out to RFC-0038. RELEASE.md's Accelerator Software section should get the same link to this RFC (follow-up PR in pytorch/pytorch).

2. CI, not CD, is the gate for shipping a CUDA version

#74 said:

  1. Update CUDA in CD (this is necessary condition to qualify for CUDA version be released as Experimental)

and defined the Experimental version as qualifying once it is "available in nightly releases (CD)". That is wrong: it makes buildable-in-nightly sufficient to ship, i.e. we could release a version nobody tested. Two principles are now stated explicitly and take precedence over the individual steps:

  1. We do not release anything that is not tested in CI. CD-only enablement produces binaries nobody has validated. A version qualifies for a release (including as Experimental) only once it has CI build and test coverage with failures tracked.
  2. We do not cut an RC for anything we are not planning to promote. The release matrix is decided before branch cut and only those versions are built in the RC.

Resulting changes to the process:

  • the Experimental definition now requires CD builds and CI build/test coverage, with failures triaged before branch cut;
  • step 3 (CD) is demoted to a prerequisite for CI enablement, explicitly not a release qualification;
  • step 4 (CI) carries the Experimental qualification;
  • step 5 (benchmarks) is gated on CI being complete, not CD;
  • new step 6 fixes the release matrix before branch cut and restricts RC builds to it;
  • deprecation step 2 keeps "drop CD before CI" and says why: we never ship a version we have stopped testing.

Also fixed: the step-1 self-link pointed at the cuda_update branch (would break once merged), the Windes typo, and a reference to conda platforms we no longer publish.

Related RFCs for CUDA update:
pytorch/pytorch#147383
pytorch/pytorch#145544
pytorch/pytorch#138609
pytorch/pytorch#134015

@albanD albanD left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly ok, just one small comment.

Comment thread RFC-0039-cuda-support.md Outdated
Comment on lines +60 to +66
3. Update CUDA in CD (prerequisite for CI enablement, **not** a qualification for release)
When: Evaluate if we have all packages for update is complete
Goal: Make sure all Linux and Windows wheel and libtorch binaries are produced on nightly. Nightly binaries on their own do not qualify the version for a release; they exist so that CI can be enabled and so users can test early.

4. Update CUDA in CI (**this is the necessary condition to qualify for a CUDA version to be released as Experimental**)
When: Update CUDA in CD is complete and nightly binaries are green
Goal: Make sure the new CUDA version has both build and test jobs running in CI. All failing tests are identified, tracked with issues, and either fixed or explicitly accepted before branch cut. A version with no CI test coverage is not eligible for the release matrix.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised about this ordering? CI doesn't actually consume the CD result?
We could enable build + test in CI first. And then once good, enable CD ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @albanD normally we do CD first since CD just builds binaries, followed up by CI which take considerably more time. Basically CD is build, CI is build+test. Resolving issues for CD help us to prepare for CI build and test. Since we would have caught the same issues in CI if we would start with CI first.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made some changes f8cd77b

@atalman
atalman force-pushed the cuda_update_policy branch from fea464d to f8cd77b Compare August 20, 2026 14:51

@albanD albanD left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok!

@atalman
atalman merged commit df68a2f into master Aug 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants