Skip to content

Update CI to cuda13 - #1950

Open
coreyjadams wants to merge 5 commits into
mainfrom
update-ci-cuda13
Open

Update CI to cuda13#1950
coreyjadams wants to merge 5 commits into
mainfrom
update-ci-cuda13

Conversation

@coreyjadams

Copy link
Copy Markdown
Collaborator

PhysicsNeMo Pull Request

This PR updates our CI from cuda12 to cuda13.

It also fixes a bug in how we check yaml files for license headers. So it looks a lot bigger than it is, because I also fixed all those license headers.

Additionally, the nightly container and pre-commit workflows are deleted:

  • nightly container was not used.
  • pre-commit is superceded by the app version of the same tool, which is faster and more efficient.

Description

Checklist

Dependencies

Review Process

All PRs are reviewed by the PhysicsNeMo team before merging.

Depending on which files are changed, GitHub may automatically assign a maintainer for review.

We are also testing AI-based code review tools (e.g., Greptile), which may add automated comments with a confidence score.
This score reflects the AI’s assessment of merge readiness and is not a qualitative judgment of your work, nor is
it an indication that the PR will be accepted / rejected.

AI-generated feedback should be reviewed critically for usefulness.
You are not required to respond to every AI comment, but they are intended to help both authors and reviewers.
Please react to Greptile comments with 👍 or 👎 to provide feedback on their accuracy.

First, we migrate the nightly and PR ci from cuda12 to cuda13

Second, we purge the unused container CI and now deprecated precommit workflow.
Note that precommit is running as a github installed app here, and we don't need it
as a dedicated action.
I updated precommit to check .yaml and .yml, and many files had incorrect license.
This fixes them.
@copy-pr-bot

copy-pr-bot Bot commented Aug 25, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS review map

Current for commit 4a028bb235d2. An approval covers every file listed for that owner; one owner is sufficient for shared files.

@coreyjadams — 21 file(s)
  • .github/ISSUE_TEMPLATE/bug_report.yml
  • .github/ISSUE_TEMPLATE/config.yml
  • .github/ISSUE_TEMPLATE/documentation_request.yml
  • .github/ISSUE_TEMPLATE/feature_request.yml
  • .github/actions/bootstrap-cudnn-ci/action.yml
  • .github/actions/replace-cache/action.yml
  • .github/actions/setup-uv-env/action.yml
  • .github/ci-requirements.txt
  • .github/regen-ci-deps-lock.sh
  • .github/workflows/blossom-ci.yml
  • .github/workflows/codeowners-info.yml
  • .github/workflows/github-multigpu.yml
  • .github/workflows/github-nightly-container.yml
  • .github/workflows/github-nightly-uv.yml
  • .github/workflows/github-pr.yml
  • .github/workflows/install-ci.yml
  • .github/workflows/merge-queue-blossom-passthrough.yml
  • .github/workflows/pre-commit.yml
  • .github/workflows/regen-ci-deps-lock.yml
  • .github/workflows/stale.yml
  • .pre-commit-config.yaml
@ktangsali — 20 file(s)
  • .github/ISSUE_TEMPLATE/bug_report.yml
  • .github/ISSUE_TEMPLATE/config.yml
  • .github/ISSUE_TEMPLATE/documentation_request.yml
  • .github/ISSUE_TEMPLATE/feature_request.yml
  • .github/actions/bootstrap-cudnn-ci/action.yml
  • .github/actions/replace-cache/action.yml
  • .github/actions/setup-uv-env/action.yml
  • .github/ci-requirements.txt
  • .github/regen-ci-deps-lock.sh
  • .github/workflows/blossom-ci.yml
  • .github/workflows/codeowners-info.yml
  • .github/workflows/github-multigpu.yml
  • .github/workflows/github-nightly-container.yml
  • .github/workflows/github-nightly-uv.yml
  • .github/workflows/github-pr.yml
  • .github/workflows/install-ci.yml
  • .github/workflows/pre-commit.yml
  • .github/workflows/regen-ci-deps-lock.yml
  • .github/workflows/stale.yml
  • .pre-commit-config.yaml
@NickGeneva — 20 file(s)
  • .github/ISSUE_TEMPLATE/bug_report.yml
  • .github/ISSUE_TEMPLATE/config.yml
  • .github/ISSUE_TEMPLATE/documentation_request.yml
  • .github/ISSUE_TEMPLATE/feature_request.yml
  • .github/actions/bootstrap-cudnn-ci/action.yml
  • .github/actions/replace-cache/action.yml
  • .github/actions/setup-uv-env/action.yml
  • .github/ci-requirements.txt
  • .github/regen-ci-deps-lock.sh
  • .github/workflows/blossom-ci.yml
  • .github/workflows/codeowners-info.yml
  • .github/workflows/github-multigpu.yml
  • .github/workflows/github-nightly-container.yml
  • .github/workflows/github-nightly-uv.yml
  • .github/workflows/github-pr.yml
  • .github/workflows/install-ci.yml
  • .github/workflows/pre-commit.yml
  • .github/workflows/regen-ci-deps-lock.yml
  • .github/workflows/stale.yml
  • .pre-commit-config.yaml

No CODEOWNER

  • .gitlab-ci.yml

Comment /codeowners-info to refresh.

@coreyjadams coreyjadams changed the title Update ci cuda13 Update CI to cuda13 Aug 25, 2026
@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR migrates GitHub CI from CUDA 12.8 to CUDA 13.3, updates CUDA-specific extras and PyG dependency handling, broadens YAML license-header checks, and removes superseded workflows.

  • Updates PR, nightly, multi-GPU, and lock-regeneration jobs to CUDA 13.3 containers and cache namespaces.
  • Builds torch_cluster from source while using CUDA 13 PyG wheels for the other graph dependencies.
  • Adds or refreshes license headers across YAML files and checks both .yaml and .yml.
  • Removes the unused nightly-container and superseded pre-commit workflows.

Important Files Changed

Filename Overview
.github/ci-requirements.txt Updates exact PyG requirements to torch 2.12/CUDA 13 builds, but leaves the committed constraints lock on incompatible torch 2.11/CUDA 12 versions.
.github/actions/setup-uv-env/action.yml Adds CUDA source-build settings for torch_cluster; its install currently reaches the stale-lock conflict before dependencies can be installed.
.github/regen-ci-deps-lock.sh Updates lock regeneration for CUDA 13 and mirrors the torch_cluster source-build configuration.
.github/workflows/github-pr.yml Migrates PR jobs and cache keys to CUDA 13.3, but the jobs consume the conflicting requirements and constraints through setup-uv-env.
.github/workflows/github-nightly-uv.yml Migrates nightly jobs to CUDA 13.3 while retaining the same affected dependency setup path.
.github/workflows/github-multigpu.yml Migrates multi-GPU jobs to CUDA 13.3 and CUDA 13 extras while indirectly consuming the affected setup action.
.pre-commit-config.yaml Correctly extends the license-header hook to include .yml files.

Reviews (1): Last reviewed commit: "While updating CI, I notice the yaml ext..." | Re-trigger Greptile

Comment thread .github/ci-requirements.txt

@peterdsharpe peterdsharpe left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great calls all around. This bundles a bunch of small smart changes.

Comment thread .github/workflows/github-nightly-uv.yml Outdated
Comment thread .github/workflows/github-nightly-uv.yml Outdated

@ktangsali ktangsali left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly good, @coreyjadams did you mention you changed something on the license testing front? I don't see those diffs here.

Comment thread .github/actions/setup-uv-env/action.yml
Comment thread .github/regen-ci-deps-lock.sh
@ktangsali

Copy link
Copy Markdown
Collaborator

Looks mostly good, @coreyjadams did you mention you changed something on the license testing front? I don't see those diffs here.

Was it this change?

  files: \.(py|yaml|yml|ci|release)$|Dockerfile$

@coreyjadams

Copy link
Copy Markdown
Collaborator Author

Yes: it was exactly the change you mentioned. The license check was only looking for .yaml files, and most of our github files are .yml, and the difference is load bearing.

After I added .yml to that line so it searched them too, it picked up many failures that we had been allowing (only half were mine, I promise!) so I fixed the ones it finds.

@coreyjadams

Copy link
Copy Markdown
Collaborator Author

/ok to test 5cbd2de

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants