Describe the bug
When running Inline Studio with an NVIDIA GeForce RTX 5070 Ti GPU, PyTorch throws a warning that the GPU's compute capability (sm_120) is not supported by the installed PyTorch version (which only supports up to sm_90). This prevents GPU acceleration and may cause the application to fall back to CPU, severely impacting performance.
To Reproduce
Steps to reproduce the behavior:
- Follow the Windows installation guide: run
webui.bat --install (which installs PyTorch from the CUDA 12.4 index).
- Launch the application with
webui.bat.
- Observe the warning in the console:
UserWarning: NVIDIA GeForce RTX 5070 Ti with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_61 sm_70 sm_75 sm_80 sm_86 sm_90.
Expected behavior
The application should recognize the RTX 50-series GPU and use it for acceleration without compatibility warnings.
Screenshots / Logs
(Attach the full log or paste the warning message)
Environment
- OS: Windows 11
- GPU: NVIDIA GeForce RTX 5070 Ti (CUDA 13.2)
- Python version: 3.11 (as specified in
.python-version)
- Inline Studio version:
inline-core==1.2.63 (latest as of today)
- PyTorch installed: from
cu124 index
Additional context
The RTX 50 series GPUs are based on the Blackwell architecture and require PyTorch compiled with CUDA 12.5+ (or 13.x) to include sm_120 support. The current install script hardcodes the CUDA 12.4 index (https://download.pytorch.org/whl/cu124), which only provides wheels built for compute capabilities up to sm_90.
I attempted to manually modify webui.bat to use the cu132 index, but then encountered dependency conflicts (e.g., torchao and bitsandbytes did not have pre-built wheels for CUDA 13.2 on Windows). A more robust solution is needed.
Possible fix
- Update the installation script to detect the GPU architecture and automatically select the appropriate CUDA version (e.g., use
cu132 for RTX 50 series).
- Alternatively, provide a command-line flag (e.g.,
--cuda-version) or environment variable (INLINE_CUDA_INDEX) to override the PyTorch index without editing the script.
- Update the dependency constraints to ensure compatibility with newer PyTorch/CUDA builds (e.g.,
torchao>=0.14 may need a version that supports CUDA 13.2).
Describe the bug
When running Inline Studio with an NVIDIA GeForce RTX 5070 Ti GPU, PyTorch throws a warning that the GPU's compute capability (
sm_120) is not supported by the installed PyTorch version (which only supports up tosm_90). This prevents GPU acceleration and may cause the application to fall back to CPU, severely impacting performance.To Reproduce
Steps to reproduce the behavior:
webui.bat --install(which installs PyTorch from the CUDA 12.4 index).webui.bat.Expected behavior
The application should recognize the RTX 50-series GPU and use it for acceleration without compatibility warnings.
Screenshots / Logs
(Attach the full log or paste the warning message)
Environment
.python-version)inline-core==1.2.63(latest as of today)cu124indexAdditional context
The RTX 50 series GPUs are based on the Blackwell architecture and require PyTorch compiled with CUDA 12.5+ (or 13.x) to include
sm_120support. The current install script hardcodes the CUDA 12.4 index (https://download.pytorch.org/whl/cu124), which only provides wheels built for compute capabilities up tosm_90.I attempted to manually modify
webui.batto use thecu132index, but then encountered dependency conflicts (e.g.,torchaoandbitsandbytesdid not have pre-built wheels for CUDA 13.2 on Windows). A more robust solution is needed.Possible fix
cu132for RTX 50 series).--cuda-version) or environment variable (INLINE_CUDA_INDEX) to override the PyTorch index without editing the script.torchao>=0.14may need a version that supports CUDA 13.2).