Failed to initialize NVML: Driver/library version mismatch

[lionelman450]

NVIDIA Open GPU Kernel Modules Version

uname -r 5.15.0-157-generic

Operating System and Version

Description: Ubuntu 22.04.5 LTS

Kernel Release

uname -r 5.15.0-157-generic

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Build Command

bash ./NVIDIA-Linux-x86_64-570.124.04.run --no-kernel-modules
cd /data/open-gpu-kernel-modules-570.124.06-p2p/
./install.sh
reboot

Terminal output/Build Log

nvidia-smi
Failed to initialize NVML: Driver/library version mismatch
NVML library version: 570.124

More Info

No response

[without-ordinary]

Getting same error with different version mismatch. NVML library version: 570.195

Installed 570.124.06 drivers, ran install.sh from 570.124.06-p2p branch, get version mismatch.

[Wang-Xiaolong]

570.124.04 != 570.124.06, you can try uninstall nvidia driver(using command nvidia-uninstall) then install 570.124.06 (https://developer.nvidia.com/datacenter-driver-570-124-06-download-archive) to get NVML updated to the version of 570.124.06 to get matched with the branch.

[pranjalsingh4550]

Please share the dmesg log output, and the strace output of the userspace executable.
Does this guide help?