Nvidia driver, CUDA tools and libraries

nvidia-logo-1

Oh no, another Nvidia driver repository? Why? This repository reflects my personal view for the way the driver should be packaged for Fedora and CentOS/RHEL. It’s somewhat different from ELRepo repositories for RHEL/CentOS and from RPMFusion packages for Fedora.

Repository installation

To install the repository on a supported Fedora distribution, run as root the following command:

dnf config-manager --add-repo=https://negativo17.org/repos/fedora-nvidia.repo

To install the repository on CentOS/RHEL:

yum-config-manager --add-repo=https://negativo17.org/repos/epel-nvidia.repo

The Nvidia software packages are available for installation by default also in Gnome Software.

Please note that the driver will show up only if your system matches one of the PCI ID supported by the driver. Otherwise, only the other Nvidia programs (mostly for CUDA development) will show up in the software center.

What’s different?

First of all the packaging is a lot simplified; more stuff is compiled from source, smaller packages and more options. This packages try to comply as maximum to the Fedora Packaging Guidelines; which means they have debuginfo packages, default Fedora’s GCC compile time options (where possible) and standard locations for binaries, data and docs.

What follows below, is a detailed explanation of all the “differences” from the various Nvidia driver packages that I was able to spot on the web and a detailed description on how to install components, etc.

Nvidia drivers

Packaging

  • nvidia-settings, nvidia-persistenced, nvidia-xconfig and nvidia-modprobe are compiled from source.
  • All RPM filters except for GL and OpenCL libraries have been removed, so there is no weird dependency option in the SPEC file. RPM pulls in all correct requirements on its own. This is to avoid pulling in the Nvidia drivers instead of the Mesa libraries or in place of the new open source OpenCL support that’s in Fedora.
  • Simplified packaging with much simpler and readable SPEC file.
  • Dependency on libva-vdpau-driver. So in Totem, or any other libVA supported application you can benefit from VDPAU acceleration.
  • Sources are generated with a script and inserted individually in the various packages; so it can be easily reproduced just by changing the version and rerunning the script.
  • nvidia-xconfig is not required on anything that uses the modular X.org directives, as it writes too much in the configuration file (keyboards, monitors, etc.) and the required entries should be written in separate configuration files under /etc/X11/xorg.conf.d. The package is still available as it’s required to speed up some configuration like multi-monitor setups with SLI Mosaic enabled from the command line, but not installed by default.
  • The NVIDIA OpenGL-based Framebuffer Capture (NvFBCOpenGL) libraries (NvFBC and NvIFR) are private APIs that are only available to NVIDIA approved partners for use in remote graphics scenarios (i.e. Steam In-Home Streaming hardware encoding); so they are packaged in another small package called nvidia-driver-NvFBCOpenGL.
  • The nvidia-settings package now builds the external libXNVCtrl.so library that can be used to control the graphic cards through the NV-CONTROL extension. This library updates the old and obsolete one in Fedora.
  • The nvidia-settings binary is compiled with GTK3 instead of GTK2 on Fedora and RHEL/CentOS 7+.
  • The driver can be installed separately from the nvidia-settings utility, so if you simply want a working driver and do not care about details, your experience should be as close as possible to the one with open source drivers.

Versioning

  • ELRepo ships 32 bit compatibility libraries in a separate package with x86_64 as the architecture and “32bit” in the name. 32 bit libraries should be like in RPMFusion, with an i686 package installable in parallel with the x86_64 one. There are no other packages in the distribution that are built for x86_64, with “32bit” in their name that contain i686 binaries (!), so Nvidia drivers should not be an exception. So no separate “32bit.x86_64” package for 32 bit libraries also on CentOS/RHEL; just install nvidia-driver-libs.i686.
  • Versions are not hidden; all packages have the same driver version.
  • No alternatives system, only the latest version which integrates CUDA support is available. For older releases nouveau works great; and anything below a GeForce 8xxx it’s in my opinion too low end to play anything modern. And Quake 3 and Doom 3 work greatly with nouveau, so that’s not a case!
  • The CentOS/RHEL repository contains the “Long Lived Branch version” where less changes occur; while Fedora repositories contains the “Short Lived Branch version”. Beta CentOS/RHEL and Fedora’s rawhide repositories will contain the “Beta Branch version”

CUDA support

  • CUDA libraries/tools for the driver are split into subpackages. There’s no need to install all the CUDA libraries and tools on a system that has only one adapter and is used for occasional gaming or for simple office use. This can save ~120 MB worth of installed libraries. nvidia-persistenced falls in this category as it’s not needed on a normal laptop or gaming system.
  • Complete packaged CUDA stack has been added for all supported distributions, all the packages provide/require/obsolete the relevant packages in the Nvidia CUDA repository; so you can enable this repository along with the official Nvidia CUDA one (x86_64 systems only).

Kernel modules

  • Multiple choice of kernel module packages; akmod (RPMFusion) for Fedora and binary kmod (Kernel ABI whitelists) for CentOS/RHEL. In addition to this, on both distributions dkms packages are available. This way all cases and personal preferences are covered for both distributions.
  • The Nvidia udev rules leverage nvidia-modprobe command in the system to create devices and set permissions even when the userspace libraries have not been loaded yet, covering the case of Wayland only sessions and compute only system without display userspace components installed.
  • The nvidia-uvm module has a soft dependency on the nvidia module, making sure that these modules are not included in the initrd (thing that would happen by using systemd’s configuration (module-s-load.d). UDev rules make sure the module has proper permissions.
  • Choice of proprietary or open source license kernel modules.

Default configuration

  • Dracut options are depending on the distribution; so no more “vga=normal is an obsolete option” at boot. Each distribution gets its own specific GRUB options for booting.
  • 96 DPI is written in the default xorg.conf config file. Why? Gnome 3 by defaults hard-codes a 96×96 DPI resolution, most of the free drivers do (intel, nouveau, etc.) as the EDID is almost never reliable (please see the excellent Adam’s Jackson post where he explains this). As an example, if you install the Nvidia drivers on a RHEL/CentOS 6 laptop where you used to have nouveau installed (96 DPI hardcoded), the fonts gets 90% of the time supersize and ugly as Gnome 2 and the Nvidia driver do not hard-code 96 DPI like Gnome 3.
  • Make X.org NVIDIA Files section to be loaded latest in case there are other packages providing a custom Files section.
  • Use new OutputClass directive on X.org server 1.16 (and later) to load the driver and do not rely on an edited /etc/X11/xorg.conf file. This also removes editing of the xorg.conf file from the package scriptlets. This does not hardcode the 96 DPI resolution.
  • Add the IgnoreABI directive by default on Fedora rawhide builds.

Kernel modesetting and Wayland support

Kernel mode setting on the nvidia-drm module is enabled by default along with the console frame buffer driver.

Distribution and Nvidia driver version support

Here is a rundown of Nvidia supported drivers and options split by distribution. Basically, CentOS/RHEL will always get a Long Lived branch release if possible, Fedora always a Short Lived branch release, and unreleased distributions will always get a Beta driver.

Operating systemCentOS / RHELFedorarawhide
Driver branchLong LivedShort Lived
Long Lived
Short Lived
Long Lived
Beta
Video Codec SDKYesYesYes
Architectures:

x86_64
aarch64
YesYesYes
Basic nvidia driver:

nvidia-driver
nvidia-driver-libs
nvidia-libXNVCtrl
nvidia-kmod-common
YesYesYes
CUDA libraries and tools:

libnvidia-ml
nvidia-driver-cuda
nvidia-driver-cuda-libs
nvidia-persistenced
YesYesYes
OpenGL Framebuffer Capture:

libnvidia-fbc
YesYesYes
Nvidia tools:

nvidia-modprobe
nvidia-settings
nvidia-xconfig

YesYesYes
Binary kernel
modules (kABI):

kmod-nvidia
YesNoNo
DKMS kernel
modules:

dkms-nvidia
YesYesYes
aKMOD kernel
modules:

akmod-nvidia
NoYesYes
32 bit compatibility on x86_64:

libnvidia-ml
nvidia-libXNVCtrl
nvidia-driver-libs
nvidia-driver-cuda-libs
YesYesYes
VDPAU librariesYesYesYes
EGLStream-based Wayland external platformYesYesYes
GBM EGL external platform libraryYesYesYes

Optimus laptops

The driver should install and operate cleanly whether you are installing it on a system which has one or more discrete Nvidia cards or an Optimus laptop with an Intel and a Nvidia card. Nothing to do to enable or configure Optimus.

This is up to the point that when the drivers are installed, you can even turn off Optimus on or off in your system Bios (if your laptop allows that) and the only difference you should see is that there’s an additional VGA card enabled in your system (check with lspci) and that the Nvidia control panel switches between a PRIME Display, like in this picture:

And a normal RandR managed one, like in this one:

Everything else should not be different from your normal experience.

Limitations with the Nvidia driver

The limitations are the same as provided by the Nvidia driver, this means that if you are running it on an Optimus laptop, the Intel card can never power off. Which means higher power consumption, unfortunately. If you have an Optimus laptop and absolutely need the proprietary drivers, my suggestion is still to disable Optimus in the Bios.

Limitations with the OSS stack

On the contrary, if you use the OSS stack (nouveau/intel) the second card can be powered off if there’s no application running on it or display directly connected to one of the card’s outputs. That’s the best reason to use the OSS drivers at all if you you’re not doing serious gaming or 3D work:

$ sudo cat /sys/kernel/debug/vgaswitcheroo/switch
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :DynOff:0000:01:00.0

You also got the nifty selection menu about running your game on the discrete card on Gnome, which is really cool:

It will power up the video card just before launching the process. Launching a program through that menu entry is like starting it from the command line with the DRI_PRIME variable declared. For example, the same as above would be:

$ DRI_PRIME=1 quake3 &
$ sudo cat /sys/kernel/debug/vgaswitcheroo/switch
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :DynPwr:0000:01:00.0

As you can see, the discrete video card is turned on. For Steam, you still need to edit each of your game to run on the Nvidia card:

SLI systems

SLI is now enabled by default with the Auto profile, there’s nothing to do if you have a SLI system. If you need any different SLI option (AA, SFR, etc.), just override it in X.org configuration files.

Nouveau fallback

With the new expanded OutputClass support for X, as carried out by Hans, it’s now super easy to switch to the OSS stack if the proprietary Nvidia driver somehow does not work. No user space component is touched, as soon as the Nvidia kernel module is not loaded (check on /sys/module/nvidia), the desktop starts with the normal OSS components you get with a normal installation. Thanks to all the work done on libglvnd, the libraries loaded are the correct one for the driver you are running.

This means that the performance of the Nvidia card would be abysmal, but still you would get a nice desktop and browser to Google around for answers on how to fix it :).

Sample installation

Here is an example. Let’s assume you have a freshly installed CentOS system with a recent Nvidia GPU and you want to:

  • Install the driver for gaming
  • Play Vulkan enabled games
  • Want to be comfortable with the Nvidia control panel
  • Play 32 bit games (Vulkan included) on a 64 bit system
  • Play 32 bit Vulkan games on a 64 bit system
# yum install nvidia-driver nvidia-driver-libs.i686
Last metadata expiration check: 0:05:30 ago on Tue 05 Feb 2019 06:52:50 AM CET.
Dependencies resolved.
================================================================================
 Package                   Arch     Version           Repository           Size
================================================================================
Installing:
 nvidia-driver             x86_64   3:415.27-4.fc29   fedora-multimedia   2.4 M
 nvidia-driver-libs        i686     3:415.27-4.fc29   fedora-multimedia    18 M
Installing dependencies:
 akmod-nvidia              x86_64   3:415.27-1.fc29   fedora-multimedia    10 M
 nvidia-driver-cuda-libs   x86_64   3:415.27-4.fc29   fedora-multimedia    25 M
 nvidia-driver-libs        x86_64   3:415.27-4.fc29   fedora-multimedia    34 M
 nvidia-kmod-common        noarch   3:415.27-1.fc29   fedora-multimedia    10 k
 akmods                    noarch   0.5.6-17.fc29     updates              22 k
 egl-wayland               x86_64   1.1.1-3.fc29      updates              29 k
 kernel-devel              x86_64   4.20.6-200.fc29   updates              13 M
 mesa-libEGL               i686     18.2.8-1.fc29     updates             112 k
 mesa-libgbm               i686     18.2.8-1.fc29     updates              39 k
 libglvnd-egl              i686     1:1.1.0-2.fc29    fedora               45 k
 libglvnd-gles             i686     1:1.1.0-2.fc29    fedora               32 k
 libglvnd-opengl           i686     1:1.1.0-2.fc29    fedora               37 k
 libglvnd-opengl           x86_64   1:1.1.0-2.fc29    fedora               39 k
 libva-vdpau-driver        x86_64   0.7.4-22.fc29     fedora               60 k
 libwayland-server         i686     1.16.0-1.fc29     fedora               38 k

Transaction Summary
================================================================================
Install  17 Packages

Total download size: 103 M
Installed size: 385 M
Is this ok [y/N]:

As you can see, this system has akmod enabled kernel modules and libraries for running 32 bit applications. The amount of data to download for the drivers is really small compared to packages that contain CUDA libraries and tools.

Package installation

If you are booting the system in UEFI mode; as a prerequisite to installing any external module (not built into the kernel package), you have to disable UEFI Secure Boot in the system configuration. All modules contained in the kernel package are signed with keys that are generated during build and deleted when packaging. If you want to preserve Secure Boot, you need to sign the modules yourself and import the keys into your hardware module. Doing so is out of scope here; if you need a decent guide just follow Red Hat’s guide for signing kernel modules.

First of all remove all the Nvidia drivers you might have on your system due to RPMFusion, ELRepo, or the Nvidia CUDA repository. This is usually accomplished with the following root command:

yum -y remove *nvidia*

Then, to install the Nvidia driver and its control panel utility in CentOS/RHEL with the binary kABI (Kernel ABI whitelist) module (this is the default on CentOS/RHEL), perform the following command:

yum -y install nvidia-driver nvidia-settings

To do the same in Fedora, using akmod modules, perform the following command:

dnf -y install nvidia-driver nvidia-settings

Specific driver installations

For both Fedora and CentOS/RHEL distributions it’s possible to install additional packages and / or variant of the basic kernel modules. This paragraph contains some examples. Make sure you have the EPEL repository enabled if you plan to use DKMS modules on CentOS/RHEL.

akmod kernel module variant (Fedora):

dnf -y install nvidia-driver akmod-nvidia

DKMS kernel module variant (Fedora/CentOS/RHEL):

yum/dnf -y install nvidia-driver dkms-nvidia

To add 32 bit libraries on a 64 bit system (for games or applications like Steam):

yum/dnf -y install nvidia-driver-libs.i686

Proprietary and open source kernel modules

Since almost a year, the Nvidia driver ships with two different implementations of the kernel modules, one proprietary and one open source. The open source one as of drivers 545.x is now considered beta quality also for the workstations, so it seems a good moment to start shipping it.

The open source one is supposed to be the only one that will be kept in the future, but at the moment both are available and both differ in terms of functionality. You can read about the main differences in terms of functionality and what chips they support in the official documentation.

I did not want to introduce another variation of the kernel modules beside akmods, kABI and DKMS, this would have created even more confusion and lots of dependencies in the SPEC files for the variations. The new akmod and DKMS packages ship both sources (MIT/GPL and proprietary kernel modules) and allow you to switch between one or the other through a configuration file.

Considering that in the long run only the open source variant will remain, I wanted to make this as transparent as possible for the users. Basically, if you don’t care and just want something that works, nothing has changed for you.

The two sources get referenced as they are referenced inside the Nvidia run file, namely “kernel” for the original proprietary kernel modules and “kernel-open” for the new open source variation.

The following instructions show you how to switch between one implementation or the other.

DKMS

Check which version you have installed:

# modinfo -l nvidia
NVIDIA

Change the type of modules you want to use and trigger a rebuild and a reinstall:

# sed -i -e 's/kernel$/kernel-open/g' /etc/nvidia/kernel.conf
# dkms build -m nvidia/545.29.02 --force
# dkms install -m nvidia/545.29.02 --force

Now check again the license and you should see that it has changed to MIT/GPL:

# modinfo -l nvidia
Dual MIT/GPL
# reboot

To switch back, change the configuration again and then trigger the same process for rebuilding installing:

# sed -i -e 's/kernel-open$/kernel/g' /etc/nvidia/kernel.conf
# dkms build -m nvidia/545.29.02 --force
# dkms install -m nvidia/545.29.02 --force
# reboot

akmods

Check which version you have installed:

# modinfo -l nvidia
NVIDIA

Change the type of modules you want to use and trigger a rebuild and a reinstall:

# sed -i -e 's/kernel$/kernel-open/g' /etc/nvidia/kernel.conf
# akmods --rebuild

Now check again the license and you should see that it has changed to MIT/GPL:

# modinfo -l nvidia
Dual MIT/GPL
# reboot

To switch back, change the configuration again and then trigger the same process for rebuilding installing:

# sed -i -e 's/kernel-open$/kernel/g' /etc/nvidia/kernel.conf
# akmods --rebuild
# reboot

Additional driver configuration to your system

To add additional configuration to your system, just create the /etc/X11/xorg.conf file. For example:

Section "Device"
Identifier "Device0"
Driver "nvidia"
Option "NoLogo" "true"
Option "DPI" "96 x 96"
Option "SLI" "Auto"
Option "nvidiaXineramaInfoOrder" "DFP-0"
Option "metamodes" "GPU-a493fbbb-7d76-86a2-8764-d76d487a75a7.DVI-I-1: nvidia-auto-select +0+0, GPU-c02960a4-be28-d5ce-8b02-be04b5e2550b.DVI-I-1: nvidia-auto-select +1680+0"
Option "BaseMosaic" "on"
EndSection

In this example we have 2 video cards with one monitor each, so we enabled SLI, Base Mosaic to have multi monitor support on SLI and make a layout with the second GPU monitor on the right of the first one. Also, we fix the DPI to 96×96, which is the hardcoded default in Gnome and in Open Source drivers.

Configuration for CUDA only systems

Your system might only be used for CUDA development and not require the X server to be running the DDX driver at all, so you might want to tweak the configuration a bit to make the system load (for example) the Intel driver as the main display and just the Nvidia driver for GPU workloads. In this case you have two options.

Option with only CUDA components installed

To install just the CUDA components of the driver and not the OpenGL libraries and all files required by the DDX part of the driver, proceed to install as follows:

# yum install nvidia-driver-cuda
Last metadata expiration check: 0:22:51 ago on Tue 05 Feb 2019 06:52:50 AM CET.
Dependencies resolved.
================================================================================
 Package                   Arch     Version           Repository           Size
================================================================================
Installing:
 nvidia-driver-cuda        x86_64   3:415.27-4.fc29   fedora-multimedia   308 k
Installing dependencies:
 akmod-nvidia              x86_64   3:415.27-1.fc29   fedora-multimedia    10 M
 nvidia-driver-NVML        x86_64   3:415.27-4.fc29   fedora-multimedia   457 k
 nvidia-driver-cuda-libs   x86_64   3:415.27-4.fc29   fedora-multimedia    25 M
 nvidia-kmod-common        noarch   3:415.27-1.fc29   fedora-multimedia    10 k
 nvidia-persistenced       x86_64   3:415.27-2.fc29   fedora-multimedia    40 k
 akmods                    noarch   0.5.6-17.fc29     updates              22 k
 kernel-devel              x86_64   4.20.6-200.fc29   updates              13 M

Transaction Summary
================================================================================
Install  8 Packages

Total download size: 49 M
Installed size: 168 M
Is this ok [y/N]:

As you can see, there are no components providing OpenGL or DDX drivers installed on the system. This will not use any X (or Wayland) configuration compared to what has been installed by default on your system.

On top of this, you can still select what kind of kernel modules you want ot have installed (kABI, akmods and dkms).

The device files are created by the udev rules that call the nvidia-modprobe command, it contains a SUID binary that creates the device files and set the appropriate permissions. The same binary is called directly by Nvidia libraries when accessing a user space component that would require device files access:

$ for i in $(rpm -ql nvidia-driver-libs.x86_64 nvidia-driver-cuda-libs.x86_64 | grep \.so); do
  strings $i | grep nvidia-modprobe > /dev/null && echo $i
done
/usr/lib64/gbm/nvidia-drm_gbm.so
/usr/lib64/libnvidia-allocator.so.1
/usr/lib64/libnvidia-allocator.so.545.29.02
/usr/lib64/libnvidia-api.so.1
/usr/lib64/libnvidia-cfg.so.1
/usr/lib64/libnvidia-cfg.so.545.29.02
/usr/lib64/libnvidia-eglcore.so.545.29.02
/usr/lib64/libnvidia-glcore.so.545.29.02
/usr/lib64/libnvidia-glsi.so.545.29.02
/usr/lib64/vdpau/libvdpau_nvidia.so.1
/usr/lib64/vdpau/libvdpau_nvidia.so.545.29.02
/usr/lib64/libcuda.so
/usr/lib64/libcuda.so.1
/usr/lib64/libcuda.so.545.29.02
/usr/lib64/libnvcuvid.so
/usr/lib64/libnvcuvid.so.1
/usr/lib64/libnvcuvid.so.545.29.02
/usr/lib64/libnvidia-opencl.so.1
/usr/lib64/libnvidia-opencl.so.545.29.02

This requires some testing and adjustments with specifics to your setup, but is definitely possible to use the integrated Intel card and or rely on a system without X installed to run the CUDA components.

CUDA

Packaging

  • Previously in the repository was included the GPU Deployment kit. This was constructed with NVML (NVIDIA Management Library) headers, docs and samples from a separate tarball. The separate tarball was using a different version number than the drivers and was packaged in the nvidia-driver-NVML and nvidia-driver-NVML-devel packages. Starting from CUDA version 8, the NVML header is provided by a CUDA subpackage (cuda-nvml-devel) and no longer provided as part of the GPU Deployment kit.
  • Included is also the Video Codec SDK (Decoder/Encoder) headers, docs and code samples. Again, this uses a different version than the drivers.
  • All the libraries are split into subpackages, much like in the original Nvidia CUDA repository. This allows you to install and build software relying on specific components without the need to install all the CUDA toolkit just to satisfy a library dependency. Also, for the same reason, static libraries have been included in each respective static subpackage.
  • In addition to the libraries bundled in the CUDA toolkit, also the cuDNN library for distributed neural networks is included in the repository. See the table below for details.

Distribution and CUDA version support

All the currently supported Red Hat Enterprise Linux versions (including rebuilds and similar forks – CentOS Stream, AlmaLinux, RockyLinux, EuroLinux, etc.) and Fedora versions are available.

In case of yet unsupported compilers in recent distributions, a compatibility cuda-gcc package is available that brings the distribution to a working level with nvcc.

By installing the cuda-gcc package, a variable is loaded automatically that adds the appropriate NVCC options to override the default system GCC when compiling:

$ cat /etc/profile.d/cuda-gcc.sh 
export NVCC_PREPEND_FLAGS='-ccbin /usr/bin/cuda'

CUDA installations

To install just a runtime CUDA support (required for running CUDA enabled programs), without DDX drivers:

yum -y install cuda nvidia-driver-cuda

To just install packages required for enabling CUDA development:

yum -y install cuda-devel

Or if you just want to enable everything:

dnf/yum -y install nvidia-driver nvidia-driver-cuda cuda-devel

A couple of examples. Just the basic tools:

# yum install cuda
Last metadata expiration check: 0:30:55 ago on Tue 05 Feb 2019 06:52:50 AM CET.
Dependencies resolved.
================================================================================
 Package                  Arch    Version              Repository          Size
================================================================================ Installing: cuda x86_64 1:10.0.130-1.fc29 fedora-multimedia 17 M Installing dependencies: cuda-libs x86_64 1:10.0.130-1.fc29 fedora-multimedia 8.6 M nvidia-driver-cuda-libs x86_64 3:415.27-4.fc29 fedora-multimedia 25 M Transaction Summary ================================================================================ Install 3 Packages Total download size: 51 M Installed size: 178 M Is this ok [y/N]:

The basic tools along with all the libraries (note that the NVML headers are included):

# yum install cuda-devel
Last metadata expiration check: 0:35:09 ago on Tue 05 Feb 2019 06:52:50 AM CET.
Dependencies resolved.
================================================================================
 Package                  Arch    Version              Repository          Size
================================================================================
Installing:
 cuda-devel               x86_64  1:10.0.130-1.fc29    fedora-multimedia  1.6 M
Installing dependencies:
 cuda                     x86_64  1:10.0.130-1.fc29    fedora-multimedia   17 M
 cuda-cublas              x86_64  1:10.0.130-1.fc29    fedora-multimedia   31 M
 cuda-cublas-devel        x86_64  1:10.0.130-1.fc29    fedora-multimedia   32 M
 cuda-cudart              x86_64  1:10.0.130-1.fc29    fedora-multimedia  135 k
 cuda-cudart-devel        x86_64  1:10.0.130-1.fc29    fedora-multimedia  533 k
 cuda-cufft               x86_64  1:10.0.130-1.fc29    fedora-multimedia   64 M
 cuda-cufft-devel         x86_64  1:10.0.130-1.fc29    fedora-multimedia  127 M
 cuda-cupti               x86_64  1:10.0.130-1.fc29    fedora-multimedia  1.4 M
 cuda-cupti-devel         x86_64  1:10.0.130-1.fc29    fedora-multimedia  226 k
 cuda-curand              x86_64  1:10.0.130-1.fc29    fedora-multimedia   38 M
 cuda-curand-devel        x86_64  1:10.0.130-1.fc29    fedora-multimedia   61 M
 cuda-cusolver            x86_64  1:10.0.130-1.fc29    fedora-multimedia   40 M
 cuda-cusolver-devel      x86_64  1:10.0.130-1.fc29    fedora-multimedia   15 M
 cuda-cusparse            x86_64  1:10.0.130-1.fc29    fedora-multimedia   27 M
 cuda-cusparse-devel      x86_64  1:10.0.130-1.fc29    fedora-multimedia   28 M
 cuda-libs                x86_64  1:10.0.130-1.fc29    fedora-multimedia  8.6 M
 cuda-npp-devel           x86_64  1:10.0.130-1.fc29    fedora-multimedia   58 M
 cuda-nvgraph             x86_64  1:10.0.130-1.fc29    fedora-multimedia   68 M
 cuda-nvgraph-devel       x86_64  1:10.0.130-1.fc29    fedora-multimedia   13 k
 cuda-nvjpeg              x86_64  1:10.0.130-1.fc29    fedora-multimedia  372 k
 cuda-nvjpeg-devel        x86_64  1:10.0.130-1.fc29    fedora-multimedia   14 k
 cuda-nvml-devel          x86_64  1:10.0.130-1.fc29    fedora-multimedia   53 k
 cuda-nvrtc               x86_64  1:10.0.130-1.fc29    fedora-multimedia  6.3 M
 cuda-nvrtc-devel         x86_64  1:10.0.130-1.fc29    fedora-multimedia   15 k
 cuda-nvtx                x86_64  1:10.0.130-1.fc29    fedora-multimedia   33 k
 cuda-nvtx-devel          x86_64  1:10.0.130-1.fc29    fedora-multimedia   41 k
 nvidia-driver-NVML       x86_64  3:415.27-4.fc29      fedora-multimedia  457 k
 nvidia-driver-cuda-libs  x86_64  3:415.27-4.fc29      fedora-multimedia   25 M

Transaction Summary
================================================================================
Install  29 Packages

Total download size: 650 M
Installed size: 1.6 G
Is this ok [y/N]:

An example where your CUDA application just uses the CUDA Runtime API and not the kernel runtime:

$ sudo dnf install cuda-cudart
Last metadata expiration check: 0:13:10 ago on Sun Oct 23 13:11:01 2016.
Dependencies resolved.
================================================================================
 Package           Arch         Version               Repository           Size
================================================================================
Installing:
 cuda-cudart       x86_64       1:8.0.44-4.fc24       fedora-nvidia       131 k

Transaction Summary
================================================================================
Install  1 Package

Total size: 131 k
Installed size: 536 k
Is this ok [y/N]:

This will avoid you pulling in all the libraries as before just because you need a single library. This is useful for example for programs that leverage just some part of the CUDA toolkit, like the Nvidia Performance Primitives for image and signal processing in FFmpeg, and similar things.

Bugs

Just open an issue to the specific package on GitHub.

1,271 thoughts to “Nvidia driver, CUDA tools and libraries”

  1. In my case (Fedora 40, Geforce RTX 3080 Ti Mobile), open source NVidia driver has memory leak triggered by Ollama. I switched back to closed source driver as workaround for the bug.

  2. Anyway despite I solved problem with open/prop NVIDIA driver. I will try to move to rpmfusion
    because I all steam games are hanging with 560 driver actually most of them continue to play music and sounds but whole image is frozen
    my current package is 3:560.35.03-4 and I’ve tried to downgrade it few numbers down till no downgrade available but it is still 560 and steam games still hanging
    so moving to rpmfusion I will try my luck there looks like there is 555 driver available….

  3. Anyone having throughput issues from Negativo17? I have been getting timeouts for the last 4 days or so? maybe since the key change on the 1st?

  4. I got this error today:

    LANG=C.utf8 dnf upgrade -y

    Last metadata expiration check: 0:08:39 ago on Sun Sep 1 07:54:29 2024.
    Dependencies resolved.

    Problem: cannot install both gstreamer1-plugins-ugly-1:1.24.6-1.fc40.x86_64 from fedora-multimedia and gstreamer1-plugins-ugly-1:1.24.6-1.fc40.x86_64 from @System
    – cannot install the best update candidate for package gstreamer1-plugins-ugly-free-1.24.7-1.fc40.x86_64

    – cannot install the best update candidate for package gstreamer1-plugins-ugly-1:1.24.6-1.fc40.x86_64

    Package Architecture Version Repository Size

    Upgrading:
    libnvidia-cfg x86_64 3:560.35.03-2.fc40 fedora-multimedia 153 k
    libnvidia-ml x86_64 3:560.35.03-2.fc40 fedora-multimedia 620 k
    nvidia-driver x86_64 3:560.35.03-2.fc40 fedora-multimedia 3.9 M
    nvidia-driver-cuda x86_64 3:560.35.03-2.fc40 fedora-multimedia 453 k
    nvidia-driver-cuda-libs x86_64 3:560.35.03-2.fc40 fedora-multimedia 40 M
    nvidia-driver-libs x86_64 3:560.35.03-2.fc40 fedora-multimedia 150 M
    Installing weak dependencies:
    xorg-x11-nvidia x86_64 3:560.35.03-2.fc40 fedora-multimedia 2.3 M

    Transaction Summary

    Install 1 Package
    Upgrade 6 Packages

    Total download size: 197 M
    Downloading Packages:
    (1/7): libnvidia-cfg-560.35.03-2.fc40.x86_64.rpm 275 kB/s | 153 kB 00:00
    (2/7): libnvidia-ml-560.35.03-2.fc40.x86_64.rpm 1.0 MB/s | 620 kB 00:00
    (3/7): nvidia-driver-cuda-560.35.03-2.fc40.x86_64.rpm 2.2 MB/s | 453 kB 00:00
    (4/7): xorg-x11-nvidia-560.35.03-2.fc40.x86_64.rpm 2.0 MB/s | 2.3 MB 00:01
    (5/7): nvidia-driver-560.35.03-2.fc40.x86_64.rpm 1.2 MB/s | 3.9 MB 00:03
    (6/7): nvidia-driver-cuda-libs-560.35.03-2.fc40.x86_64.rpm 2.2 MB/s | 40 MB 00:18

    (7/7): nvidia-driver-libs-560.35.03-2.fc40.x86_64.rpm 2.7 MB/s | 150 MB 00:56

    Total 3.4 MB/s | 197 MB 00:57
    Package xorg-x11-nvidia-560.35.03-2.fc40.x86_64.rpm is not signed
    Package libnvidia-cfg-560.35.03-2.fc40.x86_64.rpm is not signed
    Package libnvidia-ml-560.35.03-2.fc40.x86_64.rpm is not signed
    Package nvidia-driver-560.35.03-2.fc40.x86_64.rpm is not signed
    Package nvidia-driver-cuda-560.35.03-2.fc40.x86_64.rpm is not signed
    Package nvidia-driver-cuda-libs-560.35.03-2.fc40.x86_64.rpm is not signed
    Package nvidia-driver-libs-560.35.03-2.fc40.x86_64.rpm is not signed
    The downloaded packages were saved in cache until the next successful transaction.
    You can remove cached packages by executing ‘dnf clean packages’.
    Error: GPG check FAILED

    Is it a bug in repository, or, maybe, it’s a targeted attack on me?

  5. Due to change to Nvidia Open GPU, akmod-nvidia.x86_64 driver version 560.31.02 does no seem to work for GTX 10×0 or older GPUs (not supported by Open GPU). Downgraded driver again to get back display. Will you stop supporting the prioritary driver for older GPUs?

  6. I recently rebooted my reliable Media PC to discover my Fedora 39 system wouldn’t load the NVIDIA driver. Because it uses an old GeForce GT 1030, it seems it will never be supported by the “kernel-open” driver. Does anyone have any idea when support for the fully propietary NVIDIA driver might end?

    Thanks!

    For the sake of others with my problem, I’ll post the kernel error here for search engines to find:

    kernel: NVRM: The NVIDIA GPU 0000:01:00.0 (PCI ID: 10de:1d01)
    NVRM: installed in this system is not supported by open
    NVRM: nvidia.ko because it does not include the required GPU
    NVRM: System Processor (GSP).
    NVRM: Please see the ‘Open Linux Kernel Modules’ and ‘GSP
    NVRM: Firmware’ sections in the driver README, available on
    NVRM: the Linux graphics driver download page at
    NVRM: http://www.nvidia.com.
    kernel: nvidia 0000:01:00.0: probe with driver nvidia failed with error -1

      1. Hey, thanks for that, but did you notice my question?
        Does anyone have any idea when support for the fully propietary NVIDIA driver might end?

        I’m stuck using the old driver because of the age of the graphics card on my trusty old Media PC.

  7. Whenever I upgrade either the kernel or the driver, I have to manually force-rebuild nvidia akmod after the reboot, otherwise my external monitor isn’t recognized (which usually is a signal that Nvidia driver isn’t loaded, because on my laptop nouveau cannot activate the external monitor). Does it happen to anyone else? Am I missing something?

    1. If you use akmods, the kernel modules are rebuilt not at install time like for DKMS but when the system boots.

    1. A bundled copy of ccextractor used by MakeMKV is already bundled in the package:

      $ rpm -ql makemkv | grep ccextr
      /usr/bin/mmccextr
      $ mmccextr --version
      CCExtractor 0.88, Carlos Fernandez Sanz, Volker Quetschke.
      Teletext portions taken from Petr Kutalek's telxcc
      This is a reduced-functionality version optimised for DVD closed captions transcoding.
      CCExtractor detailed version info
      Version: 0.88
      Platform: x86_64-redhat-linux
      Compilation date: Sun Jun 4 20:39:02 CEST 2023
      This is a reduced-functionality version
      optimised for DVD closed captions transcoding.

  8. I have tried to install this in a fresh installation of Fedora 38. dnf install nvidia-driver nvidia-settings pulls in the akmods and everything, but rebooting the main screen is blank. I can switch to TTY and SSH, dmesg reports everything nvidia is loaded correctly.

    The card is a RTX 2070Ti.

    Am I missing something?

    1. I am now using CUDA under nouveau drivers, using nvidia-driver-cuda. It seems to work, at least, but it would be nice to know what is wrong.

      When booting on nvidia-driver, dmesg shows it loading correctly, and I can’t find anything else wrong. The main screen is just blank.

      1. I have a suspicion now. I noticed that a naive update is pulling files from rpmfusion, and some have version mismatches.

        For example, now the latest version of the driver in the repo is 535.86.05, but dnf update wants to pull nvidia-modprobe 3:535.98-1.fc38, among others.

  9. This repository has some interesting features, but it seems to lack the necessary support for Secure Boot. In my opinion a user should not have to disable Secure Boot in BIOS before updating the NVidia driver, then sign the new kernel module and finally re-enable Secure Boot in BIOS.

    ElRepo provides signed NVidia drivers and has good support for Secure Boot. Details are available on the wiki page at http://elrepo.org/tiki/SecureBootKey.

  10. Any solution for Fedora Silverblue or Kinoite? Do you Support it?
    I am running Kinoite 37. I can’t switch display to nvidia on optimus laptop using rpmfusion.

  11. what do i need todo to roll my own rpm’s of the new production branch driver, so it would be compatible with the packaging you have?

  12. Thank you for the excellent driver. Works great with szydell/system76 Copr for Fedora! Great work. Thanks again.

Leave a Reply