Nvidia driver, CUDA tools and libraries

Oh no, another Nvidia driver repository? Why? This repository reflects my personal view for the way the driver should be packaged for Fedora and CentOS/RHEL. It’s somewhat different from ELRepo repositories for RHEL/CentOS and from RPMFusion packages for Fedora.

Repository installation

To install the repository on a supported Fedora distribution, run as root the following command:

dnf config-manager --add-repo=https://negativo17.org/repos/fedora-nvidia.repo

To install the repository on CentOS/RHEL:

yum-config-manager --add-repo=https://negativo17.org/repos/epel-nvidia.repo

Starting from Fedora 25, the Nvidia software packages are available for installation by default also in Gnome Software.

Please note that the driver will show up only if your system matches one of the PCI ID supported by the driver. Otherwise, only the other Nvidia programs (mostly for CUDA development) will show up in the software center.

What’s different?

First of all the packaging is a lot simplified; more stuff is compiled from source, smaller packages and more options. This packages try to comply as maximum to the Fedora Packaging Guidelines; which means they have debuginfo packages, default Fedora’s GCC compile time options (where possible) and standard locations for binaries, data and docs.

What follows below, is a detailed explanation of all the “differences” from the various Nvidia driver packages that I was able to spot on the web and a detailed description on how to install components, etc.

Nvidia drivers

Packaging

  • nvidia-settings, nvidia-persistenced, nvidia-xconfig and nvidia-modprobe are compiled from source.
  • All RPM filters except for GL and OpenCL libraries have been removed, so there is no weird dependency option in the SPEC file. RPM pulls in all correct requirements on its own. This is to avoid pulling in the Nvidia drivers instead of the Mesa libraries or in place of the new open source OpenCL support that’s in Fedora.
  • Simplified packaging with much simpler and readable SPEC file.
  • Dependency on libva-vdpau-driver. So in Totem, or any other libVA supported application you can benefit from VDPAU acceleration.
  • Sources are generated with a script and inserted individually in the various packages; so it can be easily reproduced just by changing the version and rerunning the script.
  • nvidia-xconfig is not required on anything that uses the modular X.org directives, as it writes too much in the configuration file (keyboards, monitors, etc.) and the required entries should be written in separate configuration files under /etc/X11/xorg.conf.d. The package is still available as it’s required to speed up some configuration like multi-monitor setups with SLI Mosaic enabled from the command line, but not installed by default.
  • The NVIDIA OpenGL-based Framebuffer Capture (NvFBCOpenGL) libraries (NvFBC and NvIFR) are private APIs that are only available to NVIDIA approved partners for use in remote graphics scenarios (i.e. Steam In-Home Streaming hardware encoding); so they are packaged in another small package called nvidia-driver-NvFBCOpenGL.
  • The nvidia-settings package now builds the external libXNVCtrl.so library that can be used to control the graphic cards through the NV-CONTROL extension. This library updates the old and obsolete one in Fedora.
  • The nvidia-settings binary is compiled with GTK3 instead of GTK2 on Fedora and RHEL/CentOS 7+.
  • The driver can be installed separately from the nvidia-settings utility, so if you simply want a working driver and do not care about details, your experience should be as close as possible to the one with open source drivers.

Versioning

  • ELRepo ships 32 bit compatibility libraries in a separate package with x86_64 as the architecture and “32bit” in the name. 32 bit libraries should be like in RPMFusion, with an i686 package installable in parallel with the x86_64 one. There are no other packages in the distribution that are built for x86_64, with “32bit” in their name that contain i686 binaries (!), so Nvidia drivers should not be an exception. So no separate “32bit.x86_64” package for 32 bit libraries also on CentOS/RHEL; just install nvidia-driver-libs.i686.
  • Versions are not hidden; all packages have the same driver version.
  • No alternatives system, only the latest version which integrates CUDA support is available. For older releases nouveau works great; and anything below a GeForce 8xxx it’s in my opinion too low end to play anything modern. And Quake 3 and Doom 3 work greatly with nouveau, so that’s not a case!
  • The CentOS/RHEL repository contains the “Long Lived Branch version” where less changes occur; while Fedora repositories contains the “Short Lived Branch version”. Beta CentOS/RHEL and Fedora’s rawhide repositories will contain the “Beta Branch version”

CUDA support

  • CUDA libraries/tools for the driver are split into subpackages. There’s no need to install all the CUDA libraries and tools on a system that has only one adapter and is used for occasional gaming or for simple office use. This can save ~120 MB worth of installed libraries. nvidia-persistenced falls in this category as it’s not needed on a normal laptop or gaming system.
  • Complete packaged CUDA stack has been added for all supported distributions, all the packages provide/require/obsolete the relevant packages in the Nvidia CUDA repository; so you can enable this repository along with the official Nvidia CUDA one (x86_64 systems only).

Kernel modules

  • Multiple choice of kernel module packages; akmod (RPMFusion) for Fedora and binary kmod (Kernel ABI whitelists) for CentOS/RHEL. In addition to this, on both distributions dkms packages are available. This way all cases and personal preferences are covered for both distributions.
  • Starting from Nvidia driver version 334.16, the Nvidia DDX driver for X can also rely on the nvidia-modprobe command in the system to create devices and set permissions, so the new optional package has been added.
  • The nvidia-uvm module has a soft dependency on the nvidia module, making sure that these modules are not included in the initrd (thing that would happen by using systemd’s configuration (module-s-load.d). UDev rules make sure the module has proper permissions.
  • On Fedora, the kernel modules are compressed with XZ, like all the other kernel modules.

Default configuration

  • Dracut options are depending on the distribution; so no more “vga=normal is an obsolete option” at boot. Each distribution gets its own specific GRUB options for booting.
  • 96 DPI is written in the default xorg.conf config file. Why? Gnome 3 by defaults hard-codes a 96×96 DPI resolution, most of the free drivers do (intel, nouveau, etc.) as the EDID is almost never reliable (please see the excellent Adam’s Jackson post where he explains this). As an example, if you install the Nvidia drivers on a RHEL/CentOS 6 laptop where you used to have nouveau installed (96 DPI hardcoded), the fonts gets 90% of the time supersize and ugly as Gnome 2 and the Nvidia driver do not hard-code 96 DPI like Gnome 3.
  • Make X.org NVIDIA Files section to be loaded latest in case there are other packages providing a custom Files section.
  • Use new OutputClass directive on X.org server 1.16 (and later) to load the driver and do not rely on an edited /etc/X11/xorg.conf file. This also removes editing of the xorg.conf file from the package scriptlets. This does not hardcode the 96 DPI resolution.
  • Add the IgnoreABI directive by default on Fedora rawhide builds.

Kernel modesetting and Wayland support

Kernel mode setting on the nvidia-drm module has been disabled by default for various reasons. First of all, Wayland support in the drivers require a patched Wayland which has been refused upstream, and then the driver itself does not expose an FB driver for the console, so you won’t see any difference in the terminal output, you will still be limited to VGA.

There is a proposal for sorting everything out at XDC 2016 for hardware vendors that require to expose extensions for the drivers.

The Wayland libraries are still included in the builds, as all the dependencies are there but they are not used.

Vulkan support

vulkan_500px_june16

On supported releases, the Vulkan loader and libraries are installed automatically and you do not need to do anything to enable support in the drivers. CentOS and Red Hat Enterprise Linux 6 does not have Vulkan.

Distribution and Nvidia driver version support

Here is a rundown of Nvidia supported drivers and options split by distribution. Basically, CentOS/RHEL will always get a Long Lived branch release if possible, Fedora always a Short Lived branch release, and unreleased distributions will always get a Beta driver.

Operating systemel6 / el7 / el8f31 / f32rawhide
Driver branchLong LivedShort Lived
Long Lived
Short Lived
Long Lived
Beta
Video Codec SDKYesYesYes
Architectures:

x86_64
YesYesYes
Basic nvidia driver:

nvidia-driver
nvidia-driver-libs
nvidia-libXNVCtrl
YesYesYes
CUDA libraries and tools:

nvidia-driver-cuda
nvidia-driver-cuda-libs
nvidia-driver-NVML
nvidia-persistenced
YesYesYes
OpenGL Framebuffer Capture:

nvidia-driver-NvFBCOpenGL
YesYesYes
Nvidia tools:

nvidia-modprobe
nvidia-settings
nvidia-xconfig

YesYesYes
Binary kernel
modules (kABI):

kmod-nvidia
YesNoNo
DKMS kernel
modules:

dkms-nvidia
YesYesYes
aKMOD kernel
modules:

akmod-nvidia
NoYesYes
32 bit compatibility on x86_64:

nvidia-libXNVCtrl
nvidia-driver-libs
nvidia-driver-cuda-libs
nvidia-driver-NVML
YesYesYes
Development

nvidia-driver-devel
nvenc
nvenc-samples
YesYesYes
GLVND librariesYesYesYes
VDPAU librariesYesYesYes

Optimus laptops

The driver should install and operate cleanly whether you are installing it on a system which has one or more discrete Nvidia cards or an Optimus laptop with an Intel and a Nvidia card. Nothing to do to enable or configure Optimus.

This is up to the point that when the drivers are installed, you can even turn off Optimus on or off in your system Bios (if your laptop allows that) and the only difference you should see is that there’s an additional VGA card enabled in your system (check with lspci) and that the Nvidia control panel switches between a PRIME Display, like in this picture:

And a normal RandR managed one, like in this one:

Everything else should not be different from your normal experience.

Limitations with the Nvidia driver

The limitations are the same as provided by the Nvidia driver, this means that if you are running it on an Optimus laptop, the Intel card can never power off. Which means higher power consumption, unfortunately. If you have an Optimus laptop and absolutely need the proprietary drivers, my suggestion is still to disable Optimus in the Bios.

Limitations with the OSS stack

On the contrary, if you use the OSS stack (nouveau/intel) the second card can be powered off if there’s no application running on it or display directly connected to one of the card’s outputs. That’s the best reason to use the OSS drivers at all if you you’re not doing serious gaming or 3D work:

$ sudo cat /sys/kernel/debug/vgaswitcheroo/switch
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :DynOff:0000:01:00.0

You also got the nifty selection menu about running your game on the discrete card on Gnome, which is really cool:

It will power up the video card just before launching the process. Launching a program through that menu entry is like starting it from the command line with the DRI_PRIME variable declared. For example, the same as above would be:

$ DRI_PRIME=1 quake3 &
$ sudo cat /sys/kernel/debug/vgaswitcheroo/switch
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :DynPwr:0000:01:00.0

As you can see, the discrete video card is turned on. For Steam, you still need to edit each of your game to run on the Nvidia card:

SLI systems

SLI is now enabled by default with the Auto profile, there’s nothing to do if you have a SLI system. If you need any different SLI option (AA, SFR, etc.), just override it in X.org configuration files.

Nouveau fallback

With the new expanded OutputClass support for X, as carried out by Hans, it’s now super easy to switch to the OSS stack if the proprietary Nvidia driver somehow does not work. No user space component is touched, as soon as the Nvidia kernel module is not loaded (check on /sys/module/nvidia), the desktop starts with the normal OSS components you get with a normal installation. Thanks to all the work done on libglvnd, the libraries loaded are the correct one for the driver you are running.

This means that the performance of the Nvidia card would be abysmal, but still you would get a nice desktop and browser to Google around for answers on how to fix it :).

Sample installation

Here is an example. Let’s assume you have a freshly installed CentOS system with a recent Nvidia GPU and you want to:

  • Install the driver for gaming
  • Play Vulkan enabled games
  • Want to be comfortable with the Nvidia control panel
  • Play 32 bit games (Vulkan included) on a 64 bit system
  • Play 32 bit Vulkan games on a 64 bit system
# yum install nvidia-driver nvidia-driver-libs.i686
Last metadata expiration check: 0:05:30 ago on Tue 05 Feb 2019 06:52:50 AM CET.
Dependencies resolved.
================================================================================
 Package                   Arch     Version           Repository           Size
================================================================================
Installing:
 nvidia-driver             x86_64   3:415.27-4.fc29   fedora-multimedia   2.4 M
 nvidia-driver-libs        i686     3:415.27-4.fc29   fedora-multimedia    18 M
Installing dependencies:
 akmod-nvidia              x86_64   3:415.27-1.fc29   fedora-multimedia    10 M
 nvidia-driver-cuda-libs   x86_64   3:415.27-4.fc29   fedora-multimedia    25 M
 nvidia-driver-libs        x86_64   3:415.27-4.fc29   fedora-multimedia    34 M
 nvidia-kmod-common        noarch   3:415.27-1.fc29   fedora-multimedia    10 k
 akmods                    noarch   0.5.6-17.fc29     updates              22 k
 egl-wayland               x86_64   1.1.1-3.fc29      updates              29 k
 kernel-devel              x86_64   4.20.6-200.fc29   updates              13 M
 mesa-libEGL               i686     18.2.8-1.fc29     updates             112 k
 mesa-libgbm               i686     18.2.8-1.fc29     updates              39 k
 libglvnd-egl              i686     1:1.1.0-2.fc29    fedora               45 k
 libglvnd-gles             i686     1:1.1.0-2.fc29    fedora               32 k
 libglvnd-opengl           i686     1:1.1.0-2.fc29    fedora               37 k
 libglvnd-opengl           x86_64   1:1.1.0-2.fc29    fedora               39 k
 libva-vdpau-driver        x86_64   0.7.4-22.fc29     fedora               60 k
 libwayland-server         i686     1.16.0-1.fc29     fedora               38 k

Transaction Summary
================================================================================
Install  17 Packages

Total download size: 103 M
Installed size: 385 M
Is this ok [y/N]:

As you can see, this system has akmod enabled kernel modules and libraries for running 32 bit applications. The amount of data to download for the drivers is really small compared to packages that contain CUDA libraries and tools.

Package installation

If you are booting the system in UEFI mode; as a prerequisite to installing any external module (not built into the kernel package), you have to disable UEFI Secure Boot in the system configuration. All modules contained in the kernel package are signed with keys that are generated during build and deleted when packaging. If you want to preserve Secure Boot, you need to sign the modules yourself and import the keys into your hardware module. Doing so is out of scope here; if you need a decent guide just follow Red Hat’s guide for signing kernel modules.

First of all remove all the Nvidia drivers you might have on your system due to RPMFusion, ELRepo, or the Nvidia CUDA repository. This is usually accomplished with the following root command:

yum -y remove *nvidia*

Then, to install the Nvidia driver and its control panel utility in CentOS/RHEL with the binary kABI (Kernel ABI whitelist) module (this is the default on CentOS/RHEL), perform the following command:

yum -y install nvidia-driver nvidia-settings

To do the same in Fedora, using akmod modules, perform the following command:

dnf -y install nvidia-driver nvidia-settings

Specific driver installations

For both Fedora and CentOS/RHEL distributions it’s possible to install additional packages and / or variant of the basic kernel modules. This paragraph contains some examples. Make sure you have the EPEL repository enabled if you plan to use DKMS modules on CentOS/RHEL.

akmod kernel module variant (Fedora):

dnf -y install nvidia-driver akmod-nvidia

DKMS kernel module variant (Fedora/CentOS/RHEL):

yum/dnf -y install nvidia-driver dkms-nvidia

To add 32 bit libraries on a 64 bit system (for games or applications like Steam):

yum/dnf -y install nvidia-driver-libs.i686

Additional driver configuration to your system

To add additional configuration to your system, just create the /etc/X11/xorg.conf file. For example:

Section "Device"
Identifier "Device0"
Driver "nvidia"
Option "NoLogo" "true"
Option "DPI" "96 x 96"
Option "SLI" "Auto"
Option "nvidiaXineramaInfoOrder" "DFP-0"
Option "metamodes" "GPU-a493fbbb-7d76-86a2-8764-d76d487a75a7.DVI-I-1: nvidia-auto-select +0+0, GPU-c02960a4-be28-d5ce-8b02-be04b5e2550b.DVI-I-1: nvidia-auto-select +1680+0"
Option "BaseMosaic" "on"
EndSection

In this example we have 2 video cards with one monitor each, so we enabled SLI, Base Mosaic to have multi monitor support on SLI and make a layout with the second GPU monitor on the right of the first one. Also, we fix the DPI to 96×96, which is the hardcoded default in Gnome and in Open Source drivers.

Configuration for CUDA only systems

Your system might only be used for CUDA development and not require the X server to be running the DDX driver at all, so you might want to tweak the configuration a bit to make the system load (for example) the Intel driver as the main display and just the Nvidia driver for GPU workloads. In this case you have two options.

Option with only CUDA components installed

To install just the CUDA components of the driver and not the OpenGL libraries and all files required by the DDX part of the driver, proceed to install as follows:

# yum install nvidia-driver-cuda
Last metadata expiration check: 0:22:51 ago on Tue 05 Feb 2019 06:52:50 AM CET.
Dependencies resolved.
================================================================================
 Package                   Arch     Version           Repository           Size
================================================================================
Installing:
 nvidia-driver-cuda        x86_64   3:415.27-4.fc29   fedora-multimedia   308 k
Installing dependencies:
 akmod-nvidia              x86_64   3:415.27-1.fc29   fedora-multimedia    10 M
 nvidia-driver-NVML        x86_64   3:415.27-4.fc29   fedora-multimedia   457 k
 nvidia-driver-cuda-libs   x86_64   3:415.27-4.fc29   fedora-multimedia    25 M
 nvidia-kmod-common        noarch   3:415.27-1.fc29   fedora-multimedia    10 k
 nvidia-persistenced       x86_64   3:415.27-2.fc29   fedora-multimedia    40 k
 akmods                    noarch   0.5.6-17.fc29     updates              22 k
 kernel-devel              x86_64   4.20.6-200.fc29   updates              13 M

Transaction Summary
================================================================================
Install  8 Packages

Total download size: 49 M
Installed size: 168 M
Is this ok [y/N]:

As you can see, there are no components providing OpenGL or DDX drivers installed on the system. This will not use any X (or Wayland) configuration compared to what has been installed by default on your system.

On top of this, you can still select what kind of kernel modules you want ot have installed (kABI, akmods and dkms).

Option with DDX components installed

The Intel driver should load the modesetting driver, offload the rendering to the Nvidia driver and not use any monitor attached to the X server for the Nvidia driver.

So in this case, I would change /etc/default/grub to remove the nomodeset parameter to make the Intel KMS driver to load properly, regenerate the Grub config file, reboot and use this xorg.conf:

Section "ServerLayout"
Identifier "layout"
Screen 0 "intel"
Inactive "nvidia"
EndSection

Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID ""
EndSection

Section "Screen"
Identifier "nvidia"
Device "nvidia"
Option "AllowEmptyInitialConfiguration"
EndSection

Section "Device"
Identifier "intel"
Driver "modesetting"
EndSection

Section "Screen"
Identifier "intel"
Device "intel"
EndSection

An example of the above can also be read in the official documentation.

The device file /dev/nvidia0 is normally created when loading the Nvidia driver, so if the X driver is not loaded the device file fileis not created. You can use the nvidia-modprobe command that is in the package with the same name. It contains a SUID binary that creates the device files and set the appropriate permissions when automatic device creation is not available. It is called directly by Nvidia libraries:

$ for i in $(rpm -ql nvidia-driver-libs.x86_64); do
strings $i | grep nvidia-modprobe > /dev/null && echo $i
done
/usr/lib64/libnvidia-cfg.so.1
/usr/lib64/libnvidia-cfg.so.381.22
/usr/lib64/libnvidia-eglcore.so.381.22
/usr/lib64/libnvidia-glcore.so.381.22
/usr/lib64/libnvidia-glsi.so.381.22 /usr/lib64/vdpau/libvdpau_nvidia.so.1 /usr/lib64/vdpau/libvdpau_nvidia.so.381.22

This requires some testing and adjustments with specifics to your setup, but is definitely possible to use the integrated Intel card and or rely on a system without X installed to run the CUDA components.

CUDA

Packaging

  • Previously in the repository was included the GPU Deployment kit. This was constructed with NVML (NVIDIA Management Library) headers, docs and samples from a separate tarball. The separate tarball was using a different version number than the drivers and was packaged in the nvidia-driver-NVML and nvidia-driver-NVML-devel packages. Starting from CUDA version 8, the NVML header is provided by a CUDA subpackage (cuda-nvml-devel) and no longer provided as part of the GPU Deployment kit.
  • Included is also the Video Codec SDK (Decoder/Encoder) headers, docs and code samples. Again, this uses a different version than the drivers.
  • All the libraries are split into subpackages, much like in the original Nvidia CUDA repository. This allows you to install and build software relying on specific components without the need to install all the CUDA toolkit just to satisfy a library dependency. With the new packaging organization, the original cuda-devel and cuda-extra-libs will pull in all the specific subpackages giving you the same situation you are accustomed to. Also, for the same reason, static libraries have been included in each respective devel subpackage.
  • In addition to the libraries bundled in the CUDA toolkit, also the cuDNN library for distributed neural networks is included in the repository. See the table below for details.

Distribution and CUDA version support

Operating systemel6 / el7 / el8f31 / f32f33
CUDA cuDNNYesYesYes
GCC compatibility-cuda-gcccuda-gcc
Basic CUDA libraries/tools:

cuda
cuda-libs
cuda-extra-libs
cuda-cublas
cuda-cudart
cuda-cufft
cuda-cudnn
cuda-cupti
cuda-curand
cuda-cusolver
cuda-cusparse
cuda-npp
cuda-nvgraph
cuda-nvrtc
cuda-nvvp
YesYesYes
CUDA development:

cuda-cli-tools
cuda-devel
cuda-cublas-devel
cuda-cudart-devel
cuda-cudnn-devel
cuda-cufft-devel
cuda-cupti-devel
cuda-curand-devel
cuda-cusolver-devel
cuda-cusparse-devel
cuda-npp-devel
cuda-nvgraph-devel
cuda-nvml-devel
(also i686)
cuda-nvrtc-devel
YesYesYes
GUI programs:

cuda-nsight
cuda-nvvp
cuda-nsight-compute
cuda-nsight-systems
YesYesYes
Documentation and samples

cuda-samples
cuda-docs
(noarch)
YesYesYes

CUDA installations

To install just a runtime CUDA support (required for running CUDA enabled programs), without DDX drivers:

yum -y install cuda nvidia-driver-cuda

To just install packages required for enabling CUDA development:

yum -y install cuda-devel

Or if you just want to enable everything:

dnf/yum -y install nvidia-driver nvidia-driver-cuda cuda-devel

A couple of examples. Just the basic tools:

# yum install cuda
Last metadata expiration check: 0:30:55 ago on Tue 05 Feb 2019 06:52:50 AM CET.
Dependencies resolved.
================================================================================
 Package                  Arch    Version              Repository          Size
================================================================================ Installing: cuda x86_64 1:10.0.130-1.fc29 fedora-multimedia 17 M Installing dependencies: cuda-libs x86_64 1:10.0.130-1.fc29 fedora-multimedia 8.6 M nvidia-driver-cuda-libs x86_64 3:415.27-4.fc29 fedora-multimedia 25 M Transaction Summary ================================================================================ Install 3 Packages Total download size: 51 M Installed size: 178 M Is this ok [y/N]:

The basic tools along with all the libraries (note that the NVML headers are included):

# yum install cuda-devel
Last metadata expiration check: 0:35:09 ago on Tue 05 Feb 2019 06:52:50 AM CET.
Dependencies resolved.
================================================================================
 Package                  Arch    Version              Repository          Size
================================================================================
Installing:
 cuda-devel               x86_64  1:10.0.130-1.fc29    fedora-multimedia  1.6 M
Installing dependencies:
 cuda                     x86_64  1:10.0.130-1.fc29    fedora-multimedia   17 M
 cuda-cublas              x86_64  1:10.0.130-1.fc29    fedora-multimedia   31 M
 cuda-cublas-devel        x86_64  1:10.0.130-1.fc29    fedora-multimedia   32 M
 cuda-cudart              x86_64  1:10.0.130-1.fc29    fedora-multimedia  135 k
 cuda-cudart-devel        x86_64  1:10.0.130-1.fc29    fedora-multimedia  533 k
 cuda-cufft               x86_64  1:10.0.130-1.fc29    fedora-multimedia   64 M
 cuda-cufft-devel         x86_64  1:10.0.130-1.fc29    fedora-multimedia  127 M
 cuda-cupti               x86_64  1:10.0.130-1.fc29    fedora-multimedia  1.4 M
 cuda-cupti-devel         x86_64  1:10.0.130-1.fc29    fedora-multimedia  226 k
 cuda-curand              x86_64  1:10.0.130-1.fc29    fedora-multimedia   38 M
 cuda-curand-devel        x86_64  1:10.0.130-1.fc29    fedora-multimedia   61 M
 cuda-cusolver            x86_64  1:10.0.130-1.fc29    fedora-multimedia   40 M
 cuda-cusolver-devel      x86_64  1:10.0.130-1.fc29    fedora-multimedia   15 M
 cuda-cusparse            x86_64  1:10.0.130-1.fc29    fedora-multimedia   27 M
 cuda-cusparse-devel      x86_64  1:10.0.130-1.fc29    fedora-multimedia   28 M
 cuda-libs                x86_64  1:10.0.130-1.fc29    fedora-multimedia  8.6 M
 cuda-npp-devel           x86_64  1:10.0.130-1.fc29    fedora-multimedia   58 M
 cuda-nvgraph             x86_64  1:10.0.130-1.fc29    fedora-multimedia   68 M
 cuda-nvgraph-devel       x86_64  1:10.0.130-1.fc29    fedora-multimedia   13 k
 cuda-nvjpeg              x86_64  1:10.0.130-1.fc29    fedora-multimedia  372 k
 cuda-nvjpeg-devel        x86_64  1:10.0.130-1.fc29    fedora-multimedia   14 k
 cuda-nvml-devel          x86_64  1:10.0.130-1.fc29    fedora-multimedia   53 k
 cuda-nvrtc               x86_64  1:10.0.130-1.fc29    fedora-multimedia  6.3 M
 cuda-nvrtc-devel         x86_64  1:10.0.130-1.fc29    fedora-multimedia   15 k
 cuda-nvtx                x86_64  1:10.0.130-1.fc29    fedora-multimedia   33 k
 cuda-nvtx-devel          x86_64  1:10.0.130-1.fc29    fedora-multimedia   41 k
 nvidia-driver-NVML       x86_64  3:415.27-4.fc29      fedora-multimedia  457 k
 nvidia-driver-cuda-libs  x86_64  3:415.27-4.fc29      fedora-multimedia   25 M

Transaction Summary
================================================================================
Install  29 Packages

Total download size: 650 M
Installed size: 1.6 G
Is this ok [y/N]:

An example where your CUDA application just uses the CUDA Runtime API and not the kernel runtime:

$ sudo dnf install cuda-cudart
Last metadata expiration check: 0:13:10 ago on Sun Oct 23 13:11:01 2016.
Dependencies resolved.
================================================================================
 Package           Arch         Version               Repository           Size
================================================================================
Installing:
 cuda-cudart       x86_64       1:8.0.44-4.fc24       fedora-nvidia       131 k

Transaction Summary
================================================================================
Install  1 Package

Total size: 131 k
Installed size: 536 k
Is this ok [y/N]:

This will avoid you pulling in all the libraries as before just because you need a single library. This is useful for example for programs that leverage just some part of the CUDA toolkit, like the Nvidia Performance Primitives for image and signal processing in FFmpeg, and similar things.

Bugs

Just open an issue to the specific package on GitHub.

1,224 thoughts to “Nvidia driver, CUDA tools and libraries”

  1. Hi,
    I’m working on getting Steam to work on Fedora 32 with an NVidia discrete adapter doing the offload.
    I’m sure I’ll figure it all out eventually (combinations of VDPAU_DRIVER, and seven other environment variables, and maybe an xorg.conf.d/ file with a hidden Screen, not sure yet!).

    Since you package Steam, and you also (in a more Fedora-like way) package the nvidia driver, I think it might be a good idea to stop using the RPMFusion nvidia driver and start using yours.

    Do I need to set repo-priority or anything special to combine your repo with RPMFusion?

    1. Nvidia packages are just conflicting with the RPMFusion ones, while the multimedia repository is not compatible at all with RPMFusion.

  2. Hi, is kernel 5.8 already supported? i stuck at boot screen at ‘starting gnome display manager line’. Currently im using fedora 32 kernel 5.7.17 and its running fine. But i got stuck when im trying to boot using kernel

  3. Thanks for packaging all of the nvidia + cuda libs+tools!

    The latest version from the nvidia website is cuda-11, yet the repository you provide here is at cuda-10.2 – is there a timeline for when a update is being release for fedora 32 (if at all)?

    Much appreciated!

  4. hello,

    I have an error when updating the latest nvidia 450.57 drivers sous fedora 32 (kde) kernel 5.7.9

    Mise à jour:
    akmod-nvidia x86_64 3:450.57-1.fc32 fedora-nvidia 11 M
    Installation des dépendances:
    nvidia-kmod-common noarch 3:450.57-1.fc32 fedora-nvidia 12 k
    Ignorer les paquets en conflit :
    (ajouter « –best –allowerasing » à la ligne de commande pour forcer leur mise à niveau):
    nvidia-driver x86_64 3:450.57-1.fc32 fedora-nvidia 2.6 M
    nvidia-driver-cuda x86_64 3:450.57-1.fc32 fedora-nvidia 357 k
    Ignorer les paquets ayant des dépendances cassées :
    nvidia-persistenced x86_64 3:450.57-1.fc32 fedora-nvidia 38 k
    nvidia-settings x86_64 3:450.57-1.fc32 fedora-nvidia 809 k

    Thank You

  5. I installed updates this weekend, including the nvidia drivers -> 3.450, however compositing is broken now. I can disable it, but things are behaving well. (I am on Fedora 31 KDE spin)
    Is anyone aware of issues with this latest version?

  6. Thank you very much for providing nvidia drivers for fedora.
    There is different error trying to install on fedora 32

    $ sudo dnf install nvidia-driver
    Última verificação de data de vencimento de metadados: 0:45:52 atrás em sáb 04 jul 2020 12:53:58.
    Erro:
    Problema: conflicting requests
    – nothing provides nvidia-kmod-common = 3:440.100 needed by nvidia-driver-3:440.100-1.fc32.x86_64
    (try to add ‘–skip-broken’ to skip uninstallable packages)

    Any idea on how to install ?

    1. Answering my own question about missing nvidia-kmod-common. At first I installed only the “fedora-multimedia” repo, but the “fedora-nvidia” should also be installed. Thanks negativo17, for providing these packages.

  7. Fresh installation of Fedora 32, installed drivers from nvidia-driver. After reboot I got 1024×768 resolution and no nvidia support. When I try to sudo modprobe nvidia, I get operation not permitted:

    [root@localhost ~]# modprobe nvidia -vv
    modprobe: INFO: custom logging function 0x555c21fc9020 registered
    insmod /lib/modules/5.6.13-300.fc32.x86_64/extra/nvidia/nvidia.ko.xz NVreg_DynamicPowerManagement=0x02
    modprobe: INFO: Failed to insert module '/lib/modules/5.6.13-300.fc32.x86_64/extra/nvidia/nvidia.ko.xz': Operation not permitted
    modprobe: ERROR: could not insert 'nvidia': Operation not permitted
    modprobe: INFO: context 0x555c231714f0 released

  8. Hi, thank you for your efforts to maintain those packages. Fedora already the xorg packages patched for use optimus technoloy with nvidia propietary driver. Could you have a xorg patched version on your el8 repository to use optmus in centos as well as we use on fedora?

    1. Hi, Nvidia drivers check for X.org server 1.20.7+ to enable the additional bits required for PRIME Display Offload Sink support and GPU screens. Not a big problem per se, as this requires just an updated X.org server. Unfortunately:

      • Due to the latest policy of not supporting all subpackages, the egl-wayland-devel packages are not available (try to download one here: https://koji.mbox.centos.org/koji/buildinfo?buildID=7013)
      • Rebuilding the CentOS/RHEL 8 egl-wayland package would be fine as well, but unfortunately this does not build anymore due to some compilation issues, as the dependent packages were originally at another version

      I would suggest on desktops to update to CentOS 8 Stream, in there you already have X.org server 1.20.8: http://mirror.centos.org/centos/8-stream/AppStream/x86_64/os/Packages/

      1 – Install this: http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/centos-release-stream-8.3-3.2006.0.1.el8.x86_64.rpm
      2 – Update system & reboot

    1. After spending a day on the phone with GoDaddy, they informed me that they have an issue with the shared hosting and they can not update certificates on the various instances. According to them they had this issue on the past 3 weeks!

      Of course nobody bothered to tell me and now the certificate is expired and I have no way of fixing it. I will move everything to a separate hosting provider as soon as I can, starting from the repositories.

      Of course now it will be a dedicated box, so the cost will not be the same. Any donation is of course accepted.

  9. @SLAANESH

    Heads up!

    I am getting an expired certificate when using your repos and this website.

    Love your work and thank you!

    1. After spending a day on the phone with GoDaddy, they informed me that they have an issue with the shared hosting and they can not update certificates on the various instances. According to them they had this issue on the past 3 weeks!

      Of course nobody bothered to tell me and now the certificate is expired and I have no way of fixing it. I will move everything to a separate hosting provider as soon as I can, starting from the repositories.

      Of course now it will be a dedicated box, so the cost will not be the same. Any donation is of course accepted.

  10. Thank you for work with Nvidia and Fedora! Very much appreciated!

    Any idea when kernel 5.6.x support will be available? Fedora 31 moved from 5.5.x to 5.6.x and there no matching packages in the repo.

    Simultaneously Fedora 32 was released and I imagine it has the same issue??

    The newest packages in the repos are from April 9th, 2020.

    1. Hi, there is no binary kernel module being built for Fedora, that is only for kABI distributions like CentOS/RHEL. Either use akmod-nvidia or dkms-nvidia. Driver version 440.82 works fine with kernel 5.6.
      Just install the driver and you will be fine.

  11. Is /usr/lib64/nvidia/libOpenCL.so still a thing? I’m on Fedora 30, and am pretty sure it was here last June. Even “dnf provides” hasn’t a clue. Thanks!

    1. Can you check if you have both dkms and akmods installed? dkms is the one installing the kernel modules under weak-updates. If so, please remove dkms.

  12. Thanks for the great job.

    When i run the following commands:

    1/ dnf -y install nvidia-driver akmod-nvidia
    2/ dnf/yum -y install nvidia-driver nvidia-driver-cuda cuda-devel

    Resutls : Everything is ok.

    When i run the sample : ./deviceQuery , the result is :

    CUDA Device Query (Runtime API) version (CUDART static linking)
    cudaGetDeviceCount returned 100
    -> no CUDA-capable device is detected
    Result = FAIL

    My Laptop configuration is :
    RAM : 8 Go
    i7
    Fedora 31
    Nvidia Geforce GTX 950M

    Can you please help?

  13. On:

    lspci |grep -E “VGA|3D”

    01:00.0 VGA compatible controller: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] (rev a1)

    uname -a

    Linux localhost 5.3.11-300.fc31.x86_64 #1 SMP Tue Nov 12 19:08:07 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

    installed without errors:
    dnf install nvidia-driver nvidia-driver-libs.i686
    dnf -y install nvidia-driver nvidia-settings

    but no nvidia driver(after reboot):
    dm-x-session[2277]: (==) Matched nouveau as autoconfigured driver 0
    dm-x-session[2277]: (==) Matched nv as autoconfigured driver 1
    dm-x-session[2277]: (==) Matched modesetting as autoconfigured driver 2
    dm-x-session[2277]: (==) Matched fbdev as autoconfigured driver 3
    dm-x-session[2277]: (==) Matched vesa as autoconfigured driver 4
    dm-x-session[2277]: (==) Assigned the driver to the xf86ConfigLayout
    dm-x-session[2277]: (II) LoadModule: “nouveau”
    dm-x-session[2277]: (WW) Warning, couldn’t open module nouveau
    dm-x-session[2277]: (EE) Failed to load module “nouveau” (module does not exist, 0)
    dm-x-session[2277]: (II) LoadModule: “nv”
    dm-x-session[2277]: (WW) Warning, couldn’t open module nv
    dm-x-session[2277]: (EE) Failed to load module “nv” (module does not exist, 0)

    modprobe nvidia

    modprobe: FATAL: Module nvidia not found in directory /lib/modules/5.3.11-300.fc31.x86_64

    find /lib/modules -name “nvidia*”

    (only ethernet related)

    After this tried:
    dnf -y install nvidia-driver dkms-nvidia

    but:

    ls /var/lib/dkms/nvidia/440.44/build/

    (nothing)

    What am I missing / what should I do to get this working?

  14. Anyone know if Ray Tracing is supported on Nvidia 20xx series? I can’t seem to get either Vulkan, RT, or both working. Not sure what’s going on.

    1. “Anyone know if Ray Tracing is supported on Nvidia 20xx series? I can’t seem to get either Vulkan, RT, or both working. Not sure what’s going on.”

      To update that. These drivers have Vulkan problems. Not Negativo’s fault, but Nvidia’s. I don’t recommend these drivers for Vulkan or Ray Tracing at this time.

  15. Never mind my last comment about how to fix it. Don’t know what happened there. For Fedora 31 there’s no 32bit before 440.xx. So kind of a bad situation until the problems are fixed.

  16. I’m getting the following dnf upgrade errors on my Fedora 31 installation. Looks like some conflicts with the rpmfusion packages.

    [root@pulsar download]# dnf -y update
    Last metadata expiration check: 0:58:03 ago on Sat 09 Nov 2019 01:48:25 PM MST.
    Dependencies resolved.

    Problem: package nvidia-driver-3:435.21-2.fc31.x86_64 conflicts with xorg-x11-drv-nvidia provided by xorg-x11-drv-nvidia-3:440.31-1.fc31.x86_64
    – package akmod-nvidia-3:440.31-1.fc31.x86_64 requires nvidia-kmod-common >= 3:440.31, but none of the providers can be installed
    – cannot install the best update candidate for package nvidia-driver-3:435.21-2.fc31.x86_64

    – cannot install the best update candidate for package akmod-nvidia-3:435.21-1.fc31.x86_64

    Package Architecture Version Repository Size

    Skipping packages with conflicts:
    (add ‘–best –allowerasing’ to command line to force their upgrade):
    xorg-x11-drv-nvidia x86_64 3:440.31-1.fc31 rpmfusion-nonfree-updates 2.4 M
    Skipping packages with broken dependencies:
    akmod-nvidia x86_64 3:440.31-1.fc31 rpmfusion-nonfree-updates 26 k

    Transaction Summary

    Skip 2 Packages

    1. I ran into a similar problem as the 440.36 drivers were made available in RPMFusion.

      I fixed it by excluding all nvidia packages from RPMFusion non-free repository.

      This can be done by editing “/etc/yum.repos.d/rpmfusion-nonfree-updates.repo” and placing “exclude=nvidia” on a new-line within the “[rpmfusion-nonfree-updates]” of the file. Same can be done with “/etc/yum.repos.d/rpmfusion-nonfree.repo” and “[rpmfusion-nonfree]”.

    2. Late reply for you but might be helpful for someone else:

      Remove nvidia packages installed from rpmfusion.
      Add exclude option to rpmfusion-nonfree and rpmfusion-nonfree-updates repos config files in /etc/yum.repos.d:
      exclude=nvidia* akmod-nvidia xorg-x11-drv-nvidia*

      That option is per-repository and is a sanity saver when incompatible builds are available in multiple repos:

  17. Hi,

    sorry for asking here directly (I also asked on the Fedora forum but figured you might already know what might be the issue?)

    After upgrade to Fedora 31, I have the following problems:

    cursor blinks for a long time (5 mins?) before entering graphical mode
    nvidia-smi fails
    external monitor is not detected

    I tried commenting

    options nvidia NVreg_DynamicPowerManagement=0x02

    as recommended above (I too have a Thinkpad P51) but it didn’t help.

    Installed packages are

    nvidia-driver-cuda-435.21-2.fc31.x86_64
    dkms-nvidia-435.21-1.fc31.x86_64
    nvidia-driver-libs-435.21-2.fc31.x86_64
    nvidia-driver-435.21-2.fc31.x86_64
    nvidia-driver-NVML-435.21-2.fc31.x86_64
    nvidia-modprobe-435.21-1.fc31.x86_64
    nvidia-persistenced-435.21-1.fc31.x86_64
    nvidia-settings-435.21-1.fc31.x86_64
    nvidia-kmod-common-435.21-3.fc31.noarch
    nvidia-driver-cuda-libs-435.21-2.fc31.x86_64
    nvidia-libXNVCtrl-435.21-1.fc31.x86_64

    Many thanks for any help!!

  18. Hey, any chance for 32 bit libraries for Fedora 31 any time soon? I totally didn’t think that this might not be available when I updated today. Thanks for your work!

    1. They are now available. Had some personal issues to deal with (new kid and work!) and some hardware failures that took me much longer than planned to fix.

    1. I had the same problem too. Ended up installing F30 instead. I’d like to know what needs to be done to get 32-bit libraries installed. Steam won’t run without them.

      1. They are now available. Had some personal issues to deal with (new kid and work!) and some hardware failures that took me much longer than planned to fix.

    2. They are now available. Had some personal issues to deal with (new kid and work!) and some hardware failures that took me much longer than planned to fix.

  19. 435.21 seems to have broken fan control on secondary GPUs with no physical monitor attached. I was able to use the xorg.conf trick to make it think there was a screen attached previosuly, but this no longer works. I really hope this is a temporary regression, as we need the newer driver for DCC applications, but cannot use it without fan control for GPU rendering.

    Has anyone else come across this?

    1. I’ve done some research and it appears to be from the new Prime offloading business, which I do not need nor want. ugh.

    2. I got a response from NVidia, and it seems that the /etc/X11/xorg.conf.d/10-nvidia.conf enables prime offloading which causes issues with setup such as mine. It might be worth making a note that this is being done in case anyone else runs into the same issue.

  20. I’ve since added a bug report to the github repository: https://github.com/negativo17/nvidia-driver/issues/85

    On second thought, the issue might not even be related to those missing X.org commits. On the nvidia driver forum there is a thread about a similar issue: https://devtalk.nvidia.com/default/topic/1062922/linux/435-21-does-not-render-to-my-external-monitor/
    – it appears that the driver now defaults to render offload mode, but then it can’t use external monitors (at least on this laptop, where the HDMI output is connected to the dGPU only). I needed to put Option “PrimaryGPU” “yes” in the nvidia.conf config file in xorg.conf.d to switch it back to the old mode.

  21. It appears that the new 435.21 driver breaks hybrid graphics mode. I’m using a Thinkpad P50 with the laptop’s own display driven by the Intel chip and an external monitor connected to the HDMI output driven by the Nvidia GPU.
    Until yesterday with the previous (430.something) driver it worked nicely, today, however, the external display fails to wake up from power saving mode, and there are several other signs that the nvidia driver is somehow broken (xrandr doesn’t even report any monitors other than the laptop’s own, and nvidia-settings doesn’t start properly: when ran from the command line, it prints an error message about “ERROR: nvidia-settings could not find the registry key file.”.

    When I change the display setting in the BIOS to “discrete graphics”, which completely disables the Intel internal GPU, I’m able to use both displays, but then I don’t have graphics mode during the boot process, and in any case, I’m reluctant to switch to that mode, since there are several reports on the internet that doing so might brick the laptop.

    I’ll try downgrading the driver for now.

    1. Can you try to to comment the following line:

      # cat /usr/lib/modprobe.d/nvidia.conf | grep NV
      options nvidia NVreg_DynamicPowerManagement=0x02

      Then running depmod -a and rebooting? This makes sure the Nvidia card does not power off completely, it can just reach the lowest power state.

      Thanks.

  22. After the update to 435.17 driver and xorg 1.20.5-7.fc30@fedora-nvidia, as I understood, the prime offloading in xorg is enabled by default and system uses integrated intel gpu. But it looks like it never utilizes nvidia gpu.
    (@slaanesh, as an aside, why don’t you package nvidia-smi tool that comes with the nvidia installer?)

    Is it a misconfiguration/missing configuration, or do I need to provide extra config?

    Can you provide any pointers to resources where I can learn how this whole GPU system works? I am poking at things at random here and I don’t like that.

    Some basic info about system: https://gist.github.com/Xerkus/660574e28c7a9990318c5bf0dc79363b

    1. After the update to 435.17 driver and xorg 1.20.5-7.fc30@fedora-nvidia, as I understood, the prime offloading in xorg is enabled by default and system uses integrated intel gpu. But it looks like it never utilizes nvidia gpu.

      You have to start programs with the appropriate variables, for ex.:

      $ glxinfo | grep “OpenGL.*version string”
      OpenGL core profile version string: 4.5 (Core Profile) Mesa 19.1.5
      OpenGL core profile shading language version string: 4.50
      OpenGL version string: 3.0 Mesa 19.1.5
      OpenGL shading language version string: 1.30
      OpenGL ES profile version string: OpenGL ES 3.2 Mesa 19.1.5
      OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

      $ __NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia glxinfo | grep “OpenGL.*version string”
      OpenGL core profile version string: 4.6.0 NVIDIA 435.17
      OpenGL core profile shading language version string: 4.60 NVIDIA
      OpenGL version string: 4.6.0 NVIDIA 435.17
      OpenGL shading language version string: 4.60 NVIDIA
      OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 435.17
      OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

      All the dual GPU stuff is here: http://download.nvidia.com/XFree86/Linux-x86_64/435.21/README/primerenderoffload.html
      I will try to make a blog post, but I’m overwhelmed by work.

      (@slaanesh, as an aside, why don’t you package nvidia-smi tool that comes with the nvidia installer?)

      Just search for it, it’s packaged:

      sudo dnf provides \*bin/nvidia-smi

      1. I see. I had a wrong expectation of it working automagically. Kinda how it was for a while with the aim at the general user: just install nvidia driver packages and it is ready to go.
        Is it something that we can expect in the future when this new feature is mainlined?

        You have to start programs with the appropriate variables

        It is not too friendly for the users like my mom. She won’t know anything about gpu, much less any drivers or starting apps with different env variables.
        I found this tool https://github.com/wildtruc/nvidia-prime-select to select default active gpu. Would you consider it an appropriate solution? It seems to be using this exact new driver feature under the hood too, somehow.

        Just search for it, it’s packaged:
        sudo dnf provides *bin/nvidia-smi

        Oh, indeed. There was a bunch of cuda packages from rpmfusion and I didn’t notice one of them was actually from your repo. My bad.

        but I’m overwhelmed by work

        Thank you for finding time for this. You made my fedora experience amazing for years.

        1. I see. I had a wrong expectation of it working automagically. Kinda how it was for a while with the aim at the general user: just install nvidia driver packages and it is ready to go.
          Is it something that we can expect in the future when this new feature is mainlined?

          It is ready to go. Add the drivers, and the discrete GPU is fine and powered off until you want to use it. That’s my definition of “just works”

          It is not too friendly for the users like my mom. She won’t know anything about gpu, much less any drivers or starting apps with different env variables.

          That’s exactly the point, why should she care about the GPU then?

          Anyway you can just add this to your Bash profile or the system wide profile:

          alias nvidia-run='__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia "$@"'

          Then you can just run, for example nvidia-run doom3bfg etc.

          I found this tool https://github.com/wildtruc/nvidia-prime-select to select default active gpu. Would you consider it an appropriate solution? It seems to be using this exact new driver feature under the hood too, somehow.

          Don’t use it, it generates a lot of X config that is actually not needed. If you have one of the old laptops where some output is connected to the Nvidia GPU, the GPU powers on accordingly, so also in this case there is nothing to do.

  23. When I update to 435.21 I lose audio device GM204 ( Audio from the gpu through HDMI ) and have to downgrade back to 430.40 Is there anything I can do to fix that?

    1. 430.40 is already in the repositories since the end of July. 435.21 soon, along with the patched X server for PRIME offload and updated X server.

  24. Thanks for your nvidia drivers. Have you ever considered providing the latest “nouveau” driver, the one supplied with the fedora 30 and 31 beta is still 1.0.15 (last 1.0.16)?

  25. After upgrading from 418.56 to 430.14 on Fedora 30, mpv and ffmpeg segfault, when trying to use CUDA for hardware acceleration. Both stack traces show the fault occurring in /lib64/libnvcuvid.so.1. It was working flawlessly with the older drivers.

  26. Is there a chance that we can have different versions of CUDA installed in parallel? It would be nice to be able to install them from the repos.

    1. Was planning to start using the modules in Fedora and RHEL 8, but had no time to work on it. For now, just use Nvidia provided packages which allow parallel installation.

  27. Unable to access repo on Fedora 27, Getting error:

    Failed to synchronize cache for repo ‘fedora-nvidia’, disabling.

    Any advice?

Leave a Reply to Claudio Miranda Cancel reply