CUDA 8, cuDNN, Nvidia drivers and GNOME Software metadata

GNOME software integration

The Nvidia driver repository has been updated with AppStream metadata. From Fedora 25 onward, you will be able to search for Nvidia, CUDA, GeForce or Quadro to make the driver, control panel and other programs appear in the Gnome Software window.

As far as I know, this should be enabled by default on Fedora 25.

gnome-software-nvidia

Thanks to Richard Hughes for helping out with the metadata.

I require proper 16:10 aspect ratio pictures for both NSight and the Visual Profiler running on Fedora, so if you want to contribute just drop me an email or open an issue on the CUDA package on GitHub.

Changes to the Nvidia driver packaging

The Nvidia driver can now be installed without nvidia-settings (the control panel utility) as requested by Red Hat, in preparation for the Gnome software integration. This means the dependencies have been reversed, and that to install the driver and the control panel you need to install nvidia-settings or the driver and nvidia-settings:

dnf/yum -y install nvidia-settings kernel-devel

The libglvnd package has been updated to the latest snapshot and now features all the changes that have been introduced by Adam Jackson for the Mesa GLVND integration in Fedora 25. This means that while installing you will be prompted to install/upgrade smaller packages that contain a subset of the libglvnd libraries, this includes EGL support for the recently released beta drivers version 375.10. For anything lower than 375.10 (so Fedora 23-24 and CentOS/RHEL 6/7 at the moment of writing this) Nvidia’s last official note on EGL is:

“libEGL.so.1, while not a proper GLVND library, depends upon the GLVND infrastructure for proper functionality. Therefore, any driver package which aims to support NVIDIA EGL must provide the GLVND libraries […]”

So for now, in Fedora 23, 24 and CentOS/RHEL 6/7:

$ rpm -q --requires nvidia-driver-libs.x86_64 | grep libglvnd
libglvnd-gles(x86-64) >= 0.1.1
libglvnd-glx(x86-64) >= 0.1.1
libglvnd-opengl(x86-64) >= 0.1.1
$ rpm -q --conflicts nvidia-driver-libs.x86_64 | grep libglvnd
libglvnd-egl(x86-64) >= 0.1.1

And for Fedora 25:

$ rpm -q --requires nvidia-driver-libs.x86_64 | grep libglvnd
libglvnd-egl(x86-64) >= 0.2
libglvnd-gles(x86-64) >= 0.2
libglvnd-glx(x86-64) >= 0.2
libglvnd-opengl(x86-64) >= 0.2

Not a big deal. This accommodates the ongoing modularization in Mesa but still preserves the original EGL libraries from Nvidia. The upgrade should be transparent and you should not notice any difference except some smaller packages being installed.

vulkan_500px_june16Vulkan is now part of Fedora, so on supported Fedora releases, the Vulkan loader and libraries can be installed and you do not need to do anything to enable support in the drivers. CentOS and Red Hat Enterprise Linux do not have Vulkan yet. I’m not sure if it’s worth installing it by default along with the drivers, though.

Let’s assume you have a freshly installed Fedora 25 system with a recent Nvidia GPU and you want to:

  • Install the driver for gaming
  • Play Vulkan enabled games
  • Want to be comfortable with the control panel
  • Play 32 bit games on a 64 bit system
  • Play 32 bit Vulkan games on a 64 bit system
$ sudo dnf install nvidia-settings kernel-devel dkms-nvidia vulkan.i686 nvidia-driver-libs.i686
Last metadata expiration check: 0:33:49 ago on Mon Oct 24 14:14:30 2016.
Dependencies resolved.
=====================================================================================
 Package            Arch   Version                             Repository       Size
=====================================================================================
Installing:
 dkms-nvidia        x86_64 2:375.10-1.fc25                     fedora-nvidia   6.4 M
 libglvnd           i686   1:0.2.999-4.20161025git28867bb.fc25 fedora-nvidia   103 k
 libglvnd           x86_64 1:0.2.999-4.20161025git28867bb.fc25 fedora-nvidia   105 k
 libglvnd-egl       i686   1:0.2.999-4.20161025git28867bb.fc25 fedora-nvidia    44 k
 libglvnd-egl       x86_64 1:0.2.999-4.20161025git28867bb.fc25 fedora-nvidia    42 k
 libglvnd-gles      i686   1:0.2.999-4.20161025git28867bb.fc25 fedora-nvidia    29 k
 libglvnd-gles      x86_64 1:0.2.999-4.20161025git28867bb.fc25 fedora-nvidia    28 k
 libglvnd-glx       i686   1:0.2.999-4.20161025git28867bb.fc25 fedora-nvidia   114 k
 libglvnd-glx       x86_64 1:0.2.999-4.20161025git28867bb.fc25 fedora-nvidia   110 k
 libglvnd-opengl    i686   1:0.2.999-4.20161025git28867bb.fc25 fedora-nvidia    39 k
 libglvnd-opengl    x86_64 1:0.2.999-4.20161025git28867bb.fc25 fedora-nvidia    38 k
 libva-vdpau-driver x86_64 0.7.4-14.fc24                       fedora           61 k
 libvdpau           i686   1.1.1-3.fc24                        fedora           35 k
 nvidia-driver      x86_64 2:375.10-3.fc25                     fedora-nvidia   3.1 M
 nvidia-driver-NVML x86_64 2:375.10-3.fc25                     fedora-nvidia   397 k
 nvidia-driver-libs i686   2:375.10-3.fc25                     fedora-nvidia    15 M
 nvidia-driver-libs x86_64 2:375.10-3.fc25                     fedora-nvidia    14 M
 nvidia-libXNVCtrl  x86_64 2:375.10-1.fc25                     fedora-nvidia    26 k
 nvidia-settings    x86_64 2:375.10-1.fc25                     fedora-nvidia   935 k
 vulkan             i686   1.0.30.0-1.fc25                     updates-testing 1.5 M
 vulkan-filesystem  noarch 1.0.30.0-1.fc25                     updates-testing 8.0 k
 
Transaction Summary
=====================================================================================
Install  21 Packages
 
Total download size: 42 M
Installed size: 178 M
Is this ok [y/N]:

Note that the requirement on kernel-devel is still required as otherwise the package kernel-debug-devel is pulled in automatically in place of the normal non-debug package. There is bug opened on dnf/libsolv for this.

Changes to CUDA packaging

The CUDA packages hosted on the Nvidia repository are split into multiple subpackages, based on the library. For each library, you have the corresponding devel subpackage with the headers, the unversioned library symlink and the static library. Here, they were divided in one libs, one big extra-libs, one static and one devel subpackage for everything. Since I’m planning to enable CUDA/NVCUVID encoding/decoding in FFmpeg (I’m actually waiting to the dynamic loader patches to land in the 3.2 branch before enabling that) there should be a way to install just what is required by those functions and not the whole CUDA toolkit set of libraries.

So now, all the libraries are split into subpackages, much like in the original Nvidia CUDA repository. This allows you to install and build software relying on specific components without the need to install all the CUDA toolkit just to satisfy a library dependency. With the new packaging organization, the original cuda-devel and cuda-extra-libs will pull in all the specific subpackages giving you the same situation you are accustomed to. Also, for the same reason, static libraries have been included in each respective devel subpackage.

Example, just with the basic tools:

$ sudo dnf install cuda
Last metadata expiration check: 0:00:20 ago on Sun Oct 23 13:11:01 2016.
Dependencies resolved.
================================================================================
 Package           Arch         Version               Repository           Size
================================================================================
Installing:
 cuda              x86_64       1:8.0.44-4.fc24       fedora-nvidia        95 M
 cuda-cufft        x86_64       1:8.0.44-4.fc24       fedora-nvidia        97 M
 cuda-curand       x86_64       1:8.0.44-4.fc24       fedora-nvidia        38 M
 cuda-libs         x86_64       1:8.0.44-4.fc24       fedora-nvidia       6.4 M
 
Transaction Summary
================================================================================
Install  4 Packages
 
Total size: 236 M
Installed size: 469 M
Is this ok [y/N]:

The basic tools along with all the libraries (note that the NVML headers are included):

$ sudo dnf install cuda-devel
Last metadata expiration check: 0:10:00 ago on Sun Oct 23 13:11:01 2016.
Dependencies resolved.
================================================================================
 Package                 Arch       Version             Repository         Size
================================================================================
Installing:
 cuda                    x86_64     1:8.0.44-4.fc24     fedora-nvidia      95 M
 cuda-cublas             x86_64     1:8.0.44-4.fc24     fedora-nvidia      21 M
 cuda-cublas-devel       x86_64     1:8.0.44-4.fc24     fedora-nvidia      38 M
 cuda-cudart             x86_64     1:8.0.44-4.fc24     fedora-nvidia     131 k
 cuda-cudart-devel       x86_64     1:8.0.44-4.fc24     fedora-nvidia     659 k
 cuda-cufft              x86_64     1:8.0.44-4.fc24     fedora-nvidia      97 M
 cuda-cufft-devel        x86_64     1:8.0.44-4.fc24     fedora-nvidia      73 M
 cuda-cupti              x86_64     1:8.0.44-4.fc24     fedora-nvidia     1.2 M
 cuda-cupti-devel        x86_64     1:8.0.44-4.fc24     fedora-nvidia     213 k
 cuda-curand             x86_64     1:8.0.44-4.fc24     fedora-nvidia      38 M
 cuda-curand-devel       x86_64     1:8.0.44-4.fc24     fedora-nvidia      60 M
 cuda-cusolver           x86_64     1:8.0.44-4.fc24     fedora-nvidia      23 M
 cuda-cusolver-devel     x86_64     1:8.0.44-4.fc24     fedora-nvidia     4.1 M
 cuda-cusparse           x86_64     1:8.0.44-4.fc24     fedora-nvidia      23 M
 cuda-cusparse-devel     x86_64     1:8.0.44-4.fc24     fedora-nvidia      23 M
 cuda-devel              x86_64     1:8.0.44-4.fc24     fedora-nvidia     1.6 M
 cuda-libs               x86_64     1:8.0.44-4.fc24     fedora-nvidia     6.4 M
 cuda-npp                x86_64     1:8.0.44-4.fc24     fedora-nvidia      91 M
 cuda-npp-devel          x86_64     1:8.0.44-4.fc24     fedora-nvidia      47 M
 cuda-nvgraph            x86_64     1:8.0.44-4.fc24     fedora-nvidia     4.6 M
 cuda-nvgraph-devel      x86_64     1:8.0.44-4.fc24     fedora-nvidia      12 k
 cuda-nvml-devel         x86_64     1:8.0.44-4.fc24     fedora-nvidia      41 k
 cuda-nvrtc              x86_64     1:8.0.44-4.fc24     fedora-nvidia     6.6 M
 cuda-nvrtc-devel        x86_64     1:8.0.44-4.fc24     fedora-nvidia      16 k
 
Transaction Summary
================================================================================
Install  24 Packages
 
Total size: 655 M
Installed size: 1.4 G
Is this ok [y/N]:

The nvidia-driver-NVML-devel package, which was including the NVML header (for libnvidia-ml.so) has now been made obsolete by the new headers, which are now part of CUDA 8. So the cuda-nvml-devel package will take care of that. Again, this is the same as in the Nvidia repository. Everything that was requiring the NVML header now refers to that package instead of the previous one. I will leave it for a few releases like that and then I will remove the Obsolete/Provides tags from the various SPEC files.

The header is also required for building the latest nvidia-settings from the 375.10 source, this has been taken into account making the CUDA package buildable on i686 but generating only the cuda-nvml-devel subpackage.

Extra stuff

In addition to the libraries bundled in the CUDA toolkit, also the cuDNN library for distributed neural networks is included in the repository.

As usual, you are welcome to open bugs / request stuff / comment on the GitHub repositories.

18 thoughts on “CUDA 8, cuDNN, Nvidia drivers and GNOME Software metadata

  1. Hi, I have a problem with nvidia-settings on fedora 24. It’s the first time I try nvidia-drivers from this repository.
    When I try start nvidia-settings this happens:
    nvidia-settings: libXNVCtrlAttributes/NvCtrlAttributesNvml.c:1690: NvCtrlNvmlGetValidAttributeValues: Assertion `ret2 == NvCtrlAttributeNotAvailable’ failed.

    I installed nvidia-driver* nvidia-persistenced nvidia-settings nvidia-xconfig (and some other packages for cuda). All is working fine except nvidia-settings.

    It is a known problem or am I missing something?

  2. Could you please help me with cuda setup on F24 with NVidia 340xx drivers (GTX260)?
    I have the following packages installed:

    rpm -qa | grep cuda
    cuda-8.0.44-6.fc24.x86_64
    nvidia-driver-cuda-libs-340.98-1.fc24.x86_64
    cuda-libs-8.0.44-6.fc24.x86_64

    rpm -qa | grep nvidia
    nvidia-driver-340.98-1.fc24.x86_64
    nvidia-xconfig-340.98-1.fc24.x86_64
    kmod-nvidia-4.8.8-200.fc24.x86_64-340.98-1.fc24.x86_64
    nvidia-driver-NVML-340.98-1.fc24.x86_64
    nvidia-driver-cuda-libs-340.98-1.fc24.x86_64
    nvidia-libXNVCtrl-340.98-1.fc24.x86_64
    nvidia-driver-libs-340.98-1.fc24.x86_64
    nvidia-settings-340.98-1.fc24.x86_64
    akmod-nvidia-340.98-1.fc24.x86_64

    How do I verify that cuda is working?
    If I try installing cuda-devel I get nvidia-driver-NVML 2:375.20-1.fc24 dependency.
    Is cuda supported on 340.xx?
    Thanks,
    Vax

  3. Having installed all the cuda rpms, I am unable to build the samples. nvcc returns this error: nvcc fatal : Path to libdevice library not specified. A “find / -name “libdevice* finds nothing except the pdf and the html folder for the libdevice-users-guide. What rpm would normally have these files? Have not figured out yet where my screw-up is.
    Thanks!

    1. Sorry for the late reply:

      $ nvcc --help | grep -A3 libdevice-directory
      --libdevice-directory <directory>          (-ldir)                           
              Specify the directory that contains the libdevice library files when option
              '--dont-use-profile' is used.  Libdevice library files are located in the
              'nvvm/libdevice' directory in the CUDA toolkit.
      $ rpm -ql cuda | grep libdevice
      /usr/share/cuda/libdevice.compute_20.10.bc
      /usr/share/cuda/libdevice.compute_30.10.bc
      /usr/share/cuda/libdevice.compute_35.10.bc
      /usr/share/cuda/libdevice.compute_50.10.bc
      1. Thank you Slaanesh! A “dnf reinstall cuda” did the trick. Not sure what happened prior. OK, I can’t resist quoting Lord of the Rings: “A Wizard is never late…”

    1. Would love to, but I don’t have any hardware nor enough systems to cross-check everything (various Fedora releases and various RHEL/CentOS releases available).

  4. With Fedora 25 and your driver I get the “incompatible ABI” message. Anything you can do about that? Or anything I’m doing wrong?

    1. There’s no Nvidia driver that supports X server version 1.19, that’s part of their policy of not supporting unreleased kernels and X servers. They’re pretty fast in adding support once they are released, though. And most of the time it just works with IgnoreABI.

      I’ve pushed an updated that enables the flag also on Fedora 25.

      1. Thanks for that. Indeed I have by now tried using that flag, but X just segfaults. So I guess I’ll have to wait for X.org to do their final release.

  5. It’s been a while that I am having issues with the kernel/nvidia drivers. After these changes I have more dependency problems. Is there anything I can do to resolve this on my end?

    Packages skipped because of dependency problems:
    kernel-3.10.0-327.36.3.el7.x86_64 from updates
    kmod-20-8.el7_2.x86_64 from updates
    2:kmod-nvidia-367.57-1.el7.x86_64 from epel-nvidia
    1:libglvnd-gles-0.2.999-4.20161025git28867bb.el7.i686 from epel-nvidia
    1:libglvnd-gles-0.2.999-4.20161025git28867bb.el7.x86_64 from epel-nvidia
    1:libglvnd-glx-0.2.999-4.20161025git28867bb.el7.i686 from epel-nvidia
    1:libglvnd-glx-0.2.999-4.20161025git28867bb.el7.x86_64 from epel-nvidia
    1:libglvnd-opengl-0.2.999-4.20161025git28867bb.el7.i686 from epel-nvidia
    1:libglvnd-opengl-0.2.999-4.20161025git28867bb.el7.x86_64 from epel-nvidia
    2:nvidia-driver-367.57-4.el7.x86_64 from epel-nvidia
    2:nvidia-driver-libs-367.57-4.el7.i686 from epel-nvidia
    2:nvidia-driver-libs-367.57-4.el7.x86_64 from epel-nvidia
    2:nvidia-libXNVCtrl-367.57-1.el7.x86_64 from epel-nvidia
    2:nvidia-settings-367.57-1.el7.x86_64 from epel-nvidia

    Thanks!

    1. There’s something weird with the kmod package. It has the same files inside but somehow it does not provide anymore /sbin/depmod PRIOR to installation, but once installed, everything is fine. Can you do one of the following?

      yum --exclude=kmod update

      and then update kmod.

      After that, you can update kmod through rpm or yum. I will rework kmodtool to look for other dependencies instead of /sbin/depmod once I’m back from holiday (wednesday).

  1. I did that and at least I can now use the system again (GNOME wasn’t even logging in due to an error) but the kmod package is refusing to update. Trying to update gives me this:

    Resolving Dependencies
    –> Running transaction check
    —> Package kmod.x86_64 0:20-5.el7 will be updated
    —> Package kmod.x86_64 0:20-8.el7_2 will be an update
    –> Processing Dependency: /sbin/depmod for package: 2:kmod-nvidia-367.57-1.el7.x86_64

    Packages skipped because of dependency problems:
    kmod-20-8.el7_2.x86_64 from updates

    Trying to remove it will remove so many dependencies…

    1. I’ve pushed an update to the kmod-nvidia package that takes into account the changes that have been put into kmod. So instead of searching for /sbin/depmod, which is available once installed but not available in the yum metadata for the same kmod package, it looks for module-init-tools which is an explicit provision in kmod.

      ================================================================================
       Package            Arch   Version                            Repository   Size
      ================================================================================
      Installing:
       nvidia-driver      x86_64 2:367.57-4.el7                     epel-nvidia 3.1 M
      Updating:
       kmod               x86_64 20-8.el7_2                         updates     114 k
      Installing for dependencies:
       kmod-nvidia        x86_64 2:367.57-2.el7                     epel-nvidia  11 M
       libglvnd           x86_64 1:0.2.999-4.20161025git28867bb.el7 epel-nvidia 101 k
       libglvnd-gles      x86_64 1:0.2.999-4.20161025git28867bb.el7 epel-nvidia  27 k
       libglvnd-glx       x86_64 1:0.2.999-4.20161025git28867bb.el7 epel-nvidia 120 k
       libglvnd-opengl    x86_64 1:0.2.999-4.20161025git28867bb.el7 epel-nvidia  40 k
       nvidia-driver-libs x86_64 2:367.57-4.el7                     epel-nvidia  14 M
       nvidia-libXNVCtrl  x86_64 2:367.57-1.el7                     epel-nvidia  22 k
       nvidia-settings    x86_64 2:367.57-1.el7                     epel-nvidia 938 k
       
      Transaction Summary
      ================================================================================
      Install  1 Package (+8 Dependent packages)
      Upgrade  1 Package
       
      Total download size: 29 M
      1. Wonderful! I can finally have all packages working.

        PS: No need to apologize! It should be me thanking for your work (so here it is… thanks!!)

  • Leave a Reply