Nvidia driver, CUDA tools and libraries

nvidia-settings
Oh no, another Nvidia driver repository? Why?

This driver reflects my personal view for the way the driver should be packaged for Fedora and CentOS/RHEL. It’s somewhat different from ELRepo repositories for RHEL/CentOS and from RPMFusion packages for Fedora.

Repository installation

To install the repository on a supported Fedora distribution, run as root the following command:

dnf config-manager --add-repo=http://negativo17.org/repos/fedora-nvidia.repo

To install the repository on CentOS/RHEL:

yum-config-manager --add-repo=http://negativo17.org/repos/epel-nvidia.repo

Starting from Fedora 25, the Nvidia software packages are available for installation by default also in Gnome Software.

gnome-software-nvidia

Please note that the driver will show up only if your system matches one of the PCI ID supported by the driver. Otherwise, only the other Nvidia programs (mostly for CUDA development) will show up in the software center.

What’s different?

First of all the packaging is a lot simplified; more stuff is compiled from source, smaller packages and more options. This packages try to comply as maximum to the Fedora Packaging Guidelines; which means they have debuginfo packages, default Fedora’s GCC compile time options (where possible) and standard locations for binaries, data and docs.

What follows below, is a detailed explanation of all the “differences” from the various Nvidia driver packages that I was able to spot on the web and a detailed description on how to install components, etc.

Nvidia drivers

Packaging

  • nvidia-settings, nvidia-persistenced, nvidia-xconfig and nvidia-modprobe are compiled from source.
  • All RPM filters except for GL and OpenCL libraries have been removed, so there is no weird dependency option in the SPEC file. RPM pulls in all correct requirements on its own. This is to avoid pulling in the Nvidia drivers instead of the Mesa libraries or in place of the new open source OpenCL support that’s in Fedora.
  • Simplified packaging with much simpler and readable SPEC file.
  • Dependency on libva-vdpau-driver. So in Totem, or any other libVA supported application you can benefit from VDPAU acceleration.
  • Sources are generated with a script and inserted individually in the various packages; so it can be easily reproduced just by changing the version and rerunning the script.
  • nvidia-xconfig is not required on anything that uses the modular X.org directives, as it writes too much in the configuration file (keyboards, monitors, etc.) and the required entries should be written in separate configuration files under /etc/X11/xorg.conf.d. The package is still available as it’s required to speed up some configuration like multi-monitor setups with SLI Mosaic enabled from the command line, but not installed by default.
  • The NVIDIA OpenGL-based Framebuffer Capture (NvFBCOpenGL) libraries (NvFBC and NvIFR) are private APIs that are only available to NVIDIA approved partners for use in remote graphics scenarios (i.e. Steam In-Home Streaming hardware encoding); so they are packaged in another small package called nvidia-driver-NvFBCOpenGL.
  • The nvidia-settings package now builds the external libXNVCtrl.so library that can be used to control the graphic cards through the NV-CONTROL extension. This library updates the old and obsolete one in Fedora based on drivers version 165.
  • Starting from version 343.13, the nvidia-settings binary is compiled with GTK3 instead of GTK2 on Fedora and RHEL/CentOS 7+.
  • The driver can be installed separately from the nvidia-settings utility, so if you simply want a working driver and do not care about details, your experience should be as close as possible to the one with open source drivers.

Versioning

  • ELRepo ships 32 bit compatibility libraries in a separate package with x86_64 as the architecture and “32bit” in the name. 32 bit libraries should be like in RPMFusion, with an i686 package installable in parallel with the x86_64 one. There are no other packages in the distribution that are built for x86_64, with “32bit” in their name that contain i686 binaries (!), so Nvidia drivers should not be an exception. So no separate “32bit.x86_64” package for 32 bit libraries also on CentOS/RHEL; just install nvidia-driver-libs.i686.
  • Versions are not hidden; all packages have the same driver version.
  • No alternatives system, only the latest version which integrates CUDA support is available. For older releases nouveau works great; and anything below a GeForce 8xxx it’s in my opinion too low end to play anything modern. And Quake 3 and Doom 3 work greatly with nouveau, so that’s not a case!
  • The CentOS/RHEL repository contains the “Long Lived Branch version” where less changes occur; while Fedora repositories contains the “Short Lived Branch version”. Beta CentOS/RHEL and Fedora’s rawhide repositories will contain the “Beta Branch version”

CUDA support

  • CUDA libraries/tools for the driver are split into subpackages. There’s no need to install all the CUDA libraries and tools on a system that has only one adapter and is used for occasional gaming or for simple office use. This can save ~120 MB worth of installed libraries. nvidia-persistenced falls in this category as it’s not needed on a normal laptop or gaming system.
  • Complete packaged CUDA stack has been added for all supported distributions, all the packages provide/require/obsolete the relevant packages in the Nvidia CUDA repository; so you can enable this repository along with the official Nvidia CUDA one (x86_64 systems only).

Kernel modules

  • Multiple choice of kernel module packages; akmod (RPMFusion) for Fedora and binary kmod (Kernel ABI whitelists) for CentOS/RHEL. In addition to this, on both distributions dkms packages are available. This way all cases and personal preferences are covered for both distributions.
  • Starting of drivers version 331.17, the kernel module packages contain the main Nvidia kernel module (nvidia.ko) but also the Unified Memory kernel module.
  • Support for multiple kernel modules has been deprecated in recent driver versions (they are available up to 352.xx); but both akmod and dkms packages can be configured to build the additional modules instead of the single one, so you can still choose your preferred method.
  • Starting from Nvidia driver version 334.16, the Nvidia DDX driver for X can also rely on the nvidia-modprobe command in the system to create devices and set permissions, so the new optional package has been added.
  • The nvidia module has a soft dependency on the nvidia-uvm module, making sure the module is loaded when installing the nvidia-driver-cuda package, but making sure that these modules are not included in the initrd (thing that would happen with systemd configuration (module-s-load.d). UDev rules make sure the module has proper permissions.
  • On Fedora, the kernel modules are compressed with XZ, like all the other kernel modules.

Default configuration

  • Dracut options are depending on the distribution; so no more “vga=normal is an obsolete option” at boot. Each distribution gets its own specific GRUB options for booting.
  • 96 DPI is written in the default xorg.conf config file. Why? Gnome 3 by defaults hard-codes a 96×96 DPI resolution, most of the free drivers do (intel, nouveau, etc.) as the EDID is almost never reliable (please see the excellent Adam’s Jackson post where he explains this). As an example, if you install the Nvidia drivers on a RHEL/CentOS 6 laptop where you used to have nouveau installed (96 DPI hardcoded), the fonts gets 90% of the time supersize and ugly as Gnome 2 and the Nvidia driver do not hard-code 96 DPI like Gnome 3.
  • Make X.org NVIDIA Files section to be loaded latest in case there are other packages providing a custom Files section.
  • Starting from Fedora 21, all driver X.org configuration can be managed by simply adding/removing X.org configuration snippets in /etc/X11/xorg.conf.d.
  • Use new OutputClass directive on Fedora 21 X.org server 1.16 (and later) to load the driver and do not rely on an edited /etc/X11/xorg.conf file. This also removes editing of the xorg.conf file from the package scriptlets. This does not hardcode the 96 DPI resolution.
  • Add the IgnoreABI directive by default on Fedora rawhide builds.

Kernel modesetting and Wayland support

Kernel mode setting on the nvidia-drm module has been disabled by default for various reasons. First of all, Wayland support in the drivers require a patched Wayland which has been refused upstream, and then the driver itself does not expose an FB driver for the console, so you won’t see any difference in the terminal output, you will still be limited to VGA.

There is a proposal for sorting everything out at XDC 2016 for hardware vendors that require to expose extensions for the drivers.

The Wayland libraries are still included in the Fedora builds, as all the dependencies are there but they are not used. On CentOS/RHEL 7 packages, they are not included as this would result in missing dependencies.

Vulkan support

vulkan_500px_june16Vulkan is now part of Fedora, so on supported Fedora releases, the Vulkan loader and libraries can be installed and you do not need to do anything to enable support in the drivers. CentOS and Red Hat Enterprise Linux do not have Vulkan yet. I’m not sure if it’s worth installing it by default along with the drivers, though.

Distribution and Nvidia driver version support

Here is a rundown of Nvidia supported drivers and options split by distribution. Basically, CentOS/RHEL will always get a Long Lived branch release if possible, Fedora always a Short Lived branch release, and unreleased distributions will always get a Beta driver.

Operating systemel6 / el7f23 / f24f25
Driver branchLong LivedShort Lived
Long Lived
Beta
Short Lived
Long Lived
Driver version375.20375.20375.20
NVENC7.0.17.0.17.0.1
Architectures:

i686
x86_64
YesYesYes
Basic nvidia driver:

nvidia-driver
nvidia-driver-libs
nvidia-libXNVCtrl
YesYesYes
CUDA libraries and tools:

nvidia-driver-cuda
nvidia-driver-cuda-libs
nvidia-driver-NVML
nvidia-persistenced
YesYesYes
OpenGL Framebuffer Capture:

nvidia-driver-NvFBCOpenGL
YesYesYes
Nvidia tools:

nvidia-healthmon (x86_64)
nvidia-validation-suite (x86_64)
nvidia-modprobe
nvidia-settings
nvidia-xconfig

YesYesYes
Binary kernel
modules (kABI):

kmod-nvidia
YesNoNo
DKMS kernel
modules:

dkms-nvidia
YesYesYes
aKMOD kernel
modules:

akmod-nvidia
NoYesYes
32 bit compatibility on x86_64:

nvidia-libXNVCtrl
nvidia-driver-libs
nvidia-driver-cuda-libs
nvidia-driver-NVML
YesYesYes
Development

nvidia-driver-devel
nvenc
nvenc-samples
libvdpau-devel
YesYesYes
GLVND libraries

libglvnd
libglvnd-egl
libglvnd-gles
libglvnd-glx
libglvnd-opengl
libglvnd-core-devel
libglvnd-devel
YesYesYes
VDPAU libraries

libvdpau
1.1.11.1.11.1.1

Sample installation

Here is an example. Let’s assume you have a freshly installed Fedora 25 system with a recent Nvidia GPU and you want to:

  • Install the driver for gaming
  • Play Vulkan enabled games
  • Want to be comfortable with the control panel
  • Play 32 bit games on a 64 bit system
  • Play 32 bit Vulkan games on a 64 bit system
$ sudo dnf install nvidia-settings kernel-devel dkms-nvidia vulkan.i686 nvidia-driver-libs.i686
Last metadata expiration check: 0:33:49 ago on Mon Oct 24 14:14:30 2016.
Dependencies resolved.
============================================================================================
 Package              Arch     Version                                Repository       Size
============================================================================================
Installing:
 dkms-nvidia          x86_64   2:375.10-1.fc25                        fedora-nvidia   6.4 M
 libglvnd             i686     1:0.2.999-3.20161017gita813b56.fc25    fedora-nvidia   103 k
 libglvnd             x86_64   1:0.2.999-3.20161017gita813b56.fc25    fedora-nvidia   105 k
 libglvnd-gles        i686     1:0.2.999-3.20161017gita813b56.fc25    fedora-nvidia    29 k
 libglvnd-gles        x86_64   1:0.2.999-3.20161017gita813b56.fc25    fedora-nvidia    28 k
 libglvnd-glx         i686     1:0.2.999-3.20161017gita813b56.fc25    fedora-nvidia   114 k
 libglvnd-glx         x86_64   1:0.2.999-3.20161017gita813b56.fc25    fedora-nvidia   110 k
 libglvnd-opengl      i686     1:0.2.999-3.20161017gita813b56.fc25    fedora-nvidia    39 k
 libglvnd-opengl      x86_64   1:0.2.999-3.20161017gita813b56.fc25    fedora-nvidia    38 k
 libva-vdpau-driver   x86_64   0.7.4-14.fc24                          fedora           61 k
 libvdpau             i686     1.1.1-3.fc24                           fedora           35 k
 nvidia-driver        x86_64   2:375.10-1.fc25                        fedora-nvidia   3.1 M
 nvidia-driver-NVML   x86_64   2:375.10-1.fc25                        fedora-nvidia   397 k
 nvidia-driver-libs   i686     2:375.10-1.fc25                        fedora-nvidia    15 M
 nvidia-driver-libs   x86_64   2:375.10-1.fc25                        fedora-nvidia    14 M
 nvidia-libXNVCtrl    x86_64   2:375.10-1.fc25                        fedora-nvidia    26 k
 nvidia-settings      x86_64   2:375.10-1.fc25                        fedora-nvidia   935 k
 vulkan               i686     1.0.30.0-1.fc25                        fedora          1.5 M
 vulkan               x86_64   1.0.30.0-1.fc25                        fedora          1.4 M
 vulkan-filesystem    noarch   1.0.30.0-1.fc25                        fedora          8.0 k
 
Transaction Summary
============================================================================================
Install  20 Packages
 
Total download size: 43 M
Installed size: 184 M
Is this ok [y/N]:

As you can see, this system has dkms enabled kernel module and libraries for running 32 bit applications. The amount of data to download for the drivers is really small compared to packages that contain CUDA libraries and tools. All packages have an higher Epoch set; so they should never be upgraded on your system when you enable this repository along the RPMFusion or ELRepo ones.

Package installation

If you are booting the system in UEFI mode; as a prerequisite to installing any external module (not built into the kernel package), you have to disable UEFI Secure Boot in the system configuration. All modules contained in the kernel package are signed with keys that are generated during build and deleted when packaging. If you want to preserve Secure Boot, you need to sign the modules yourself and import the keys into your hardware module. Doing so is out of scope here; if you need a decent guide just follow Red Hat’s guide for signing kernel modules.

First of all remove all the Nvidia drivers you might have on your system due to RPMFusion, ELRepo, or the Nvidia CUDA repository. The packages should already take care of this for you, as they should be completely compatible; but better be safe than sorry. This is usually accomplished with the following root command:

yum -y remove \*nvidia\*

Then, to install the Nvidia driver and its control panel utility in CentOS/RHEL with the binary kABI (Kernel ABI whitelist) module (this is the default on CentOS/RHEL), perform the following command:

yum -y install nvidia-driver nvidia-settings

To do the same in Fedora, perform the following command:

dnf -y install nvidia-driver nvidia-settings kernel-devel

Requirement on kernel-devel is required as otherwise the package kernel-debug-devel is pulled in automatically in place of the normal non-debug package. There is bug opened on dnf/libsolv for this.

Specific driver installations

For both Fedora and CentOS/RHEL distributions it’s possible to install additional packages and / or variant of the basic kernel modules. This paragraph contains some examples. Make sure you have the RPMFusion repository enabled if you plan to use akmod kernel modules on Fedora or EPEL if you plan to use DKMS modules on CentOS/RHEL.

akmod kernel module variant (Fedora):

dnf -y install nvidia-driver kernel-devel akmod-nvidia

DKMS kernel module variant (Fedora/CentOS/RHEL):

yum/dnf -y install nvidia-driver kernel-devel dkms-nvidia

To add 32 bit libraries on a 64 bit system (for games or applications like Steam):

yum -y install nvidia-driver-libs.i686

Additional driver configuration to your system

To add additional configuration to your system, just create the /etc/X11/xorg.conf file if it does not exist (by default it exists only in Red Hat Enterprise Linux / CentOS 6 systems). For example:

Section "Device"
    Identifier  "Device0"
    Driver      "nvidia"
    Option      "NoLogo" "true"
    Option      "DPI" "96 x 96"
    Option      "SLI" "Auto"
    Option      "nvidiaXineramaInfoOrder" "DFP-0"
    Option      "metamodes" "GPU-a493fbbb-7d76-86a2-8764-d76d487a75a7.DVI-I-1: nvidia-auto-select +0+0, GPU-c02960a4-be28-d5ce-8b02-be04b5e2550b.DVI-I-1: nvidia-auto-select +1680+0"
    Option      "BaseMosaic" "on"
EndSection

In this example we have 2 video cards with one monitor each, so we enabled SLI, Base Mosaic to have multi monitor support on SLI and make a layout with the second GPU monitor on the right of the first one. Also, we fix the DPI to 96×96, which is the hardcoded default in Gnome and in Open Source drivers.

Configuration for headless systems

Your system might only be used for CUDA development and not require the X server to be running the driver at all, so you might want to tweak the configuration a bit to make the system load (for example) the Intel driver as the main display and just the Nvidia driver for calculation. In this case, the Intel driver should load the modesetting driver, offload the rendering to the Nvidia driver and not use any monitor attached to the X server for the Nvidia driver.

So in this case, I would change /etc/default/grub to remove the nomodeset parameter to make the Intel KMS driver to load properly, regenerate the Grub config file, reboot and use this xorg.conf:

Section "ServerLayout"
    Identifier "layout"
    Screen 0 "intel"
    Inactive "nvidia"
EndSection
 
Section "Device"
    Identifier "nvidia"
    Driver "nvidia"
    BusID "<BusID for NVIDIA device here>"
EndSection
 
Section "Screen"
    Identifier "nvidia"
    Device "nvidia"
    Option "AllowEmptyInitialConfiguration"
EndSection
 
Section "Device"
    Identifier "intel"
    Driver "modesetting"
EndSection
 
Section "Screen"
    Identifier "intel"
    Device "intel"
EndSection

An example of the above can also be read in the official documentation.

The device file /dev/nvidia0 is normally created when loading the Nvidia driver, so if the X driver is not loaded the device file fileis not created. You can use the nvidia-modprobe command that is in the package with the same name.
It contains a SUID binary that creates the device files and set the appropriate permissions when automatic device creation is not available. It is called directly by Nvidia libraries:

$ for i in $(rpm -ql nvidia-driver-libs.x86_64); do strings $i | grep nvidia-modprobe && echo $i; done
/usr/bin/nvidia-modprobe
/usr/lib64/libnvidia-cfg.so.1
/usr/bin/nvidia-modprobe
/usr/lib64/libnvidia-cfg.so.375.20
/usr/bin/nvidia-modprobe
/usr/lib64/libnvidia-eglcore.so.375.20
/usr/bin/nvidia-modprobe
/usr/lib64/libnvidia-glcore.so.375.20
/usr/bin/nvidia-modprobe
/usr/lib64/libnvidia-glsi.so.375.20
/usr/bin/nvidia-modprobe
/usr/lib64/vdpau/libvdpau_nvidia.so.1
/usr/bin/nvidia-modprobe
/usr/lib64/vdpau/libvdpau_nvidia.so.375.20

This requires some testing and adjustments with specifics to your setup, but is definitely possible to use the integrated Intel card and or rely on a system without X installed to run the CUDA components.

CUDA

Packaging

  • Previously in the repository was included the GPU Deployment kit. This was constructed with NVML (NVIDIA Management Library) headers, docs and samples from a separate tarball. The separate tarball was using a different version number than the drivers and was packaged in the nvidia-driver-NVML and nvidia-driver-NVML-devel packages. Installing these, the gpu-deployment-kit dependency provided by the CUDA repositories was preserved. Starting from CUDA version 8, the NVML header is provided by a CUDA subpackage (cuda-nvml-devel) and no longer provided as part of the GPU Deployment kit.
  • Along with NVML, the nvidia-healthmon and nvidia-validation-suite package is provided to monitor TESLA GPU clusters (x86_64 systems only). For whatever confusing reason, these are not part of the CUDA 8 release. I will leave them in the repository for a while before removing them.
  • Included is also the NVENC (Nvidia Encoder) header, docs and code samples. Again, this uses a different version than the drivers.
  • All the libraries are split into subpackages, much like in the original Nvidia CUDA repository. This allows you to install and build software relying on specific components without the need to install all the CUDA toolkit just to satisfy a library dependency. With the new packaging organization, the original cuda-devel and cuda-extra-libs will pull in all the specific subpackages giving you the same situation you are accustomed to. Also, for the same reason, static libraries have been included in each respective devel subpackage.
  • In addition to the libraries bundled in the CUDA toolkit, also the cuDNN library for distributed neural networks is included in the repository. See the table below for details.

Distribution and CUDA version support

Operating systemel6 / el7f23 / f24 / f25
CUDA branch/version8.0.448.0.44
CUDA cuDNN version5.15.1
Basic CUDA libraries/tools:

cuda
cuda-libs
cuda-extra-libs
cuda-cublas
cuda-cudart
cuda-cufft
cuda-cudnn
cuda-cupti
cuda-curand
cuda-cusolver
cuda-cusparse
cuda-npp
cuda-nvgraph
cuda-nvrtc
cuda-nvvp
YesYes
CUDA development:

cuda-cli-tools
cuda-devel
cuda-cublas-devel
cuda-cudart-devel
cuda-cudnn-devel
cuda-cufft-devel
cuda-cupti-devel
cuda-curand-devel
cuda-cusolver-devel
cuda-cusparse-devel
cuda-npp-devel
cuda-nvgraph-devel
cuda-nvml-devel
(also i686)
cuda-nvrtc-devel
YesYes
Java GUI programs:

cuda-nsight
cuda-nvvp
YesYes
Documentation and samples

cuda-samples
cuda-docs
(noarch)
YesYes

CUDA installations

To install just a runtime CUDA support (required for running CUDA enabled programs):

yum -y install cuda nvidia-driver-cuda

To install packages required for enabling CUDA development:

yum -y install cuda-devel

or, if you need to develop some application that requires multiple libraries:

yum -y install cuda-devel

A couple of examples. Just the basic tools:

$ sudo dnf install cuda
Last metadata expiration check: 0:00:20 ago on Sun Oct 23 13:11:01 2016.
Dependencies resolved.
================================================================================
 Package           Arch         Version               Repository           Size
================================================================================
Installing:
 cuda              x86_64       1:8.0.44-4.fc24       fedora-nvidia        95 M
 cuda-cufft        x86_64       1:8.0.44-4.fc24       fedora-nvidia        97 M
 cuda-curand       x86_64       1:8.0.44-4.fc24       fedora-nvidia        38 M
 cuda-libs         x86_64       1:8.0.44-4.fc24       fedora-nvidia       6.4 M
 
Transaction Summary
================================================================================
Install  4 Packages
 
Total size: 236 M
Installed size: 469 M
Is this ok [y/N]:

The basic tools along with all the libraries (note that the NVML headers are included):

$ sudo dnf install cuda-devel
Last metadata expiration check: 0:10:00 ago on Sun Oct 23 13:11:01 2016.
Dependencies resolved.
================================================================================
 Package                 Arch       Version             Repository         Size
================================================================================
Installing:
 cuda                    x86_64     1:8.0.44-4.fc24     fedora-nvidia      95 M
 cuda-cublas             x86_64     1:8.0.44-4.fc24     fedora-nvidia      21 M
 cuda-cublas-devel       x86_64     1:8.0.44-4.fc24     fedora-nvidia      38 M
 cuda-cudart             x86_64     1:8.0.44-4.fc24     fedora-nvidia     131 k
 cuda-cudart-devel       x86_64     1:8.0.44-4.fc24     fedora-nvidia     659 k
 cuda-cufft              x86_64     1:8.0.44-4.fc24     fedora-nvidia      97 M
 cuda-cufft-devel        x86_64     1:8.0.44-4.fc24     fedora-nvidia      73 M
 cuda-cupti              x86_64     1:8.0.44-4.fc24     fedora-nvidia     1.2 M
 cuda-cupti-devel        x86_64     1:8.0.44-4.fc24     fedora-nvidia     213 k
 cuda-curand             x86_64     1:8.0.44-4.fc24     fedora-nvidia      38 M
 cuda-curand-devel       x86_64     1:8.0.44-4.fc24     fedora-nvidia      60 M
 cuda-cusolver           x86_64     1:8.0.44-4.fc24     fedora-nvidia      23 M
 cuda-cusolver-devel     x86_64     1:8.0.44-4.fc24     fedora-nvidia     4.1 M
 cuda-cusparse           x86_64     1:8.0.44-4.fc24     fedora-nvidia      23 M
 cuda-cusparse-devel     x86_64     1:8.0.44-4.fc24     fedora-nvidia      23 M
 cuda-devel              x86_64     1:8.0.44-4.fc24     fedora-nvidia     1.6 M
 cuda-libs               x86_64     1:8.0.44-4.fc24     fedora-nvidia     6.4 M
 cuda-npp                x86_64     1:8.0.44-4.fc24     fedora-nvidia      91 M
 cuda-npp-devel          x86_64     1:8.0.44-4.fc24     fedora-nvidia      47 M
 cuda-nvgraph            x86_64     1:8.0.44-4.fc24     fedora-nvidia     4.6 M
 cuda-nvgraph-devel      x86_64     1:8.0.44-4.fc24     fedora-nvidia      12 k
 cuda-nvml-devel         x86_64     1:8.0.44-4.fc24     fedora-nvidia      41 k
 cuda-nvrtc              x86_64     1:8.0.44-4.fc24     fedora-nvidia     6.6 M
 cuda-nvrtc-devel        x86_64     1:8.0.44-4.fc24     fedora-nvidia      16 k
 
Transaction Summary
================================================================================
Install  24 Packages
 
Total size: 655 M
Installed size: 1.4 G
Is this ok [y/N]:

An example where your CUDA application just uses the CUDA Runtime API and not the kernel runtime:

$ sudo dnf install cuda-cudart
Last metadata expiration check: 0:13:10 ago on Sun Oct 23 13:11:01 2016.
Dependencies resolved.
================================================================================
 Package           Arch         Version               Repository           Size
================================================================================
Installing:
 cuda-cudart       x86_64       1:8.0.44-4.fc24       fedora-nvidia       131 k
 
Transaction Summary
================================================================================
Install  1 Package
 
Total size: 131 k
Installed size: 536 k
Is this ok [y/N]:

This will avoid you pulling in all the libraries as before just because you need a single library. This is useful for example for programs that leverage just some part of the CUDA toolkit, like the Nvidia Performance Primitives for image and signal processing in FFmpeg, and similar things.

Bugs

Just open an issue to the specific package on GitHub.

594 thoughts on “Nvidia driver, CUDA tools and libraries

  1. The gdm login screen shows up correctly using your nvidia driver. And in 4K resolution too.
    But when I try to login with a user I only get a black screen.
    Looking at the xorg.1.log i see:
    nvidia(gpu-0): failed to acquire modesetting permission
    so I assume the problem is that gdm is starting two X sessions one on vt0 and another on vt1.
    is this supposed to work?
    I did disable wayland in /etc/gdm/custom.conf
    anything else I need to do?
    I also selected gnome on Xorg in gdm when logging in as a user.

    1. So, multiple issues here. If the system detects that you are using the Nvidia binary driver, X is still executed as root and not with normal user privileges. This translates in just one X instance starting.

      In your case this means that is loading some other driver (like the Intel KMS one), which would also explain the “failed to acquire modesetting” error. Why it does work at login I don’t know, you need to check your logs. If you’re on an Optimus laptop then that’s another topic and you need to configure it.

      1. It’s not an optimus laptop, it’s a desktop machine with GTX1080. First I installed rpmfusion drivers, discovered they don’t work (I get the gdm something went wrong message), then I found your page, removed rpmfusion *nvidia*, and installed your drivers. I was happy to finally get the gdm login screen in full 4K, but it would not allow me to login because it spawns a second xorg which fails, and from logs i can see that other drivers are also enabled on the second xorg. I don’t know if this is some leftovers from rpmfusion, I can try making a fresh fedora 25 install + your drivers if that would help. Also note, while installing fedora 25 I had to choose boot with basic video or I would not get to the installer, maybe that left some differences on the system…

        1. That’s strange, I have a couple of desktops with Nvidia cards and have to do absolutely nothing apart from installing the packages. The system detects that you are using the Nvidia driver; the non-root X servers do not start and there is only one X instance running as root.
          Does your system also have an Intel card integrated on the motherboard? You can check that there are no other driver using kms with the following command. It should only return this output if only the Nvidia driver is loaded:

          $ lsmod | grep kms
          drm_kms_helper        151552  1 nvidia_drm
          drm                   344064  7 nvidia_drm,drm_kms_helper

          In case there is something left over by other packages or manual configurations I think you can clean it up without reinstalling anything.

          1. I tried the lsmod | grep kms and get exactly the same output as you have pasted (in single or multi-user mode, without X).

            I tried a fresh fedora 25 install and this time with only your multimedia repo, no rpmfusion but it did not help. I installed your nvidia drivers immediately after installing, so without updating the system (dnf update), and it did not work, then I updated the system and still the same issue.

            Yes my i5-2500K does have onboard graphics and the motherboard supports that, but I don’t think that’s the issue.

            I have uploaded Xorg.0.log:
            http://pastebin.com/7aPNgJks

            and Xorg.1.log:
            http://pastebin.com/ttTj5VfC

            If you can make some sense out of that?

            What else can I try?

            My dual-boot fedora 24 install works fine (with rpmfusion nvidia) but I would like to fedora 25 working too so I can move to that.

    2. I have a GTX970 and I got black screen too. The screen completly turn off. Never append before. I start thinking to join to the RedTeam (AMD), I’m really tired of maxwell gpu on linux.

      1. And It 2016 and I want a wayland compositor.
        I’m tired of X and the compositors based on X, It’s hard and embarrassing works on a desktop with lag, tearing and stutter all the time, even on discrete powerfull gpu.

    1. Not sure if this will help you, but *every time* I do a fresh install of nvidia drivers from Negativo17, the first reboot stalls for a while and then fails with the GNOME “Oh no! Something happened!” error message (GDM doesn’t even start). If I simply reboot by switching to a console and running “reboot” as root, it works just fine after that. It seems to be a first-run issue…

  2. So I was previously using the driver obtained and installed from nvidia.com on Fedora 24, then yesterday I decided to make the switch both to Fedora 25 and to this repo (because I was getting tired of having to manually fix stuff every time kernel updated).

    Having some issues:

    I previously had it configured to use KMS with the following command line `rd.driver.blacklist=nouveau nvidia-drm.modeset=1 video=vesa:off`

    I also added the nvidia modules to my initrd using dracut.

    Now I see that installing the packages has changed some things, notably `nouveau.modeset=0 nomodeset gfxpayload=vga=normal`. No matter which configuration I use (yours, without my changes or mine, without yours), I cannot get GDM to show up.

    If I boot with the kms flags (and with the modules in initrd), it will give me a bunch of garbage in dmesg about failing to convert NVKMS memory to GEM objects (or so) every time I try to switch VT (it seems any time a KMS context-switch occurs), and if I boot with the nomodeset flags (and without the modules in initrd), then gnome-session hits a segfault.

    `segfault at 0 ip 00007f0311775229 … in libgtk3.so.0…`

    Totally at a loss…

    1. Hello,

      > I also added the nvidia modules to my initrd using dracut.

      You should not, otherwise at the next update you require an initrd rebuild. They should be loaded after systemd switches the root filesystem on boot.

      To clean everything up, if you come from a manually installed driver:

      – Remove stuff from /etc/default/grub
      – Remove stuff from /etc/{modprobe.d,depmod.d}
      – Remove stuff from grub configuration files
      – Remove all packages and manually installed files from Nvidia
      – Remove all Nvidia kernel module source code in /usr/src
      – Reinstall packages providing libGL.*
      – Reboot with Nouveau and standard Fedora setup

      Then you can install the packages. Can’t help if you have untracked Nvidia files scattered around your system, sorry.

      1. I’m perfectly comfortable rebuilding my initrd’s with dracut after updates. I’m not worried about that. I need the KMS support for a wayland-based application I’m working on. I have already fully uninstalled the old nvidia drivers. (use of the –uninstall flag in the installer was sufficient). I also did try to use the modules without my patched-together initrd and grub settings, and it failed with the same error below:

        The problem actually was that GLX wasn’t being initialized. Xorg attempted to load the wrong GLX module (the X.org Foundation version). If GLX isn’t available, libGTK will apparently just segfault. If anyone from future internet is encountering this problem, try running an X server with e.g. `Xorg :2` and a terminal in that server with `DISPLAY=:2 xterm` or so. Then try to run `glxinfo` in that terminal. It’ll tell you if GLX is missing on the display.

        When I saved an X configuration using nvidia-settings, it created a Files section. When I removed this section, GDM could start. I presume that it was overriding somehow the Files section in the xorg.conf.d config file.

        1. I think you have some leftovers around. The packages install everything that is required to have a working X driver out of the box:

          $ cat /etc/X11/xorg.conf.d/99-nvidia-modules.conf 
          # Load with order 99 as the Files section needs to be the very last
          Section "Files"
          	ModulePath   "/usr/lib64/nvidia/xorg"
          	ModulePath   "/usr/lib64/xorg/modules"
          EndSection
           
          Section "Module"
              Disable "glamoregl"
          EndSection

          and:

          $ cat /usr/share/X11/xorg.conf.d/10-nvidia-driver.conf 
          Section "OutputClass"
              Identifier "nvidia"
              MatchDriver "nvidia-drm"
              Driver "nvidia"
          EndSection
  3. Thanks for the all of your efforts – on previous Fedoras, I’ve rolled-my-own updates, but thought that it would simplify things if I could do it using your repo for Fedora 25. I’m “just” looking to do CUDA/tensorflow stuff on a monitorless box.

    dnf install kernel-devel dkms-nvidia
    dnf install cuda-devel cuda-cudnn-devel

    and reboot, gives me an nvidia driver in “lsmod”.

    But when I try and do something with tensorflow, it says that the cuda_XXX.so are all loaded fine, but fails to find a /dev/nvidia0 device (and I don’t see anything useful in the /dev/ listing). Could someone who has a working CUDA configuration tell me what I’m missing? Or is it the tensorflow end – though this is the first error it gives :

    I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:147] no NVIDIA GPU device is present: /dev/nvidia0 does not exist

    Many Thanks, Martin

    1. Hello, you’re missing one step. The package nvidia-driver-cuda is missing in your commands. That’s what loads the nvidia-uvm module and pulls in the other CUDA libraries.

      So, based from your above commands:

      dnf install kernel-devel dkms-nvidia nvidia-driver-cuda
      1. I added nvidia-driver-cuda as suggested (and that pulled in nvidia-persistenced & opencl-filesystem). The install I had done earlier had already pulled in nvidia-uvm as a dependency. After a reboot, still no /dev/nvidia0 (though /dev/nvidia-uvm was there).

        However, running the script at http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#runfile-verifications did create a /dev/nvidia0 that TensorFlow liked, upon which the following python snippet :

        python
        import tensorflow as tf
        sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

        gives the output :

        I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcublas.so locally
        I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcudnn.so locally
        I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcufft.so locally
        I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcuda.so.1 locally
        I tensorflow/stream_executor/dso_loader.cc:111] successfully opened CUDA library libcurand.so locally

        I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:925] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
        I tensorflow/core/common_runtime/gpu/gpu_device.cc:951] Found device 0 with properties:
        name: GeForce GTX 760
        major: 3 minor: 0 memoryClockRate (GHz) 1.137
        pciBusID 0000:01:00.0
        Total memory: 1.98GiB
        Free memory: 1.94GiB
        I tensorflow/core/common_runtime/gpu/gpu_device.cc:972] DMA: 0
        I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] 0: Y
        I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 760, pci bus id: 0000:01:00.0)
        Device mapping:
        /job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 760, pci bus id: 0000:01:00.0
        I tensorflow/core/common_runtime/direct_session.cc:252] Device mapping:
        /job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: GeForce GTX 760, pci bus id: 0000:01:00.0

        After which everything looks good (i.e. GPU operations run on the GPU).

        What rpm should be doing the node creation ?

        1. There’s something wrong. The node creation is done once you load the nvidia module and the X driver is loaded. I’ve never done anything in that regard:

          $ ls -al /dev/nvidia*
          crw-rw-rw-. 1 root root 195,   0 Nov 25 09:04 /dev/nvidia0
          crw-rw-rw-. 1 root root 195,   1 Nov 25 09:04 /dev/nvidia1
          crw-rw-rw-. 1 root root 195, 255 Nov 25 09:04 /dev/nvidiactl
          crw-rw-rw-. 1 root root 195, 254 Nov 25 09:04 /dev/nvidia-modeset
          crw-rw-rw-. 1 root root 239,   0 Nov 25 09:04 /dev/nvidia-uvm
          1. Hello, I’ve read your blog posts, I think you can simplify the configuration. Of course that requires some testing.

            I would not use intel.modeset=1 and nomodeset like you did.

            After that, just rely only on the xorg.conf configuration to load the modesetting driver. In the xorg.conf configuration file you can remove all the InputDevice, Monitor, ServerLayout and Files sections, they should be not needed, the InputDevice sections for sure. Your Files section is also wrong, and does not contain the Nvidia GLX server module, which is already defined in the package /etc/X11/xorg.conf.d/99-nvidia-modules.conf:

            http://us.download.nvidia.com/XFree86/Linux-x86/375.20/README/randr14.html
            http://negativo17.org/nvidia-driver/ (example down below)

            So in your case, I would change /etc/default/grub, regenerate the Grub config file, reboot and use this xorg.conf:

            Section "ServerLayout"
            	Identifier      "X.org Configured"
            	Inactive 	"Card0"
            EndSection
             
            Section "Device"
            	Identifier  "Card0"
            	Driver      "nvidia"
                    Option      "AllowEmptyInitialConfiguration"
            	BusID       "PCI:1:0:0"
            EndSection
             
            Section "Device"
            	Identifier  "Card1"
            	Driver      "modesetting"
            	BusID       "PCI:0:2:0"
            EndSection
             
            Section "Screen"
            	Identifier "Screen0"
            	Device     "Card1"
            EndSection

            Also, regarding the device file creation. The device file is normally created when loading the Nvidia driver, so if X is not running the driver is not loaded and the device file is not created. As per the documentation you posted, you can use the nvidia-modprobe command that is in the package with the same name.
            It contains a SUID binary that creates the device files and set the appropriate permissions when automatic device creation is not available. It is called directly by Nvidia libraries:

            $ for i in $(rpm -ql nvidia-driver-libs.x86_64); do strings $i | grep nvidia-modprobe && echo $i; done
            /usr/bin/nvidia-modprobe
            /usr/lib64/libnvidia-cfg.so.1
            /usr/bin/nvidia-modprobe
            /usr/lib64/libnvidia-cfg.so.375.20
            /usr/bin/nvidia-modprobe
            /usr/lib64/libnvidia-eglcore.so.375.20
            /usr/bin/nvidia-modprobe
            /usr/lib64/libnvidia-glcore.so.375.20
            /usr/bin/nvidia-modprobe
            /usr/lib64/libnvidia-glsi.so.375.20
            /usr/bin/nvidia-modprobe
            /usr/lib64/vdpau/libvdpau_nvidia.so.1
            /usr/bin/nvidia-modprobe
            /usr/lib64/vdpau/libvdpau_nvidia.so.375.20

            I think I will make a blog post about this someday, for now I’m updating the driver page with this information.

          2. > I would not use intel.modeset=1 and nomodeset like you did.

            That was not clear sorry. I mean, remove intel.modeset=1 and also _remove_ nomodeset like you did.

  4. Hi, I’ve installed nvidia-driver, kernel-devel and akmod-nvidia on fedora 25 cinnamon. After rebooting whenever I log in, t cinnamon desktop crashes and fall into fallback mode. Can you suggest how to fix this?

  5. Hello,
    do you know if there is a way to install not the latest nvidia driver but a specific version ? We are currently constraint to use vendors certified drivers for our applications and that’s often the n-1 or n-2 drivers version.
    many thanks

    1. I’m sorry but don’t know how I could do otherwise. I would need a lot of specific repositories for the different versions.
      What I’m offering is what is described in the table in the Nvidia driver page, that is:

      – RHEL/CentOS: long-lived branch release
      – Fedora: short-lived branch release
      – Fedora rawhide: beta

      By chance, as of today, the latest Nvidia driver version (375.20) is a long-lived branch release, so it’s everywhere.

  6. I’m a little confused by the wording in the first post, but does KMS work? I wanted to try out Prime Sync with F25 and Xorg 1.19 but trying to enable KMS with nvidia-drm.modeset=1 causes the driver not to load. I’m not sure if I’m just missing something or what.

    1. Kernel modesetting is not compatible with SLI, it does not have a framebuffer driver (i.e. no console on fb devices like normal KMS drivers) and it requires a patched Mutter to operate with Gnome which is not yet merged:

      https://fedoraproject.org/wiki/Wayland_features#Nvidia_driver_support
      http://us.download.nvidia.com/XFree86/Linux-x86/375.20/README/kms.html

      So basically yes, it’s useless now. That’s why it is disabled in my packages. I will enable it once it can be made to work out of the box.

  7. Hi, running Fedora 24 and upgraded to 375.20 with an Nvidia 770 GTX. OpenGL runs fine, but whenever I try to launch Dota 2 or The Talos Principle in Vulkan, I get this

    [ 87.671044] dota2[2537]: segfault at 48 ip 00007fad8b2e5ad9 sp 00007ffef7576e00 error 4 in libnvidia-glcore.so.375.20[7fad89f4a000+13e2000]

    And Talos just gives me a segmentation fault after trying to load Vulkan.

  8. It seems the 375.20 driver update is broken, Xorg complains that the kernel module has a different version (375.10) than the userspace driver (375.20), even after removing and reinstalling the kernel modules (and driver).

      1. I just tried again – and, additionally, did a “dracut –regenerate-all –force” – which seems to have fixed the problem. Sorry for the noise.

  9. I made it work on fedora 25 with xorg 1.18. You have to downgrade Xorg from 1.19 to version 1.18 from fedora 24. After it’s working.

    dnf downgrade xorg-x11-server\* –releasever=24 –allowerasing
    echo “exclude=xorg-x11*” >> /etc/dnf/dnf.conf

    After reinstall your nvidia driver.

  10. I’m having difficulty getting the drivers to work for me on my system.

    Its a fedora 25 system that I upgraded from Fedora 24.

    At the moment I get a segfault during X startup, aparently:

    [ 25.641] (EE) Backtrace:
    [ 25.642] (EE) 0: /usr/libexec/Xorg (OsLookupColor+0x139) [0x59e229]
    [ 25.642] (EE) 1: /lib64/libpthread.so.0 (__restore_rt+0x0) [0x7ffbd2ad75bf]
    [ 25.642] (EE) 2: /lib64/libnvidia-glcore.so.375.10 (nvidiaAddDrawableHandler+0x54599e) [0x7ffbcd868a2c]

    I put the complete xorg log here:
    https://gist.github.com/jave/3df01788d014d6bc5999dd4ea5d1228b

    1. As explained in the previous comments, driver 375.10 does not support the unreleased X server 1.19 that is now in Fedora 25. Even setting the IgnoreABI flag makes the X server crash. I’m sorry but you have to wait for a new driver.

  11. [ 1493.223] (EE) Backtrace:
    [ 1493.224] (EE) 0: /usr/libexec/Xorg (OsLookupColor+0x139) [0x59e2d9]
    [ 1493.224] (EE) 1: /lib64/libpthread.so.0 (__restore_rt+0x0) [0x7f03434d15bf]
    [ 1493.224] (EE) 2: /lib64/libnvidia-glcore.so.375.10 (nvidiaAddDrawableHandler+0x54599e) [0x7f033e262a2c]
    [ 1493.224] (EE)
    [ 1493.224] (EE) Segmentation fault at address 0

    1. Nvidia drivers 375.10 do not yet suppport X.org 1.19, the driver installation is relying on the “IgnoreABI” directive in its configuration. Of course this does not give you any guarantee. Nvidia does not support unreleased X servers, so if you can not downgrade the X server to 1.18.x there’s not much you can do.

      1. I was caught by this a couple of days ago while upgrading from fc24 to fc25.
        I eventually managed to downgrade a large chunk of the system to get to a hybrid fc24¾

        Looking through my bash history I found some commands I tried with varying degrees of success that may help someone else in the same situation, but do check to see what other packages may be removed before committing. I can’t remember which were successful!
        I use dnf with cache enabled and mounted on an nfs share, useful with 6 machines to maintain.

        dnf reinstall nvidia\* –release=24
        dnf downgrade xorg\* –releasever=24
        dnf downgrade gdm\* –releasever=24
        dnf remove mlt-freeworld
        dnf downgrade gnome\* –releasever=24 –allowerasing –exclude=mlt\*
        dnf install –releasever=24 akmod-nvidia nvidia-driver nvidia-driver-cuda nvidia-driver-cuda-libs nvidia-driver-cuda-libs.i686 nvidia-driver-libs nvidia-driver-libs.i686 nvidia-libXNVCtrl nvidia-modprobe nvidia-persistenced nvidia-settings
        akmods –force
        grub2-mkconfig -o /boot/grub2/grub.cfg
        dnf install vlc vlc-core –releasever=24 –allowerasing

        Now I got my main X system working again, but without firefox as it’s caught in a Catch 22 situation.
        All my servers are running fc25 fine in text mode.

        My advice to anyone wanting to upgrade to fc25 and use X with the nvidia drivers is to hold off just a little bit longer until we know it’ll work.

  12. Any chance of updating the CUDA packages for Fedora 24 to 8.0? I need the new version but I’d rather not upgrade to Fedora 25 while it’s still in beta.

    Either way though, much appreciated!

  13. On Fedora 25, this happens with the new libglvnd packages. Some libraries still need to be hidden unless you are going to start recompiling mesa as well:

    Error: Transaction check error:
    file /usr/lib64/libGL.so.1 from install of libglvnd-glx-1:0.2.999-2.20161017gita813b56.fc25.x86_64 conflicts with file from package mesa-libGL-12.0.3-2.fc25.x86_64

      1. I had a conflict with 32-bit wine/playonlinux because the copr repo doesn’t provide the i686 packages on x86_64. I worked around it by manually creating a new .repo file pointing to the i386 packages. I’m also currently withholding the new xorg packages due to nvidia incompatibilities. Any idea if the new beta driver supports xorg 1.19?

      2. Actually, I still seem to be having upgrade issues:

        Error: package nvidia-driver-libs-2:370.28-6.fc25.x86_64 conflicts with libglvnd-egl(x86-64) >= 0.1.1 provided by libglvnd-egl-1:0.2.999-2.20161017gita813b56.fc25.x86_64.
        package nvidia-driver-libs-2:370.28-6.fc25.i686 conflicts with libglvnd-egl(x86-32) >= 0.1.1 provided by libglvnd-egl-1:0.2.999-2.20161017gita813b56.fc25.i686.
        package nvidia-driver-libs-2:370.28-4.fc25.i686 requires nvidia-driver = 2:370.28-4.fc25, but none of the providers can be installed

        1. Ok, Mesa with GLVND enabled and Nvidia drivers without a libEGL.so.1 can not co-exist, as mesa-libEGL requires libglvnd-egl. I hope Nvidia will release soon drivers compatible with libglvnd’s implementation or it will be a complete mess. Nvidia library is *similar* to the one in libglvnd but different. If a similar driver reaches the release date of Fedora 25 there will be *two* libEGL.so.1 libraries based on libglvnd. Argh.

          As written here:

          https://devtalk.nvidia.com/default/topic/915640/unix-graphics-announcements-and-news/multiple-glx-client-libraries-in-the-nvidia-linux-driver-installer-package/

          “libEGL.so.1, while not a proper GLVND library, depends upon the GLVND infrastructure for proper functionality.”

          I’m reverting the change in libglvnd.

          1. I replied a bit late, but per my comment below, Nvidia just released 375.10 driver with what appeared to be a different libEGL, so you might want to test with that first?

          2. I built 375.10, and indeed it now contains 2 different libEGL.so libraries, one standalone and one based on libglvnd, pretty much like their 2 different libGL.so libraries.

            The documentation/release notes still do not report this change and still use the above wording. I made a test with libglvnd‘s EGL library and it does not work yet:

            $ eglinfo 
            eglinfo: eglInitialize failed
            
            $ strings libEGL.so.1 | grep -i glvnd
            1.5 libglvnd
            /etc/glvnd/egl_vendor.d:/usr/share/glvnd/egl_vendor.d
            %s/.glvndXXXXXX

            So situation is that libglvnd’s EGL is not supported.

            Regarding the additional changes, I made the CUDA package to build on i686 for just the cuda-nvml-devel subpackage, so now nvidia-settings can be built. Will release the drivers in a moment, as soon as I reset EGL status.

      3. It installs fine using that Mesa build. I can’t test if it is actually working yet though.
        I also just updated to Nvidia’s 375.10 driver which requires a few changes.

  14. Hello. I have installed your repo drivers on fresh fedora 25 with 4.8.1-1.fc25.x86_64 with 01:00.0 VGA compatible controller: NVIDIA Corporation GF119M [GeForce GT 520M] (rev a1). Gdm is not starting with “Oh, no” error. Can you help me?

  15. Have been using your NVIDIA repo for nearly a year, and it has been awesome. I use both nvidia-driver and the cuda package for deep learning purposes, and ffmpeg (from your other repo) as well to preprocess data. Now we are going to have a bunch of GTX 1080, CUDA 8.0 support would be nice! Thanks for the amazing work!

      1. I install the latest Fedora anywhere I can; but some of machines are managed by CentOS 7. Only in Fedora we use your repositories.

        The best thing about your CUDA package is the host_config.h file you changed. It can compile even with the constantly updated GCC in Fedora. Some times I need to append a -std=c++03 for legacy cuda code, but other than that everything works smoothly.

        I personally think the compiler version check is complete BS from NVIDIA’s end. Forbidding the user from compiling code using a new compiler is utterly wrong. The correct approach is to give linking or run-time error if the code compiled does not work (but why on earth wouldn’t it work if the ABI is used correctly, regardless of compiler version?). Thanks to your amazing work this is not a problem in Fedora at all.

          1. Thanks! I just saw the new versions from dnf today. Will test in the next week and let you know how it goes.

            Thanks for adding cudnn as well, BTW!

          2. Every works smoothly for the driver and CUDA, but only one problem for cudnn on Fedora 24 x64:

            # dnf install cuda-cudnn
            Last metadata expiration check: 3:16:26 ago on Wed Oct 26 12:18:50 2016.
            Error: nothing provides cuda = 1:8.0 needed by cuda-cudnn-1:5.1-1.fc24.x86_64
            (try to add ‘–allowerasing’ to command line to replace conflicting packages)

            I use my own libcudnn.so for now so it is not a serious issue. Thanks for the great work!

          3. Yep, that’s a leftover of a useless dependency. You can just install it with rpm -Uvh --nodeps http://negativo17.org/repos/nvidia/fedora-25/x86_64/cuda-cudnn-5.1-1.fc25.x86_64.rpm.
            New build coming in a few minutes.

            Thanks for reporting.

          4. I used the cuda 8.0 repo for a few days (including cudnn with fixed dependency) for training my neural nets, and everything works smoothly. Just want to say thanks!

    1. Yeah, just saw that as well. libglvnd 0.2.x It is not supported by the Nvidia drivers which still require version 0.1. Since there is support for nothing yet, if installed will just make things crash. Maybe it was better to coordinate the update containing libglvnd with the one containing Mesa support into one.

      I’m rebuilding 0.1.1 with an updated Epoch, so Nvidia drivers will keep running.

      1. Many thanks for the response! My whole lab uses your Fedora CUDA repo for ML/Deep Learning research so we cannot thank you enough. Some software, such as Theano, expects the CUDA normal directory structure but that is the only real problem we have had. Mostly we use TensorFlow now so it’s really not a problem.

        Thanks again.

          1. Amazing work., thank you very much! I also see that you have added CuDNN 5.1 into the repo as well. Is it recommended to remove the old CUDA install before upgrading?

  16. Hello,

    I reinstalled the drivers (# dnf install nvidia-driver dkms-nvidia nvidia-driver-libs.i686) (I tested nouveau driver) and I noticed that nvidia-settings wasn’t installed. I installed it manually (# dnf install nvidia-settings). Is it intended ?

    Thanks !

    1. Hello, yes it is, since a few days. Unfortunately I had no time to create a blog post about this.
      I’ve been contacted by Red Hat, they asked me to add AppStream metadata for the Nvidia repository (Fedora 25+) and make nvidia-settings optional. Will try to make a post this weekend.

  17. After F24 did update the kernel to 4.7.2-201, I think all my computers with Nvidia cards, had the graphical start-up destroyed. I have a mixed setup on several computers and is using both the 340xx driver and the 367 depending of the computer.
    On one of the work computers (Geforce GT650, using akmod and the 67.44-1.fc24@fedora-nvidia), shutdown takes a looooooong time.. like 6-10mins (sometime start-up is also slow)

    After I did read the /var/log/messages, I realized that the system tries to build a (a)kmod driver for each kernel on shutdown, but fails for some of the “extra/older” kernels I have in my boot list.

    Like here (from /var/log/messages, only akmod log entries) some builds succeed and some fails

    Sep 8 07:32:59 akmods-shutdown: Building modules for all installed kernels.
    Sep 8 07:32:59 akmods-shutdown: Checking kmods exist for 4.6.3-300.fc24.x86_64[ OK ]
    Sep 8 07:34:21 akmods-shutdown: Building and installing nvidia-kmod[ OK ]
    Sep 8 07:34:21 akmods-shutdown: Checking kmods exist for 4.6.3-300.fc24.x86_64+debug[ OK ]
    Sep 8 07:35:30 akmods-shutdown: Building and installing nvidia-kmod[FAILED]
    Sep 8 07:35:30 akmods-shutdown: Building rpms failed; see /var/cache/akmods/nvidia/367.44-1-for-4.6.3-300.fc24.x86_64+debug.failed.log for details
    Sep 8 07:35:31 akmods-shutdown: Hint: Some kmods were ignored or failed to build or install.
    Sep 8 07:35:31 akmods-shutdown: You can try to rebuild and install them by by calling
    Sep 8 07:35:31 akmods-shutdown: ‘/usr/sbin/akmods –force’ as root.
    Sep 8 07:35:33 akmods-shutdown: Checking kmods exist for 4.6.4-301.fc24.x86_64[ OK ]
    Sep 8 07:35:33 akmods-shutdown: Checking kmods exist for 4.6.4-301.fc24.x86_64+debug[ OK ]
    Sep 8 07:36:42 akmods-shutdown: Building and installing nvidia-kmod[FAILED]
    Sep 8 07:36:42 akmods-shutdown: Building rpms failed; see /var/cache/akmods/nvidia/367.44-1-for-4.6.4-301.fc24.x86_64+debug.failed.log for details
    Sep 8 07:36:42 akmods-shutdown: Hint: Some kmods were ignored or failed to build or install.
    Sep 8 07:36:42 akmods-shutdown: You can try to rebuild and install them by by calling
    Sep 8 07:36:42 akmods-shutdown: ‘/usr/sbin/akmods –force’ as root.
    Sep 8 07:36:44 akmods-shutdown: Checking kmods exist for 4.7.2-201.fc24.x86_64[ OK ]
    Sep 8 07:36:44 akmods-shutdown: Checking kmods exist for 4.7.2-201.fc24.x86_64+debug[ OK ]
    Sep 8 07:37:54 akmods-shutdown: Building and installing nvidia-kmod[FAILED]
    Sep 8 07:37:54 akmods-shutdown: Building rpms failed; see /var/cache/akmods/nvidia/367.44-1-for-4.7.2-201.fc24.x86_64+debug.failed.log for details
    Sep 8 07:37:54 akmods-shutdown: Hint: Some kmods were ignored or failed to build or install.
    Sep 8 07:37:54 akmods-shutdown: You can try to rebuild and install them by by calling
    Sep 8 07:37:54 akmods-shutdown: ‘/usr/sbin/akmods –force’ as root.

    Then sometime on boot, the system again tries to build for older kernels. (and the boot takes loooOOoong time)

    Is it somehow possible to configure for only building for the newest kernel ?

    (BTW. I wonder why the 4.7.2-201… says “OK” for exist of the kmod, but fails on installing.. (int the log above), when the computer on boot, boots really fast with that driver..)

    Oh.. And thanks for a great site, and all the work you put into it.

    1. From the logs above it seems you are building for both kernel and kernel-debug. Are you booting into it or the normal one?
      If you don’t need it, can you remove all kernel-debug components and see if that happens?

      # dnf remove kernel-debug\*
    1. Sure, I will do. The problem is that it has to be supported by both Gstreamer and FFMpeg. I haven’t yet checked if it’s compatible.

      1. NVENC 7.0.1 is available. It is compatible with Gstreamer bad plugins 1.8, 1.9 and FFMpeg 2.8.7 and 3.1.2.
        Rebuilds of all those components are now available.

  18. Hello! Fedora newbie is here.
    After fresh installing and updating F24 I tried to install nvidia drivers for my GeForce GTX 560. But failed. What I’ve done:
    installed your repository
    after that “dnf install nvidia-driver dkms-nvidia” like in your example.
    Reboot… Blank white screen with error cycling.

    Do you have step-by-step instructions? Should I follow steps under “specific driver installation”?

    1. Apart from that, you should make sure you are installing kernel-devel from the command line. If you are relying on its own dependencies, as explained in the comments above, you hit a depsolv bug that actually pulls in kernel-debug-devel. If it did on your system, remove it, and install the one without debug. It’s listed in the commands required in the driver page.

Leave a Reply