CUDA enabled programs

There is a new repository available with CUDA enabled programs in package format. This contains programs that have been linked to CUDA libraries or have CUDA support enabled. At the moment this is available only on Fedora 21, if there is sufficient feedback I will enable it also for other distributions.

The repository is available here.

At the moment of writing, the repository contains the following:

  • Nvidia Encoder (NVENC) enabled FFMpeg
  • CUDA and FFMpeg enabled Blender
  • ccminer (multi cryptocoin miner)

Please keep in mind that these packages update already available packages that are in Fedora and RPMFusion, so you might step into other dependency issues if you have tons of other programs using these libraries on your system.

Also, CUDA 6.5 works only on fairly recent GPUs, so your old models might not be compatible. Follow this Nvidia link to check if your GPU is CUDA capable. Basically, anything with Compute Capability >= 2.0 is ok. Also the Wikipedia article on CUDA has all the Compute Capability information available.

The Blender build looks for libcuda and the Nvidia Unified Video Memory module on the system to enable CUDA support. By installing the Nvidia driver from my repository with CUDA support, you should have a working Blender installation that is able to use your GPU for rendering.

Screenshot from 2015-04-27 20:03:57

As you can see from the picture, I only have the basic Nvidia driver plus CUDA support installed, there is no need to install the full CUDA stack. The runtime is required only for additional specific libraries (like ccminer that uses some specific CUDA math libraries).

This build of Blender looks for libcuda.so.1; so if you are using the official Blender build from blender.org, you also need to have the full nvidia-driver-devel package installed as it contains also the libcuda.so symlink.

To recap, this is what is required to have a fully working Blender with CUDA and FFMpeg support enabled on Fedora 21:

# yum-config-manager --add-repo=https://negativo17.org/repos/fedora-cuda-programs.repo
# yum-config-manager --add-repo=https://negativo17.org/repos/fedora-nvidia.repo
# yum install kernel-devel akmod-nvidia nvidia-driver nvidia-driver-cuda blender

Then reboot, and make sure the nvidia-uvm.ko module is loaded.

Repositories update

Another batch of changes has reached the repositories in the last weeks. The updates have touched everything:

  • MakeMKV has been updated to version 1.9.2.
  • HandBrake has been updated to the current 0.10.1 for Fedora 20, 21 and 22.
  • libdvdnav and libdvdread are based on 5.0.x releases. Also, all required libraries for MakeMKV and HandBrake have been rebuilt for Fedora 22 until RPMFusion catches up, at which point I will remove them from the repository.
  • The Nvidia driver is now at version 346.59 for all CentOS/RHEL and Fedora variants and at 349.12 (beta) for Fedora 22. According to the release notes, the 319.42 drivers will try to copy the framebuffer console’s contents out of /dev/fb0 making the Plymouth transition even better on UEFI systems.
  • The Nvidia driver is now using libvdpau 1.1 which contains support for decoding H.265/HEVC video streams (VDPAU Feature Set F) on supported GPUs.
  • The GPU Deployment kit (NVML headers and docs) is now at version 346.46. Also, nvidia-healthmon is now 64 bit only.
  • The CDRtools suite has been updated to version 3.01a28.
  • The Flash plugin package has been updated to version 11.2.202.451 and it has also been pushed to RPMFusion in the form of lpf package.
  • Spotify it’s at version 0.9.17.1 for CentOS/RHEL 7 and Fedora 20, 21 and 22. Instead of bundling Ubuntu’s libgcrypt library; now it uses the compat-libgcrypt package from ssaavedra/compat-libcrypt COPR repository. The package is already included where appropriate.
  • Wimms’ ISO tools have been updated to 2.30a.
  • RAR has been updated to version 5.2.1.

As always, any issue just let me know. I’m in the process of moving and starting a new job; so please bear some patience if I don’t reply to you immediately.

Big update to the Nvidia driver repository (346.xx, 340.xx compat, CUDA)

My personal Nvidia repository has seen quite a few updates on versions, CUDA enablements, legacy drivers and Delta RPMS.

Long Lived branch

Version 346.35 is now the new Long Lived branch release, this, plus the fact that is the newest made it to all supported distributions (CentOS/RHEL 6/7, Fedora 20/21/rawhide).

Here is the table that lists the current versions:

Operating systemCentOS / RHELFedorarawhide
Driver branchLong LivedShort Lived
Long Lived
Short Lived
Long Lived
Beta
Video Codec SDKYesYesYes
Architectures:

x86_64
YesYesYes
Basic nvidia driver:

nvidia-driver
nvidia-driver-libs
nvidia-libXNVCtrl
nvidia-kmod-common
YesYesYes
CUDA libraries and tools:

nvidia-driver-cuda
nvidia-driver-cuda-libs
nvidia-driver-NVML
nvidia-persistenced
YesYesYes
OpenGL Framebuffer Capture:

nvidia-driver-NvFBCOpenGL
YesYesYes
Nvidia tools:

nvidia-modprobe
nvidia-settings
nvidia-xconfig

YesYesYes
Binary kernel
modules (kABI):

kmod-nvidia
YesNoNo
DKMS kernel
modules:

dkms-nvidia
YesYesYes
aKMOD kernel
modules:

akmod-nvidia
NoYesYes
32 bit compatibility on x86_64:

nvidia-libXNVCtrl
nvidia-driver-libs
nvidia-driver-cuda-libs
nvidia-driver-NVML
YesYesYes
Development

nvidia-driver-devel
nvenc
nvenc-samples
YesYesYes
GLVND librariesYesYesYes
VDPAU librariesYesYesYes
EGLStream-based Wayland external platformYesYesYes
GBM EGL external platform libraryYesYesYes

CUDA stack

A complete packaged CUDA stack has been added for all supported distributions. This now includes all CUDA libraries and tools at version 6.5.19 (includes NVML / GPU deployment kit). You can easily install CUDA 6.5 on CentOS/RHEL 6/7 and Fedora 20/21/rawhide!

All the packages provide/require/obsolete the relevant driver packages in the RPMFusion repository and all the CUDA packages in the Nvidia repository; so you can enable this repository along with the official Nvidia CUDA one and RPMFusion at the same time. Packages will get upgraded accordingly.

Nvidia is slowly fading out 32 bit support from CUDA, and you can see it reflected in the various packages. The Unified Video Memory kernel module (nvidia-uvm.ko has been removed in version 346.16, CUDA graphical programs are 64 bit only, many libraries and compilers are available in 64 bit only, etc.

Feedback from users has been integrated, where possible.

List of components by distribution:

Operating systemCentOS / RHELFedorarawhide
CUDA cuDNNYesYesYes
GCC compatibility
(if needed)
-cuda-gcccuda-gcc
Basic CUDA libraries/tools:

cuda
cuda-libs
cuda-cuda
cuda-cli-tools
cuda-cublas
cuda-cudart
cuda-cufft
cuda-cupti
cuda-curand
cuda-cusolver
cuda-cusparse
cuda-extra-libs
cuda-libs
cuda-npp
cuda-nvjpeg
cuda-nvrtc
cuda-nvtx
cuda-nvvp
cuda-samples
cuda-sanitizer
YesYesYes
CUDA development:

cuda-cccl-devel
cuda-cudart-devel
cuda-cudnn-devel
cuda-cupti-devel
cuda-cuxxfilt-devel
cuda-devel
cuda-nvml-devel
cuda-nvprof-devel
cuda-nvrtc-devel
cuda-nvtx-devel
cuda-profiler-devel
cuda-sanitizer-devel
YesYesYes
GUI programs:

cuda-nsight
cuda-nvvp
cuda-nsight-compute
cuda-nsight-systems
YesYesYes

Legacy drivers 340.xx

A compatibility repository for drivers on 340.x, the new legacy release for cards up to 9xxx chipsets has been introduced. It’s in the same place, just follow the instructions by appending -340 to the repository file. This repository does not include the CUDA packages, just the enablement on the drivers.

The repository itself it’s not guaranteed to stay online forever; the GTX 9xxx series are from 2008 and I don’t guarantee I will maintain it for long.

Delta RPMS

Delta RPMS have been introduced, to reduce the time and data required for upgrades. Driver packages can reach 90 mb and CUDA packages can span even 650 mb. This would save you a lot of time into upgrading them. For now, delta RPMS have been generated for the new 346.35 drivers, and this reduced nearly 80% the download size on Fedora 21.

We’ll see some real gain when updating the CUDA packages.

Ending words

Along this, there is the usual assortment of packages refinement (syntax, RPMLint, optimizations, etc.). For additional details, please see the Nvidia driver page.

As time permits, new CUDA enabled packages will be added to the repository, namely Blender, ccMiner, NVENC enabled ffmpeg, etc.

As usual, any feedback is much appreciated!

Experimental CUDA packages

nv-cuda-2014header-updated

I’ve created experimental CUDA packages that try to follow Fedora packaging guidelines as close as possible. Those have been updated to the Nvidia Fedora 20 repository, and are installable through normal yum commands.

To install them, you need to use my repository that contains the latest drivers.

32 bit support

Nvidia is slowly fading out 32 bit support from CUDA, and you can see it reflected in the various packages. The Unified Video Memory kernel module (nvidia-uvm.ko has been removed in version 346.16, CUDA graphical programs are 64 bit only, many libraries and compilers are available in 64 bit only, etc.

Package testing

I’ve uploaded packages only to the Fedora 20 repository, as they are very big and this is what I’m using at the moment as my main desktop. To help test these, I’ve also added a package for ccminer, a CUDA cryptocurrency miner that links to the packages and requires them to be built. By installing it, all required CUDA runtime libraries should be installed as well.

If all goes well, my plan is to enable CUDA packages for all supported Fedora/CentOS/RHEL distributions and add also package software that in the current form do not use the Nvidia libraries.

As an example, the Blender package in Fedora does not (obviously) link to the CUDA libraries, so no CUDA rendering. On the contrary, the binary that you can download from the Blender website is linked statically to the CUDA libraries at compile time.

After some feedback I will enable them for all the other distributions. So if you need them, please test them.

Packages available

A brief recap on the packages, here we have the full list of drivers and CUDA packages that are available inside the repository folder:

$ ls -1 *nvidia* *cuda*
akmod-nvidia-343.22-2.fc20.x86_64.rpm
cuda-6.5.19-2.fc20.x86_64.rpm
cuda-cli-tools-6.5.19-2.fc20.x86_64.rpm
cuda-devel-6.5.19-2.fc20.i686.rpm
cuda-devel-6.5.19-2.fc20.x86_64.rpm
cuda-docs-6.5.19-2.fc20.noarch.rpm
cuda-extra-libs-6.5.19-2.fc20.i686.rpm
cuda-extra-libs-6.5.19-2.fc20.x86_64.rpm
cuda-libs-6.5.19-2.fc20.i686.rpm
cuda-libs-6.5.19-2.fc20.x86_64.rpm
cuda-nsight-6.5.19-2.fc20.x86_64.rpm
cuda-nvvp-6.5.19-2.fc20.x86_64.rpm
dkms-nvidia-343.22-2.fc20.x86_64.rpm
kmod-nvidia-343.22-2.fc20.x86_64.rpm
nvidia-driver-343.22-1.fc20.x86_64.rpm
nvidia-driver-cuda-343.22-1.fc20.x86_64.rpm
nvidia-driver-cuda-libs-343.22-1.fc20.i686.rpm
nvidia-driver-cuda-libs-343.22-1.fc20.x86_64.rpm
nvidia-driver-devel-343.22-1.fc20.i686.rpm
nvidia-driver-devel-343.22-1.fc20.x86_64.rpm
nvidia-driver-libs-343.22-1.fc20.i686.rpm
nvidia-driver-libs-343.22-1.fc20.x86_64.rpm
nvidia-driver-NvFBCOpenGL-343.22-1.fc20.i686.rpm
nvidia-driver-NvFBCOpenGL-343.22-1.fc20.x86_64.rpm
nvidia-driver-NVML-343.22-1.fc20.i686.rpm
nvidia-driver-NVML-343.22-1.fc20.x86_64.rpm
nvidia-driver-NVML-devel-340.29-1.fc20.i686.rpm
nvidia-driver-NVML-devel-340.29-1.fc20.x86_64.rpm
nvidia-healthmon-340.29-1.fc20.x86_64.rpm
nvidia-libXNVCtrl-343.22-1.fc20.i686.rpm
nvidia-libXNVCtrl-343.22-1.fc20.x86_64.rpm
nvidia-libXNVCtrl-devel-343.22-1.fc20.i686.rpm
nvidia-libXNVCtrl-devel-343.22-1.fc20.x86_64.rpm
nvidia-modprobe-343.22-1.fc20.x86_64.rpm
nvidia-persistenced-343.22-2.fc20.x86_64.rpm
nvidia-settings-343.22-1.fc20.x86_64.rpm
nvidia-xconfig-343.22-1.fc20.x86_64.rpm

Package bundles

From the above list, packages can be grouped as follows for an x86_64 system. For additional details, please see the repository page.

Kernel modules, in both akmod and dkms variants. Instantiated kernel modules are available as rebuild in both by enabling the appropriate configuration on your system.

akmod-nvidia
dkms-nvidia
kmod-nvidia

These are the basic driver packages, they are what is required along the kernel module packages to have accelerated drivers and full OpenGL support for a normal desktop. That is gaming, office use, etc. But no CUDA support.

nvidia-driver
nvidia-driver-libs.i686
nvidia-driver-libs
nvidia-libXNVCtrl
nvidia-libXNVCtrl.i686
nvidia-settings

Then we have the CUDA libraries and tools that are part of the drivers and not of the full CUDA toolkit, as they are closely tied to the drivers:

nvidia-driver-cuda
nvidia-driver-cuda-libs.i686
nvidia-driver-cuda-libs
nvidia-persistenced

Then there are extra tools and libraries, like Framebuffer Compression OpenGL libraries, GPU Deployment Kit (NVML, also called Nvidia Management Library), command line configuration for very specific X.org setups and a tool that leverages the NVML library to perform health checks on GPU clusters.

Please note that the Nvidia Management Library headers and tools do not follow the same versioning of the main driver set as they are provided by Nvidia in a separate bundle that is compatible across multiple releases of the drivers. For example, at the time of writing this, we have 340.58 (long lived), 343.22 (short lived) and 346.18 (beta) drivers available.

All works with version 340.29 of the NVML libraries.

nvidia-driver-NvFBCOpenGL.i686
nvidia-driver-NvFBCOpenGL
nvidia-driver-NVML.i686
nvidia-driver-NVML
nvidia-healthmon
nvidia-modprobe
nvidia-xconfig

Lastly, we have all the development files (unversioned library symlinks, headers and documentation) for compiling programs that link to the above driver libraries.

nvidia-driver-devel.i686
nvidia-driver-devel
nvidia-driver-NVML-devel.i686
nvidia-driver-NVML-devel

After those, that have been provided here for more than a year, I’ve now added CUDA packages. These can be splitted into multiple components as well; first group contains most runtime components for simply running CUDA enabled programs:

cuda-libs.i686
cuda-libs
cuda-extra-libs.i686
cuda-extra-libs

Then we have development files (headers, stub libraries, documentation, compilers, etc.) for compiling programs that link to the CUDA libraries:

cuda
cuda-cli-tools
cuda-devel.i686
cuda-devel
cuda-docs.noarch

And then finally, we have Java GUI programs (debuggers, etc.):

cuda-nsight
cuda-nvvp

Official Nvidia repositories

All the packages provide/require/obsolete the relevant driver packages in the RPMFusion repository and all the CUDA packages in the Nvidia repository; so you can enable this repository along with the official Nvidia CUDA one and RPMFusion at the same time. Packages will get upgraded accordingly.

Fedora Nvidia drivers: OutputClass for X server 1.16, GPU deployment kit, CUDA enablement, RHEL7 kABI modules

The Nvidia repository has been updated; here is the table that lists the current versions:

Operating systemCentOS / RHELFedorarawhide
Driver branchLong LivedShort Lived
Long Lived
Short Lived
Long Lived
Beta
Video Codec SDKYesYesYes
Architectures:

x86_64
YesYesYes
Basic nvidia driver:

nvidia-driver
nvidia-driver-libs
nvidia-libXNVCtrl
nvidia-kmod-common
YesYesYes
CUDA libraries and tools:

nvidia-driver-cuda
nvidia-driver-cuda-libs
nvidia-driver-NVML
nvidia-persistenced
YesYesYes
OpenGL Framebuffer Capture:

nvidia-driver-NvFBCOpenGL
YesYesYes
Nvidia tools:

nvidia-modprobe
nvidia-settings
nvidia-xconfig

YesYesYes
Binary kernel
modules (kABI):

kmod-nvidia
YesNoNo
DKMS kernel
modules:

dkms-nvidia
YesYesYes
aKMOD kernel
modules:

akmod-nvidia
NoYesYes
32 bit compatibility on x86_64:

nvidia-libXNVCtrl
nvidia-driver-libs
nvidia-driver-cuda-libs
nvidia-driver-NVML
YesYesYes
Development

nvidia-driver-devel
nvenc
nvenc-samples
YesYesYes
GLVND librariesYesYesYes
VDPAU librariesYesYesYes
EGLStream-based Wayland external platformYesYesYes
GBM EGL external platform libraryYesYesYes

Quite a few things have changed, the packages are going towards providing the complete packaged CUDA stack. So far, only the GPU deployment kit has been inclued; and the packages allow for parallel installation with the Nvidia CUDA repositories by osboleting/updating packages as required. Here is some details on the things that have been implemented.

X.org configuration

  • Starting from Fedora 21, all driver X.org configuration can be managed by simply adding/removing X.org configuration snippets in /etc/X11/xorg.conf.d.
  • Use new OutputClass directive on Fedora 21 X.org server 1.16 (and later) to load the driver and do not rely on an edited /etc/X11/xorg.conf file. This also removes editing of the xorg.conf file from the package scriptlets.
  • Add the IgnoreABI directive by default on Fedora rawhide builds.

Kernel modules

  • Add a new UDev rule in nvidia-driver-cuda for the nvidia-uvm module and make X.org NVIDIA Files section to be loaded latest in case there are other packages providing a custom Files section (thanks Jan P. Springer for spotting these).
  • The binary nvidia-modprobe is now SETUID, but its package is no longer a mandatory requirement for the drivers, so it will not get installed by default.
  • Now that both Redhat Enterprise Linux 7 and CentOS 7 have been released, binary modules (kABI) are now provided for these distributions.

CUDA support

  • Added the GPU Deployment kit to the repository. This is constructed with NVML (NVIDIA Management Library) included with the drivers plus headers, docs and samples from a separate tarball. The separate tarball is using a different version number than the drivers. This is packaged in the nvidia-driver-NVML and nvidia-driver-NVML-devel packages. Installing these, the gpu-deployment-kit dependency provided by the CUDA repositories is preserved.
  • Along with NVML, the nvidia-healthmon package is provided to monitor TESLA GPU clusters.

Along this, there is the usual assortment of packages refinement (syntax, RPMLint, optimizations, etc.). For additional details, please see the Nvidia driver page.

If you would like to test the CUDA packages please contact me and I will point you to a repository hosting the CUDA packages.

Another batch of big updates for the repositories (Nvidia, HandBrake, Steam, etc…)

Another batch of changes for the repositories. The HandBrake repository has seen some updates:

  • MakeMKV has been updated to version 1.8.11.
  • HandBrake has been updated to the current development version for Fedora 21. It contains additional/different bundled libraries, but the situation is not that bad.
  • libdvdnav is now based on a 5.0.0 snapshot that contains all the fixes required to avoid the HandBrake crash while opening a DVD for scanning with the default settings. Removal of the flag “Use dvdnav (instead of libdvdread)” in the preferences panel is no longer required.

Please test it, I found it much more stable than the previous version. If it works, I might build it also for current Fedora releases.

handbrake-svn

Regarding the other updates:

  • The Nvidia driver is now at version 340.24 for all CentOS/RHEL and Fedora variants.
  • Nvidia Fedora 21 packages have now aKMOD support
  • Nvidia CentOS/RHEL 7 packages have now binary kABI modules, so you can install the binaries directly without relying on DKMS.
  • The CDRtools suite has been updated to version 3.01a24.
  • The Steam package has been updated to version 1.0.0.48-2, the same build has been pushed to RPMFusion. The repository hosted here still contains other SteamOS packages.
  • The Flash plugin package has been updated to version 11.2.202.378 and Skype has been updated to version 4.3.0.37. Both versions have also been pushed to RPMFusion in the form of lpf packages.
  • Spotify it’s still at version 0.9.10; as the x86_64 and i386 builds share the same data package. Upstream has updated the x86_64 build to 0.9.11 but at the same time has reverted the i386 build to 0.9.4, so all Ubuntu users will not get the update anymore. Until this is sorted out, I’m going to leave the current package versions as 0.9.10 for systems that are both 32 and 64 bit. RHEL/CentOS 7 (which is 64 bit only) has the new 0.9.11 version.

As always, any issue just let me know.

Complex setup with Nvidia Optimus / Nouveau Prime on Fedora 20

This is a re-edit of my previous post about using and configuring an Optimus enabled laptop in Fedora 20/21. It should work on other distributions as well.

optimus_technology_badge

My laptop at work is a Dell Latitude E6430. Comes loaded with features and I really like it. Among the various features there’s the fact that this is an Nvidia Optimus enabled laptop, sporting both an Intel video card and an Nvidia one:

$ lspci | grep -i vga
00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor Graphics Controller (rev 09)
01:00.0 VGA compatible controller: NVIDIA Corporation GF108GLM [NVS 5200M] (rev a1)

This one is a muxless laptop of the worst kind: video outputs are connected only to specific chips!

Update 23rd May 2014: UEFI console (using efifb) on external monitors has been fixed with bios update version “A14”. Now the UEFI console is also rendered on external monitors if the lid is closed; so I’ve updated the guide.

LVDS (Internal panel)Intel
VGA (not usable along with the docking station one)Intel
VGA (Docking station)Intel
DVINvidia
DVI (Docking station)Nvidia
DisplayPort (Docking station)Nvidia
HDMINvidia

So to use an external HDMI connection at home you need to drive it through the Nvidia card, it doesn’t matter if Optimus is enabled or not. I regularly use it docked with the lid closed, external keyboard and mouse and 2 external monitors connected to the VGA and DVI outputs of the docking station. Basically while I’m at the office it looks like a normal desktop computer; but sometime I need to disconnect it to go on a meeting; and sometimes I use it at home to play games as well.

Guess what? Free drivers, proprietary drivers, UEFI, UEFI secure boot, multi monitor, outputs changing on the fly… all sorts of fun! I’m impressed by the fact that it all works together.

There are four modes on which I can operate the system:

  • Optimus enabled, free drivers for both Intel and Nvidia cards (implementation is called “Prime”)
  • Optimus enabled, free driver for Intel and proprietary driver for the Nvidia card
  • Optimus disabled, free driver for the Nvidia card
  • Optimus disabled, proprietary driver for the Nvidia card

Each one has its drawbacks, so let’s explain each setup a bit. At the end of the post I’ve made a table with all the pros and cons of each solution.

My current setup is:

  • Fedora 20 x86_64
  • Kernel 3.14.4 (stock Fedora)
  • Nouveau DDX 1.0.9 (stock Fedora)
  • Intel DDX 2.21.15 (stock Fedora)
  • Nvidia proprietary drivers 337.19 (from my repository)
  • VDPAU library 0.7 (stock Fedora)
  • Mesa libraries 10.1.3 (20140509 prerelease, stock Fedora)

UEFI / legacy bios

If secure boot is enabled; there’s no way to use the proprietary Nvidia driver without fiddling with UEFI keys. The module is built separately from the kernel package; so there’s no way for it to have the same signature as the kernel.

When UEFI is enabled, the free drivers work fine and replace the efifb framebuffer driver with their own; thus giving proper modesetting at the correct resolution and a speedy and responsive terminal.

With the proprietary Nvidia driver, the efifb is not replaced; so the console still operates with it and the Nvidia driver only operates the X part. Unfortunately, using this method, the framebuffer console is really slow, the resolution is not optimal, and the EFI framebuffer is exposed onto external monitors only from bios version “A14”. Before the update, pressing CTRL+ALT+Fx jumped me to the console that is shown in the closed laptop lid on the docking station; making it pretty useless.

What UEFI could bring you is the Intel Rapid Start Technology which has been included in kernel 3.11; so make your choices depending on what you need.

Optimus disabled (Nouveau or Nvidia)

When Optimus is disabled, I can freely use the proprietary Nvidia driver or the free Nouveau driver.

Both solutions work; unfortunately performance and feature wise Nouveau cannot compete with the proprietary Nvidia driver.

My main issue is power management; with the Nvidia driver the battery lasts a lot more and the performance difference is abysmal. Nouveau performance is really poor with 3D games (especially Steam commercial ones, with Doom 3 it works fine) and there’s absolutely no power management; at least on my laptop. By playing with performance levels I was only able to overheat the card.

Another thing that does not work with Nouveau is the docking station removal. With the Nvidia proprietary driver I’m able to do the following:

– Disconnect from the docking station: output goes from the external VGA and DVI monitors to the internal LVDS display.

– Reconnect to the docking station: internal LVDS display gets shut off and output goes to VGA and DVI monitors as they were before; one next to the other. I can even close the lid and the computer doesn’t go in standby.

With Nouveau, I’m able to disconnect from the docking station but when reconnecting I need to reconfigure the monitors in their place; and after this, when closing the lid I need to wake up again the computer because it goes on standby.

With the recent Xrandr support to the proprietary drivers I don’t even need to edit che X.org configuration file. Whether I use nvidia-settings or Gnome Displays panel the result is reflected in both implementations and preserved across boots.

Optimus enabled (Nvidia)

To configure Optimus with proprietary drivers perform the following. First of all install the proprietary driver as normal. Now edit the /etc/grub2.cfg file and remove some parameters from the kernel command line. This is required because the Intel driver still need to operate with its KMS driver. So, from this:

nouveau.modeset=0 rd.driver.blacklist=nouveau nomodeset gfxpayload=vga=normal

you should go to this:

nouveau.modeset=0 rd.driver.blacklist=nouveau

After this, edit/recreate the /etc/X11/xorg.conf file with the following contents:

Section "Device"
    Identifier "nvidia"
    Driver "nvidia"
    Option "NoLogo" "true"
    Option "DPI" "96 x 96"
    # Specify Nvidia PCI device
    BusID "PCI:1:0:0"
    # Make sure X starts also when no outputs are connected to the Nvidia chip
    Option "AllowEmptyInitialConfiguration"
EndSection

# Slave device
Section "Device"
    Identifier "intel"
    # Simple output, no full Intel driver
    Driver "modesetting"
    # BusID "PCI:0:2:0"
EndSection

Section "Screen"
    Identifier "intel"
    Device "intel"
EndSection

Section "Screen"
    Identifier "nvidia"
    Device "nvidia"
EndSection

# Make sure the Nvidia device is the first in the server
Section "ServerLayout"
    Identifier "layout"
    Screen 0 "nvidia"
    Inactive "intel"
EndSection

Make sure to set the correct bus ID for the Nvidia card; for instructions look in the Nvidia documentation. Make sure the integrated Intel card is using modesetting driver and not the native one.

The AllowEmptyInitialConfiguration directive in the Nvidia section is required to let X start with the Nvidia driver without any output attached to the chip, for example while using the internal LVDS monitor.

Upon reboot, you will see KMS running for the Intel card (Plymouth screen) and then the login manager appears on the Nvidia attached panels, while the Intel outputs shut off.

After logging in, you can also check that both drivers are running with the following commands:

$ lsmod | egrep "i915|nvidia"
nvidia              10411255  41 
i915                  796218  2 
i2c_algo_bit           13257  1 i915
drm_kms_helper         50652  1 i915
drm                   283747  5 i915,drm_kms_helper,nvidia
i2c_core               38656  7 drm,i915,i2c_i801,drm_kms_helper,i2c_algo_bit,nvidia,videodev
video                  19261  1 i915

To light up the other display some xrandr command is required:

$ xrandr --setprovideroutputsource Intel NVIDIA-0
$ xrandr --auto

Your Intel monitor should now have an extended desktop managed by the Nvidia card. Move windows around, and launch some commands to see that wherever you go you’re using the Nvidia accelerated driver:

$ glxinfo| grep "OpenGL version string"
OpenGL version string: 4.4.0 NVIDIA 337.19
$ vdpauinfo | grep -i string
Information string: NVIDIA VDPAU Driver Shared Library  337.19  Tue Apr 29 19:51:41 PDT 2014

Everything seems to work, except output manipulation. Xrandr, Gnome and Nvidia drivers have a different view.

Xrandr view:

$ xrandr -q | grep conn
VGA-0 disconnected (normal left inverted right x axis y axis)
LVDS-0 disconnected (normal left inverted right x axis y axis)
DP-0 disconnected (normal left inverted right x axis y axis)
DP-1 connected primary 1680x1050+0+0 (normal left inverted right x axis y axis) 474mm x 296mm panning 3360x1050+0+0
HDMI-0 disconnected (normal left inverted right x axis y axis)
DP-2 disconnected (normal left inverted right x axis y axis)
DP-3 disconnected (normal left inverted right x axis y axis)
LVDS-1-0 connected (normal left inverted right x axis y axis)
VGA-1-0 connected 1680x1050+1680+0 (normal left inverted right x axis y axis) 474mm x 296mm

This is what I have in the Nvidia settings panel and in the Gnome Displays panel for the monitors; in one case I see only one of the external monitors, in the other one I have all monitors:

Primary monitor assignment does not work as well. I usally have the Gnome panel on the left monitor. If I try to move it from the Nvidia output I get this feedback:

$ xrandr --output VGA-1-0 --primary
X Error of failed request:  BadMatch (invalid parameter attributes)
  Major opcode of failed request:  139 (RANDR)
  Minor opcode of failed request:  30 (RRSetOutputPrimary)
  Serial number of failed request:  51
  Current serial number in output stream:  53

Putting monitor problems aside, running in this mode does not really give any benefit compared to running it with Optimus disabled and the proprietary Nvidia driver installed. Both cards are running with power management, but the Nvidia card is never shut off, so it doesn’t use less power than when running standalone.

There’s no way to turn off the card with vga_switcheroo, all libraries come from the Nvidia drivers and your desktop is being rendered by the Nvidia card.

GDM configuration for Optimus (Nvidia)

Let’s assume you want to create the above setup in GDM (Gnome Display Manager), making it run automatically the xrandr commands mentioned above.

You need to create a text file with the display name (usually “:0”) to start the commands upon Init time the appropriate GDM configuration directory:

$ cd /etc/gdm/Init/
$ cat \:0 
#!/bin/sh

# Get the xrandr providers
output="$(xrandr --listproviders)"

src=$(echo "$output" | grep " Source" | head -n1 | awk '{print $NF}' | cut -d: -f2)
sink=$(echo -e "$output" | grep " Sink" | head -n1 | awk '{print $NF}' | cut -d: -f2)

# Pass provider or sink and source
xrandr --setprovideroutputsource "$sink" "$src"

# Make sure xrandr sees all the outputs
# xrandr --auto

# Do not move up. Only now xrandr shows the outputs
lvds=$(xrandr | grep -i "lvds" | head -n1 |cut -d " " -f 1)
xrandr --output "$lvds" --off
xrandr --output "$lvds" --auto

The “–off” and then “–auto” is actually redundant, but it helped in a couple of cases.

Prime enabled (Nouveau)

The Open Source implementation of this is actually called “Prime” for obvious reasons.

Here comes the juicy part. With enough maturity on the Nouveau side this would be the perfect setup. To start with this implementation; nothing is required, just install Fedora and everything should be already set up by default. Booting it shows the Plymouth logo on both outputs.

Login in the system, and check that both drivers are running:

$ lsmod | egrep "i915|nouveau"
nouveau               943445  1 
i915                  651861  4 
mxm_wmi                12865  1 nouveau
ttm                    79865  1 nouveau
i2c_algo_bit           13257  2 i915,nouveau
drm_kms_helper         50239  2 i915,nouveau
drm                   274480  8 ttm,i915,drm_kms_helper,nouveau
i2c_core               34242  7 drm,i915,i2c_i801,drm_kms_helper,i2c_algo_bit,nouveau,videodev
wmi                    18697  3 dell_wmi,mxm_wmi,nouveau
video                  19104  2 i915,nouveau

Poking around with xrandr will give you totally different outputs from the Nvidia driver:

$ xrandr -q | grep conn
LVDS2 connected 1600x900+0+0 (normal left inverted right x axis y axis) 309mm x 174mm
VGA2 connected 1680x1050+1600+0 (normal left inverted right x axis y axis) 474mm x 296mm
LVDS-1-1 disconnected (normal left inverted right x axis y axis)
DP-1-1 disconnected (normal left inverted right x axis y axis)
DP-1-2 connected (normal left inverted right x axis y axis)
HDMI-1-1 disconnected (normal left inverted right x axis y axis)
VGA-1-1 disconnected (normal left inverted right x axis y axis)

But at least they’re consistent with the Gnome Displays panel:

For reasons I don’t understand the Nvidia card appears twice in 2 different but identical providers:

$ xrandr --listproviders
Providers: number : 3
Provider 0: id: 0x96 cap: 0xb, Source Output, Sink Output, Sink Offload crtcs: 3 outputs: 2 associated providers: 2 name:Intel
Provider 1: id: 0x66 cap: 0x7, Source Output, Sink Output, Source Offload crtcs: 2 outputs: 5 associated providers: 2 name:nouveau
Provider 2: id: 0x66 cap: 0x7, Source Output, Sink Output, Source Offload crtcs: 2 outputs: 5 associated providers: 2 name:nouveau

With the tests I made, there’s no apparent difference when using one or the other. Usage of one card or the other is driven by the DRI_PRIME environment variable. If it’s set to 0, commands run on the Intel card, if it’s set to 1 they will run on the Nvidia card. For example:

$ DRI_PRIME=1 vdpauinfo | grep -i string
Information string: G3DVL VDPAU Driver Shared Library version 1.0

Or even better, to check OpenGL status:

$ glxinfo | grep -e 'OpenGL.*string.*'
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) Ivybridge Mobile 
OpenGL core profile version string: 3.3 (Core Profile) Mesa 10.1.3
OpenGL core profile shading language version string: 3.30
OpenGL version string: 3.0 Mesa 10.1.3
OpenGL shading language version string: 1.30
$ DRI_PRIME=1 glxinfo | grep -e 'OpenGL.*string.*'
OpenGL vendor string: nouveau
OpenGL renderer string: Gallium 0.4 on NVC1
OpenGL core profile version string: 3.3 (Core Profile) Mesa 10.1.3
OpenGL core profile shading language version string: 3.30
OpenGL version string: 3.0 Mesa 10.1.3
OpenGL shading language version string: 1.30

Unfortunately the desktop is very slow, it’s rendered by the Intel driver and put on the Nvidia card for display. I’ve tried changing priority in vga_switcheroo prior to starting X, setting the DRI_PRIME=1 variable at boot, use xrandr to change the provider output source etc. to no avail; the desktop can run only on the first card or it doesn’t work. Usually I get a black screen upon GDM start.

There’s no power management as well, so the Intel card runs normally but the Nvidia one is always on and stuck in an intermediate performance level.

When docking it; I get cloned outputs on all external displays at a very low resolution. Same issue with the Optimus disabled Nouveau driver; the outputs need to be rearranged, the lid closed and the computer needs to be woken up from standby.

Prime power management (Nouveau)

Power management for automatic powerup/shutdown of cards in Optimus systems and runtime management has been introduced in kernel 3.12. Is on by default on Optimus enabled laptops and is enabled by default.

The situation has improved a lot in 3.13 and 3.14, and at least on my system it works flawlessly:

# cat /sys/kernel/debug/vgaswitcheroo/switch 
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :DynPwr:0000:01:00.0

As you can see the second card is dynamically powered. Try to undock the system and check the status again: the second output is no longer needed so the second card shuts off:

# cat /sys/kernel/debug/vgaswitcheroo/switch 
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :DynOff:0000:01:00.0

Now, with the laptop undocked, launch a command on the second card:

# DRI_PRIME=1 vdpauinfo | grep -i string
Information string: G3DVL VDPAU Driver Shared Library version 1.0
# cat /sys/kernel/debug/vgaswitcheroo/switch 
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :DynPwr:0000:01:00.0

You will notice a slight delay before the command output is returned, but the card is powered on again! This is awesome. Now, after 1 or 2 seconds look again at the card:

# cat /sys/kernel/debug/vgaswitcheroo/switch 
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :DynOff:0000:01:00.0

It’s shut off! Dock the laptop again and the monitor should come up again.

Keep in mind that powering up and down cards is a totally different things than power managing and adjusting clocks etc. for a running card. This make the Nvidia card shutdown automatically, not regulate its power levels during usage.

Dual cards can be shut down or powered up on demand through vga_switcheroo. For example, login in your system as root without X running and no outputs connected to the Nvidia chip. Look at the card status with the following command:

# cat /sys/kernel/debug/vgaswitcheroo/switch
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :DynOff:0000:01:00.0

This will tell you that the Integrated Graphics Display (IGD) is powered up (Pwr) and that is the primary display (+), while Discrete one (DIS) is dynamically off. To turn on the secondary video card, a single command is required:

# echo > /sys/kernel/debug/vgaswitcheroo/switch
# cat /sys/kernel/debug/vgaswitcheroo/switch
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :On:0000:01:00.0

This will power on the Nvidia card. A look at the battery will tell you now that you have half the power because the Nvidia card sucks power along the integrated Intel one.

Turn off the integrated card (IGD), and switch the framebuffer console to the discrete one (DIS):

# echo DDIS > /sys/kernel/debug/vgaswitcheroo/switch
[  879.436727] i915: switched off
# cat /sys/kernel/debug/vgaswitcheroo/switch
0:IGD: :Off:0000:00:02.0
1:DIS:+:Pwr:0000:01:00.0

This will move the framebuffer and your shell to the other Nvidia driven monitor and shut down the Intel card. You will see the framebuffer console on the Nvidia output. Sweet, isn’t it?

Summary

A Prime enabled laptop with open source drivers does not have any configuration and does not require any manual configuration. The fact that the Nvidia card can power down itself is great and doubles my battery duration! On the screen I have KMS consoles without huge fonts and can have UEFI secure boot enabled! This is really awesome.

Unfortunately though, without proper Nouveau power management and performance improvements added to the fact that I need to reconfigure monitors everytime I move (sometime the output gets all black as well when docking); the experience is not that great. I don’t know why, but when I’m undocked and using only the LVDS internal panel, the Intel performance is fantastic. Problems arise only when it’s docked and Nouveau is enabled as well.

OptimusDisabledDisabledEnabledEnabled (Prime)
DriverNvidiaNouveauIntel/NvidiaIntel/Nouveau
ConfigurationVery easy.Already set up.Very complexAlready set up.
Card power
management
Perfect!Poor performance, no power management.Nvidia card always powered up, renders for all screens.Dynamic video card switching works fine, Nouveau performance not.
Optimus card
power management
N.A.N.A.Nvidia card can't power down.Perfect!
Docking / UndockingPerfect!Manual intervention requiredManual intervention required, unreliableManual intervention required
PerformancePerfect!Pretty bad.Very good, some tearing when moving windows.Bad when using the Nvidia card for output, otherwise perfect!
Bios ConsoleVGA, no KMS.Perfect (KMS)!Perfect (KMS on Intel).Perfect (KMS)!
UEFI ConsoleUses efifb. Somewhat slow.Perfect (KMS)!Perfect (KMS on Intel).Perfect (KMS)!
UEFI secure bootCan't work.Perfect!Can't work.Perfect!

Summing up, my current choice is for the Optimus disabled setup with Nvidia drivers. I can play games, dock, undock, power management works ok and I can drive all outputs easily. And if I need to go in a meeting I don’t need to be extra cautious in shutting down virtual machines, because the system might not go up again. It’s kinda retro style when booting with the text console and battery does not last more than 3 hours, but I can bear it.

If you’re not hunger for games or 3d stuff except the usual desktop compositing, just stick with the default OpenSource drivers and components, there is no setup required and everything works out of the box. My battery charge lasts usually 6 hours on a mixed usage case.

Nvidia has started contributing to the Nouveau driver with support for the GK20A (Nvidia K1) chip and it will be merged in kernel 3.16; let’s hope it will do the same for the other chips and components.

Updated Nvidia drivers

The Nvidia repository has been updated; here is the table that lists the current versions:

Operating systemCentOS / RHELFedorarawhide
Driver branchLong LivedShort Lived
Long Lived
Short Lived
Long Lived
Beta
Video Codec SDKYesYesYes
Architectures:

x86_64
YesYesYes
Basic nvidia driver:

nvidia-driver
nvidia-driver-libs
nvidia-libXNVCtrl
nvidia-kmod-common
YesYesYes
CUDA libraries and tools:

nvidia-driver-cuda
nvidia-driver-cuda-libs
nvidia-driver-NVML
nvidia-persistenced
YesYesYes
OpenGL Framebuffer Capture:

nvidia-driver-NvFBCOpenGL
YesYesYes
Nvidia tools:

nvidia-modprobe
nvidia-settings
nvidia-xconfig

YesYesYes
Binary kernel
modules (kABI):

kmod-nvidia
YesNoNo
DKMS kernel
modules:

dkms-nvidia
YesYesYes
aKMOD kernel
modules:

akmod-nvidia
NoYesYes
32 bit compatibility on x86_64:

nvidia-libXNVCtrl
nvidia-driver-libs
nvidia-driver-cuda-libs
nvidia-driver-NVML
YesYesYes
Development

nvidia-driver-devel
nvenc
nvenc-samples
YesYesYes
GLVND librariesYesYesYes
VDPAU librariesYesYesYes
EGLStream-based Wayland external platformYesYesYes
GBM EGL external platform libraryYesYesYes

For details, please see the Nvidia driver page. Fedora 19 and 20 nvidia-settings package has been updated on par with the other channels to use the system provided jansson library instead of the bundled one.

Updated repositories for Steam, Nvidia, HandBrake/MakeMKV, CDRtools

Another batch of updates to the repositories:

  • MakeMKV has finally been updated to version 1.8.7-2, which contains a small tweak as suggested in MakeMKV‘s forums to re-enable internal SSL support. This should solve all SCSI errors when decrypting BluRay discs on recent Fedora OpenSSL releases that do not ship all EC curves.
  • The HandBrake build with some of the bundled libraries removed in favour of system libraries (~50% of them) has been pushed as the supported build.
  • The nvidia-settings package now builds the external libXNVCtrl.so library that can be used to control the graphic cards through the NV-CONTROL extension. This library updates the old and obsolete one in Fedora based on drivers version 165.
  • The CDRtools suite has been updated to version 3.01a21.
  • The Steam package now produces an additional steam-noruntime subpackage that contains all the library requirements for running Steam without using the Steam Runtime. More details on the Steam repository page. This build has also been pushed to RPMFusion and will become the next update; so users which have the Steam package installed from the RPMFusion repositories will still have a Valve supported configuration with the Runtime enabled.