Updated packages

There is a new update for HandBrake, it now reverts to using bundled libAV in place of the system FFMpeg. UTF subtitles were not recognized properly when scanning files, and VP8 encoding is not working properly.

Also, there is a new update for libbluray and a FFMpeg update that enables hardware acceleration for Intel Quick Sync Video hardware. That is, you can now use hardware acceleration for encoding on both Nvidia cards (NVENC) and Intel CPUs (QSV). CDRtools has also been updated to the very latest.

Notes and information on the HandBrake repository page has been updated accordingly.

Updated packages and new Samsung repository

As some of you may have noticed, a lot of updates and new stuff has been pushed.

Highlights from the updates

  • The Nvidia driver version 358.16 has been promoted as “Short Lived” for Fedora, and it has support for X.org Video ABI 20. So it is now the default for Fedora 21, 22 and 23.
  • Driver version 352.63 is the new “Long Lived” driver for RedHat/CentOS.
  • Both drivers have a fallback mechanism for the SELinux regression introduced in 358.09, so you don’t need anymore to disable SELinux for GDM refresh problems.
  • The legacy driver has been updated to 340.96, and this as well has been enabled for Fedora 23 as it also supports X.org 1.18
  • CDRtools after years of an alpha 3.01 has finally updated to… an alpha 3.02 😀
  • HandBrake has been updated to the very latest snapshot and is relying only on separate source tarballs that are not statically linked. Unfortunately some are of dubious licenses as usual. Please note that there are some issues with subtitle tracks embedded in Matroska files not being correctly interpreted.
  • The Nvidia NVML library and CUDA packages have seen some small update.
  • Steam has been updated to version 1.0.0.51 and has updated UDev rules for the Steam Controller. If you are one of the owners of such controller, you will find support out of the box. The same changes have also been pushed to the RPMFusion package.

New repository

There’s a new repository for Samsung printers, multifunction printers and scanners that carries the “Samsung Unified Linux Driver” for both the scanning and printing functions. Just by installing it and enabling the ports in the firewall should be enough to get every device running in no time. Refer to the appropriate page for details.

Fully fledged FFMpeg binaries

Also, due to popular request, there is now a custom built FFMpeg package that drops in as a replacement for RPMFusion ones and that enables linking and support for all the codecs/encoders/decoders that would result in an unredistributable binary. This is now available the HandBrake repository. The following codecs/encoders/decoders/transports have been enabled:

  • VP8 and VP9 de/encoding
  • WebP encoding
  • AAC (Fraunhofer, LibVO and other variants) de/encoding
  • OpenAL 1.1 capture support
  • BluRay reading
  • AMR-WB de/encoding
  • AMR-NB de/encoding
  • RTMP[E] support
  • H.264 encoding (OpenH264, Cisco variant)
  • OpenGL rendering
  • SSH transport
  • Fontconfig/fribidi text support

This sobstitutes the ffmpeg binary that was provided as part of the CUDA enabled programs, as it is now tied to all the other multimedia libraries available in this repository. The support for Nvidia H.264/H.265 hardware encoding/decoding is enabled here as well and the required packages are now installed through the use of RPM weak dependencies.

The package has a different Epoch so it is not overwritten by normal updates. The idea is to have all the possible codecs supported out of the box, so also expect HEVC kvazaar (H.265), Intel Quick Sync Video (H.265) hardware support through libmfx/mfx_dispatch, CIFS transport, and other stuff, as soon as I have more time.

The same repository will be used to host the CUDA and FFMpeg enabled Blender, as it makes no sense to have a separate repository for it. Either you have a full blown multimedia collection with each component strictly tied to each other, or you just have the plain Fedora repositories with no license/patent encumbered options.

Enjoy!

Fedora 23 packages now live

All the repositories have been updated for Fedora 23, so if you trigger an update, everything should update properly. CUDA enabled programs are still building.

A few notes:

  • HandBrake has been updated to a pre-release of 1.0 for Fedora 23. Updated x264/x265/FFmpeg libraries should give a speed bost to all encoding operations.
  • The Spotify 0.9.x repository has been removed. It will never receive updates anymore, and now the 1.x builds are on feature parity, including 32 bit support. If you haven’t upgraded, just do it now.
  • Nvidia drivers version 358.09 do not yet support X.org driver ABI 20, so you’re probably going to have some lock ups or random issues.
  • The SteamOS and X-Box replacement driver have been updated to the latest upstream.

Please let me know if you have any upgrade issue.

BluRay playback and ripping on Fedora (AACS, BD+, BD-J)

There are multiple ways that one can try to make BluRay decryption and playback in Linux. Of course, after some googling and fiddling around, you can see that is quite simple even if not the different instructions are quite confused as they span across multiple iterations of software development.

First of all a brief introduction. It is all explained here in great detail on the BluRay Wikipedia page, this is just an extract.

There are two different types of encryption:

  • AACS, performed using keys)
  • BD+, performed using a small embedded Java VM that executes programs and decrypts contents

And one type of “enhanced” java embedded menu (even small apps, like internet connected browsers, etc.) that can run on the BluRay player:

  • BD-J, using a GEM standard Java VM that includes also BDlive, for internet access

If any of these is implemented in a disk you own, the thing you need to do is simply insert a disk and run the bd_info command (from the libbluray-utils package) against your BD/DVD device.

Screenshot from 2015-08-27 10-59-57

This, of course, does work if you have the proper decryption keys or software installed on your system. That is, by default, nothing. 🙂

So, to enable decryption of the discs, you can proceed in two different ways, one using only open source software (not really, as you need anyway keys and a JVM dump of a BluRay Player) or going through commercial software (free at the moment while in beta).

Unfortunately none of my BluRays worked with open source tools on my system. I was not even able to get the required keys with the aacskeys program; so the proprietary software is my current choice.

Required repositories

In both cases, first of all enable the RPMFusion and the HandBrake/MakeMKV repository. Most of the keys required can be fetched from labDV, where you can also contribute back yours.

For the following examples I’m taking VideoLan as the main player, as it has good support (and a nice interface) to fiddle around with the various settings and is already available in RPMFusion.

Open Source AACS/BD+ decryption

First of all install libaacs and libbdplus from RPMFusion:

dnf install libaacs libbdplus

Then put the required AACS keys in a place, so that your AACS client can find them. For example:

mkdir -p ~/.config/aacs
wget -c http://www.labdv.com/aacs/KEYDB.cfg -O ~/.config/aacs/KEYDB.cfg

Now, VLC, using libbluray, should dynamically load the library if it finds it on the system and then decrypt the AACS protected content if the key for the specific disc is available.

Similar to AACS, put the required BD+ JVM dump in the appropriate folder, so that your BD+ client can find them. For example:

mkdir -p ~/.config/bdplus/
wget -c http://www.labdv.com/aacs/libbdplus/bdplus-vm0.bz2 -O ~/bdplus-vm0.bz2
tar -xvjf ~/bdplus-vm0.bz2 -C ~/.config/

Again, now libbluray should dynamically load the library if it finds it on the system and then decrypt the BD+ protected content using the VM dump in your personal folder.

Do not forget to update the AACS keys and the VM dump every once in a while!

Moving keys and dumps in a system wide location

This is required if you have multiple user for the system and you want to avoid each user into maintaining its own set of keys and dumps.
After installing everything in your personal folder, you will end up with a structure like the following:

$ find .config/bdplus .config/aacs | sort
.config/aacs
.config/aacs/KEYDB.cfg
.config/bdplus
.config/bdplus/vm0
.config/bdplus/vm0/aes_keys.bin
.config/bdplus/vm0/device_discovery_1.bin
.config/bdplus/vm0/device_discovery_2.bin
.config/bdplus/vm0/device_discovery_3.bin
.config/bdplus/vm0/device_discovery_4.bin
.config/bdplus/vm0/device_discovery_5.bin
.config/bdplus/vm0/ecdsa_keys.txt
.config/bdplus/vm0/mem_area_02.bin
.config/bdplus/vm0/mem_area_03.bin
.config/bdplus/vm0/mem_area_04.bin
.config/bdplus/vm0/mem_area_05.bin
.config/bdplus/vm0/mem_area_06.bin
.config/bdplus/vm0/mem_area_07.bin
.config/bdplus/vm0/mem_free.bin
.config/bdplus/vm0/memory.map
.config/bdplus/vm0/mem_player_executable.bin
.config/bdplus/vm0/mem_player_name.bin
.config/bdplus/vm0/mem_player_version.bin

To move up everything to a common location, just performs the following commands:

sudo mv ~/.config/{aacs,bdplus} /etc/xdg
sudo chown -R root:root /etc/xdg/{aacs,bdplus}

That’s it!

Proprietary AACS/BD+ decryption

This is done through MakeMKV, available from the HandBrake/MakeMKV repository, with the registration key available publicly from their forum. The software is fast, works really well, and although not open source is the only way to properly get access to your disks.

Support for external decoders has been added in libbluray some time ago, and allows you to specify different libraries than libaacs/libbdplus for decrypting content.

To proceed, remove the libaacs/libbdplus libraries if you have them installed, and install MakeMKV:

dnf remove libaacs libbdplus
dnf install libbdplus

As explained in the repository page, the MakeMKV package sets up environment variables that override the default libraries using for decryption and just uses MakeMKV’s libmmbd to launch a hidden MakeMKV instance for decryption in the background. To load the environment, logoff and login again, so it is picked up by any program starting.

After logging in, check that the environment is loaded with the following command:

$ env | egrep "LIBAACS_PATH|LIBBDPLUS_PATH"
LIBBDPLUS_PATH=/usr/lib64/libmmbd.so.0
LIBAACS_PATH=/usr/lib64/libmmbd.so.0

Then start VLC, select to open a BluRay disc, with or without the menus. Both will smoothly work as long as it is not a BD-J, as you can see in the pictures below. Menu is navigable, reactive to selections and clickable.

menus

Now also the command bd_info works fine, using the same mechanism.

BD-J Java menus

BD-J support is still in its infancy, at the moment of writing, with the latest libbluray 0.8.1 I am able to get part of the menus working. Most of the time I just got a video loop where the menu should be but not the actual menu. This is much better than having just VideoLan crashing, though. 🙂

Support for this is enabled by installing the libbluray-bdj package. The package puts an OpenJDK compiled Java Archive inside the classpath of the Java VM, so no further configuration is required. To install the package, just do:

dnf install libbluray-bdj

After this, in VideoLan you can just select “Open Disc”, “Blu-Ray” as the source and deselect “No disc menus”. As I said, support is not perfect, in the screnshot below I was able to see the menus but not click on them and in the background I was having a lot of errors.

bdj

I’m tracking the upstream project, so if the library can be rebuilt without breaking compatibility with the additional Fedora packages, I will add it to the repository along with MakeMKV. Before version 0.8.1 I was not even able to open a BD-J enabled BluRay disc with VideoLan.

One final curiosity

All my BD-J enabled discs have a Playstation 3 firmware update on the disc. Don’t know why, but probably because Sony owns the BluRay format so they can do as they please. Meh!

As always, feedback is welcome.

NVIDIA repository improvements

I’ve just pushed a big update to the Nvidia repository. The list of changes is quite big, so if you are a user of the repository please take your time to read through it.

CUDA

CUDA has been replaced with version 7 for all supported RHEL/CentOS/Fedora releases, with the main difference being no more support for 32 bit systems. From now on, there will be no compatibility packages for the i686 architecture. So, after upgrading, make sure to remove all the i686 pacakges (that is, yum/dnf remove cuda\*.i686).

Apart from this, CUDA 7 packages introduce new stuff, improves on the packaging and can now run correctly on all Fedora systems, including Fedora 22, which was not supported by CUDA 6.5. As announced previously, the only “regression” is when enabling C+11 on GCC 5.1 (i.e. Fedora 22).

The new packages take into consideration the Nvidia provided ones, and replace them accordingly; so if you have their packages installed it should upgrade them where appropriate leaving no traces of the former. Local changes include the pkg-config files for development packages (not included in their self installer and in my 6.5 packages) and the segregation of static libraries in their own subpackage, thus reducing installed size greatly. The next step is to proceed like for the open components of the Nvidia driver: replacing all the pre-built binaries with source compiled stuff. At the moment, this includes cuda-gdb, for which sources do exist.

NVIDIA driver

The Nvidia driver has seen a repackaging of the main components, where the biggest change is the library layout.

All the unique libraries are now in standard locations, leaving only the duplicate ones under /usr/lib{,64}/nvidia. Also, X configuration files that the user should avoid touching, have been moved under /usr/share/X11/xorg.conf.d. Library dependencies have been reduced, so you can now compile a program against the NVML library (libnvidia-ml.so.1) or CUDA without needing to install the full driver or using the Nvidia provided stubs.

This makes for a simpler package with a simpler filter for conflicting libraries and is also propedeutic work to enable hardware accelerated encoding with Steam In-Home streaming and Nvidia drivers. Hardware decoding in Steam has been put in place, so now it’s time for the (only) supported hardware encoding.

NVIDIA beta driver (355.xx)

The biggest change comes with the 355.06 driver in Fedora 23, which introduces partial support for the GL Vendor-Neutral Dispatch library (libglvnd), a new kernel module building system (in preparation for their modeset driver after the 355.x series) and dropped support for Nvidia instanciated modules.

The new beta driver requires the GL Vendor Neutral Dispatch library in place for proper operation, and being this a separate open source project (only a prototype in Mesa, at the moment) it has been built from source. So there is now a required libglvnd package that only contains the libraries required by the driver for running. The more they are integrated with the driver, the more I will enable in the package. This is a transitional package only, as sooner or later Mesa and the X server will introduce the same mechanism, making the transitional package obsolete and just using distribution provided packages.

This with could eventually lead to system running different OpenGL implementations at the same time, solving the Optimus and multiple cards from multiple vendors combination issues.

The Nvidia control panel can be optionally linked to NVML, so I’ve tried to enable it in the beta drivers. I don’t see any changes so far, but probably this is due to the fact that I don’t have new enough cards to be compatible with NVML.

The beta driver has been pushed to the Fedora 23 branch only, as explained in the Nvidia repository page.

As always, feedback is welcome.

CUDA 7.0 enabled programs for Fedora 22

nv-cuda-2014header-updatedI’ve udpated the CUDA version in the Fedora 22 Nvidia repository, it now contains CUDA 7.0.28 along with the cuFFT 7.0.35 patch. Note that from this version, CUDA is x86_64 bit compatible only, so there are no more i386 packages. There is still the cudart library available for 32 bit, but I don’t think it’s worth packaging.

The packages hosted here should correctly upgrade and obsolete the ones in Nvidia’s own repository, so it should be possible to go straight from one version to the other, if you need.

The static libraries (according to packaging guidelines) have been placed in a cuda-static package, thus reducing by an order of magnitude the size of the packages containing libraries. The toolkit can of course be installed and used to create CUDA binaries on systems where there is no Nvidia adapter installed.

The Nvidia compiler (nvcc) throws an error if the GCC version detected is higher than 4.9 (Fedora 22 default is 5.1.1), but the removing the check makes the compiler run fine until you enable C++11 support. If you need to enable C++11 support you need to use a separate GCC, older than 5.x, for compilation. See this comment here for details.

As part of the update, the repository that contains CUDA enabled programs (Blender, CCMiner and NVENC enabled FFMpeg) has been updated for Fedora 22. This is completely optional, so you can have Nvidia packages on your system and still use RPMFusion’s FFMpeg and Fedora’s Blender. I’ve tried to submit Blender updates to the appropriate package maintainer but did not receive any answer.

I will rebase all distributions to CUDA 7.0 as soon as the next long lived driver release will be branched by Nvidia.

As always, feedback is welcome. If you have any issue or would request an enabled CUDA package to add to the repository, just write in the comments or write me an email.

Steam made easy

The Steam repository has received some love; changes to the Steam package will also be pushed in RPMFusion after a bit of feedback.

Updated X-Box gamepad driver

The SteamOS X-Box Gamepad driver has been updated with the new Valve rebase on the 3.18 kernel, with new hardware support and bugfixes. Remember to disable Secure Boot if you plan to use it on an UEFI system.

In-Home Streaming

There are now instructions for setting up In-Home streaming, including the firewall rules required for enabling your desktop to accept connections from other Steam clients that are available on your network.

Also, hardware decoding is available if you are streaming from another Steam client and you are not running the Ubuntu Runtime.

Disabling Ubuntu Runtime

It is now possible again to have the steam-noruntime package that pulls in all (known) 32 bit dependencies and allows you to run Steam without the Ubuntu Runtime. Remember this is not supported, so while getting the benefits of it, you might as well encounter additional problems.

Also, some games pull in additional dependencies (the Witcher 2 or Anodyne, for example), so without the Runtime you are on your own to figure out what is missing.

But running Steam without the Ubuntu Runtime has also its benefits, for example you use DTS 7.1 sound through PulseAudio.

The steam-noruntime package also pulls in as the dependencies the required libraries for the hardware accelerated decoding with In-Home Streaming. This makes streaming much less flickery client side.

Those features normally work only with the Ubuntu/Debian libraries on the system, so avoiding the runtime is the only choice.

170 games and counting!

Nvidia driver modeset kernel module

As part of the latest Nvidia driver update at version 352.09, there is now code supporting a new nvidia-modeset kernel module that should be running on a system and that interfaces with the usual nvidia kernel module.

Evidence of this is in the kernel module sources and in the nvidia-modprobe command code that is hosted in Github.

From the nv-modeset-interface.h header in the kernel module “sources”:

/*
 * This file defines the interface between the nvidia.ko and
 * nvidia-modeset.ko Linux kernel modules.
 *
 * A minor device file from nvidia.ko's pool is dedicated to
 * nvidia-modeset.ko.  nvidia-modeset.ko registers with nvidia.ko by
 * calling nvidia_register_module() and providing its file operation
 * callback functions.
 *
 * Later, nvidia-modeset.ko calls nvidia.ko's nvidia_get_rm_ops()
 * function to get the RMAPI function pointers which it will need.
 */

Let’s hope that modesetting support in the driver is near and we will not have to wait additional years for it. Also, let’s hope that the firmware images required for the latest hardware on Nouveau will be released soon, without further delays.

As soon as it will be delivered, I will implement it in the packages according to the driver table in the repository page.