FreeBSD 10; new X.org, KMS, PKGNG, poudriere…

freebsdLately I’ve been playing with FreeBSD and its new technologies; PKGNG & poudriere and the new KMS/X.org stack that has been pushed to HEAD.

This is just my short how to for updating a FreeBSD CURRENT installation (10.0 at the moment of writing) with the new source code available on the FreeBSD servers.

Starting from FreeBSD 10.0 updating the base system through sources is performed with Subversion; and a lite version of it is shipped in the base system.

Get the source

First of all we must proceed in downloading the full sources from the FreeBSD servers:

svnlite checkout svn://svn0.eu.FreeBSD.org/base/head /usr/src
svnlite checkout svn://svn0.eu.FreeBSD.org/ports/head /usr/ports
svnlite checkout svn://svn0.eu.FreeBSD.org/doc/head /usr/doc

The ports directory; which is quite large; can also be downloaded and put in place with the following commands:

portsnap fetch
portsnap extract

This will populate all the directories with the required source code.

Define configuration for binaries

After getting the sources; we need to define some variables in the /etc/make.conf file to play with the new technologies being introduced.

Many things like PKGNG, CLANG, etc. have been enabled by default in recent builds, so the list is quite small:

# These lines are required to make sure that the make update command works
SVN=/usr/bin/svnlite
SVN_UPDATE=yes
 
# These lines enable the new X.org server (1.12.4) new Mesa libraries, KMS support, etc.
WITH_NEW_XORG=yes
WITH_KMS=yes
WITH_GALLIUM=yes
 
# Line for custom kernel configuration
KERNCONF=CUSTOM

FreeBSD still ships by default with X.org server 1.7; Mesa 7.x and a lot of outdated components. The new X.org components have not been switched yet to default; so specific lines need to be put in place for compiling the required port versions.

After this file, we need to create the kernel configuration file that is specified by the KERNCONF variable. I’m happy with the GENERIC kernel as I run it on multiple systems; so I will not change any option inside the configuration. What I do is turn off debugging support that is enabled by default on unrealeased FreeBSD versions.

So let’s create /usr/src/sys/amd64/conf/CUSTOM and put the following content inside:

include GENERIC
ident CUSTOM
 
# Disable debugging options
nooptions DDB
nooptions GDB
nooptions DEADLKRES
nooptions INVARIANTS
nooptions INVARIANT_SUPPORT
nooptions WITNESS
nooptions WITNESS_SKIPSPIN
nooptions MALLOC_DEBUG_MAXZONES

This kernel configuration file does nothing except specifying a new name and disabling the debugging options already enabled by default in the FreeBSD kernel. This way, when I update the sources; I don’t need to pay attention at which features are new or change in the GENERIC kernel. Pretty smart system.

Update the base system

To update the system, launch the following commands as root (you can of course build the system without being root, but for this short how to this is much clear):

cd /usr/src
make -s update
make -s -j5 buildworld
make -s -j5 buildkernel

The various make commands will make sure that you update the system with the latest sources and build all the toolchain required to compile the system and all the system commands and libraries. After this we’re good to go to install the new kernel and reboot the system:

make -s installkernel
shutdown -r now

When rebooting, choose “Boot in single user mode” (boot -s at the prompt) and perform the following commands to mount the root system in read/write mode:

mount -u /
mount -a -t ufs

Then check for conflicting configuration files that are to be updated (in my case 90% of the time I just need to press “i” to install the new configuration file); install the updated binaries and reboot.

mergemaster -p
cd /usr/src
make -s installworld
mergemaster
reboot

Configure poudriere for generating packages

FreeBSD 10 does not ship anymore with legacy package management tools but only with PKGNG; so the old way of building ports feels “obsolete” after you’ve been using a Linux distribution with package management for years. Thanks, poudriere comes to the rescue.

First of all we install the bleeding edge poudriere package (hey, we’re on 10.0-CURRENT; so we are already bleeding!). To install the package, go to the ports directory and install it:

cd /usr/ports/ports-mgmt/poudriere-devel
make install

After installing, we need to configure it and create a FreeBSD jail for building packages and some variables in poudriere’s configuration file.

cd /usr/local/etc/
cp poudriere.conf.sample poudriere.conf

Personally, this is what I have in my configuration file:

NO_ZFS=yes
FREEBSD_HOST=ftp://ftp.freebsd.org
RESOLV_CONF=/etc/resolv.conf
BASEFS=/share
USE_PORTLINT=no
USE_TMPFS=yes
DISTFILES_CACHE=/usr/ports/distfiles
CHECK_CHANGED_OPTIONS=yes
CHECK_CHANGED_DEPS=yes
NOLINUX=yes

As you can see I don’t have a ZFS filesystem, I’m not interested in Linux support and I’m sharing the downloaded source tarball directory with the ports system.

The /share directory is a separate mount point in my system where I link the base FreeBSD source folder, the ports directory, the documentation source directory and all the poudriere jails, work queues and package repositories. Make sure to have plenty of space!

We need to replicate the /etc/make.conf of our base system inside the jail to make sure all the packages get the correct settings. Create the file /usr/local/etc/poudriere.d/10-amd64-make.conf with the following contents:

WITH_NEW_XORG="yes"
WITH_KMS="yes"

Now we’re ready to start with poudriere. Issue the following commands:

poudriere ports -c
poudriere jail -c -m ftp -v 10.0-CURRENT -j 10-amd64

This will download a recent copy of the ports tree and create a FreeBSD jail taking the most recent snapshot of 10.0-CURRENT directly from the FreeBSD ftp servers! Awesome.

To update the ports tree, just issue the following command:

poudriere ports -u

Now that we’re ready to go, it’s time to list what ports we would like to build. Create a text file and put the list of ports you want to build inside of it; for example:

x11/xorg
x11/xorg-minimal
x11/nvidia-driver
x11/nvidia-settings
ports-mgmt/poudriere-devel

If you want some additional specific per-port configuration options, you can use the options subcommand. For example; to disable Linux support in the Nvidia driver, perform the following command:

poudriere options x11/nvidia-driver

and deselect Linux; much like you would do using standard ports.

Generate package repositories from the ports collection

Now that all is in place it’s time to trigger the builds. Perform the following command:

poudriere bulk -f portslist.txt -j 10-amd64

Poudriere will take care of setting a parallel task for each cpu you have and make sure all the dependencies are built in the correct order.

After munching for a long time (better go hack some Ingress portals or leave it running night long…) you will have a repository ready for use on your system. Depending on the paths/names you’ve chosen for the FreeBSD jail and for the configuration file you will have something like the following:
# cd /share/data/packages/10-amd64-default
# ls -l
total 176
drwxr-xr-x  2 root  wheel   8704 Sep 13 18:35 All
drwxr-xr-x  2 root  wheel    512 Sep 13 11:51 Latest
-rw-r--r--  1 root  wheel  12508 Sep 13 18:35 digests.txz
-rw-r--r--  1 root  wheel  46344 Sep 13 18:35 packagesite.txz
-rw-r--r--  1 root  wheel  97092 Sep 13 18:35 repo.txz

You just need to add the repository to your system and you’re ready to install packages! The repository also contains the latest pkg package for distribution.

This folder can be published on the web, left locally or wherever you choose. The only thing you need to do is enable the repository on your system(s) through the /usr/local/etc/pkg.conf file.

As an example, if you want to use the current package repository directly on the system where you’ve built everything just leave the following line in the configuration file:

packagesite: file:///share/data/packages/10-amd64-default

Using package repositories

Finally a great package management for FreeBSD. First of all update the repository definition:

pkg update

and then install the packages you want! For example:

pkg install xorg nvidia-driver

PKGNG will take care of everything.

Updating packages

To update packages, just re-update the ports tree with poudriere, rebuild with the bulk command (only the changed packages or the packages with changed dependencies will be rebuilt!) and upgrade your system!

poudriere ports -u
poudriere bulk -f portslist.txt -j 10-amd64
pkg update
pkg upgrade

Awesome.

Using OpenConnect with RSA Software Tokens in Fedora / RHEL / CentOS

OpenConnect is the Open Source alternative for the proprietary Cisco AnyConnect client. Our company is using the Cisco AnyConnect client along with PIN protected RSA Software Tokens for the authentication.

Looks like a complicated solution if you don’t have your corporate Windows client around; there’s no Cisco AnyConnect client for Linux with RSA Software Token support and the RSA application itself it’s only for Windows.

Guess what? It is not complicated at all and does not require WINE or any other tool to run the Windows app. It does not require the broken Cisco AnyConnect client either.

Update 10th June 2014: New instructions for new RSA Token Converter 3.0.
Update 10th October 2014: RSA Token Converter no longer needed.

Installation

Install the required software to import the token and the OpenConnect client itself:

# yum -y install NetworkManager-openconnect stoken-cli stoken-gui

If you’re using RHEL or CentOS up to 6.3 please reboot your system as the bundled NetworkManager is not able to reload plugins through dbus; otherwise if you’re running Fedora or RHEL/CentOS 6.4 or later you can directly proceed.

Token manipulation

Use stoken to import your token, choosing between the various options:

$ stoken import --token 2000123456...
$ stoken import --token com.rsa.securid.iphone://ctf?ctfData=2000123456...
$ stoken import --file mytoken.sdtid

Leave blank for the password request if you don’t want to password protect your token. Launch the command line program or the graphical program to get the passcode. The pin must be identical to the one you’ve set when first connecting to the VPN; otherwise the generated passcode will not match with the one generated on the VPN server:

$ stoken
Enter PIN: 1234
32031342

Alternatively you can use the GTK based stoken-gui program:

Stoken

Remove token PIN

To further speed up things; you can issue the following command to remove the pin request when opening the token (both command line and graphical):

$ stoken setpin
Enter new PIN: 1234
Confirm new PIN: 1234
$ stoken
32031342

Configuration

Create a new OpenConnect VPN through the VPN wizard of NetworkManager; the only required parameter is the server name.

If you have at least Fedora 20 or (probably) RHEL 7 with NetworkManager-openconnect 0.9.8 you can also paste the RSA Soft Token in the text box or use the Stoken file for passcode generation.

Openconnect1

Connection

Upon connection, you will be asked your username and RSA passcode. If you have enabled Soft Token integration with the PIN saved in ~/.stokenrc you will be asked only the username.

Openconnect2

Initiating the connection from the command line can be used as well. This method also enables you to avoid entering the PIN; offering the same functionality as NetworkManager-openconnect 9.8.0 on all Fedora releases and CentOS/RHEL 6:

sudo cp ~/.stokenrc /root
sudo openconnect --token-mode=rsa vpn.example.com

Pretty easy, huh?

Steam on Fedora 20

steamI’ve updated the Steam repository with Fedora 20 packages and removed Fedora 17 bits from the SPEC files since it has now gone EOL.

Fedora 19 and up now have SDL 2 in the main repositories; though the Steam client it’s shipping it’s own library that cannot be deleted or it will get downloaded again upon the client startup.

Enabling Cisco WebEx in Fedora 19/20

Enabling Cisco WebEx on a Fedora system is actually a lot easier than it looks by searching on Google. Pretty usual uh? Every time you look for something Linux related, a plethora of posts tell you that you need to compile, download, hack and modify.

This was probably true 10 years ago, but now setting up everything is much more easier than it sounds and usually involves a couple of settings and a couple of packages.

These are the steps required to setup Cisco WebEx on a Fedora 19 system; whether it be x86_64 or i686:

# yum install icedtea-web java-1.8.0-openjdk \
    pangox-compat.i686 libXmu.i686  libgcj.i686 mesa-libEGL.i686 \
    gtk2.i686 libpng2.i686
# setsebool -P unconfined_mozilla_plugin_transition=off mmap_low_allowed=on

The first packages are by good chance already installed on your system and should be the same of your system architecture; while the others need always to be the i686 variant as the WebEx program is compiled for 32 bit processors.

webex

30th October 2013:

Updated information with additional packages for latest WebEX update.

8th January 2014:

As reported in the comments, due to recent Mesa updates, if you don’t have Mesa’s libEGL installed you have to add it. Added to the list of packages required for installation.

No more secondary architectures

I was notified I can not use Fedora Koji builders for doing personal builds of Fedora forbidden items. This means I have to drop the following architectures from the repositories:

  • CentOS/RHEL 5 – ppc
  • CentOS/RHEL 5 – ppc64
  • Fedora 20 – armv7hl

I don’t have any hardware (or access) to any system like those; so my only chance is to drop support for them. All those architectures are bound to disappear from the repositories with the next round of updates.

CDRtools repository updated

New version 3.01a17 has been released. Starting from this version packages are built with all architectures enabled; this means that Fedora 20 has inherited armv7hl support and CentOS/RHEL 6 and CentOS/RHEL 5 have now respectively ppc64 and ppc support.

Along with the update, all Fedora 17 packages have been removed now that the distribution has gone EOL.

ARM builds available for Nvidia driver

Nvidia repository has finally received the armv7hl builds and patches for kernel 3.10. I’ve added a new table that depicts the supported functionalities by distribution.

Operating systemel6 / el7f24 / f25f26
Driver branchLong LivedShort Lived
Long Lived
Beta
Short Lived
Long Lived
Driver version375.66381.22381.22
NVENC7.1.97.1.97.1.9
Architectures:

i686
x86_64
YesYesYes
Basic nvidia driver:

nvidia-driver
nvidia-driver-libs
nvidia-libXNVCtrl
YesYesYes
CUDA libraries and tools:

nvidia-driver-cuda
nvidia-driver-cuda-libs
nvidia-driver-NVML
nvidia-persistenced
YesYesYes
OpenGL Framebuffer Capture:

nvidia-driver-NvFBCOpenGL
YesYesYes
Nvidia tools:

nvidia-healthmon (x86_64)
nvidia-validation-suite (x86_64)
nvidia-modprobe
nvidia-settings
nvidia-xconfig

YesYesYes
Binary kernel
modules (kABI):

kmod-nvidia
YesNoNo
DKMS kernel
modules:

dkms-nvidia
YesYesYes
aKMOD kernel
modules:

akmod-nvidia
NoYesYes
32 bit compatibility on x86_64:

nvidia-libXNVCtrl
nvidia-driver-libs
nvidia-driver-cuda-libs
nvidia-driver-NVML
YesYesYes
Development

nvidia-driver-devel
nvenc
nvenc-samples
libvdpau-devel
YesYesYes
GLVND libraries

libglvnd
libglvnd-egl
libglvnd-gles
libglvnd-glx
libglvnd-opengl
libglvnd-core-devel
libglvnd-devel
YesYesYes
VDPAU libraries

libvdpau
1.1.11.1.11.1.1

Root the Zopo ZP980 / C2 Android phone using adb

Rooting the Zopo C2/ZP980 device requires the execution of a local executable that grants you root access to the phone.

The procedure here depicts using Fedora for rooting the phone; to do the same on Windows or on another platform make sure to have appropriate drivers and adb (Android Debug Bridge) installed. The procedure is much more simple than most people try to do and it does not involve shitty Windows graphical programs and a plethora of different tools installed on the phone.

Install Android tools on the system

Super easy task for Fedora:

yum -y install android-tools

Enable USB debugging on the system

To enable the development menu, go to System Settings, About Phone and tap repeatedly the Android version field. A new menu under System Settings called Developer Options should appear.

Under Developer Options flag USB debugging.

Download the hack

Download the Galaxy S4 hack from here unpack it somewhere. Original hack with a lot of files can be downloaded from RootzWiki; but most of the files are not needed.

Getting root access.

Connect the phone to the USB cable, make sure the phone is NOT in USB mass storage mode (MTP or CDROM work fine) and launch the following commands from the folder where you unpacked the previous archive:

adb push su /data/local/tmp/
adb push pwn /data/local/tmp/

Then launch a shell on the phone:

adb shell

And then these commands:

cd /data/local/tmp
chmod 0755 ./pwn
./pwn
mount -o rw,remount /emmc@android /system
cp su /system/bin/
chmod 6755 /system/bin/su
cp su /system/xbin/
chmod 6755 /system/xbin/su
mount -o ro,remount /emmc@android /system

Voilà, the phone is rooted. Make sure to install any Superuser app from the Play Store. My preference is for the Superuser app from the Clockworkmod / RomManager developer.

Now you can replace the super ugly boot animation and logo. Just make sure to delete or replace the following files:

/system/media/bootanimation.zip
/system/media/bootaudio.mp3

I’ve prepared a custom boot animation that only prints the Zopo logo without the stupid music and the animation. It takes less time to boot and is much more sober. You can download it here.

Installing ClockWorkMod Recovery

Again this does not involve Windows graphical programs, a plethora of different tools, etc. you just need to download the file and overwrite a specific partition on the flash. Too simple, huh?

First of all download the custom ClockWorkMod recovery from here and unpack it somewhere; then push the recover image file to the SD card:

adb push recovery.img /storage/sdcard0/

After this launch a shell on the phone:

adb shell

And then flash the recovery partition:

shell@android:/ $ su
shell@android:/ # dd if=/storage/sdcard0/recovery.img of=/dev/recovery
shell@android:/ # exit
shell@android:/ $ exit

After this, just reboot into your new recovery image.

adb reboot recovery

Plase note, that after flashing a custom recovery image you can not apply Off the Air (OTA) updates anymore. The wireless update creates files that are not compatible with the custom recovery image, even if you attempt to manually flash them.

To revert back to the original custom recovery image, download your original firmware from the Zopo website and use the above procedure to flash the original recovery,img image.