Nov 062019
 

My activities in October were mostly related to updating my COPR repositories for CentOS 8 and cleaning up the old repositories:

  • I updated the ganto/jo COPR repository to support CentOS 8.
  • I updated the ganto/vcsh COPR repository to support CentOS 8 and added package builds for the alternative architectures (aarch64 and ppc64le).
  • Thanks to the help of jmontleon I was finally able to build LXD which is available in my ganto/lxc3 repository for CentOS 8. I also updated the RPM for the latest stable release LXD 3.18.
  • After years of development the distrobuilder tool which is meant to replace the shellscript-based LXC templates was tagged in a first 1.0 release that should now also be able to build CentOS 8 container images. Of course I updated the corresponding RPM in the ganto/lxc3 COPR repository accordingly. I’m not sure how they decide to do new releases therefore I might decide to go back building regular git snapshot releases of this tool in the future.
  • I updated the ganto/goaccess COPR repository to support CentOS 8 and also increased the built goaccess version to a git snapshot from May 2019 based on version 1.3. Unfortunately the official Fedora package is still only at version 1.2. I was first testing the latest git snapshot but then found that it is affected by a bug (GitHub issue 1575) which would fail to render the access graphs properly.
  • The last COPR repository pending an update for CentOS 8 is ganto/umoci which still fails because of go-md2man missing from EPEL 8.
  • I deleted some outdated COPR repositories (ganto/lxc, ganto/lxd, ganto/lxdock) and archived the related GitHub repositories holding the RPM spec files.

Then I was also experimenting with adding Debian machines to a CentOS FreeIPA identity management server via Ansible. Years ago I wrote an Ansible role freeipa-client which was able to do that but still required manual setup of the Kerberos keytab on the client machine. I plan to replace that with a collection of new roles trying to blend-in with DebOps as much as possible. But unfortunately nothing ready to show yet.

Finally, as always, I updated a lot of ebuilds in my linuxmonk-overlay Gentoo overlay.

Oct 012019
 

I’m starting a new series of blog posts summarizing my various activities regarding free software projects. There might not be every month something worth mentioning, but this month I was quite busy what might be interesting for some of you.

Following I’ll list some of the activities I was involved regarding free software projects in September:

  • After the official release of CentOS 8, I started rebuilding the packages in my lxc3 COPR repository for CentOS 8. The lxd package is still missing and I’m planning to provide it for CentOS 8 together with the pending update to lxd-3.17. A rebuild of the packages in my various other COPR repositories can be expected in the coming weeks.
  • Being the package maintainer of the spectre-meltdown-checker package in Fedora and EPEL, I followed the instructions to request a package branch for epel-8. This was approved a few hours ago, so the packages is now available via Koji and awaits approval in Bodhi for inclusion into the EPEL testing and eventually stable repository. Please give some karma if you’d like to accelerate this.
  • I merged some pull-requests in the Gentoo go-overlay git repository where the original maintainer entrusted me with commit permissions. Because he didn’t participate since last December, I used the chance to cleanup the repository to pass the repoman checks again and eventually merged a PR for the latest traeffik 1.x (1.7.18) release.
  • I put some effort into packaging the Gnome 3.34 release in my personal Gentoo linuxmonk-overlay. Of course I’m running it on my main workstation on top of Wayland without any major issues so far. Give it a try if you can’t wait for the official ebuilds to be ready.
  • I released version 0.1.2 of my acme-tiny Ansible role which fixes an annoying bug. It could happen that if the certificate renewal was unsuccessful, a still valid certificate would have been overwritten with an empty file. Now the role will make a backup copy of the old certificate by default and validate the new certificate before replacing the old one.
Apr 202014
 

Finally the day arrived, that I can say, I’m a Linux user since 10 years. On April 19 2004, I started my stage1 Gentoo Linux installation on my main workstation box. Since then, I changed my main desktop to Gnome, I adapted my work flow according to what’s possible with open source tools and I never looked back again to my old time with proprietary Windows tools.

However, I already made my first Linux experience a few years earlier than 2004. As far as I still remember, my first Linux distribution ever was SUSE Linux 7.0, which I was able to install without problems on an old computer of mine. I was kind of lost with this new concept that every small functionality is a program by itself, and why are these applications called so weird anyway? A bit later, I wanted to build a dedicated game and file server for LAN parties and it should run with an easy to maintain and lightweight alternative to Windows. The component that I was proud of the most in this server was a Promise SX4000 RAID controller which supported RAID5. I made sure that it was supporting Linux before buying it. At that time I learned the hard way, that Linux support doesn’t magically mean that the driver code is integrated into the upstream Linux kernel and therefore is supported by every Linux distribution. First, there were only some binary modules available, so my distribution for the server was Red Hat Linux 7.2. At that time RPMs were still some kind of annoying black magic to me, as there was no automated dependency resolving available by default. I didn’t know about yum which was optionally available already. Eventually, I never really could make the server work properly in the way how I imagined. Also, I learned from my geek friends, that there is Gentoo, which would be the best Linux distribution anyway. With Gentoo Linux I finally succeeded to install my RAID adapter and I started learning a basic principle, which I still think it’s true today: “If some open source software doesn’t work how you want, you simply don’t try hard enough”. I then also started using Gentoo on my Apple iBook G3, which released me from some Apple Java 1.4 Swing weirdness I experienced during the University programming exercises. These achievements showed me… Linux, ehmn Gentoo Linux, is the way to go. 🙂

You may wonder, why I still remember exactly when I originally setup my Gentoo workstation. Simply, because thanks of the rolling release model of Gentoo, I’m still running the same installation since then. I still have my emerge.log around with the entire update history since day one. You want some goodies?

  • First emerged package:
         Mon Apr 19 14:47:13 2004 >>> sys-apps/portage-2.0.50-r6
           merge time: 28 seconds.
    
  • Original toolchain:
         Mon Apr 19 15:00:50 2004 >>> sys-devel/binutils-2.14.90.0.7-r4
           merge time: 4 minutes and 45 seconds.
    
         Mon Apr 19 15:41:29 2004 >>> sys-devel/gcc-3.3.2-r5
           merge time: 39 minutes and 34 seconds.
    
         Mon Apr 19 16:09:44 2004 >>> sys-libs/glibc-2.3.2-r9
           merge time: 28 minutes.
    
  • Original desktop environment and browser:
         Tue Apr 20 14:51:33 2004 >>> x11-base/xfree-4.3.0-r5
           merge time: 17 minutes and 26 seconds.
    
         Wed Apr 21 02:10:24 2004 >>> gnome-base/gnome-2.4.2
           merge time: 3 seconds.
    
         Wed Apr 21 11:18:36 2004 >>> net-www/mozilla-firefox-0.8-r2
           merge time: 42 minutes and 9 seconds.
    

It’s really interesting to dig around in this file. As you can see, it took a few days to compile the entire system, but at the end, I had a system which satisfied my expectations and still serves me well today.

With help of the emerge.log, I can also make some interesting comparisons on how PC hardware evolved. Building OpenOffice.org back then on a single core AMD Athlon XP and LibreOffice today on a six core AMD Phenom II, which is also rather antique already:

     Thu Sep 23 23:56:56 2004 >>> app-office/openoffice-1.1.2
       merge time: 6 hours, 18 minutes and 15 seconds.

     Sun Feb  9 11:35:57 2014 >>> app-office/libreoffice-4.1.4.2
       merge time: 1 hour, 2 minutes and 32 seconds.

If you are interested in more details of my Gentoo emerge history or if you know some tools to automatically analyze or graph the emerge.log, please leave me a comment below.

So how did I evolve during the last 10 years using Linux? I became an IT professional for Linux engineering, I was witnessing how open source software was gaining acceptance in the most conservative IT environments on one side and driving innovation and efficiency on the other side. This wouldn’t have been possible without Gentoo, which taught me to dig into documentation and community reports to solve the problems. A big thanks to Gentoo and the entire Linux and open source community for all their support and motivation! I had a great time with you for the last 10 years. At the beginning, I could have never imagined that today the majority of people are running a Linux-based mobile phone or that Linux is evolving so rapidly as gaming platform. I’m looking forward to the next 10 years with Linux and open source software…

Dec 022012
 

FreeIPA is an integrated user, host and service identity management solution combining 389 Directory Server (LDAP), MIT Kerberos, a BIND DNS server and the Dogtag Certificate Authority service with a simple but powerful Web GUI and an extensive command line interface for easy administration. It claims to become something like an Active Directory for Linux and Unix environments and is heavily pushed by Red Hat, which also integrates it as IPA server in their Red Hat Enterprise Linux distribution. A nice overview can be found in this presentation.

After having the pleasure of playing around with the Red Hat IPA server on RHEL and CentOS for the past few weeks, I also wanted to use this excellent identity management platform with my Gentoo Linux boxes. Some years ago, a bug report was opened in the Gentoo bugzilla (#297665), to coordinate the inclusion of FreeIPA in Gentoo. Andreis Vinogradovs, another Gentoo user, then started an effort, to write some of the necessary ebuilds for building FreeIPA, however they are still far from complete and therefore haven’t made it into the official Gentoo repository yet. This means that FreeIPA is unfortunately still not fully available for Gentoo.

Based on Andreis’ work, I started another effort, to update and polish the FreeIPA ebuild and its dependencies, so that they can be used on a Gentoo Linux box. The server part has dozens of dependencies not yet officially integrated in Gentoo, and the available ebuilds are mostly outdated, so I haven’t put too much effort yet into integrating the server parts on Gentoo. Especially the entire PKI infrastructure is still missing.

However, I succeeded to configure a Gentoo box as full-featured FreeIPA client, including OpenRC support for `authconfig` and `ipa-client-install`. I also found and reported some bugs in official Gentoo ebuilds (#445394, #445478), where you have to work-around in case you try out the setup yourself.

Of course you are curious now, where you can find the ebuilds. Because the work on them and especially the testing is still ongoing, I created a repository on Github so that everybody who is interested can have a look at ebuilds and provide constructive feedback in terms of pull requests.

I’m especially looking for people who would like to try the FreeIPA client with a Gentoo systemd or/and a hardened SELinux system.

How can you test the FreeIPA client on your Gentoo box?

You have to begin with setting up a (Free)IPA server, which is currently only possible on a Red Hat-based distribution. The easiest way is to setup a CentOS 6 VM, then run:

[root@centos6 ~]# yum install ipa-server
[root@centos6 ~]# ipa-server-install

More information can be found in the upstream installation guide.

Then add the ‘freeipa-overlay’ to the layman configuration of your Gentoo client. How you do this is described here.

ATTENTION: This guide is meant to be for experimental testing only. Don’t do this on your workstation if you are not familiar with FreeIPA and its technologies. I don’t take any responsibility if you blow up your machine. You have been warned!

Finally you are ready to emerge FreeIPA. Make sure tho have a look at the various
USE flags. They don’t have too much influence on build-time functionality but
on run-time dependencies. So you can slim down your installation in case you
already know, that you don’t need another DNS server or winbind support for
example. Set the ‘minimal’ USE flag for only building the IPA client
(Update 07.12.2012: This USE flag was replaces with ‘server’, so the client will be installed by default):

gentoo ~ # emerge -av freeipa

Some keyword unmasking may be required when you run a stable Gentoo installation.

Before you can start your IPA client installation, you have to make sure, that an empty NSS certificate database exists. This is expected to be under /etc/pki/nssdb. Gentoo however puts all the SSL stuff under /etc/ssl. I solved this by creating a symlink:

gentoo ~ # ln -s ssl /etc/pki
gentoo ~ # certutil -N -d /etc/pki/nssdb

Eventually the IPA client can be configured. E.g.:

gentoo ~ # ipa-client-install --mkhomedir --no-dns-sshfp
Discovery was successful!
Hostname: gentoo.example.com
Realm: EXAMPLE.COM
DNS Domain: example.com
IPA Server: centos6.example.com
BaseDN: dc=example,dc=com

Continue to configure the system with these values? [no]: yes
User authorized to enroll computers: admin
Synchronizing time with KDC...
Password for admin@EXAMPLE.COM:

Enrolled in IPA realm EXAMPLE.COM
Created /etc/ipa/default.conf
Domain example.com is already configured in existing SSSD config, creating a new one.
The old /etc/sssd/sssd.conf is backed up and will be restored during uninstall.
Configured /etc/sssd/sssd.conf
Configured /etc/krb5.conf for IPA realm EXAMPLE.COM
SSSD enabled
Configured /etc/openldap/ldap.conf
NTP enabled
Configured /etc/ssh/ssh_config
Warning: Installed OpenSSH server does not support dynamically loading
authorized user keys. Public key authentication of IPA users
will not be available.
Configured /etc/ssh/sshd_config
Client configuration complete.

That’s it! Your system is now able to use user accounts created on the IPA server. Check it with:

gentoo ~ # id admin
uid=155960000(admin) gid=155960000(admins) groups=155960000(admins)

As you can see in the generated /etc/pam.d/system-auth, pam_unix will be checked before pam_sssd. This means your local user accounts still have precedence towards the IPA accounts.

Happy testing… 🙂

Oct 312012
 

As a Linux enthusiast and Gentoo user I was always looking for the perfect boot experience. While I managed to boot my kernel with EFI and grub 2 (as described in my wiki), I still had some troubles with OpenRC playing nice with my LVM-only setup initialized by dracut. Tonight I finally figured out the missing configuration pieces to shut up all warnings on system init.

Initial situation
All my Linux partitions are stored in a single LVM volume group, to stay as flexible as possible:

merkur ~ # lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
[...]
└─sda5 8:5 0 49.5G 0 part
├─vg_merkur-slash (dm-0) 253:0 0 2.5G 0 lvm /
├─vg_merkur-boot (dm-1) 253:1 0 200M 0 lvm /boot
├─vg_merkur-tmp (dm-2) 253:2 0 6G 0 lvm /tmp
├─vg_merkur-swap (dm-3) 253:3 0 4G 0 lvm [SWAP]
├─vg_merkur-var (dm-4) 253:4 0 4G 0 lvm /var
├─vg_merkur-usr (dm-5) 253:5 0 12.8G 0 lvm /usr
└─vg_merkur-opt (dm-6) 253:6 0 8G 0 lvm /opt

My boot toolset currently consists of grub-2.00-r1, kernel-3.6.4, dracut-024, lvm-2.02.95-r4 and openrc-0.11.2

Kernel Configuration
Before compiling the kernel, make sure to include all the required configurations. For this setup, the most important ones are:

CONFIG_BLK_DEV_INITRD
CONFIG_DEVTMPFS
CONFIG_MODULES
CONFIG_SYSVIPC

Dracut Configuration
Before installing dracut, the desired modules have to be configured in /etc/make.conf. :

DRACUT_MODULES="caps lvm mdraid syslog"

For this setup at least the “lvm” module is mandatory. Further dracut was built with the “device-mapper” USE flag enabled.

Altough some Linux developers (especially from Red Hat/Fedora) advice against a separate /usr partition because of many boot time dependencies on this system path, I didn’t bother much to change my years old setup. Since version 014, dracut includes a module to fill this gap (/usr/lib/dracut/modules.d/98usrmount/mount-usr.sh). It simply mounts the /usr partition right after the root file system early in the boot process. Therefore we have to make sure that the dracut modules “usrmount” and “lvm” are included in the initramfs, which was possible without any manual modification of /etc/dracut.conf, when generating the boot image with:

dracut -H

Kernel Command Line Configuration
Dracut runtime parameters are given on the kernel command line in the Grub configuration. To automatically enable the LVM Volume Group and spawning a debug shell in case the boot should fail, I added the following parameters in grub:

root=/dev/vg_merkur/slash rd.lvm.vg=vg_merkur rd.shell

LVM Configuration
Since dracut is now responsible to enable our volume group, the corresponding init script has to be disabled:

rc-update del lvm boot

Fsck and Fstab
When booting the system now, the /etc/init.d/fsck script will complain that it cannot check the file systems which are already mounted. Fortunately, the init script allows us to define that fsck should be only run when specific “fs_passno” values are set. I therefore this value to “1” for the file systems which are mounted by dracut and to “2” for all the file systems which should be checked by OpenRC. Take care, when specifying a value of “0”, the file system will be never checked for consistency:

# [fs] [mountpoint] [type] [opts] [dump/pass]
/dev/vg_merkur/boot /boot ext2 noatime,nosuid,nodev 0 2
/dev/vg_merkur/slash / ext4 noatime,discard 0 1
/dev/vg_merkur/usr /usr ext4 noatime,discard,nodev 0 1
/dev/vg_merkur/var /var ext4 noatime,discard,nosuid,nodev 0 2
/dev/vg_merkur/opt /opt ext4 noatime,discard,nosuid,nodev 0 2
/dev/vg_merkur/tmp /tmp ext4 noatime,discard,nosuid,nodev 0 2
/dev/vg_merkur/swap none swap sw 0 0

In /etc/conf.d/fsck we then can define, that the fsck init script should only care about file systems with a “fs_passno” larger than “1”:

fsck_passno=">1"

That’s it… If you have some questions or hints, please leave a comment.

Oct 032007
 

Recently I wanted to set up a testing server for the different virtualization techniques for Linux. For this I have an Asus P5LD2 mainboard with an Intel dual core Pentium D 3,2 GHz which supports the Virtual Machine Extensions (VMX). Thanks to this I can compile Xen with the ‘hvm’ USE-flag and run fully virtualized guest operating systems on my Xen supervisor. This means I could run nearly every i386 compatible operating system (even Windows 😉 ) in my Xen environment. Without such hardware every guest operating system has to have a Xen enabled kernel.

Another approach with the same result is the open source project QEMU. Its abstraction level is higher than with Xen and it can even emulate different target architectures from your current x86 host. So far x86_64, ARM, SPARC, PowerPC, MIPS and M68k target systems are supported. Its guest operating system does not need any single change to run on QEMU. This makes it very comfortable to test new live CDs or operating system images. But it is not so trivial to setup QEMU and Xen together on a Gentoo machine.

How to setup QEMU on 32bit Gentoo in Xen dom0?

If you compile Xen on a 32bit host you have to add ‘-mno-tls-direct-seg-refs’ to your CFLAGS. That is because the glibc TLS library is implemented in a way that will conflict with how Xen uses segment registers. For compiling the non-patched QEMU 0.9.0 you have to use a gcc version 3.x. The nowadays default gcc 4.x is not yet supported. After several compile failures I finally found to setup QEMU the following way:

1. For compiling gcc-3.x remove the ‘-mno-tls-direct-seg-refs’ from /etc/make.conf and set the ‘nossp’ and ‘nopie’ USE-flags. Otherwise gcc or later qemu will not compile.

2. Switch to gcc-3.x before compiling qemu-softmmu, qemu-user and qemu. In my case it’s: gcc-config i686-pc-linux-gnu-3.3.6

3. Check your CFLAGS again because the optimization flags for gcc 4.x are not always backwards compatible to gcc-3.x. In my case the make.conf looks like this:

# gcc-3.x
CFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer -mno-tls-direct-seg-refs"

# gcc-4.x for compiling gcc-3.3
#CFLAGS="-march=prescott -O2 -pipe -fomit-frame-pointer"

# gcc-4.x
#CFLAGS="-march=prescott -O2 -pipe -fomit-frame-pointer -mno-tls-direct-seg-refs"

4. Now you can compile QEMU. Do not forget to switch back to your original CFLAGS and gcc-4.x after successfully emerging QEMU. I recommend to you to also build the QEMU kernel accelerator module kqemu which has to be compiled with the same compiler as the kernel itself.

Now Xen and QEMU are able to run whatever operating system image you give them. Have fun with playing around…

Additional links: