Linux on the desktop has come a long way. The Gnome and KDE communities have built themselves a big, very powerful set of tools to build on. And using these tools, they created an enormous amount of software for a large number of different purposes.

Then they discovered that there is a lack of formality in the RPC mechanisms available under UNIX like operating systems. The Shared Memory IPC provides just shared memory and a little flow control, which is tedious. The sysvmsg API is still very inconvenient when communicating between various different processes, especially if they're arbitrary. Sockets work much better in that respect and have a well-defined API, but it is still relatively hard to exchange data over them.

UNIX offered the SUN RPC API which was used to implement NFS, among other things. However, it was ”just“ an RPC implementation and not a real service based middleware. Especially among Universities, a rather complex method of doing RPC had become fashionable: CORBA.

At this time, the only available Open Soure CORBA ORB was Mico. However, Mico still lacked some of the desired features and didn't support a lot of programming languages so the Gnome developers decided to implement their own ORB called ORBit.

The KDE people faced a very similar issue. However, instead of implementing all of CORBA, they developed a much smaller, more lightweight protocol called DCOP. DCOP was more tailored to be used for communicating between the different applications.

So the Gnome developers wanted to reduce the complexity of their protocol as well and started working on a protocol which was supposed to join the advantages of DCOP and CORBA. The result was called the Desktop Bus (dbus) protocol. Instead of complete remote objects it just offers remote interfaces with functions that can be called.

esd and PulseAudio

The sound system underwent a similar development. Initially, the operating system provided an API called the Open Sound System. It is based on a read-write device in /dev and a number of IOCTLs. It was the same API which is found in the BSD and Solaris operating system.

However, the Linux incarnation of OSS was a particularly simplicistic one which only supported one sound channel at the same time and only very rudimentary mixing. As a workaround, the community came up with a daemon which accepted sound samples and mixed them in software: the Enlightened Sound Daemon (esound). This daemon even acquired network capabilities so people could stream whatever they wanted over the network to other computers and play it there without having to resort to systems like the Networked Audio System (NAS). The KDE developers went even further and implemented an audio system on top of DCOP called MCOP.

Over time, the Linux kernel developers came up with a new API to control the various details of the sound card, have many different volume settings and to be able to mix in hardware. It was called the Advanced Linux Sound Architecture (ALSA).

Then, Gnome and KDE developed APIs to abstract the uses of OSS, esound and ALSA: gstreamer for Gnome and Phonon for KDE. Since gstreamer depended heavily on the Gnome libs and phonon on the KDE libraries, the rest of the community had to either adopt or try to keep up with the ever changing sound backends. Esound was deprecated and replaced with Pulseaudio, which triggered yet another shift of APIs.

Over time, more and more subsystems started getting DBus based frontends. hald was added to detect hardware properties. It turned into an official dependency for X.Org and was subsequently replaced with DeviceKit. PackageKit was added as a generic API to instruct the system to find and install packages through a DBus interface. ConsoleKit replaced the regular session and pseudoterminal management environment. PolicyKit imposed additional restrictions on system calls and allowed other people to do privileged operations without changing to the superuser explicitly. sssd is now in the process of replacing PAM as an authentication framework which is also contacted through DBus, without the need to use the system authentication and session management API (PAM and NSS, mostly).

NetworkManager added a new abstract way to configure network devices, such as network cards, wireless LAN or 3G. Like everything else, it provides a DBus interface for executing various operations, such as discovering wireless LANs, connecting to a network and awareness if the computer is currently connected to a network. Various GUI programs such as Firefox, Pidgin, Gajim and similar tools use NetworkManager to clean up their caches and reconnect after the network connection was terminated. They also go to some sort of offline mode if NetworkManager tells them to, in which they don't attempt to connect to the network and try to do whatever you want locally (e.g. queuing messages to be sent, displaying web pages from cache, etc.)

Another addition was systemd, which now replaces System V init and all other types of init derivates on various distributions. It is an init daemon which reads services from a database, somewhat like Solaris' svcadm. However, for starting and stopping services and telling the system to shut down or do something else, systemd has a DBus interface. The old /dev/initctl interface is no longer supported.

And most recently, there was a new addition to the pool: Journal is a service which replaces syslog and which exposes a DBus API for logging messages into a binary log (instead of a plain text log like syslog did). The adoption rate among desktop and other programs is great because now finally everybody can use their favorite API to log, grant log permissions, search logs etc. The world is becoming more awesome every single day.

Or did it? RedHat Enterprise Linux (RHEL) is a Linux distribution tailored for long-term support environments — which includes corporate desktops — and servers. Ubuntu LTS is following the same goal: to provide a modern distribution with long-term support for use in corporate workstations and servers.

NetworkManager

The latest versions of both Ubuntu LTS and RHEL ship with NetworkManager for managing their network connectivity. If you don't use NetworkManager, a number of programs will refuse to connect to the Internet or behave in various weird ways. More so, a lot of system services now depend on NetworkManager and won't start unless it is running. And if you run NetworkManager, it starts periodically messing up any local system configuration. So you're basically bound to use NetworkManager.

So you install a server in headless mode (Wait, the installer won't typically let you do that anymore. But let's assume you do it nonetheless because your server doesn't have a graphics card anyway, it's attached to a Cyclades SSH serial port adapter like any other one of your UNIX servers.) Then you try to figure out how to configure NetworkManager from the command line. There's no tool in the entire distribution which lets you do that.

So from some time in the past you remember that you used to use a program called cnetworkmanager to operate NetworkManager from the command line. You install it and — the DBus API changed since the program was written, so the DBus call fails with a not-very-helpful error message.

So the only way to actually use NetworkManager is to use nm-applet, an X11 system tray application. You install your i3, you install your stalonetray and you start nm-applet — hey, it works! Now you can finally connect to the network. And if you wonder how you were supposed to install these packages without network access: by periodically calling ifconfig and ip route add until you finally managed to fetch all the data before NetworkManager would mess it up again.

So you have a bit of a more complex network configuration and need to add routes or, even better, use OSPF to find routes to some targets which don't go straight via some default gateway. (Why? Perhaps because you wanted to run Linux on your default gateway.) Fire up Quagga and you will see how Quagga struggles to add routes while NetworkManager struggles to remove them again. Part of your packets make it to their destination. Also keep in mind that you're now running X11 and a network management GUI on your router!

Now to your road warrior laptop. For simplicity sake and because you already exchanged SSH keys anyway, you decide to connect to your company via SSH-based VPNs. If you do that from the command line, NetworkManager gets very angry with you and does stupid things to ensure you can't put your default route over the new VPN device, or even use it at all.

So you have to use NetworkManager, which only supports vpnc and OpenVPN. However, Open Source vpnc servers are pretty much nonexistent and OpenVPN requires you to either set up a complete PKI or live with ridiculous preshared key algorithms. tinc supports simple public/private key algorithms, but it is not supported by NetworkManager. So the only way to make VPN work is to migrate to OpenVPN and to maintain your own PKI.

Choice

There are many more such effects with the new interfaces but these examples should suffice for now.

The effect of all those changes are numerous. For one, it is no longer possible to run the system without a graphical user interface unless you plan to invest a huge amount of work and to throw out most of your system support. If you want to get vendor support, this is not the way you will want to go.

You also can't implement complex network or authentication setups anymore. The number of possible combinations in the configuration has been significantly reduced by removing options which are not typically used for desktop systems. Also, since the APIs have a tendency to change very frequently, typically, only genuine supported Gnome or Ubuntu/Fedora software tends to work on the long run. If you try to use an alternative which has an user interface you prefer or has a feature you want, you will find very frequently that it is trying to call some DBus interface which is no longer implemented or has a different set of parameters.

Even worse is if you try to use any window manager that is not KDE or Gnome. Both KDE and Gnome launch a very large amount of daemons which are required by a number of applications — pulseaudio, an user dbus session (in addition to the system dbus instance), gnome-settings-daemon, etc. pp. Many programs also require support from applications which exist as tray icons, so you need to find an application to emulate the Gnome tray — and not all of them do it correctly.

Also, many of the advanced features like suspending the laptop when closing the lid or other ACPI events are no longer implemented as shell scripts in /etc, but have moved to be DBus APIs implemented by Gnome and KDE. The reason is that it becomes much easier to display things on the screen, but it also means that the /etc scripting API is rotting away and will not work on the long run. So if you want your laptop to suspend when you close the lid, your window manager must implement it.

Even worse, some of the applications don't react very well under window managers which are not KDE and Gnome because they don't implement the original X11 protocol directly and rely on so-called window manager hints.

Debugging DBus based systems

Which brings us to debugging. Even if your API uses DBus, it is not necessarily bug free. So under DBus based systems, you will sometimes see very weird interactions which seem to come out of nowhere, and it is absolutely not clear to you what happened.

You can use dbus-monitor to get an idea of what is going on on the DBus, but if you have some weird interaction you typically have no clue what the name of the DBus call you're looking for may be, so you start dbus-monitor without any filters only to discover that there is a huge amount of traffic, some of which is log messages.

So you try to read it from the logs but they are binary in some format Journal is writing…

This makes the system appear very opaque to everybody who's trying to take a serious look at it and fix problems. The result is that even people like me start going for solutions like ”Restart the application“ or ”Delete the configs“, because debugging a problem becomes extremely time consuming and the interactions between the different applications are no longer well-defined and obvious. This was one of the very basic design principles of UNIX.

Effects on other operating systems

A very common reaction when people hear that the Linux distributors are doing something crazy is to say, ”Who cares, I'll just use my NetBSD/FreeBSD so this won't affect me“. This, however, is only partially true.

The problem is that even users of FreeBSD and NetBSD want to use some of the software which was implemented for one of the desktop environments will have to find ways to make the DBus services work and react in the correct way. Jared McNeill attempted this with the NetBSD port of DeviceKit, but most operating systems aren't designed to support the kind of APIs involved. As a result, it becomes extremely difficult to support such software, and makes all operating systems more like Linux if they want to be able to run this type of software.

This is the exact opposite of the design principles of standards like POSIX and the Single UNIX specification. These specifications set a common high-level ground for all operating system interfaces, but leave the implementation details up to the systems. In order to honor the thought put into these design principles, the system shouldn't depend on anything other than a C API either so the implementation details are entirely up to the implementor.

More than that, this again affects choice and diversity. One of the biggest arguments the Linux community has brought up for migration from Linux was diversity, but right now, Linux implementors are competely ignoring this plea of their own. Instead, they come up with, well, ”proprietary“ Open Source software which locks people to use Linux and Gnome/KDE.

And this change in design principles is something which should be changed very soon. The current tendency towards DBus interfaces is actively harming the more proficient users in the various ways they made for themselves. Linux and UNIX have always been about the ease of customization. Gnome and KDE are both based on the idea that this only confuses the first-time users and should not be offered, which is fine. However, these window managers are now forcing themselves onto the users, and limiting the user's choice of operating system to just one. This is harmful and obliterates most of the advantages UNIX and Linux systems have given us.

So if you believe in the principles behind UNIX and Open Source, please don't write software which requires any of the Gnome/KDE and DBus API. Writing X11 programs with xcb and proper RPC APIs like SUNRPC or Thrift should be more than good enough. So, please support choice and freedom by implementing programs the right way instead of the Linux/Gnome/DBus way.

Since a lot of the world still evolves around PHP, and a friend of mine asked me to host a PHP based blog, I was confronted with a typical PHP application which, like all typical PHP applications, enjoys heavily writing to its installation directory and keeping its own configuration there.

Being the automation freak I am, I rolled an RPM package of the PHP application which installs it into /usr/share. Then, a Puppet rule creates an Apache vhost in /home/www and union mounts the shared installation into htdocs, with a vhost subdirectory named confdata as read-write layer.

It turned out I had better used NetBSD for the task. The only unionfs implementation available in CentOS 5.5 is fuse based and called funionfs. However, funionfs doesn't support SElinux contexts, so everything ends up in the context fusefs_t, leaving it inaccessible to Apache. A small SElinux module fixed that:

module serendipity 1.0;

require {
    type httpd_t;
    type fusefs_t;
    class lnk_file read;
    class dir { read write remove_name getattr create search add_name };
    class file { read write getattr create setattr rename };
}

#============= httpd_t ==============
allow httpd_t fusefs_t:dir { read write remove_name getattr create search add_name };
allow httpd_t fusefs_t:file { read write getattr create setattr rename };
allow httpd_t fusefs_t:lnk_file read;

This might not be the most secure solution but nothing other than Apache runs on this VM anyway, so I didn't care enough. It's still better than turning off SElinux entirely.

In order to allow the software to access the database, I had to flip another SElinux switch:

httpd_can_network_connect_db --> on

Now things almost worked. However, installing templates via the web interface does not, so I went on to investigate:

% cd /home/www/s9y.zrh.internetputzen.com/htdocs/templates
% mkdir test
mkdir: cannot create directory `test': No such file or directory
% ls -ld test
ls: test: No such file or directory
% touch test
touch: setting times of `test': File exists
% ls -ld test
-rw-r--r-- 1 root root 0 May 24 20:49 test
% rm test

It's impossible to create directories in the funionfs. Apparently it's some kind of bug. Creating the template in confdir worked but it means the web interface is not working.

Looking forward to aufs2 in later versions of CentOS.

Yesterday and today I tried using an USB stick as a medium for installing Fedora 13 on my netbook (a HP Mini 110). In the course, I discovered a bug in liveusb-creator which appears to umount the target file system (type vfat) properly before running syslinux in order to install the boot code. As syslinux also scribbles on the device, the result is a botched file system.

Thinking I had found the cause of my problems, I went on to create my own bootable USB stick — unsuccessfully of course. The /boot partition, which is located at the beginning of the device, always remounted itself read-only when I tried to install the kernel or grub. Sometimes it even showed up as read-write in /proc/mounts, yet it wasn't writable. So I started looking for SElinux context problems — none.

Attempting to set an explicit SElinux context for the /boot partition upon mounting, I unmounted the /boot partition, only to get some terrible notifications in my dmesg:

 Mar 10 11:51:54 methusalix kernel: ext3_abort called.
 Mar 10 11:51:54 methusalix kernel: EXT3-fs error (device sdb1): ext3_put_super: Couldn't clean up the journal

Yet somehow, it was unmounted. I tried to mount the partition again, which yielded:

 Mar 10 11:56:24 methusalix kernel: journal_bmap: journal block not found at offset 12 on sdb1
 Mar 10 11:56:24 methusalix kernel: Aborting journal on device sdb1.
 Mar 10 11:56:25 methusalix kernel: __journal_remove_journal_head: freeing b_committed_data
 Mar 10 11:56:41 methusalix kernel: ext3_abort called.
 Mar 10 11:56:41 methusalix kernel: EXT3-fs error (device sdb1): ext3_journal_start_sb: Detected aborted journal

In the end, the file system was not mounted. Great. So I ran a file system check, which crashed with ”Too many errors in inode …“

So my next idea was to run badblocks, so I tried to umount the root partition of the new system on the stick, and got some more nice kernel messages:

 Mar 10 11:57:01 methusalix kernel: EXT4-fs error (device sdb2): ext4_mb_generate_buddy: EXT4-fs: group 0: 6936 blocks in bitmap, 7095 in gd
 Mar 10 11:57:01 methusalix kernel: JBD: Spotted dirty metadata buffer (dev = sdb2, blocknr = 0). There's a risk of filesystem corruption in case of system crash.
 Mar 10 11:57:01 methusalix kernel: JBD: Spotted dirty metadata buffer (dev = sdb2, blocknr = 0). There's a risk of filesystem corruption in case of system crash.

Mount returned ”umount: /mnt: device is busy.“ although lsof did not reveal any users of the file system. All I could do at this point was to reboot and run badblocks thereafter. After running for approximately 1 ½ hours, it finally displayed:

 3921914
 3921915
 3921916
 3921917
 3921918
 3921919
 done                                
 Pass completed, 798052 bad blocks found.
 tonnerre@methusalix:~% 

I don't think there's any rescue for this USB stick.

In a FAQ (Frequently Answered Questions), people usually expect answers to the questions they might ask. Ubuntu is taking things to a new level there and simply repeats the question, adding a new one in the end:

Question: Ubuntu One Client FAQ #588: "My Account says that I have had an account for x months. What does this mean?"

Answer: The My Account page says that I have had an account for x months. This is confusing. What does it mean? Does this represent the age of my subscription?

Source

Thanks for clarifying, Matt Griffin!