2011-12-13 01:48:00

The destructive desktop — Linux in trouble?

Linux on the desktop has come a long way. The Gnome and KDE communities have built themselves a big, very powerful set of tools to build on. And using these tools, they created an enormous amount of software for a large number of different purposes.

Then they discovered that there is a lack of formality in the RPC mechanisms available under UNIX like operating systems. The Shared Memory IPC provides just shared memory and a little flow control, which is tedious. The sysvmsg API is still very inconvenient when communicating between various different processes, especially if they're arbitrary. Sockets work much better in that respect and have a well-defined API, but it is still relatively hard to exchange data over them.

UNIX offered the SUN RPC API which was used to implement NFS, among other things. However, it was ”just“ an RPC implementation and not a real service based middleware. Especially among Universities, a rather complex method of doing RPC had become fashionable: CORBA.

At this time, the only available Open Soure CORBA ORB was Mico. However, Mico still lacked some of the desired features and didn't support a lot of programming languages so the Gnome developers decided to implement their own ORB called ORBit.

The KDE people faced a very similar issue. However, instead of implementing all of CORBA, they developed a much smaller, more lightweight protocol called DCOP. DCOP was more tailored to be used for communicating between the different applications.

So the Gnome developers wanted to reduce the complexity of their protocol as well and started working on a protocol which was supposed to join the advantages of DCOP and CORBA. The result was called the Desktop Bus (dbus) protocol. Instead of complete remote objects it just offers remote interfaces with functions that can be called.

esd and PulseAudio

The sound system underwent a similar development. Initially, the operating system provided an API called the Open Sound System. It is based on a read-write device in /dev and a number of IOCTLs. It was the same API which is found in the BSD and Solaris operating system.

However, the Linux incarnation of OSS was a particularly simplicistic one which only supported one sound channel at the same time and only very rudimentary mixing. As a workaround, the community came up with a daemon which accepted sound samples and mixed them in software: the Enlightened Sound Daemon (esound). This daemon even acquired network capabilities so people could stream whatever they wanted over the network to other computers and play it there without having to resort to systems like the Networked Audio System (NAS). The KDE developers went even further and implemented an audio system on top of DCOP called MCOP.

Over time, the Linux kernel developers came up with a new API to control the various details of the sound card, have many different volume settings and to be able to mix in hardware. It was called the Advanced Linux Sound Architecture (ALSA).

Then, Gnome and KDE developed APIs to abstract the uses of OSS, esound and ALSA: gstreamer for Gnome and Phonon for KDE. Since gstreamer depended heavily on the Gnome libs and phonon on the KDE libraries, the rest of the community had to either adopt or try to keep up with the ever changing sound backends. Esound was deprecated and replaced with Pulseaudio, which triggered yet another shift of APIs.

Over time, more and more subsystems started getting DBus based frontends. hald was added to detect hardware properties. It turned into an official dependency for X.Org and was subsequently replaced with DeviceKit. PackageKit was added as a generic API to instruct the system to find and install packages through a DBus interface. ConsoleKit replaced the regular session and pseudoterminal management environment. PolicyKit imposed additional restrictions on system calls and allowed other people to do privileged operations without changing to the superuser explicitly. sssd is now in the process of replacing PAM as an authentication framework which is also contacted through DBus, without the need to use the system authentication and session management API (PAM and NSS, mostly).

NetworkManager added a new abstract way to configure network devices, such as network cards, wireless LAN or 3G. Like everything else, it provides a DBus interface for executing various operations, such as discovering wireless LANs, connecting to a network and awareness if the computer is currently connected to a network. Various GUI programs such as Firefox, Pidgin, Gajim and similar tools use NetworkManager to clean up their caches and reconnect after the network connection was terminated. They also go to some sort of offline mode if NetworkManager tells them to, in which they don't attempt to connect to the network and try to do whatever you want locally (e.g. queuing messages to be sent, displaying web pages from cache, etc.)

Another addition was systemd, which now replaces System V init and all other types of init derivates on various distributions. It is an init daemon which reads services from a database, somewhat like Solaris' svcadm. However, for starting and stopping services and telling the system to shut down or do something else, systemd has a DBus interface. The old /dev/initctl interface is no longer supported.

And most recently, there was a new addition to the pool: Journal is a service which replaces syslog and which exposes a DBus API for logging messages into a binary log (instead of a plain text log like syslog did). The adoption rate among desktop and other programs is great because now finally everybody can use their favorite API to log, grant log permissions, search logs etc. The world is becoming more awesome every single day.

Or did it? RedHat Enterprise Linux (RHEL) is a Linux distribution tailored for long-term support environments — which includes corporate desktops — and servers. Ubuntu LTS is following the same goal: to provide a modern distribution with long-term support for use in corporate workstations and servers.

NetworkManager

The latest versions of both Ubuntu LTS and RHEL ship with NetworkManager for managing their network connectivity. If you don't use NetworkManager, a number of programs will refuse to connect to the Internet or behave in various weird ways. More so, a lot of system services now depend on NetworkManager and won't start unless it is running. And if you run NetworkManager, it starts periodically messing up any local system configuration. So you're basically bound to use NetworkManager.

So you install a server in headless mode (Wait, the installer won't typically let you do that anymore. But let's assume you do it nonetheless because your server doesn't have a graphics card anyway, it's attached to a Cyclades SSH serial port adapter like any other one of your UNIX servers.) Then you try to figure out how to configure NetworkManager from the command line. There's no tool in the entire distribution which lets you do that.

So from some time in the past you remember that you used to use a program called cnetworkmanager to operate NetworkManager from the command line. You install it and — the DBus API changed since the program was written, so the DBus call fails with a not-very-helpful error message.

So the only way to actually use NetworkManager is to use nm-applet, an X11 system tray application. You install your i3, you install your stalonetray and you start nm-applet — hey, it works! Now you can finally connect to the network. And if you wonder how you were supposed to install these packages without network access: by periodically calling ifconfig and ip route add until you finally managed to fetch all the data before NetworkManager would mess it up again.

So you have a bit of a more complex network configuration and need to add routes or, even better, use OSPF to find routes to some targets which don't go straight via some default gateway. (Why? Perhaps because you wanted to run Linux on your default gateway.) Fire up Quagga and you will see how Quagga struggles to add routes while NetworkManager struggles to remove them again. Part of your packets make it to their destination. Also keep in mind that you're now running X11 and a network management GUI on your router!

Now to your road warrior laptop. For simplicity sake and because you already exchanged SSH keys anyway, you decide to connect to your company via SSH-based VPNs. If you do that from the command line, NetworkManager gets very angry with you and does stupid things to ensure you can't put your default route over the new VPN device, or even use it at all.

So you have to use NetworkManager, which only supports vpnc and OpenVPN. However, Open Source vpnc servers are pretty much nonexistent and OpenVPN requires you to either set up a complete PKI or live with ridiculous preshared key algorithms. tinc supports simple public/private key algorithms, but it is not supported by NetworkManager. So the only way to make VPN work is to migrate to OpenVPN and to maintain your own PKI.

Choice

There are many more such effects with the new interfaces but these examples should suffice for now.

The effect of all those changes are numerous. For one, it is no longer possible to run the system without a graphical user interface unless you plan to invest a huge amount of work and to throw out most of your system support. If you want to get vendor support, this is not the way you will want to go.

You also can't implement complex network or authentication setups anymore. The number of possible combinations in the configuration has been significantly reduced by removing options which are not typically used for desktop systems. Also, since the APIs have a tendency to change very frequently, typically, only genuine supported Gnome or Ubuntu/Fedora software tends to work on the long run. If you try to use an alternative which has an user interface you prefer or has a feature you want, you will find very frequently that it is trying to call some DBus interface which is no longer implemented or has a different set of parameters.

Even worse is if you try to use any window manager that is not KDE or Gnome. Both KDE and Gnome launch a very large amount of daemons which are required by a number of applications — pulseaudio, an user dbus session (in addition to the system dbus instance), gnome-settings-daemon, etc. pp. Many programs also require support from applications which exist as tray icons, so you need to find an application to emulate the Gnome tray — and not all of them do it correctly.

Also, many of the advanced features like suspending the laptop when closing the lid or other ACPI events are no longer implemented as shell scripts in /etc, but have moved to be DBus APIs implemented by Gnome and KDE. The reason is that it becomes much easier to display things on the screen, but it also means that the /etc scripting API is rotting away and will not work on the long run. So if you want your laptop to suspend when you close the lid, your window manager must implement it.

Even worse, some of the applications don't react very well under window managers which are not KDE and Gnome because they don't implement the original X11 protocol directly and rely on so-called window manager hints.

Debugging DBus based systems

Which brings us to debugging. Even if your API uses DBus, it is not necessarily bug free. So under DBus based systems, you will sometimes see very weird interactions which seem to come out of nowhere, and it is absolutely not clear to you what happened.

You can use dbus-monitor to get an idea of what is going on on the DBus, but if you have some weird interaction you typically have no clue what the name of the DBus call you're looking for may be, so you start dbus-monitor without any filters only to discover that there is a huge amount of traffic, some of which is log messages.

So you try to read it from the logs but they are binary in some format Journal is writing…

This makes the system appear very opaque to everybody who's trying to take a serious look at it and fix problems. The result is that even people like me start going for solutions like ”Restart the application“ or ”Delete the configs“, because debugging a problem becomes extremely time consuming and the interactions between the different applications are no longer well-defined and obvious. This was one of the very basic design principles of UNIX.

Effects on other operating systems

A very common reaction when people hear that the Linux distributors are doing something crazy is to say, ”Who cares, I'll just use my NetBSD/FreeBSD so this won't affect me“. This, however, is only partially true.

The problem is that even users of FreeBSD and NetBSD want to use some of the software which was implemented for one of the desktop environments will have to find ways to make the DBus services work and react in the correct way. Jared McNeill attempted this with the NetBSD port of DeviceKit, but most operating systems aren't designed to support the kind of APIs involved. As a result, it becomes extremely difficult to support such software, and makes all operating systems more like Linux if they want to be able to run this type of software.

This is the exact opposite of the design principles of standards like POSIX and the Single UNIX specification. These specifications set a common high-level ground for all operating system interfaces, but leave the implementation details up to the systems. In order to honor the thought put into these design principles, the system shouldn't depend on anything other than a C API either so the implementation details are entirely up to the implementor.

More than that, this again affects choice and diversity. One of the biggest arguments the Linux community has brought up for migration from Linux was diversity, but right now, Linux implementors are competely ignoring this plea of their own. Instead, they come up with, well, ”proprietary“ Open Source software which locks people to use Linux and Gnome/KDE.

And this change in design principles is something which should be changed very soon. The current tendency towards DBus interfaces is actively harming the more proficient users in the various ways they made for themselves. Linux and UNIX have always been about the ease of customization. Gnome and KDE are both based on the idea that this only confuses the first-time users and should not be offered, which is fine. However, these window managers are now forcing themselves onto the users, and limiting the user's choice of operating system to just one. This is harmful and obliterates most of the advantages UNIX and Linux systems have given us.

So if you believe in the principles behind UNIX and Open Source, please don't write software which requires any of the Gnome/KDE and DBus API. Writing X11 programs with xcb and proper RPC APIs like SUNRPC or Thrift should be more than good enough. So, please support choice and freedom by implementing programs the right way instead of the Linux/Gnome/DBus way.

Flattr this