October 2008 Archives

2008-10-31 16:29:30

If you forgot screen in irssi

There's always something new to learn on #netbsd. If you happened to forget to start irssi inside screen, it suffices to run:

/upgrade /usr/pkg/bin/screen /usr/pkg/bin/irssi

…and your session will be running inside screen.

Posted by Tonnerre Lombard | Permanent link | File under: general

2008-10-29 13:45:42

Iterating hot air

Python allows to create iterators through the builtin function iter. This function has two ways of being called:

  • iter(collection) -> iterator
  • iter(callable, sentinel) -> iterator

The first call is rather unproblematic for sure. It calls a class method, it gets its object and that's it.

The second variant is however utterly useless. The callable is not called with any object — in fact, the call to iter does not even know what object is being iterated. This requires the programmer to store the state externally in some kind of global variable, or not to have any state at all. The latter would render the iterator rather pointless, since a crucial part of the purpose of an iterator is to externalize the state.

Posted by Tonnerre Lombard | Permanent link | File under: broken, programming

2008-10-28 16:08:20

Why Greylisting is harmful (2)

An ISP has a mail cluster solution which is distributed over a number of hosts in a /24 subnet. The SPF record correctly reads:

$ host -t TXT isp.com isp.com descriptive text "v=spf1 ip4:subnet/24" $

Since the mail cluster has to deliver a large amount of mails per day, it is attached to a SAN and distributes message delivery over the various servers. A random server picks up a message and attempts to deliver it. Locking works well so no double attempts to deliver it are made, ever. This effectively prevents messages with delivery problems from clogging up the queue on a specific server.

Here's why the ISP can forget their great mail server: greylisting. A lot of implementors don't investigate in any way what they whitelist — there's a variety of options ranging from SPF over the RIPE database to server name wildcards (which would be nasty, though) — but instead whitelist one single host. Then, however, the likelyhood of the resend attempt being performed by the same server is fairly low, so the next server will also hit the greylisting barrier. This can continue for a long time until the mail is finally delivered — or, if things go really bad, rejected.

While this is not an argument against greylisting itself, it is one against a majority of implementations.

Posted by Tonnerre Lombard | Permanent link | File under: network

2008-10-27 11:16:54

Integer() is an int, but be aware that it's not

In Python's external ASN.1 toolkit features a special type, Integer(), which can be written to or fetched from an ASN.1 structure transparently. The resulting object, despite being an instance of pyasn1.Integer, behaves like any other integer: all operations can be applied as one would apply them to an integer.

All operations? Not quite! A single function parameter is holding out, strong as ever, against the integer compatibility. Life is not easy for the programmers attempting to make use of the resulting objects.

In fact it's quite funny: the problem is with the function pow. As the documentation says:

In [1]: pow.__doc__
Out[1]: 'pow(x, y[, z]) -> number\n\nWith two arguments, equivalent to x**y. With three arguments,\nequivalent to (x**y) % z, but may be more efficient (e.g. for longs).'

Since crypto wants speed, and ASN.1 is frequently used in crypto, we now try to combine pyasn1 and the amk crypto toolkit, since Python itself does not contain any crypto. We use the DSA sign function, which basically only calls pow() for us. This is where we get bitten. The reason is simple and astonishing:

In [2]: i = pyasn1.type.univ.Integer(23)
In [3]: pow(i, 23)
Out[3]: Integer('20880467999847912034355032910567')
In [4]: pow(23, i)
Out[4]: Integer('20880467999847912034355032910567')
In [5]: pow(23, 23, i)
TypeError Traceback (most recent call last)

/home/tonnerre/<ipython console> in <module>()

TypeError: unsupported operand type(s) for pow(): 'int', 'int', 'instance'

In [6]:

Astonishingly, the Integer instance is allowed as the first and second argument to pow(), but not as the third one. More astonishingly, all of them are supposed to be ints. Even more weirdly, if pow is called with 3 pyasn1 Integer()s, the error message is still the same: the first two Integer()s are said to be int, the last one remains an instance.

Posted by Tonnerre Lombard | Permanent link | File under: broken, programming

2008-10-23 13:16:39

ASN.1 for DSA public and private keys

I spent some hours today looking for an adequate description of the ASN.1 structure of a DSA public and private key. In theory, RFC3279 should cover this, and it does, in some way or other: section 3.2.3 has very confusing information about the structure.

But even though I am coding Python at this point in time, Perl has once again saved me: the Crypt::DSA::Key::PEM module contains the ASN in its source code:

DSAPrivateKey ::= SEQUENCE {
    version INTEGER,
    p INTEGER,
    q INTEGER,
    g INTEGER,
    pub_key INTEGER,
    priv_key INTEGER

DSAPublicKey ::= SEQUENCE {
    inner SEQUENCE {
        DSAParams SEQUENCE {
            p INTEGER,
            q INTEGER,
            g INTEGER
    pub_key BIT STRING

DSAPubKeyInner ::= INTEGER

Thanks a ton, Perl!

Posted by Tonnerre Lombard | Permanent link | File under: programming

2008-10-22 08:33:43

Free Software Mag: "10 ways to attract women to your FS project"

The Free Software Magazine has published an article about Ten easy ways to attract women to your Free Software project.

Judging from the suggestions, it is absolutely unclear if the article is for real or just a prank. Some arguments are perfectly valid, such as ”Informal apprenticeship with men may be awkward“. One can observe the effect at geek events already; women are usually asked which man they ”belong to“, apparently people find it hard to believe that the woman might attend the event out of interest. This goes so far that women even introduce themselves as ”the girlfriend of …“

Other arguments just appear to be plainly insulting (such as women having less time due to childcare, an argument which appears to regard women as ”birth giving machines“. Following this argument, women would not be present in any job) or simply bullshit (such as avatars in web fora giving people a ”face“ – a lot of women actually appreciate the anonymity of the web because it protects them from seduction-seeking men).

Some people even go so far to say that only women can really judge the problems. While they certainly cannot be excluded from the process, we cannot really expect the tiny minority of our community which they constitute to solve our problem for us. It is a bad excuse of men to keep out of the issue.

I have, myself, already invested a large amount of time into the matter over time, since I am also seeing the low percentage of women in the IT as a big problem. I see a lot of these points as well, but I am seeing a variety of problems here ranging from unfair treatment over stupid male procedures to simple education issues on both sides. My opinion on it all can, however, be summarized in two short sentences:

  1. Men should wisen up slightly and not treat the women in any special way. They don't need nor want it, it only drives them away.
  2. A little less conversation, a little more action, please (© Elvis Presley)

Posted by Tonnerre Lombard | Permanent link | File under: chaos

2008-10-20 01:32:38

Linux wrecking fest

On Saturday, I attended the Linux install fest of the GNU Generation at EPFL. Despite the fact that it was all about penguins, it was actually quite a bit of fun meeting all different types of people.

During the preparation phase, we wired all installation tables with power and ethernet cables of different lengths. For that purpose, we used GNU/multi power sockets.

GNU/multi power socket

Nathalie ensured that I got a sufficiently twisted case, so in my case it was not about an installation but rather to make a Firewire hard disk work on an existing installation. (Yes, Linux install fests are also for fixing problems.)

After ensuring that it is not a problem with the Firewire OHCI, I discovered that plugging the disk into the OHCI caused an interrupt storm. On some rare occasions, however, it would work as usual. On a Windows and a MacOS of nearby users, the same occurred. Thus, it seemed that the firewire client interface on the disk was broken.

Another patient nearby had problems with the network card not transmitting. After a reboot, it disappeared entirely from the PCI device list. Deactivating and reactivating it in the BIOS did not help, it remained gone, even on the Windows installation of the notebook.

But on the upside, I also managed to make a compiz installation work, showed people how to use zsh, SSH keys, sshfs and gmailfs, and apparently left a good general impression: my client gave me the best possible evaluation.

Other features from the event were an eekpc cluster…

EekPC cluster

…and a foldable keyboard.

Foldable keyboard

Posted by Tonnerre Lombard | Permanent link | File under: chaos

2008-10-19 23:52:15

Petition against biometric passport a success

The federal chancellery announced recently that the freedom campaign against biometric passports and ID cards was a success. The Freedom Campaign itself has a press release on the subject.

Out of 64'064 collected signatures, 63'733 signatures were considered valid. This greatly exceeds the 55'000 signatures submitted by the freedom campaign itself, apparently, signatures have as well been submitted by other campaigns.

Congratulations to such a great success! But the petition is only the first step. When the referendum takes place, it must be won, and should it be won, then alternative legislation should be proposed so we won't end up with the same situation in a couple of years.

Posted by Tonnerre Lombard | Permanent link | File under: politics

2008-10-19 23:13:16

Military invasion at the local Coop

When my train arrived from Geneva tonight, I went to the Coop Pronto to get some bread. And then the Coop suddenly started to get crowded with military. As Miguel appropriately noticed, these people appear to live on shrink-wrapped sandwiches and barley juice.

Military invasion at the Coop

As it turned out, they were apparently all waiting for a train to take them to their training camp.

Posted by Tonnerre Lombard | Permanent link | File under: general

2008-10-10 14:56:32

Ugliest code snippet of the day: MySQL initialization

The ugliest code snippet I could find today is the code required to initialize a MySQL database:

MYSQL *mysql = malloc(sizeof(MYSQL));
/* Note: we omit the NULL pointer check here because MySQL does it too. */
if((mysql=mysql_init(mysql))==NULL) { (error handling) }

Posted by Tonnerre Lombard | Permanent link | File under: broken, programming

2008-10-10 13:39:08

PyException P-p-p-p-panic

Python as a programming language features exceptions, as most modern languages do nowadays. However, since it is Python, these exceptions of course have to be named slightly differently from the way it is done everywhere else.

Usually, exceptions are thrown using throw new Exception(); and the construction to catch them is usually:

try {
catch (SomeException se)

In Python, the construct does not deviate all that much logically, but the terminology is entirely different. Instead of throwing exceptions, Python raises them like signals (raise SomeException()). Also, they are not caught but excepted:

except SomeException, se:

Funnily, the indentation varies between the version. If a finally is added in Python 2.4, it cannot coexist in the same indentation level with except:

>>> try:
...     raise Exception()
... except:
...     print('Ugh')
... finally:
File "<stdin>", line 5
SyntaxError: invalid syntax

This is of course fixed in Python 2.5, but since large amounts of wide-spread Python applications such as Zope still depend on Python 2.4, compatibility is a rather delicate problem here.

Moreover, however, exceptions don't seem to work stably even in the latest versions of Python. I have experienced various cases where Python would simply segfault when an exception is thrown:

$ python __init__.py
2008-10-10 13:34:25,572│__init__.py[37]: INFO: Initializing communication server
Segmentation fault (core dumped)

The corresponding code:

log.info("Initializing communication server")
    (some code)
except Exception, e: # This is where the core dump occurrs
    log.critical('unable to unregister original servant')

One has to wonder how to handle this case from within Python; probably not at all. All one can do is to guess what the problem was and try to fix it.

Posted by Tonnerre Lombard | Permanent link | File under: broken

2008-10-09 12:06:00

Partial results are valid but processing is subverted

People keep wondering why I changed to git-svn as a subversion client, even though I tend to dislike git. The answer is simple: it is the only usable Subversion client.

Hearing the protests in my ears already, I think I'll simply paste some of the sessions I recently had with Subversions “svn” client. All of these appear to occur regularly.

Of course, the proper procedure would be to debug these, but after a short visit to the code I had to decide that I should concentrate on getting my work done rather than to fix this bunch of, well, code.

svn update me to fail

$ svn up
U src/toolbox/peersarray.py
svn: In directory 'src/toolbox/pypath'
svn: Error processing command 'delete-entry' in 'src/toolbox/pypath'
svn: Can't read directory 'src/toolbox/pypath/idl/.svn/tmp': Partial results are valid but processing is incomplete
Segmentation fault (core dumped)
$ svn up
svn: Working copy '.' locked
svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details)
Segmentation fault (core dumped)
$ svn cleanup
svn: Can't read directory 'tests/t/.svn/tmp': Partial results are valid but processing is incomplete
Segmentation fault (core dumped)

More svn update woes

$ svn up
svn: Directory 'src/toolbox/trunk/idl/.svn' containing working copy admin area is missing

svn check this out!

$ svn co svn+ssh://… test
A test/testfile
Segmentation fault (core dumped)
$ cd test
$ svn cleanup
svn: Can't read directory '.svn/tmp': Partial results are valid but processing is incomplete
Segmentation fault (core dumped)

One has to wonder how these people actually get any work done.

Posted by Tonnerre Lombard | Permanent link | File under: broken

2008-10-09 11:58:34

Helpless MySQL

Trying to set up a MySQL instance to test a backup script against, I encountered a nice endless social engineering loop:

mysql> help contents

Nothing found
Please try to run 'help contents' for a list of all accessible topics


Funnily, this fits nicely with the commercial support I'm used to from MySQL A/B.

Posted by Tonnerre Lombard | Permanent link | File under: broken

2008-10-04 01:14:57

courier-authlib: how not to use libtool

Due to security issues with the old version, I upgraded courier-authlib to version 0.61.0. In the course of the upgrade, I discovered a number of library related weirdlesses, though.

The symptom was that /usr/pksrc/lib/courier-authlib/libcourierauth.so.0 was a symlink on libcourier.auth.so rather than the other way around. Closer inspection of the Makefile showed that evidently someone had a very weird understanding of libtool.

The .la rules all contianed the flag -avoid-version, removing the .0.0 version suffix from the library version. However, it seems that the authors nevertheless wanted these suffixes, so they re-added them using the for loop:

for f in $(pkglib_LTLIBRARIES); do . $$f; rm -f $(DESTDIR)$(pkglibdir)/$$dlname.0 $(DESTDIR)$(pkglibdir)/$$dlname.0.0; ln -s $$dlname $(DESTDIR)$(pkglibdir)/$$dlname.0; done

This line reads every .la file as a shell script and tries to symlink the unversioned library to the versioned one, since the addition of -avoid-version prevents libtool --mode=install from taking care of the symlinks themselves. Even worse, on some platforms these versioned files should not even exist. It only ever works by pure chance.

This is just one of the countless ways people found to abuse libtool in horrible ways to break portability. It seems that the funniest part about libtool is the number of ways in which people don't understand it.

Posted by Tonnerre Lombard | Permanent link | File under: programming