2008-02-17 02:45:19

The OpenLDAP way of concurrency

OpenLDAP is a multithreaded implementation of an LDAP server, licensed under a special derivate of a 2-clause BSD license called «OpenLDAP license». It uses LDBM or BerkeleyDB as a storage backend or can alternatively be configured to read its information from an SQL database or other formats (which effectively disables writing to the database). Like for BSDprojects.net, BerkeleyDB is frequently used for user management (authorization, authentification, mail addresses, etc.).

Up to this point, all attempts to upgrade to OpenLDAP's BerkeleyDB backend have failed. There was always the same error: when the database has been filled with some data, any LDAP search request would crash the LDAP server. The stack trace usually ended in some call instruction or some other instruction immediately following a call.

In an answer to NetBSD PR 37997, I was reminded by Christos Zoulas that the problem might actually be no crash at all but an issue with the rlimit facility of NetBSD, so I experimented a bit with the stack limit.

By default, NetBSD imposes a limit of 2 MB on the program stack. However, not even 4 or 8 MB seemed enough. A stack limit of 32 MB however solved the problem entirely.

It seems that OpenLDAP attempts to avoid locking problems between threads by putting all information on the function local stack. This only works well with a low amount of data or with low parallelism. However, in the case of an LDAP server, shared data structures would indeed be desirable.

LDBM itself does not use so much stack space, so it did not trigger the stack space limit. However, with parallel requests being handled by multiple LDAP threads, iit would undoubtedly only be a matter of time.

Let's see how well the 32 MB stack limit works...


Posted by Tonnerre Lombard | Permanent link | File under: programming, general

2008-02-17 01:50:38

OSS Jam Reloaded in Zurich

After the success of the first OSS Jam, Google invites people to a second round in its new Zurich office on February 28th, 2008. Once again, participants are invited to present their projects in a short 5-minute time frame, trying to find future project contributers.

This time, a topic has been set for submissions: «The desktop in the past, present and future». People who are seeking for participation in their desktop projects are invited to present them at the jam.

OSS Jam and Google

The question is of course how this fits into Google's search engine business. To this question, there are two different, unrelated answers.

Firstly, Google has extended its scope beyond search engines quite some time ago. Like Yahoo delivered widgets for web applications, Google also delivered the Google Widget Toolkit for Java web applications and similar products, and is expanding its scope to Open Source and the community.

Secondly, Google's OSS Jam provides the Open Source community with ways to find new participants for their projects, and as such, there is a philosophical relation to the search business.

Either way, it is going to be interesting to watch the future development of this tradition.


Posted by Tonnerre Lombard | Permanent link | File under: programming

2008-01-20 20:31:02

EU Commission launches next stage of DRM consultation

The European Commission has launched the next stage of the Digital Restrictions Management consultation. In this stage, the current proposal has been published and is open to comments from all stakeholders until February 29th, 2008.

The time for these measures appears particularly badly chosen, especially in the light of the fact that the big producers, such as Sony BMG or Warner Music, have recently decided that there is no added value in the use of DRM in copyright protection, and no longer DRM protect their media. Under these circumstances, the entire process should be questioned again before pointless measures are taken.


Posted by Tonnerre Lombard | Permanent link | File under: news, politics

2008-01-20 20:05:46

RIAA linking to pirated software?

Rumour has it that the RIAA is linking to pirated software. This is an interesting step to take for them, especially because it has been decided earlier in the DeCSS case that even the act of linking to web sites containing legal infringements is a legal infringement.

The impact of this case is going to be interesting. This is not the first time that the RIAA, which is known for their lawsuits against countless users of file sharing software and Linux video players, is itself in infringement of the rights they claim to defend.

Update: This is just a case of cross site scripting, there is no infringement on behalf of the RIAA.


Posted by Tonnerre Lombard | Permanent link | File under: general, politics

2008-01-14 19:33:14

SwissIX Outage - Rumours

Starting from 14:12 on Sunday, the 20th of January, massive packet loss started to occurr on all links at SwissIX. Shortly after that, all connectivity from many autonomous systems stopped. Swisscom GPRS fallback worked well though, apparently since Swisscom had enough peerings outside of SwissIX.

Investigating the situation in SwissIX, there were various incarnations of always the same packets flooding the network – it looked entirely like a switch loop. In theory, however, all switches used in datacenters should do spanning tree in order to avoid this situation, so this explanation appears somewhat vague. A mail on layerone-customer however also claims that a switch loop in InterXion was causing the failure. Maybe an unusual situation triggered a bug, causing STP to fail?

Cyberlink restored connectivity pretty quickly, once the interfaces at SwissIX were shut down, the traffic ran over Germany and similar fallback routes. swissix.ch remained unreachable, also via Swisscom GPRS. The SwiNOG coordination network was split into two pieces, rendering it ad absurdum.

Solnet took more than 2 hours to recover. Around 15:30, the full packet loss after the main Zuchwil router converted to a routing loop with the nexthop. Around 16:30, connectivity started to get back to normal. Maybe there are too few non-SwissIX peers available for this case?

Unfortunately, the outage falls right between the moving of the NGAS.ch network monitoring infrastructure, so there are no concise graphs available showing what exactly had happened. This has been corrected immediately afterwards.


Posted by Tonnerre Lombard | Permanent link | File under: news, network