February 2008 Archives

2008-02-17 02:45:19

The OpenLDAP way of concurrency

OpenLDAP is a multithreaded implementation of an LDAP server, licensed under a special derivate of a 2-clause BSD license called «OpenLDAP license». It uses LDBM or BerkeleyDB as a storage backend or can alternatively be configured to read its information from an SQL database or other formats (which effectively disables writing to the database). Like for BSDprojects.net, BerkeleyDB is frequently used for user management (authorization, authentification, mail addresses, etc.).

Up to this point, all attempts to upgrade to OpenLDAP's BerkeleyDB backend have failed. There was always the same error: when the database has been filled with some data, any LDAP search request would crash the LDAP server. The stack trace usually ended in some call instruction or some other instruction immediately following a call.

In an answer to NetBSD PR 37997, I was reminded by Christos Zoulas that the problem might actually be no crash at all but an issue with the rlimit facility of NetBSD, so I experimented a bit with the stack limit.

By default, NetBSD imposes a limit of 2 MB on the program stack. However, not even 4 or 8 MB seemed enough. A stack limit of 32 MB however solved the problem entirely.

It seems that OpenLDAP attempts to avoid locking problems between threads by putting all information on the function local stack. This only works well with a low amount of data or with low parallelism. However, in the case of an LDAP server, shared data structures would indeed be desirable.

LDBM itself does not use so much stack space, so it did not trigger the stack space limit. However, with parallel requests being handled by multiple LDAP threads, iit would undoubtedly only be a matter of time.

Let's see how well the 32 MB stack limit works...

Posted by Tonnerre Lombard | Permanent link | File under: programming, general

2008-02-17 01:50:38

OSS Jam Reloaded in Zurich

After the success of the first OSS Jam, Google invites people to a second round in its new Zurich office on February 28th, 2008. Once again, participants are invited to present their projects in a short 5-minute time frame, trying to find future project contributers.

This time, a topic has been set for submissions: «The desktop in the past, present and future». People who are seeking for participation in their desktop projects are invited to present them at the jam.

OSS Jam and Google

The question is of course how this fits into Google's search engine business. To this question, there are two different, unrelated answers.

Firstly, Google has extended its scope beyond search engines quite some time ago. Like Yahoo delivered widgets for web applications, Google also delivered the Google Widget Toolkit for Java web applications and similar products, and is expanding its scope to Open Source and the community.

Secondly, Google's OSS Jam provides the Open Source community with ways to find new participants for their projects, and as such, there is a philosophical relation to the search business.

Either way, it is going to be interesting to watch the future development of this tradition.

Posted by Tonnerre Lombard | Permanent link | File under: programming