For those using redland for RDF in the wild

Not so much if, but when redland corrupts its db4 files here is a series of commands that you might want to shove into a script. Though you really want to setup daily db_dumps that are archived if you use redland for data you care about.

I have attempted and posted a patch and nagged to get this fixed mainline. But I'm not holding my breath that it will be fixed any time soon. I might have to dump redland, fork it proper, or some other nasty solution to stop libferris users complaining about apps "crashing" which is directly due to redland getting its db4s stuffed.

You get the pleasure of running this on all your redland db4 files to try to recover (at least you can run client apps again without a segv in the redland code).

$ db_dump -r myrdf-so2p.db > myrdf-so2p.dump
db_dump: myrdf-so2p.db: DB_VERIFY_BAD: Database verification failed
$ rm -f myrdf-so2p.db
$ db_load -f myrdf-so2p.dump myrdf-so2p.db
$ rm -f myrdf-so2p.dump

The public bug is here
http://bugs.librdf.org/mantis/view.php?id=104

Manufacturing Descent

The joys of changing one of the base libraries of the libferris suite. I had to release 8 packages at once because the base library had a few cleanups that were not backward compatible. I thought it was better to just break things a little now before the OBS packages for Fedora, openSUSE and Ubuntu are publicised for general consumption.

In other news, you can now get two sets of EA for results from a search on a PostgreSQL EA index. inside either the "idx:" or "index:" namespace are EA versions that are taken directly from the database itself. So for example you can show the MD5 checksum for resulting files by using idx:md5 as the result column. This is very handy for checksums and the like but also handy when you don't actually want to go and touch the original file at all to present results. There are still a few little changes to be made for full, totally offline, removable SATA drive index result presentation but things are very close to that level now.

Also checkout the apps/phpsearchinterface directory in the latest libferris release for two versions of a web interface to libferris searching. The first is a simple example that uses XSLT on client side to present XML search results to the user. The second example uses an AJAX interface together with the dhtmlxSuite grid control (not included) to present a nicer, more modern interface. These two clients are just examples and are in their infancy but they do give (particularly the latter example) an idea of how libferris can be used on a LAN environment for index searching. Think "libferris Intranet search" ;-)

Eclipse... now it runs ok, but...

Yes, with heaps of RAM and a ulimit higher than 90% of the apps I execute, eclipse runs up... Don't get me wrong, it can have 2gb of RAM if it wants, if it works well...

The emacs bindings will leave you in despair. Some bindings that an emacs user expects to exist at a fundamental level are C-x C-b which almost happens. But you are left with a dropdown of buffer names and the most recent non active buffer is not highlighted by default.

Cap SenSitive search and replace seems to be coming in 2008. And although you can start an incremental search with C-s without a dialog hitting C-s C-w C-w will not grab a nice chunk of identifier to isearch-forward with. The s-and-w key being so close one quickly gets into the habit of that for isearching. The search and replace bug makes me wonder how folks actually use eclipse to edit code, though I guess you don't miss what you didn't have.

So ye olde emacs will likely live on for another 2-5 years as my C++ editor of choice. Either that or somebody will drop enough cash to actually get eclipse-emacs working to some retro level that can be somewhat tolerated. I guess emacs users are not in the target demographic of eclipse anyway.

Keeping up with the joneseses

After noticing that the boost::serialization binary format is platform dependent recently some folks told me that I should maybe just use the text format for b::serial. The claims of relative efficiently being hard to tell came up of course. So this time I actually benchmarked with valgrind.

For text archive, 223,879,650 instructions and 2.97% overall runtime but using binary archive 2.1% relative runtime and 140,057,074 instructions. The relative runtime is for a short app interaction, not just application startup.

Since the b::ser is only used for loading data that is changed very infrequently but is loaded all the time I might end up having the text version as a fallback just in case you change architectures libferris can try to load the binary version, fail and then try to load the text version and force a resave right at that point. This means loading after an architecture switch will be about 5-6 times slower, but only the first time you run any libferris app after the arch change.

Fun to look ahead to the maemo / moko invasion of the ferris.

I added a few little optimizations here and there to libferris for the next release too. Many of these make the filemanager more responsive in some use cases I have... YMMV.

That's one hell of a package diane

Now I have packages of libferris for Fedora 7 and 8 and opensuse 10.3. So the next obvious move for a someone who has rolled many an rpm would be to jump into the deep end of debian packaging...

root@ubuntu710:/tmp/PKG/libferris-1.2.1# ls -lh  ../*deb
-rw-r--r-- 1 root root 2.4M 2007-11-28 04:39 ../ferris_1.2.1-1_i386.deb
-rw-r--r-- 1 root root 5.4M 2007-11-28 04:40 ../libferris1_1.2.1-1_i386.deb
-rw-r--r-- 1 root root 228K 2007-11-28 04:39 ../libferris-dev_1.2.1-1_i386.deb
-rw-r--r-- 1 root root 249K 2007-11-28 04:40 ../libferrisui1_1.2.1-1_i386.deb
-rw-r--r-- 1 root root  64K 2007-11-28 04:40 ../libferrisxslt1_1.2.1-1_i386.deb

I still need to actually add some pre/post scripts and the likes but it is looking somewhat decent so far. The downside is that the default builds for all three distros are somewhat conservative leaving out many of the higher level, higher dependency functionality. But for folks just wishing to mount xml, db4 and do some desktop search these packages should work. Though you will get reduced selection of what index formats are supported etc.

Open-C for s60, so close but yet so far

Like the vast tribes who look over /. from time to time I noticed the new rewards for developing for Google's new mobile platform. I found this quite interesting considering the recent openc comp and the differences between the pecuniary motivation offered by the two camps.

I tried to port over rsync to s60 using open-c. The massive problems with the API not offering anything to cleanly emulate fork() and exec() are bad enough, try submitting a patch to an open source project with ugly stuff in a #ifdef clause to try to emulate these calls with pthreads and see how far you get. I haven't tried this with the rsync guys yet as I haven't managed to get it running yet :/

The killer, so to speak is the lack of signal() and kill() calls. The self IPC that is quite common of having an app fork() itself and signal itself or wait for itself at a later time. Waiting is one thing, that can be quite readily ported to pthreads type calls using IPC mechanisms. When a parent sends for example USR2 signal to the child is where you run into unbounded joy. Especially where all of this is happening inside a do_recv() function which is used quite heavily in both client and server mode.

I might throw together a kludge for it at some point because it actually compiles (though will not run because I haven't ported some fork() code yet). There are three or so places that use fork() which will need attention. It compiles because I have a nasty pthreads implementation of fork() in an effort to avoid dumping a huge bunch of garbage boilerplate code into apps to be ported. I don't know why the symbian dudes didn't include a int s60_pthreads_fork() implementation....

Berkeley db and libstldb4 for s60

It turns out the problem I was having with bdb on s60/openc was that the default stack size is very small (by desktop standards). Adding EPOCSTACKSIZE 65536 to the mmp file of the test client made things work as one would expect when the sis file is installed on the device itself.

If your stack is too small then the application just magically goes away on the device and you are left wondering why. Not only that but if the function that smashes the stack with auto variables is tracked down you will have a function that, once entered, will have a corrupt stack right from the start. Functions like open(2) will not work properly because things are already on their way to the bit bucket. These sorts of things take a while to track down when you are used to stacks that adjust their size when they get full.

So, long and the short, stldb4 now works on s60/openc :)

I notice that there is also SDL for symbian which when taken together with the fact that evas has a backend engine for SDL makes edje/evas GUIs for desktop and s60 seem like a good choice. Being able to debug the look and feel of the app on a desktop using a window of device size and then just running the edje on the s60 to see if any special effects run fast enough.