Everything is a filesystem. Everything is an emacs buffer.

I finally started hacking on TRAMP to add libferris support. I notice that in 2009 GVFS support was added to TRAMP which is very cool. I was originally poking around the lisp sources for the various things that TRAMP can use as a virtual filesystem when a fairly clean design hit me. Instead of bringing TRAMP to libferris, why not do the opposite?

So I started digging into the 8,000 odd line main tramp.el file and tracing ssh connections to see what commands were executed in the background. The plan being to add any of the POSIX commands that tramp wanted which libferris didn't already have clones of. It turns out there weren't too many, mainly ftest and fstat. These are needed to poke around and obtain details from the filesystem in order for tramp to know if things exist and more information about them. The twist of course if that much of the "stat" data is not available for some libferris filesystems. For example, an element in an XML file might not have a proper octal access mask, or indeed any explicit user/group information.

I currently have load/save working for subelements in XML files, and keys in an ISAM file like a berkeley db4. The patch to tramp itself is around 10-15 lines. Much of this can be abstracted out so you can nominate to use "the libferris command" instead of the normal coreutils one. eg, using fstat instead of stat.

$ fcreate --create-type=db4 --rdn=isam.db .
$ date | ferris-redirect -T isam.db/subfile1
$ db_dump -p isam.db
VERSION=3
format=print
type=btree
db_pagesize=4096
HEADER=END
subfile1
Fri Feb 26 18:24:55 EST 2010\0a
DATA=END

And to edit the subfile1 inside that berkeley db file over the network:
$ emacs "/monkeyiq@myserver:/tmp/isam.db/subfile1"

For performance, it is handy to play with version control and how backup files are made. The later is actually quite important for editing part of an XML file, because you can't have "elementname~" as a valid element in an XML document.

This works OK, but having the backups on the local machine is even faster.
(setq backup-directory-alist
'(("^/[a-z]*:monkeyiq@myserver:.*"
. "/monkeyiq@myserver:/tmp/.saves")))

And now to get editing data from postgresql happening too, and oh, editing a buffer in an emacs process on a remote host which is mounted by libferris?

Five Star General

I've been thinking about proxy patterns a Soprano lately. It seems like a great idea to create a Soprano::Model shim which logs mutable operations to an SQLite database. This would allow intermittent syncs between devices, so a maemo device could have a full local mirror of RDF and communicate with your laptops, servers and desktop machines to keep semantic knowledge in sync.

There are a bunch of nice little edge cases which make this deceptively complex. bnode handling for one changes the rules because identity must be preserved.

I'm not sure if/when I'll hack on such functionality, but I thought I'd blog about it in case somebody else is also kicking around the same ball :)

Libferris: now and then...

Those who have been following libferris developments over the recent years will have noticed that I switched over to using Soprano for RDF support in 2009. I also added support for mounting relational databases (using QtSql) and converted the http and ftp mounting over to use Qt for network IO. This later point also extends to the support for mounting flickr, vimeo, google spreadsheets and docs etc. So Qt is slowly marching closer to the "kernel" of libferris.

I've been toying around with switching over some of the core parts of the libferris kernel to using Qt. Things like using Qt's smart pointers instead of the policy based ones from Modern C++ Design, switching over to using Qt signal/slots, and perhaps offering both iostream and QByteArray data access methods in parallel.

There are a number of "smaller" tasks to support this. For example, creating a stldb4 like library allowing access to Berkeley db4 and other ISAM files with a "Qt native" style API. Stldb4 only supports bdb files, whereas allowing other ISAM like gdbm, tdb, and WhateverDB to be accessed through the hypothetical QISAM library would be nice. If anyone has recommendations along these lines please drop me a line or leave a comment. I mention smaller in quotes above because I know this task is quite difficult to do well, things like ordering and prefix ordering of keys come up as well as support for secondary indexing and other goodies which are really needed to milk performance out of custom ISAM solutions.

The harder part would be porting over the core kernel of libferris to Qt stuff. As it would be extremely hard to do piecewise it would require a contiguous block of free time. It is also a fairly hard thing to try to get financial support for because the only "feature" it adds is the possibility of ports for other platforms. My point of mentioning this is that at some stage, libferris ports for Symbian and other Qt supported platforms will become much simpler. Finally I could "cp" an image directly from the camera of my s60 phone to flickr without the annoying third party apps.

As an aside, I was wondering how long it has been since I started using RDF on the desktop. The November 2003 version 1.1.12 release mentions mounting RDF... so it's been a while. RDF is extremely handy for separating data access from personal metadata. If you want to add an extended attribute to a read-only NFS share, with libferris and RDF you can, and you just use the same API as normal ;)