libferris available for debian arm hard float

To compliment my existing packages of the libferris virtual filesystem and index/search suite for soft float debian, I now offer hot off the press, debian hard float! The distinction between how floating point is handled probably doesn't make a big difference to the operation of libferris, but if you are installing debian on an odroid-u2 then you are likely running hf, and as such having debs which are hf makes installing libferris a whole bunch simpler.

With the eMMC card on the u2, it is a really enjoyable little server machine to play around with. So far I've done the rudimentary test that XML is mountable as a filesystem and created one or two indexes with the Qt/SQLite index plugin for libferris. Note that in recent releases the sqlite backend is transaction backed which gives a huge performance increase, and on really IO constrained machines this is even more noticeable. This is a little tip for those using QtSQL, transactions are not just for making operations atomic, you may find that the whole show runs faster when it is transaction protected.

If you haven't played with libferris, things are auto mounted where possible, and there are many coreutils like tools to make interacting with ferris simple. The ferris-redirect is like the bash redirection but can write to any filesystem that libferris knows how to mount.

ben@odroidu2:/tmp/test$ cat example.xml
<top>
<node name="foo">bar
</node>
</top>
ben@odroidu2:/tmp/test$ fls example.xml
top
ben@odroidu2:/tmp/test$ fls example.xml/top
foo
ben@odroidu2:/tmp/test$ fcat example.xml/top/foo
bar
ben@odroidu2:/tmp/test$ date | ferris-redirect -T example.xml/top/foo
ben@odroidu2:/tmp/test$ cat example.xml
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<top>
  <node name="foo">Tue Apr 30 14:57:26 PDT 2013
</node>
</top>

The above interaction would also work for mounted Berkely DB and other filesystems of course.


I have noticed one of the binary scoped destructors doesn't like the hard float build for whatever reason. This can cause some of the command line tools to not exit gracefully, which is a shame. I can't get a good backtrace for the situation either, which makes tracking it down a nice day long adventure into trial and error.

So something productive has been generated during the last round of jet lag after all!

The goods http://fuuko.libferris.com/debian/debian-armhf/


Save Ferris!

FontForge: Rolling Type Design with No Save Using Collab

FontForge has some support for collaborative type design. It's early days, but things are moving along in the right direction. In order to test things I've been looking at the python scripting support with an eye to moving control points around etc, or doing design and modification from the script and having the collab server keeping up with things. This way I can "save" the type that is being designed at certain points and compare the saved font with that I would expect the result of multiple collaborators (the previous python scripts) should be. Automated testing for the win!

So, No Save is only for the "doing" scripts. I of course want to save the current type at my leisure :)

You might be wondering what a script that creates a glyph might look like. I had to do a bit of trial and error to figure out how to use the scripting API myself. With that in mind, I might roll some of my scripts into the mainline FontForge git repo so others can enjoy the little snippits to base their own scripts on.

Anyway, the following script will load a font and create a new capital "C" glyph. The core of this that wasn't that intuitive to me is that you have to set g.layers[] to the layer you got earlier from g.layers[] or the new contour will not show up in the /tmp/out.sfd file. See the MUST comment for the needed line.

import fontforge

f=fontforge.open("test.sfd")       
fontforge.logWarning( "font name: " + f.fullname )

g = f.createChar(-1,'C')
l = g.layers[g.activeLayer]
c = fontforge.contour()
c.moveTo(100,100)   
c.lineTo(100,700)   
c.lineTo(800,700)   
c.lineTo(800,600)   
c.lineTo(200,600)   
c.lineTo(200,200)   
c.lineTo(800,200)   
c.lineTo(800,100)   
c.lineTo(100,100)   
l += c
g.layers[g.activeLayer] = l  #### MUST do this for changes to show up

f.save("/tmp/new.sfd")

At first blush I was expecting to do something like
c = layer.createContour()
c.moveTo()
...
c.lineTo()
and then not have to do anything special. At that stage calling f.save() should know about the new contour etc and save. But without setting the g.layers[active] to the layer that contains the contour you will not see it.

Digging into the C code in python.c I see that point, contour etc are all basically abstractions for python use only. When you assign to a layer in the "g.layers[] = l" call, the C function PyFF_LayerArrayIndexAssign() calls PyFF_Glyph_set_a_layer() which uses something like SSFromLayer() to convert the python only data structure (contour or what have you) into a native "c" SplineSet object.

The good news is that with all this mining into python.c I now have some collab sprinkles in there. So when you do "g.layers[] = l" the FontForge in script mode will send updates to the layer off to the server as a collab update message.

The test is quite easy to run. As three consecutive scripts, start the collab server process (collab server remains running, script ends). Next attach to the collab server and update the C glyph, and finally attach to the collab server and grab all its data and save a out.sfd file.

fontforge -script collab-sessionstart.py
fontforge -script collab-sessionjoin-and-change-c.py
fontforge -script collab-sessionjoin-and-save-to-out.sfd.py

The middle script connects to the collab server and makes its changes with the python API and then exits. No Save. To know if the changes made it to the collab server, the last script grabs all the updates etc and builds the "current" font to save into /tmp/out.sfd.

It took a bit of hacking in the python code, but now the little changes to the contour (path) of the C glyph are sent to the server as one would expect.

The python scripts are still out of repo. Since they are interesting in and of themselves I'll likely put them into my fontforge fork as a prestage to having them mainline.

Now to move on to the next thing that needs to be send to the collab server and updated in all clients.

FontForge Design: Two is a party!

FontForge can now allow multiple people to collaborate on designing a font in real time. This is all still rather alpha level code, and in fact the pull request including this code was only pushed over the fence in the last hour ;) If real time collaboration sounds interesting to you, or something in this post seems of interest, then you might like to attend an upcoming Interactivos Workshop which is right after the Libre Graphics Meeting in Madrid (10-27 April).

At interactivos I will be expanding on the current collab code and discussing future directions and possible cross tool network specifications to allow different font tools to talk to each other. Even if you are not convinced that you want anyone else joining your collab session, you might like something like a Web sink watching your work and updating a page on your tablet as you modify the font so you can see in near real time if changes are going to work well on a target device and design or not. Why burden your workflow having to have your local fontforge export to an OTF and slow you down when a server can do all that for you in the background. There is no reason that the processes in the collab session can't all be running on your behalf.


If you are running osx then I have a binary drop for you which bundles all the needed stuff for collab at:
http://fuuko.libferris.com/osx/packages/201303/05_1630/


To start a server select the menu item Collaborate/Start Session... a dialog will show you the IP address the server will run on so you can start fontforge on another computer (+other OS) and connect to that address using Collaborate/Connect to Session...

Things are in very early days and focus has been only on the glyph view. It seems that there is some fluff holding the undo system from working on the metrics view (thus on kerning etc).

The design uses FontForge's undo/redo system to know what the changes are in the font, and since those changes can be serialized to an SFD format that code is reused to send and receive the undo information across a zeromq broadcast server.

The design is inline with the MCT (Merge Enabled Change Tracking) for ODF that is up on the OASIS lists. In FontForge's case, s baseline full SFD snapshot and sequential fragments thereafter describing updates.

If you are compiling from sources then you'll need czmq and zeromq version3 development stuff installed. Then grab master from github and enjoy.
https://github.com/fontforge/fontforge

Redland && Fedora 18

It appears that redland might be busted in some respects in Fedora 18. After a fresh install I recompiled libferris as per usual after an upgrade, only to find that the RDF/Soprano engine for metadata was busted. Digging into this, looking at the soprano sources and with many versions of soprano and sopranocmd I thought maybe the issue was one layer closer to the metal. Switching to rdfproc from redland I noticed that I couldn't create a new on disk RDF/db store!

$ rdfproc myrdf add \
  'http://witme.sf.net/libferris-core/0.1/subj' \
  'http://witme.sf.net/libferris-core/0.1/pred' \
  'http://witme.sf.net/libferris-core/0.1/obj1'

rdfproc: Failed to open hashes storage 'myrdf'

After downloading redland's source rpm, recompiling it and installing the resulting rpm files I could then create RDF/db again. Same command, different result:

$ rdfproc myrdf add \
  'http://witme.sf.net/libferris-core/0.1/subj' \
  'http://witme.sf.net/libferris-core/0.1/pred' \
  'http://witme.sf.net/libferris-core/0.1/obj1'
rdfproc: Added triple to the graph

I haven't verified this on a second Fedora 18 machine yet, but it might well mean that anyone trying to use the Berkeley db backends of redland on F18 would need an update to redland or to do some tinkering.

Cross platform package building..?

Sorry for the chatter post, but if anybody has recommendations for a tool that can build for win, osx, and lin that would be great. The project is an autofools one, mainly coded in 100s of kloc of C++. Builds like a treat on a Fedora machine, can be beaten unto submission to build on osx, and I assume on a suitably tainted windows machine it will gcc into binaries too. At least it has built on those other platforms in the past.

I've had great success with OBS, but that was mainly for Linux packages. It seems OBS can do mingw too, but I've not walked the valley of darkness into building for the more closed platforms on OBS before.

The saucelabs looks pretty cool, but it seems targeted to web code if I am reading it correctly.

The initial plan is to get 24hr rolling packages for all platforms and have feedback as to which day a github commit has broken the package build. It might be nice to have it for each github commit, but I think it would be easy enough to bisect a break given a 24 hour window unless an armada of contributors rushes at the ship.

A separate build issue I've been tinkering with in my mind for a while is grabbing from a github repo and creating android packages. Different code base for this though, mainly some of my n9 apps, as such, preferably for a mixed C++/QML app. But I think for that project I'll wind up taking my chisel and hammer and coming back with a cron job.

KDE, Qt, and the segv

Short story: qt-4.8.4 on Fedora 16 might offer you crashes when clicking the menu in the panel. I had this on my main user account, but not another account. Downgrading to 4.8.3 solved this with the exact same .kde, .local et al in place.

The crashes offered my g_iterator type things in the backtrace. Nothing juicy that I could try to work backward from. Digging around seemed to show theme stuff, pointing at the nvidia drivers (which always comes up in searches), and there was a bug I found along the way mentioning qt 4.8.4, so I reverted to 4.8.3 from a mirror that had the old packages still around.

I decided to update the laptop before a trip on thursday, and this was of course when I discovered this little bundle of joy. I'm not sure which part of my config was causing this, but at least I can now use the laptop with my desktop of choice while away!

Libferris as a KIO Slave

The libferris virtual filesystem can now be exposed as a KIO Slove too. This allows you to use KDE applications to list, read and write the vast number of virtual filesystems that libferris makes available. For those who don't know, ferris is a little project I've been working on over the last decade to make everything a file: XML, db, relational data, web services, and even applications like XWindow, Amarok, pulseaudio, and gstreamer.

The following will create a Berkeley db4 file from the command line and show it to you. To dig into such files with libferris you can just read the file directly. So in this case I just grab the "base" directory in this db4 file with konq. You should see the sample and file2 files in that directory view and be able to load and save those "files" into kwrite. Sorry about the video being a tad jumpy, I have to work out what part of my desktop is causing that :/


Using libferris through the KIO interface from Ben Martin on Vimeo.

The commands as plain text:

$ cd /tmp
$ db45_load -T -t btree foo.db
base/sample
value here
base/file2
contents

$ db_dump -p foo.db
$ konqueror ferris:/tmp/foo.db/base

little tricks I found during the hacking, your listDir() method might call listEntry( e, false ) with a final call using ( e, true ). The last call is impotent with regard to "e" and just a finalizer call. The get() method uses data() to deliver bytes to the KIO user (application). The put() method uses dataReq() in a loop to grab chunks frrom the KIO user. Currently I have lazy methods, for example the put() just grabs everything from the KIO user and then operates on the data once it has everything. Really bad for 4gb files, but for smallish files to get a feel for things it works quite well.

Also, if you are using library init, in my case using gmodule to dynamically load some plugins from libferris itself, you might be in for a world of fun and games. Currently I spawn processes and interact with them from the KIO slave to get around this issue. I imagine for a more efficient implementation being able to tell KDE to load my KIO Slave as an application with a normal full init leading to a main() but haven't looked at that little technicality yet.

I've mostly been tinkering with kioclient, konq, and kwrite on libferris files at the moment. Things are turning out well, though there are still many glitches to this early days integration. This will be released in the next libferris version once I clean it up a bit more.