Sinzui

I spent the past few days reviewing my opportunities to switch Medusa's backend DB and get the indexer handling more content types. The Sutra metadata idea requires a real DB, and while Medusa can use it, to work well, Medusa would really become the filesystem indexer. Another VFS search is needed to provide the features needed for both file and meta data queries. Seth's ideas look good, and they look so close to RDF semantics, that I think RDF should be the default means for both meta and file data to be indexed/imported. I think we need a query object/language suitable for applications and users. I've continued playing with complex search layouts, and I think I've got one for files. Because of the flexibility needed for meta queries, I leaning toward some means of query command line completion. I don't think Seth will agree, but users do need some search now. I'm still not 100% on how to fit My ideas of a meta db into Seth's. I've been reading some code to see how it could work. I don't want to create new tables, rather, I want a set a attributes and relations that fit comfortably within Seth's plans.

I've been tinkering with a new complex search sidebar for Nautilus. I've decided that advanced search cannot really to simplified with a set of controls to help the user select options to make queries. I've nixed the user and owner clauses since Medusa is meant to index the user's home directory. Advanced search will have to be freely typed into a text entry. Now I could simplify the query operators. I could come up with a command line/code completion feature like good terminals and editors. But Seth has grander plans, and I should not do anything on this issue until I really understand what he wants to do.

The metadata database that Marco and I have been discussing (a summary of Sutra is available) shares a lot of ideas that Seth has been exploring. I think it's time for me to sign up as Seth's toady (AKA sub-commander sparky) since his plans are more mature. I still want to deliver search for GNOME 2.6, so I'll need to finish black boxing Medusa's search controls, then decide what parts can be switched to the new DB. In my original plan, I wasn't going to switch the DB until after 2.6, but since one might be ready to go, I might be able to do this sooner. I strongly feel that Medusa and what ever it becomes needs an incremental indexer to discover data changes on the filesystem–I'll lobby to get this one completed for 2.6.

I finally got my thoughts about Marco's metadata problem into words. I can sleep now, but I should get back to modeling search GUIs for Nautilus.

The three fates of Greek mythology governed the thread of life and wove it into a cloth representing a person's actions and relations.  Sutra is a Sanskrit word meaning thread, or connective cord, it shares the same root as suture, and it means a narrative.  The Sutra metadata database provides the means for users to store, organize, retrieve relevant personal data.  The data will change with every use,  strengthening and weakening its relationships to reflect the user's life.

Sutra has the attributes of several kinds of applications including directory servers, PIM apps, customization and state data, personalization service, and search engine.  It broadens the features of applications like Dashboard and Medusa, and offers new facilities to applications like Epiphany, Rhythmbox, GThumb.  It can extend and alter the development of applications like Evolution, or the Desktop.  It does not store the files or resources that the data pertains to, not does it offer utility to a group of users.  Few users will know that Sutra exists, but their applications will use it, and act as a proxy for the user.

You can read the full article if your interested.

Indy and I watch an episode of Doctor Who a few evening ago. It was Sylvester McCoy playing the Doctor in Paradise Towers. She hasn't stopped talking about it so I'll start recording them for her (because she really shouldn't be staying up to 2:00 AM to watch them). I think Sylvester was the best Doctor, and I'm sure she'll really like Ace when she is introduces in a few episodes.

I tried to take a break yesterday, but I was too distracted by Medusa developments. I didn't get to sleep until 3:00 AM, and I got up late to work, and I spent the day reading Medusa code instead of working on my company's problems. I blame Marco. I've got to throttle back or I'll burn out.

I played with OTS (open text summarizer) that Marco and Dom recommended. It is a very clever tool. I plan to use it inside the indexer so that only the most important words are indexed. This will significantly reduce the size of the content DB. The ratio feature will let me assign different ratios to mime-types, because source code should be more verbose to be accurate. This beats the snot out the of stop-list I was contemplating. All I need to do is guarantee that the mime-type parser always delivers UTF-8, and that was the plan.

Coupling OTS in the back and an enhanced GSF in the front, will make mime-type parsers easy to create. I'll extend GSF so I can get metadata, summary, and attributes from office, documents, PDFs, images, and music files for special handling. I'll then get the file data from GSF. I'll need to make some mime-type handlers, but that isn't hard since I'm just extracting text at for this phase, and I hope to borrow the importers from Gnumeric and Abiword.

But Marco put two things in my head. Most of last night was focused on his need for a metadata DB. Medusa is focused on filesystem metadata, not personal data. Epiphany needs a place store bookmarks and history and it must be accessible to all apps. I was planning to make a smart indexer to handle bookmarks and history like a filesystem, but his approach is better. Since I wanted to get all the personal information into Medusa's DB in a future phase, I was going to toss the DB. Well his summary of the problem really helps me focus on where I want to go. In the end, Medusa will be a much smaller tool, and the Metadata DB will become the primary repository and service. I can't stop thinking about it, but I really cannot think about it until I've got Medusa fully working in Nautilus ,and engineered so I so can re-architect Medusa without screwing with Nautilus. I doubt I can start rebuilding the DB, split Medusa into an indexer and storage project, until next year, but Marco wants this started sooner.

I looked at DrWright's activity monitor code today at work. I think GNOME needs an idle monitor. Apps like Medusa and DrWright can connect to it. GNOME should split the screensaver, power management, and system lock and make them clients of the idle monitor.

I've got a wireless home network, a laptop, and a porch, but they are awkward to use togather. When I'm on the porch, I have to aim my NIC at my daughter's window because the signal cannot get through the brick on the exterior of the house and the tile in the bath. I should make an antenna that sends the signal to a tree in the backyard, then bounce the signal to the porch (and spa).

I learned that my former boss is considered too technical by a few of his job prospects. He just a tech savvy business man. I wonder if I ruined him when I put RedHat on his laptop, and set up the Python tutorial.

I watched some old Scooby Doo cartoons with my daughter Indy–Velma is still a hottie after all these years.

I'm getting some nonsense about ignoring pragma when I compile the Medusa's idle code on Solaris. It has to do the Solaris 8's X intrinsics.h I took at quick look at replacing the code idle taken from x-screensaver. Teuf recommended looking at xautolock. It looks good, both simple and portable. GNOME really needs to ditch x-screensaver. It needs an idle lib that screen savers, power management, and system locking can can use. I don't want to rewrite the idle code now, but I may need to, to get Medusa running on a Sun

I've been modeling new search GUIs. I want to move the simple and complex search GUI code from Nautilus to Medusa so I ca make underlying changes to Medusa without Nautilus needing to be rebuilt. Bonobo looks to be the right way to encapsulate Medusa's search and status controls [rm -rf `msearch -u 'search:[file:///]content includes_all_of bonobo flame` ]. I'm not going to try to make the complex search match all of Medusa's capabilities; after an exchange with Seth, I'm convinced only natural language will work, and the ain't gonna happen for GNOME 2.6. Instead, complex search needs to do the kind of queries that Nautilus users need. I think I can subvert the simple search control to accept all Medusa criteria to perform very complex queries. I need a clever analyzer that checks the entry box text for viable clauses, and parse them into a URL instead of assuming that we are doing a file_name contains query.

I was reviewing the Medusa bugs (to avoid doing more GUI work) and decided to make a quick fix for the directory_name clause, which never returns the directory in the result, just the children. I discovered that both of the reported bugs are not in the code, but the documentation. The directory_name clause is supposed to return all files and directories that are beneath the matching directory. A better name for it might be location, parent_directory, or path. So I fixed the erroneous (and sparse) documentation instead. I also noticed that the directory_name clause has the full URL in it so you can test for Documents/art or file:///home. Since the URL's protocol can be queried in regular clauses, there is no need for the document root section of the URL '[file:///,webdav:///home/curtis]'. All the code ignores it, the brackets aren't URL friendly, but the search VFS module requires it. Once I've moved the URL generating controls into Medusa, it'll be safe for me to remove the document root from VFS, and rename the directory_name clause.

I just applied a patch sent 23 months ago to fix a compilation error on Solaris. The compiler warning was obvious, the code looked wrong, and the patch looked right. It took a few minutes to check that it wouldn't compile, well I lie. Medusa doesn't compile on Sun for lots of reasons, but setting those aside, the compiler warnings and patch were clear. I wonder if there is way to pull all patches from open bugs and apply them on an integration server. I shouldn't speak so harshly. I've known that the patch would work for a few months, I just never had a Sun to test it on–thanks Time Life! I couldn't have done without borrowing that Sparc left idle from the layoffs.

The evil idle code reared it's ugly head. Sun hates that too. I loath to reconcile Solaris to the ancient and dubious code pulled from x-screensaver. My candidates to replace the idle code are from GnoTime, DrWright, and xautolock. I'm not particular, I just want to know it will work on a lot of platforms, or better still find something that is already running and use it. X-screensaver is on most systems, so I could use the watch switch to attach to it, but I don't think I would be able to get Medusa's indexer to work without system configuration.

With glib and GTK at it's foundation, and mission of bring a free desktop to all UNIX, I'm still surprised when I read GNOME code that uses nothing but system libs. I made good progress on getting Medusa to compile on Solaris today. I hunted for some time looking for a portable replacement for setenv. The problem was reported in lots of places, and I even found a patch the George Lebl made for glib, but I could find the replacement for sometime. I looked high and low, well the other way round, they didn't make it into glib, they're in libgnome.

I discovered the error, 'attempt to put segment in horiz list twice', that I introduced when I reconnected Medusa to Nautilus is from libart_lgpl. Well I wasn't anywhere near that. I messed around with the GTK, and some signals to display it. One error is definitely tied to to getting the emblems/keywords working again. I wonder if the error in really not in the pixbuf stuff, but in the chain of events displaying it.

A co-worker remarked that the US should agree to take up the metric system in exchange for the UK to stop driving on the left. Nice plan, but some adjustment is needed. The UK isn't exactly metric friendly. The US will use metric if Japan stops driving on the left. With the UK, the US will agree to stop mangling the English language, and start mangling Spanish. If that isn't enough, the US will agree to continue putting down France, but will also continue buying all its great export products. The trouble is Australia and New Zealand. I don't think there is anything the US can do to persuade them to drive on the right side of the road.