Category Archives: web

Face painting and historical Web development

I spent my last two weekends working with ancient web technologies. Anne decided she wanted to update her Happy Faces web to attract more face painting business. I agreed to create a form for customers to inquire about rates and availability. The hosting service only offered Python 2.4,  CGI, and sendmail.

I really did not want to write a script that I knew others had written. I just wanted to make a pretty little form to help Anne. I found one library that almost worked.  I knew I could write a better library, and I would not trust any library without a test suite. I decided to write my own that would manage the request and response form, sending emails, and storing them if sendmail failed. I wrote a simple means to define a schema of what fields are in the form and how to validate them. The test module even provides a test web server that runs the CGIs you place in the root directory. This was a fine meditative exercise, but I would not describe it as fun.

I did have fun creating Anne’s contact form. Defining the schema for her form took minutes, and it was painless to make changes as we discussed what was optional and what the wording should be. The web page is not historical, it is very modern in fact. It uses HTML 5 form inputs and JavaScript to collect the information. The new  form inputs work in HTML 4 browsers like IE and FireFox, but HTML 5 browsers like Chromium and Safari see widgets that require less typing, less thinking about what to input.

I posted my CGI contact library on Launchpad for anyone who finds they need to work with historical technology.

A Practical Guide to Bug Triage

Or: “Why I don’t classify bugs as medium”

The process of triaging issues (bugs, features, and tasks) has one crucial principle: Prioritise the work according to need and certainty.

Work is prioritised because there are not enough engineers to do all the work. Some features will never be completed, some bugs will never be fixed. Triage determines which bugs can and will be fixed, which features can and will be implemented. Need is generally understood, when planning work, but certainty is not, and that often leads to wasted work and unmet expectations.

By need, I mean a measure of severity. What percentage of users does the issue affect, and how severely does it impede them from completing their task.

By certainty, I mean a measure of how certain the engineers are that they can address the issue. Time is also a factor in this measure, the longer an issue takes to address, the more likely that the conditions that were first judged will change.

The act of triage is separating work into groups that are being worked on now, next and last. There can only be as many “now” bugs or features as there are engineers. The number of “next” work is limited to the velocity of the engineers and how infrequently plans change. The bugs that are last will probably never be addressed, the last features may never be started.

The corollary to this rule is that there are a finite number of bugs or features in the first two groups. There cannot be more work in these groups than there are engineers to do for the given period of time; otherwise the engineers, businesses and users are being misinformed about when issues will be addressed.

An Example

Consider there is one engineer and two bugs. He can only work one bug at a time. One bug is more important than the other. The risk is that he may not be able to fix one of the bugs before users are disappointed and abandon the application. He risks disappointing all users if he does not fix either bug because he choose the one with the most need over the one he was certain he could address.

If he does not know how to fix the bug with the most need, or that the fix takes a long time, he is wasting time he could have spent fixing the bug with more certainty. The only way he can address the bug with the most need is to employ a hack to reduce the need, to meet the expectations of some users. The hack is also used to gain time to understand the problem, thus increase certainty.

Only Assign Work that You Are Committing to do in the Near Future

When a work is assigned to an engineer, he is committing to complete the work in the near future. What the “near future” means is different for each project. I suggest 3 releases is the “near future”, because when work is planned, the engineer is thinking about now, next, and last. For some projects this period might be 6 weeks, for others, 6 months.

I prefer to plan for the current release, and the next one. As work is reprioritised, it may be rescheduled to the third release. I do not think it is wise to plan a bug or feature to be completed in the third releases because if it slips to the fourth or fifth released, I doubt the it was correctly prioritized as high.

Any high work that is assigned to a engineer for more than 3 releases was not high. If it were, the work would have been reassigned to someone who could complete it in the scheduled time. Any other work that is assigned for more than 1 release is also misprioritised. You are lying to yourself, and the the project’s users, when you assign work that you are not committing to fixing.

Practical Classifications of Importance

Work is often classified in relative terms. It is better to classify work according to how it are managed to convey when and under what terms the bug will be fixed or a feature will be complete. There are three priorities that work can be classified as:

Critical
The bug dramatically impairs users. Users may lose their data. Users cannot complete crucial tasks. The feature is needed to encourage adoption or prevent abandonment of the project.

Synonyms: required, essential, now, must do

The work is immediately assigned to a engineer. It is his top priority to fix. Team members help the engineer to plan and do the work. The work is released as soon as it is deployable; in the case of a bug, it is released outside of the release schedule.
High
The bug prevents users from completing their tasks. The feature provides new kinds of tasks or new ways of completing tasks.

Synonyms: expected, next, can do, should do

The work is assigned to a engineer to be completed in the next 3 releases. The engineer may choose to do other work if he believes it is within the scope of the high priority work.
Medium
The bug is inconvenience for many users. The feature provides new ways of completing tasks.

Synonyms: preferred

The work is not scheduled, though it is intended to be completed. When the work is assigned, it may also be scheduled, but there is no commitment to complete it for the stated release. The engineer may choose to postpone the work in favour of more important work.
Low
The bug is an inconvenience to users, but it does not prevent them from completing their tasks. The feature is a convenience to users.

Synonyms: optional, last, may do

The engineer may assign the work to himself while working on a high priority work because the high work provides an opportunity to complete the low priority work at less cost. If the low work in any way jeopardises the high priority work, the low work is unassigned. The engineer is thus certain that the work can be fixed quickly and without difficulty. A corollary to this rule is that low work that is assigned to a engineer must be “in progress” or “fixed” states.

The Problem with “Medium”

It might be argued that when the engineer has an opportunity to fix a low or a medium bug, he must choose the medium one. This rules does not define a practical distinction between medium and low. There is no commitment to fix the medium bug; it will not be scheduled for fixing. A engineer chooses to undertake a low bug because he sees an opportunity to fix it while working in the affected code. The engineer is choosing to do unscheduled work because he is certain it does not jeopardise his scheduled work. The engineer might see an opportunity to fix a medium and a low bug at the same time, but that is unlikely.

It can also be argued that ‘critical’ is ‘high’ and that ‘high’ is ‘medium’. True, that is a matter of semantics. The crux of the issue is that there are three practical classifications of work. The words chosen to describe the classifications could use the tofu scale of hard, firm, and soft. People who are unfamiliar with triage will appreciate names that convey the kind of attention the issue will receive.

Some teams with a large number of bugs prefer to keep a pool of medium work from which releases are planned. Items in the pool may be escalated to high if it is perceived that once work is started, there should be a commitment to complete it as scheduled. This work is different from low work because the work makes a substantial improvement to the application, but like low, there is no commitment when the work will be completed. It can be argued that work starts on medium bugs and features because of changes to other priorities, certainties, or the number of users it affects.

Consequences of Misprioritised Work

Stakeholders often use reports that list the prioritised work for a release and for each engineer. When work is misclassified there are two commonly observed consequences: a decreased in certainty, and a decrease in communication.

In the first consequence, the engineer’s effort may be wasted; there are issues that have more need and certainty. Engineers, and other stakeholders, are often tempted to complete the misdirected work after the misclassification is discovered because it is assumed that it is better to always deliver something finished than nothing at all. This is a risky choice, because it jeopardises work in future releases. By working on less important work, the engineer is decreasing the certainty of the more important work.

The second consequence is that the engineer ignores the list and he works on issues according to some other source, such as the opinion of another stakeholder. While the engineer is working on the correct issue, it is unclear to other parties what work is going on and when will it be completed. Users may abandon the project in frustration. Planners cannot coordinate all the stakeholders.

The first consequence is possibly a failure to do re-prioritisation during the triage process, but second consequence is a total failure in the triage process. Why would anyone do triage if the prioritisation will be ignored? How can work be coordinated if the work is unknown to all stakeholders? Why would users trust a project if it does not do what it says it will do?

Work must be reprioritised during the triage process to ensure that engineers are working on the issues with the most need and certainty. Engineers must work from the list or prioritised issues.

Indicators of Misprioritised Work

The rules of practical classification provide tests for misprioritised bugs, features, or tasks.

  • The work is critical, but it is not assigned and targeted for release.
  • The work prioritised as high, but it is not assigned and for a release.
  • The work is high, but have not been worked on in 3 releases.
  • The work is low and unassigned, yet it is targeted for a release.
  • The work is low and assigned, but the engineer is not working on it.
  • The work is considered to be triaged, but it’s priority is not critical, high, or low.
  • An engineer is assigned more work than he can accomplish in 3 releases, and it cannot be reassigned.

Control-Click to select, WTF?

My group is rebuilding one of our websites, and the UI is getting some serious attention. The business and technical groups came to an early decision that we would not be using multi-select listboxes. They alway require special instruction to use, and they are difficult to use for many users. I don’t know how long that MacOS has supported this feature. Windows has had the feature for more than 15 years. GTK has a similar behavior in the treeview. But Why, oh why, are we perpetuating such a bad solution.

We use radio button for mutually exclusive options, and checkboxes for inclusive options. I have no issue with the listbox to restrict the view of a large list, but why change the rules of the exclusive/inclusive operation? The problem is the size of the list, not how the user interacts with it. The listbox could, maybe should, display exclusive items as radio buttons, and inclusive items as checkboxes. The user should know by the presentation of the items in the list how the listbox will behave.

Using GTK’s treeview to implement the behavior doesn’t take a lot of work–just add use a radio button or a checkbox instead of text, and ignore the SELECTION_MULTIPLE property. Putting the checkboxes/radio buttons in a scrollpane is even easier to implement. regardless of the implementation, all listboxes in my GNOME/GTK desktop should behave consistently.

GTK  exclusive and inclusive listbox examples

Until I thought about this matter I had not noticed that Gecko/gtkmozembed is forging a listbox control for GTK. Since the listbox was removed from GTK few years ago, Gecko must be using a its own widget to imitate the missing behavior. Still, this behavior, while consistent with other OSes, it is not consistent within each OS. I’m considering using a div with fixed dimensions and overflow to fix the behavior. I’m somewhat uncomfortable introducing non-standard UI behavior, particularly to a Web browser. But I feel that things are so broken, that it would be negligent of myself and my group not to take some action.

To anyone who has any influence of the Mac/Windows interface, please help end the control-click to select madness.

Replacing the GNOME software map

I'm making a repository and publisher tool that will replace the aging and crippled GNOME software map. Edd Dumbill is working on DOAP that will allow projects to publish their vital information so that it can be aggregated by sites like GNOME and Freshmeat. By registering a public DOAP file/feed with the relevant sites, the project maintainer will only need to update the feed to send updates to all the sites.

After the initial registration with the GNOME DOAP repository (redland), the tool will check for periodic updates from the project's public DOAP file. When an update is found, the tool will use XSLT to regenerate the GNOME directory of apps, as well as the application summary pages. This will give the Web site a consistent presentation for the GNOME apps that should be easier for users to explore. We could consider adding search capabilities to the Web site in the future.

But what to name the tool? It's difficult to stay politically correct when dealing with DOAP: 'Dealer', 'Stash', 'Junkie' the bad allusions go on and on. I like 'Dealer', no DOAP, because its job is to server the project data to the users.

A few notes in between sneezes.

Last week I was struck down my allergies. I started making mistakes converting developer.gnome.org to UTF-8 and XHTML. Nor could I focus in my day job.

I spent a lot of time in Bugzilla reviewing patches. By Sunday morning, we had gone from 1703 to 978 unreviewed patches. I don't think there are many obsolete or incomplete patches in the db now. On Friday I started using jhbuild to list the modules I should patch review, and by Saturday night I started going through my own menus.

I came across this remark while doing the patch review:

OK, perhaps you've proved that short-circuiting that improves performance is possible, but you've done much violence to the Pango code base in the process.
–Owen Taylor 2003-02-28 10:26

Beware of the Monkey God

I brought most of www.gnome.org up to XHTML compliance last weekend. There is some PHP changes to do, but we may just pull PHP from the site all together. The GST and GNOME-Network projects may want to start making plans to switch from PHP to flat HTML.

Last night I was looking at the strange markup the News Summaries on developer.gnome.org. HTML <hr> elements were appearing instead of XHTML <hr /> elements. Unable to locate the source of the phantom elements I went to bed. I awoke 60 minutes later at 12:30 AM with the Monkey God whispering answers in my ear. Beware the Monkey God's code; it is a zen-like language that is often undecipherable the morning after. This morning I had indeed fixed the <hr> problem by changing the xsltproc HTML option to XML plus a few rule refinements, but created wacky empty anchors in the process. The News Summaries were valid, but would not display correctly in the browser. I couldn't figure what I was doing last night, but I was vaguely aware that the Monkey God made no mention in the technical requirements that the pages would actually work.

Thanks to jamesh who provide a patch to fix the problem. No doubt he can decipher the Monkey God's code.

Converting GNOME Web content

I've been converting the encoding and markup of the GNOME Web sites in an effort to avoid doing some real work. We are moving the GNOME sites to UTF-8 and XHTML to catch up with standards. The Foundation and GUADEC sites are complete. The main portion of www.gnome.org is complete, the PHP portions of the w.g.o and the projects are my next tasks. If you maintain the pages for a project on w.g.o and you don't want me touching your content, send me an email. I'm fixing the developer Web site last because it has the most content, and the content is in a very unpredictable state.

We have a number of ways to generate Web content, some special to the Web, other common like DocBook and gtk-doc. We need to regenerate some of the content as XHTML. Older content that that isn't worth regenerating can be updated by hand, or we can label it as deprecated (or even historical).

Converting the character encoding isn't too difficult, but there are some inconveniences. Source code is stored in CVS in the encoding of the last developer or tool; some code in GNOME CVS is not UTF-8 and will not display right on cvs.g.o. This is a brief sample of invalid UTF-8:

./gnome-vfs/libgnomevfs/gnome-vfs-job-queue.c ./esound/esdplay.c ./libgnome/libgnome/gnome-config.h ./libgnomecanvas/libgnomecanvas/gnome-canvas-text.c ./libbonoboui/samples/compound-doc/container/container-io.h ./libgnomeui/libgnomeui/gnome-scores.c ./gnome-themes/gtk-themes/Mist/src/mist-style.c ./gnome-applets/battstat/acpi-linux.c
./gtkhtml2/libgtkhtml/gtkhtml.h 

GNOME Web sites needs more attention

I set the month of December aside for Web hacking, but I did nothing. I took every opportunity I had and played with Medusa, and planned my Storage changes for 2004. So I'm making my negligence public. There are dozens of quick fixes to make. The store application must be built over the next few days. The reorganization of the site and general cleaning simple must be done in January. I really don't have the time to do all of this, but there are people willing to help if I make the time to do some organization of it.

GNOME needs a primer for newbies

Murray, I think you are partially right about Anjuta promoting Glade code. Without a comprehensive primer about the GNOME development process that provides best practices and a checklist, we will continue to see such mistakes. I lurked on the gnome lists for years, and learned a lot about good and bad practices. When It came time for me to contribute code, I was very frustrated trying to locate good information. In the case of glade, there are dozens of tutorials available, most for Python, and most use glade-generate code. I knew this was wrong, but there is not a good document about what is right. In the case of my recently created msearch-gui, I wanted to reuse code from gnome-search-tool, but I found glade-generated code. I was savvy enough to realize that just because I found glade-generated code in the base GNOME packages, that does not mean I should use it.

There are lots of articles and tutorials about GNOME development, many are wrong, and many are on the developer.gnome.org site. We are reorganizing developer.gnome.org, and we will take the opportunity to remove old documents or label them obsolete. The Web site needs to do a better job at integrating the information into a cohesive set of documents that clearly define how to develop without developers needing to hunt for the answers to their questions.

I would like to create an outline of the GNOME development process, and commit it as a project. It would be good to capture the experience of the seasoned developers, and update project as the development process evolves. Providing a checklist will really help developers avoid making revisions to get their code in shape for a public release. Tools like Anjuta and Scaffold should reinforce the GNOME development process by providing features that reinforce the process, and make it difficult to deviate from the best path.

GNOME contact page

I've scraped a lot of addresses, lists, and channels off the gnome.org sites. Too much for the contact page. I need to better define it's scope to limit the content so I can settle on how best to present it. We may want to break the contact page into one for users and another for developers. I think we should also include http://www.gnomesupport.org/ since it has a lot of feature users need that are not provided by gnome.org.