One of those mornings

6am. Our eldest child woke me from my slumbers with "Daaaad, the toilet's overflowed!". Turns out it had thoroughly clogged itself late last night and then a leaky tank valve had allowed water to gradually drip into the bowl overnight. Ugg. However, the rather questionable design of many American low-flush toilets is not the point of this post. In the process of clearing the clog I realized that this was where an education comes in useful. I asked her to flush another toilet in the house which created a partial vacuum in the sewer line and helped my efforts with the plunger. It worked wonderfully! A discussion of the physics involved ensued, but the best bit was seeing my daughter continue to believe that her Dad could fix anything.

Which letters are never doubled in English words?

There is now a handy website for searching an English dictionary using regular expressions. It appears that the only letters that are never doubled are 'j', 'q', and 'x'. Mind you, you have to go to Uzbekistan to get their unit of currency which is named a "tyyn".

Perhaps I should make up a word such as "sajjaqquixx" to fix the missing ones. I'm not sure what it means yet, and neither is Google. Suggestions welcome.

Happy Christmas,

~Matt

Testing, testing

This log post should also appear on my Facebook page, along with a picture of our house as it was in 1948.

Christmas book ideas

I've recently read two different graphic novels that I've been very impressed by. I recommend both of them for smart teens and older.

Logicomix, Apostolos Doxiadis and Christos Papadimitriou

The story of Bertrand Russell and his mathematical (and other) passions. An entertaining story with enough of the history of logic to make me think. Beautiful art of Athens too.

Grandville, Bryan Talbot

A well-written steampunk detective story using animals. But it's more than that, lots of references to all sorts of interesting things and people. The art is captivating and left me wanting more of it.

BitBake and reusing existing ipkg files

I've spent a few days helping a customer with BitBake, a python-based build tool for embedded systems. One of the distributions that uses BitBake is OpenEmbedded, a "build-every-single-thing-from-source" Linux distro.

The customer had a number of existing ipkg packages and wanted to add them to the BitBake'd image. That turns out to be an undocumented use case, so here's the information to save other people the time we spent.

For a target architecture of armv5te, the recipe file for a package with an ipkg file named mypackage-1.2.0.3-1_armv5te.ipk looks like the code below.

DESCRIPTION = "My Package"
DEPENDS = ""
LICENSE = "GPL"

PN = "mypackage"
PV = "1.2.0.3"
PR = "1"
PACKAGES = "${PN}"

# Just fetch and place the ipkg in the correct place to build the rootfs.

# As used in openembedded/classes/image.bbclass via rootfs_ipk.bbclass
PACKAGE_INSTALL_append = ${DEPLOY_DIR_IPK}/armv5te/${P}_armv5te.ipk

SRC_URI = "\
  http://ipkg-repository.example.com/subdir/${P}_armv5te.ipk \
  "

do_unpack() {
  echo "Null unpack method"
}

do_patch() {
  echo "Null patch method"
}

do_configure() {
  echo "Null configure method"
}

do_compile() {
  echo "Null compile method"
}

do_stage() {
  echo "Null stage method"
}

do_install() {
  oenote Creating directory ${WORKDIR}/install/${PN}
  # A directory is needed for each member of PACKAGES
  mkdir -p ${WORKDIR}/install/${PN}
}

do_package() {
  echo "Null package method"
}

do_package_stage() {
  echo "Null package_stage method"
}

do_distribute_sources() {
  echo "Null distribute_sources method"
}

do_package_write() {
  oenote Copying the downloaded ipkg package from ${DL_DIR} to\
 ${DEPLOY_DIR_IPK}/armv5te
  install ${DL_DIR}/${P}_armv5te.ipk ${DEPLOY_DIR_IPK}/armv5te
}

Version: BitBake 1.8.3

It doesn't look like much by itself, but the work behind it was, er, significant. BitBake and OpenEmbedded have a good number of layers, moderate documentation and great power.


Other tips:

Don't execute BitBake in a subdirectory or it will rebuild the world in a new tmp directory there.

OpenEmbedded apparently doesn't check the return code from the ipkg install commands, so if your files aren't appearing in your image (rootfs) directory, then check their dependencies or run the explicit ipkg command that can be found in the relevant run* command file.

If ipkg asks for user input about overwriting a configuration file, then the OpenEmbedded do_rootfs() function will hang and you will need to kill both the ipkg and bitbake commands manually.

See http://bec-systems.com/web/content/view/79/9/ for more information about adding new packages to images.

There's no such thing as a unique name

Reading an old post by James Clark about Thai personal names, I wondered how many different names I am addressed by. Surprisingly, I came up with eight.

John Matthew Simon Doar (birth certificate)
Matthew Doar (school name, birth family usage)
Matt Doar (work name)
Matthew Baginski Doar (family name after marriage)
Matthew B. Doar (shortened family name used for publications)
Matthew Baginski (used by systems that can't handle two words in a family name)
J. Matthew Doar (occasional form letters)

and strangest of all is the U.S. Immigration Service who refer to me as "John M. Doar" and once forced me to make up a signature for it on the spot! All I really want these days is to be unique enough for Google.

Handling large numbers of JIRA projects

Once you have a large number of projects in JIRA creating an issue can become a bit tedious, scrolling through the long list of projects to choose just the right one. JIRA (Enterprise edition) already provides Project Categories that you can group projects into, so without further ado, here's a JSP hack to reduce the size of that list of projects by letting you select a project category first.

Installation

The modified file for JIRA 3.13 is createissue-start.jsp and replaces the file of the same name in your JIRA instance. There should be no need to restart JIRA. The usual disclaimers apply, drink responsibly, don't drive and derive etc.

My first build tool question

gnu.misc.discuss

The post above from me in August 1994 asking for help with gnumake reminds me that I never did learn how to use a build tool in college. Fifteen years later, I still don't know the answer to the question that I asked, but I do know that make is still not my favourite build tool.

Still, at least the books about Make have improved with John Graham-Cumming's GNU Make Unleashed and Robert Mecklenburg's Managing Projects with GNU Make, Third Edition. That "Third Edition" part is important.

~Matt

Should you go to AtlasCamp? Hint: yes!

This post is a friendly sales job for AtlasCamp, the Atlassian developers get-together that's happening October 21-23 in Half Moon Bay, California.

Cons

- Costs a bit of money this year ($200 instead of free)
- er ...
- that's it!

Pros

+ If your company depends on JIRA or Confluence and you've changed them somehow, this is the best place I've found for technical questions
+ Friendly people who write useful code at this conference, not just vapid schmoozing
+ Good presentations about what's changing in the products you depend on
+ Plenty of opportunities to tell the core Atlassian developers and top management exactly what you think of their products

See you there.

~Matt

She found it 13 years later

Some stuff from old web pages when "Lizi", aka Elizabeth Shardon Baginski Doar, was first born. And indeed she did find them, all by herself, just the other day. I'm not sure she exactly loved it, but it was certainly a surprise!

---

I was born October 14th 1996 and am much loved by my parents. I like not sitting still and asking "why?". Below is an excerpt from my first email.

>fuukyiki7z x z>8jlj .l[
>'00 uyjkklglylewl;ghju7j 

An interesting graph of contraction times during our daughter's birth. I think she'll love this page one day.

Does this bug tracker make me look whiny?

I submitted a bug about JIRA to Atlassian recently and wondered who reported most of the bugs in their very public database. No surprise, about half of the ones in the last year are from Atlassian employees.

Over the last three years, it's a tie between Neal Applebaum and Sergiy Lizenko (ILS-Ukraine) who each submitted 81 issues, and myself trailing in third place with 39 issues.

~Matt

AtlasCamp 2009

AtlasCamp is back, and it's moved to Half Moon Bay. It also costs something this year, but after the great time I had last year I'm looking forward to it.

Python Unicode oddness

If you have a Python string with an invalid character for the current character set, then other characters may get removed unexpectedly. For instance:

>>> a = 'FOO\xe0BAR'
>>> print '%r' % a
'FOO\xe0BAR'

>>> print '%r' % unicode(a, 'utf8')
Traceback (most recent call last):
 File "", line 1, in 
 File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/encodings/utf_8.py", line 16, in decode
   return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 3-5: invalid data

Fair enough, \xe0 isn't a valid utf8 character. But if I tell the decoder to just ignore any characters it doesn't understand, it also eats the "B" and "A" characters!

>>> print '%r' % unicode(a, 'utf8', 'replace')
u'FOO\ufffdR'
>>> print '%r' % unicode(a, 'utf8', 'ignore')
u'FOOR'

That was unexpected. One day I'm sure someone will explain why it happens to me.

Making it easier to maintain JIRA workflows

This is one for JIRA admins.

I've had to track down a few problems recently that were related to fields not appearing on screens, especially when issues were changing their status. The cause turned out to be customized Transition Screens. A Transition Screen is the screen that is shown while you are moving from one status to another, and it's easy to forget which screen goes with each transition.

Sure, you can click on each transition and look for which screen is being used, or you can look at the list of screens and see which transitions used each screen. However, as the number of transitions and screens begins to grow, workflows become harder to maintain. The precision hack shown here helps with this. Click on the image to enlarge and look at the "Transitions" column. Each line has had the text "with the screen "added.




When you view any JIRA workflow with the "View Workflow Steps" screen, the transitions between steps (statuses) now have their screen information shown as well. At a glance, you can now look at a single workflow and see which of your transitions have those troublesome custom transition screens.

The file to change is

atlassian-jira/secure/admin/views/workflow/viewworkflowsteps.jsp

and the modified version can be found here. This was tested with JIRA 3.13.2 and is also a good example of what you can do with JSP if you have to.

Remedy - free the attachments!

I've inspected a lot of issue tracking databases in the last few months - JIRA, Bugzilla, TeamTrack, ClearQuest, Trac, GNATS and Remedy¹ - but the one I want to consider in this post is Remedy (a.k.a. the "Remedy Action Request System", part of BMC Remedy Service Management from BMC).

Remedy stores attached files in the database and compresses them to save space. That's fine, and they provide a compiled API library to talk to the running server application and extract and decompress the files. But if you're trying to get the data straight from the database, you're out of luck. Searching suggests that the file compression is some proprietary form of zlib but that's it. No source, no algorithm, nothing!

If you're providing an API to extract the data, why not show the customers the source? I suppose they might want to hide changes in the compression algorithm from users, but I can handle seeing that. I'm all for using carefully constructed APIs but not at the cost of the data being locked into one database.

¹ I know, I know, you're thinking "wow, those toolsmiths sure do live life in the large!"

I found a bolt-hole

I finally rented an office near home. The main things I needed were:

* an internet connection
* reasonable airconditioning and heating
* peace and quiet

The last item is why working at home isn't always as easy as one might imagine. And it's not just the kids!

~Matt

Names for Bug States

There is an interesting discussion going on over in a Bugzilla group about changing the default workflow (Open, Fixed, Closed etc) that comes with Bugzilla. My thoughts are that whatever they choose, everyone will want to change it to match their own thoughts on what states a bug should have.

What is really needed are some state names to guide users on how to use a workflow. So in the spirit of the day, I propose doing away with those vague and language-specific words such as "Open" and "Closed" and using integers instead. The first state that a bug starts in would be 0 (or perhaps 1 for non-engineering groups), then the next state should be 1, and then 2, and so on. Next, remove all transitions between states except to the next state and the possibly the previous state.

We now have a simple measure of the state of a bug, acceptable to all!

Tracking Those Personal Issues

Geek Hero Comic

I suspect that some people use bug trackers for even odder things. JIRA has a personal license for just those sort of, well, personal, things.

I'm off to the summit!

I went to the San Jose Atlassian User Group yesterday in the San Jose Museum of Art (great place for a meeting). And I won the drawing for a ticket to the Atlassian Summit!

Hope to see you there.

Neat solution for CSS footers

When navigating a website, it's a bit disconcerting to see the text at the bottom of the page moving up and down the screen as the amount of text on each screen changes.

There's a neat fix using CSS over at http://www.cssstickyfooter.com. It apparently works on every browser I could think of.

~Matt