Making Bugzilla Read-only

I finally got this working after a few frustrating attempts. So, what not to do first.

Change database-level user permissions

I actually got this working once but it took three or four iterations since Bugzilla needs to be able to write to some tables to let you log in. Also, once you have messed with the permissions, I find restoring them to the prior state to be fiddly. This approach is really at the wrong level.


The editbugs group

There's a system level group in Bugzilla named "editbugs" that is apparently related to allowing a user to edit bugs. However removing a user from the group does nothing that I could see.

What Worked For Me

1. For each product, check the Closed checkbox so that the product does not appear in the list of products where bugs can be created. This effectively stops new bugs from being created, though perhaps using the right URL might still work.

2. Define a new group named something like "CanStillEdit" that can be used for bugs and add it to all products. Don't add anyone to the new group yet.

3. For each product, edit the group memberships at the end of the configuration page and set the "Can Edit" checkbox for just the CanStillEdit group. Leave all of the other groups unchanged. Now only members of the CanStillEdit group can edit bugs. It took a few passes through the documentation about this feature to see what they meant.

If all goes well then trying to save a change to a bug will produce a big red box with an error message in it. Usually the error message is helpful ("you do not have permission to do that"), but sometimes it says things like "you tried to update X from to 123" which is confusing but harmless. Search works just as before.

Other details:

1. The other settings for the groups I was working with were Shown/NA.
2. Administrators can also still edit issues.
3. All this is for Bugzilla 2.x - I would hope that there are better ways to do this in the latest versions.

Big Red Button as a Psych Test

In a recent post Raymond Chen talks about Big Red Switches and various people weigh in with recollections about pressing such things. Well, I have in my possession a Big Red Button that I found at the excellent Triangle Machinery & Tool Co in San Jose. When I had a desk, I used to have it mounted with a thick cable running from it to under my desk. It wasn't connected to anything but it told me lots about the people around me.

Most people would ask "what's that for?" or "is it connected?", but occasionally there would be someone who would press the button and then ask the same questions. Those are the people you should never let into a server room!

Emacs, OS X and the EDITOR variable

This took a bit of fiddling to get right, so I offer it up to the Great Google Index with the hope that it may save someone else some time. The EDITOR environment variable is what is used by Unix applications (such as Subversion and git) when they want to start an editor for input. Starting emacs from the command line in OS X uses the open command.

To get this working with emacs and OS X Leopard I had to make the file /usr/bin/emacs contain:

#!/bin/sh
open -nWa /Applications/Emacs.app/Contents/MacOS/Emacs "$@"

This opens a file using a new instance of emacs and waits until emacs exits.

Three Hard Lessons in Consulting

1. Don't start the project until everything is ready. If a machine needs more memory to run the next version of an application, wait for the memory.

2. Stop working after the agreed period of time. It's better to disappoint a client and renegoiate than to surprise them with unexpected billable hours.

3. Windows infrastructure has a price. Locked directories, having to use Notepad, Wordpad or install Emacs, debugging complex service configurations: all of these things add up to about 25% more time taken during major tasks such as upgrades.

I'm officially and practically platform-independent in the services I provide. Linux, Unix, Windows, OS X, I do them all. But from now on I'm going to start charging different rates for Windows environments. Not a lot, just enough to offset the extra time and effort that they require.

My Dash Express

Funnily enough, I'm not much a tech fan-boy. Generally what happens is that our friends are first adopters, and then we use their cast offs a few years later. I wrote a book on a discarded laptop and I watch films on a second-hand screen, while sitting on a pre-loved sofa. It's not just that I'm cheap, it's also a green thing.

So what on earth persuaded me to spend $400 last month on a new GPS, the Dash Express? That's just what my dearest wife wanted to know as well. Now, she has no use for such navigation devices having spent most of her life in the Bay Area. But now that I travel to different clients for work I was getting lost more often than seemed sensible, even for my relaxed approach to navigation.

The attractions of the Dash over other GPS systems for me are:

- GPS, cellphone and wifi antennae are all used. The GPS is for positioning, the cellphone for traffic information and internet connections while on the road, and wifi for updates at home
- I can right click on an address in a web page and send it to my car. No more typing on tiny screens. This is great.
- I can search while I'm on the road. For example, when I get near my destination I realize I have to park somewhere, so I can search for parking right then.

The downsides are:

- $10/month subscription fee, though the first three months are free
- Routing is slow and favors highways
- I had to screw a bracket onto my dash to secure it since CA doesn't allow anything to be mounted on the windscreen.


So far I've been happy with it, though I'm expecting great things from the first few software updates later this year. It has certainly saved me from some major blunders since I've been using it. And my wife refers to the Dash as my mistress!

Atlassian User Group, Palo Alto

Well, I enjoyed the Atlassian User Group yesterday at Stanford. I'd forgotten just how complex getting a parking spot can be there, but I made it with only a few surprised comments from my new mistress. I gave a summary of my JIRA MultiSite post, but the things that I enjoyed most were:

- seeing what's coming up in JIRA 4.0 (nice work from Nick Menere and others)

- asking Mike (CEO) both technical and business questions and seeing him handle both well

- meeting lots of smart people using JIRA, with an eye to new clients of course.

The sponsors Stanford and WANdisco provided a good spread of food, which always helps such things. I'd recommend any JIRA administrators in the Bay Area come to the next one.

More bugs than revisions

(Debian bug count from http://master.debian.org/~ajt)

Heard as an aside: "that project's got more bugs than revisions!"

Which made me think what kind of project might reasonably expect that statement to be true. The space shuttle software as a whole, perhaps? Lots of testing implies lots of bugs, and scrupulous code reviews means fewer commits.

At the other end of things, many small projects might get a few hundred revisions and then a few hundred bugs over time. Of course, most bugs will end up closed in one way or another.

Anybody able to point to a real project with more bugs than revisions?

Just costs you double

Me: "just" costs you double.
Them: Huh?
Me: Every time you use "just" to describe a feature or a process it tells me you've made a gross assumption about what I'll need to do ...

That comment on a post titled All I Need Is A Programmer made me nod in agreement. "Just" is a warning sign in conversations for me, like "always", "never", and the dreaded "trust me".

All those phrases make me think of exceptions. "Always? Well, there is that edge case ...". "Just? Why 'just'? What about ..." and I begin to wonder whether the person really knows what they're talking about.

Just say 'no' to 'just'. (You knew that was coming, right?)

The Mac Mini as a Laptop and VMWare Fusion

I needed a new laptop because the one I had borrowed had disk errors in the Windows registry, as disks do after too long running Windows. Since I find laptop screens, keyboards and mice uniformly awkward, I bought a laptop without the keyboard, mouse or screen - a Mac Mini. (Frys, US$750)

The upsides, apart from being less than half the price of a decent Mac laptop, are that it really is compact and there are some decent virtualization choices (Bootcamp, Parallels and Fusion). I went with VMWare Fusion and overall, it behaves as you might wish. Easy to install, cheap at $100 and runs most VMWare images.

The downsides of not buying a laptop are that power outages become more significant without a battery, and perhaps the disk drive isn't mounted in a suitable manner for carrying the box in a bag every day? Swapping from VM to VM, or VM to host OS sent the swap rate soaring, so I installed an extra 1GB of RAM to help with that. While installing the extra memory, I took a good look at the hard disk and it seems to be mounted just as it would be in a laptop, so I'm hoping that the disk will survive being moved every day. If anyones else has experience with mac mini hard disk lifetimes, I'd be interested to hear from you.

And so today, I had another "virtualization is great" moment. The server that I had traveled an hour to work on was busy having its motherboard replaced. Since I actually wanted to work on was the OS configuration, so I booted the Windows server image with Fusion on my little Mac Mini, did the necessary work, and moved right along. Very neat.

The February issue of Python Magazine is out and contains my article "Using Python and SOAP to create a CLI for JIRA" about the Python CLI that I wrote for JIRA a while ago. The article's summary reads:

Many web applications include an API that lets you interact with them from the command line as well as with a browser. In this article, Matthew shows how to build a command line interface for JIRA, a well-known issue tracking system, using Python and SOAP. JIRA is a Java application, but using SOAP allows you access to many of its features using just Python.

One hint that I wish I had remembered to add is that when you have a redirect to your JIRA server, for instance when http://jira.mycompany.com is redirected by Apache or IIS to http://jira.mycompany.com:8080, you may see your login mysteriously fail. The answer is that you have to use the redirected URL with the JIRA CLI. You can find out what the redirected URL is by running the CLI with the argument -v10 to increase the logging verbosity and look at the line that starts with "Host". This example shows that the port to use is 8080.

*** Outgoing HTTP headers **********************************************
POST /rpc/soap/jirasoapservice-v2 HTTP/1.0
Host: localhost:8080

Finally, a use for those JIRA user properties

One of the most useful JIRA plugins I've found is the JIRA Toolkit, described as "a bunch of neat custom fields Atlassian have developed for their own use". As an aside, if they're so neat and useful then why aren't they in the core product?

One of the more recently developed fields is the View User Property custom field, which is currently only documented in the issue that last link refers to. This handy little field allows you to display properties that you previously added to a user, as a read-only field in each issue.

For example, add a property named "Company" to some of your users in JIRA, then install the JIRA toolkit and create an instance of the View User Property field. Now configure it with "reporter:Company". Add the new field to some screens and the value of the reporter's Company field will show up in the issue. This also works with "assignee:Company". You can even get a user name with "My Custom Field:Company". This will use the user name found in the custom field named "My Custom Field". Just use the JIRA custom field name, including any spaces.

I've just used this to associate a company name with every customer user, and I'm sure there are a number of other pieces of per-user information that could be displayed with this field. One missing piece for this field is the ability to search for issues with particular values.

Evaluating JIRA Multisite

Given the number of organizations already using JIRA across a WAN, there is plenty of interest in finding ways to improve the experience. I've had a few clients suggest using distributed databases, changing HTTP caching behaviour or not using HTTPS. None of these are really great fixes, and are complicated by the fact that JIRA keeps much of its data in a local Lucene index outside the database for performance reasons.

So when WANdisco announced a beta of JIRA Multisite last November in partnership with Atlassian, I was interested to see what it would do. It's billed as a high availability solution and in doing that it gives you local JIRA servers with all your data nicely synchronized. There is another approach that was announced at about the same time, the JIRA clustering solution Scarlet. I haven't evaluated Scarlet yet but it appears to have a single point of failure by default.

I contacted WANdisco to ask for an evaluation copy and they were happy to help. They have an existing replication tool for CVS and Subversion that they've connected to JIRA. You need their tool and their instance of JIRA. As an aside, though they are keeping up with each release of JIRA, I'd rather have instructions about how to modify my existing instance of JIRA to work with their tool, but I'll take what I can get for now.

To provide high availability you have to have 3 or more instances of JIRA, but since I was mainly interested in how each sites' performance changed, I just set up two instances of JIRA, one in San Jose, CA and the other in Bangalore, India. The connection between the two sites is a clogged T1 at best and the team in India often have sluggish response times from JIRA.

Setup Experience

WANdisco wanted to set the tool up, but I did it myself in an hour for the two nodes. Instructions were beta quality, but not bad. After that piece of stubbornness, their tech founder worked out what I had done wrong in about an hour, and then together we had it all working in another hour. Three hours from scratch is pretty good as these things go.

Testing

I modified a bug in San Jose and watched the change appear in India a second or two later. Then I modified a bug in India and saw the change locally in about the same time. Just as expected. Then we stopped one of the JIRA servers, made some changes, waited a bit, restarted the server and saw the changes all get synchronized. Other users updated issues over the next month and the changes appeared just as expected. The big win was that the users in India saw their local response improve dramatically. The underlying WANdisco replication tool was rock solid for the month's evaluation.


Restrictions

The version I tested didn't synchronize attachments, but that has been added since then. You do have to use the same OS (and database I believe) for all the instances of JIRA. This was not a problem for me, but if you have a Windows server in one location and Linux in another, it won't work.

I didn't try https, but I did set up LDAP authentication and that worked as expected

I'm pretty sure that if I wanted to go back to one instance of JIRA I could have exported the data and then reimported it into a non-multisite instance of JIRA.

Cost

Pricing is public and is US $7500 per instance of JIRA, which is about 50% more than the current Enterprise license cost. This seems about right given the cost of the tool and the target customers. Support comes from WANdisco and JIRA, in that order.

Summary

JIRA Multisite is still in its early stages, but it is very promising. It worked well for me with little effort, and provides good value for the price.

Perspective of a 3-year old

A few choice observations from my youngest son in the past few months. I'm posting them as reminders that not everyone thinks as we expect them to.

During the minor earthquake in San Jose, we all went outside. Afterwards he kept running outside then back inside again. When asked why, he said he was "looking for more earthquakes." (I guess we found one outside the first time?)

We came across a dried-up deer carcass, and I was teaching him to use a stick instead of his hands to touch roadkill. This lead to a discussion about death and whether the deer would come back to life. I gently explained that the deer was done with its body now. He looked thoughtful for a moment, and then drove the stick through the carcass, exclaiming "you stay dead then!"

When asked during the Christmas Pageant rehearsal what kinds of animals were in the manager with Jesus, he piped up with "pterodactyls!". To his disappointment, he is going to be a lamb instead.

Choosing Project Names

The discussion What' in a Project Name? over at Coding Horror reminded me of section 3.6.1 of my book Practical Development Environments:

Project names are usually chosen by engineering groups, with one name for each significantly different version of the products that they are working on. There should be no need to change a project's name once it has been chosen. Product names, on the other hand, are the names that customers see, and these names are usually chosen to help a product sell or to become popular. Product names can change at the whim of a market research poll or a new VP of Sales.

Some general guidelines for choosing names for projects are:

Keep it short

Since project names may appear in filenames or source code, shorter project names are preferable; four to six characters is common. Longer names will only be abbreviated anyway, and usually in two different ways.

Use distinctive sounds

Project names should sound different from each other when spoken aloud by people whose native language is not the one used by the rest of the group. Even if everyone speaks English, having two projects named "ctest" and "seebest" is too close for comfort.

Use low-frequency letters

It's much easier to be confident that all references to a project name can be found if the name contains characters that are less common in the local language. This is a good argument for choosing project names that use unusual characters, such as the letters q and z for English.

Apocryphal aside: a few years ago there was a project named IDS that apparently had a function named IDSConnect. Then the project was renamed DIS and all its functions were renamed accordingly, which led to their function for creating connections being renamed to DISConnect. The letters d, i, and s are too common in English to simply reuse them in
such an anagram.

Make it unmarketable

Sometimes a project name will be reused as a product name, but not if it is already trademarked, or if you make it odd or crude enough! Project names don't have to have a theme, though that can be fun. They don't even have to be meaningful, just memorable with an obvious way of pronouncing the word. You can choose a number of suitable names once and then let people decide which one they want to use next. Names of stars, types of sushi, rare diseases, and characters from comic books are some ideas to start with for project names.

Unexepected side effects in shell scripts

Shell scripts that fail to preserve the expected environment are a waste of everyone's time. For instance, you call a function which changes the current directory to somewhere else for its own purposes but fails to change back on all exit paths from the function. Trying to remember which directory you might be in later on becomes combinatorially difficult as more conditional statements are added over time. To make it more concrete:

BAD:

function MyFunction() {
   cd some_random_directory
   # Do something useful
}

GOOD:

function MyFunction() {
   # Change to the necessary directory
   pushd some_random_directory

   # Do something useful

   # Now pop the directory off the stack    
   cd -
}

If you have conditional statements, then make sure you catch them too:

BAD:

function MyFunction() {
   # Change to the necessary directory
   pushd some_random_directory

   # Do something useful and test something
   if [ "${test}" == "value" ]; then
       return # Oops, this function left us somewhere unexpected
   fi

   # Now pop the directory off the stack    
   cd -
}

GOOD:

function MyFunction() {
   # Change to the necessary directory
   pushd some_random_directory

   # Do something useful and test something
   if [ "${test}" == "value" ]; then
       # Don't forget to pop the directory off the stack    
       cd -
       return
   fi

   # Now pop the directory off the stack    
   cd -
}

Basically, preserve an expected and known state between function calls, just like any modern other programming language.

Why tags are rarely used as you would expect

As a recent introduction to version control explains, a tag is a way to record a snapshot of the state of some files at a moment in time. When I ask clients why they want to tag their source code, one reason is always "so we can reproduce a build later on". And we generally nod our heads and move on.

The funny thing is that I've never actually seen a tag used for that purpose. Tags do get used to decide whether a particular bug fix was in a certain build, or as references for released builds. But as a developer if I need an earlier build, it's probably to debug a customer problem, which means that the software was already released. Since my build process will have carefully preserved any build artifacts such as debug symbol files, I already have the files that I need.

The other observation is that build systems are designed to be able to use different branches. So if I really needed to recreate a build, I could always branch from the tag and then build using the head of the new branch. Of course, this assumes that nothing else in the environment has changed.

In summary, tags are necessary, but maybe it's not for the reasons that we usually assume.

Thoughts on Making Things

It's time to get some thoughts out of my head about making things. Physical things, software things, relationship things. You know, things. That's what engineers do after all.


Nothing is Monolithic

Nothing created by people is a monolith. It's always made up of smaller things. Something may seem greater than the sum of its parts, but only by how we treat it. Really, it is just a number of things put together.
For instance, a beautiful painting may well move me greatly, but at one level it is still flecks of paint on canvas.

Everything is a Hack

"Hack" in the sense of changing a thing to work around a problem. Nothing created by humans springs fully formed into existence; everything is an extension of what came before. Whatever you see around you has taken multiple versions to become what it is now.

Everything looks different when you think about how it was made

This was an epiphany for me at eighteen, when I began to look around me and ask "how was that thing made?". Once you consider how a thing could have been produced, you understand that thing in a different way. And just as children are no less marvelous after you've read a book on reproductive science, software and hardware is still amazing even when you're the one creating it.

Python + bash + ssh = too many layers!

I was reminded by a recent project that many (most?) hardware failures occur at the connections between things. That's why percussive maintenance gets results. I was writing a tool to execute lots of commands remotely on multiple machines, with the different commands being synchronized from one machine. I chose to write the tool in python, and that made creating it nice and easy.

But the shell commands to be executed had to be communicated to each machine. Now I could have written all the commands and their arguments (some of which contained the dreaded spaces) into separate files, copied them over with scp and executed them remotely. Except that there was a fair amount of analyzing of results and deciding what to run next based on the results of the analysis. This is generally tedious to do with bash and can be hard to maintain.

So I decided to keep the control logic in the python tool and use ssh to send each command separately to the remote machines. Which works, but the extra headache of getting all the spaces escaped and quoted added a major amount of maintenance pain to the tool. I could have encapsulated it all more cleanly, I'm sure, but it all "just grew" in scope and size.

The final block in this tottering tower was that some of the commands had to be run using sudo, but not all of them. Getting all those sudo strings in the right place took a few hours of my life from me without feeling that I got much for it. So it goes.

I've tried using CORBA, WebServices etc over the years, but they feel pretty heavyweight for this sort of thing. So the question I'm asking myself is: how could I have done it better?

JDiff 1.1.0 released

I finally got around to incorporating a few patches into a release for JDiff, the Java doclet I wrote years ago to generate Javadoc-like reports about the changes between two versions of a Java API. I still think it's a cool tool, in my opinion. And in the opinion of Apple, Oracle and Sun too.

I took the opportunity to rearrange all the source files to make it easier to work on, and combined all the Ant files into one big build.xml file. I noticed when I was updating the documentation that I had to tell some Java developers what Ant was in 2001, which is not the case now. I'm still trying to wrestle the repository into Subversion in SourceForge.

Scott McCloud at Hijinx Comics

I had a great time last night hanging out with my son Jacob (8) at our local comic store Hijinx Comics with Scott McCloud, his family and a bunch of other locals in the know. Scott McCloud is probably the most self-aware and thoughtful person creating comics at the moment. I liken the year-long tour he is just finishing to a professor travelling the world to spread his knowledge. His original book Understanding Comics has been worth re-reading every few years - it's one of those books, like Edward Tufte's. Any kind of collaboration between them would be an amazing thing to behold.

The other thing we really enjoyed was his family being there. Jacob was amazed to find any girl who liked Pokemon as much as he does, and Scott's wife Ivy is also a great presence in any room. Many thanks to Dan Shahin for organizing the get-together.