News like that is really hard to process. I wanted a second opinion and ended up receiving 5 or 6 of them from numerous specialists. The consensus was unanimous …

“Go out and enjoy every minute of the time you have left”.

Paradoxically, the worse the news, the easier it turns out to decide what to do. While I’m truly blessed with the best job in the world, it’s time to spend less time in front of a computer and more time with my family and friends, particularly my 3 kids. From Monday, I’ll be working part-time (for the first time since 1988).

Time also for a really good holiday. There are places I’d like to see and, quite frankly, it’s now or never! Given the nature of travel insurance, I’m now unlikely to see many of those places overseas but that still leaves a huge list of amazing places in Australia to visit.

Some people would say I’m lucky to know in advance that my time is nearly up and that many others never get that forward notice. I’m in two minds about that. The down side is that thinking about one’s death becomes terribly paralyzing at times. I frequently get angry about dying so young. There is still so much I want to do and so much I want to see. I strongly believe that we’re all responsible for making the world a better place. I’m proud of what I’ve achieved but, given another 43 years, it could be so much more.

Here’s hoping that I can prove the doctors wrong and live for many more years instead of just a few months. Miracles do happen. I plan to stay out of pain, focus on really good nutrition, rest a lot, enjoy every moment and hope the cancer spreads slowly (or disappears altogether!). In the meantime, here’s hoping someone has a mega-brainwave and cures cancer. How hard can it be?

As you can imagine, I have a lot on my mind right now. Two things keep popping into my head though: bacteria under rocks in Antarctica and a quote by Steve Jobs. I keep thinking of the first thing because it reminds me that, no matter how harsh the circumstances, life finds a way to survive. The quote by Steve is essentially this: “We’re put on this Earth to make a dent”. My job in this lifetime is far from done so I plan being around for a long time yet.

Bazaar Explorer is a cross-platform desktop application that runs on Windows, GNOME, KDE and OS X. It doesn’t try to be a poor implementation of a file manager or a brain-dead IDE. Instead, it focuses on version control stuff: managing branches, managing changes and collaboration. While it’s not cooked enough yet for me to completely abandon the command line, it’s coming along nicely and is proving just as productive, if not more so, for some common version control tasks. Here’s a brief introduction by pictures.

On start-up, a Welcome page is presented if a location isn’t specified:

As shown above, I’ve created bookmarks to several of the repositories I have on my PC, one repository per project. I also have some bookmarks that navigate straight to commonly accessed branches. (“core” is the core Bazaar project btw.)

Double-clicking on a repository opens the repository view:

At the top are the branches (and other objects) nested inside the repository. Below that are details about the currently selected object. Double-clicking on a branch opens it:

This gives a status report on the current working tree: any conflicts found, what’s new, what’s changed, etc. From the report, you can click on a file to open it in your editor or see the per-file diff. The full diff, together with a heap of other branch operations, is available by clicking on the relevant toolbar button.

Easy access to Bazaar’s various configuration files is provided by the Settings menu:

Furthermore, you can define your own tools and launch them from the Tools menu. In Explorer, a tool is either a special bzr command (like lp-open or pqm-submit), a local application (like KCacheGrind or Qt Designer) or a web site.

Explorer recognises that many users have different needs at different times: one open source project vs another, work vs home, one client vs another, advanced user vs trainer, etc. As such, you can create, download and switch “hats” – collections of tools and bookmarks you want to use together.

Even better than defining your own tools is reusing a set of tools that someone else has already put together! If you’re a core developer or team leader on a project, you can define a hat for others to use and include links to all the important websites they’ll need: the issue tracker, wiki, build server, qa results, etc. That ought to mean less ramp up time for new contributors on open source projects or new staff on in-house projects. See http://bazaar-vcs.org/BzrExplorer/Hats for details.

In summary, I think Bazaar Explorer is pretty cool. For a weekend project that only started in June, I’m blown away by how quickly it’s come together. I can thank the combination of cool technology (Python, Qt, bzrlib, QBzr, bzr-gtk) and keen early adopters for that. A special thanks goes to Alexander Belchenko who has helped heaps, removing bugs almost as quickly as I’ve put them in. If you haven’t already, please give Bazaar Explorer a try. If you like it, please consider translating it to your native language and/or joining the Bazaar Explorer Developers team. We’d love to have more developers on board, particularly if you’re a KDE or OS X user.

To start the ball rolling, I’m pleased to announce the Manifesto for Community-Agile Software Guidance. Anyone can sign this – simply go to the page and add a comment indicating that you agree! More importantly, a paper explaining the values, principles and practices is available in numerous formats including html and pdf. (I was planning to present this paper at OSCON 2008 but unfortunately can’t travel at this time.)

To get involved, visit https://launchpad.net/~community-agile and join the team!

When I started this blog in 2007, I explicitly made the decision to focus it on professional topics and avoid making it about life in general. I gave it the title Agile Teams, Open Software, Passionate Users .. Life is too short for anything else. You can tell it’s not about life in general because I left Good Wine out of the title. There are times like now though when separating professional from personal just doesn’t make any sense. We simply spend such a large percentage of our waking hours working that happiness at work is directly related to personal happiness for many of us, and good personal health directly impacts our productivity and relationships at work.

In my case, I’m extremely lucky to be doing the work I do at Canonical. It’s something I feel deeply passionate about: making it easier to produce great software more efficiently. I also have the privilege of working with a bunch of really smart people and I learn something new from them each and every week, if not every day.

My family and I have been overwhelmed by the amount of support everyone has offered. Almost every one we know has contacted us on hearing the news asking what they can do to help. I’m writing this article because I want to let people know, regardless of where they live, that they can help. Here’s how:

1. Don’t take your health for granted like I did. Early detection of many diseases in the only real defence so go and get those tests done you’re been putting off because you feel fine.
2. Do something you enjoy and do it well. Life truly is too short to be working in a job you hate or to be wasting time using unproductive processes and tools. (If you use a computer, try Ubuntu. If you develop software, try Bazaar.)
3. Take care of the people close to you.

We live in a society where talking about one’s butt simply isn’t done – no-one ever goes to the toilet in any novel I’ve ever read! It’s not easy telling people that I have rectal cancer, but it’s common and often fatal. If sharing my story means other people catch it or another disease sooner, then I’m pleased to have done it.

Update: The final version is now uploaded and linked above (and it’s really PDF, not PostScript, this time).

#5 on my list of criteria for evaluating version control tools is Integration. Software exists to get things done and it rarely, if ever, exists in isolation. The more successful software is, the more pressure there is to integrate it with other tools and systems. I believe lack of mature integration with other systems will be the #1 reason for many teams delaying their move from a central VCS tool (like CVS and SVN) to a distributed VCS tool (like Bazaar, Git or Mercurial) in the next 12 months. The good news is that the new breed of VCS tools all do a lot right in terms of enabling integration but we need to do much more. Firstly, we could and ought to be doing more at the core of the new products. Secondly, we need to get behind the really important integration add-ons and help them reach maturity faster.

At the core product level, Design for Integration comes down to four key things …

1. The software must be open.
2. The software needs a layered architecture.
3. The total solution needs to be manageable.
4. A rich domain model is highly desirable.

Closed systems are extremely difficult to integrate with. To (partially) quote my own blog title …

Open Software – life is too short for anything else.

At a minimum, systems need to support open protocols or open APIs so other products can talk to the product in a stable way. Ideally, the system will be open source so integrators can access, understand and improve the integration points.

The benefits of a layered architecture where the UI is decoupled from back-end services is well understood these days. As well as multiple layers though, multiple communication models are needed – both synchronous and asynchronous. In other words, systems wanting to integrate cleanly with a product usually need some way of hearing about interesting events in the core product – even if that is just via email – in addition to the more common request-response communication model.

Once the various pieces exist and talk to each other as desired, application management issues become really important. The big issues here are things like security (e.g. sharing authentication credentials) and dependency management between the core product and add-ons. Do all the pieces have to be upgraded at exactly the same time or can the core be upgraded then the other pieces at another time?

Finally, it greatly helps if the core product has a rich semantic model at its core. This makes it easier for external products to communicate in a logical way and minimises the amount of data loss when interchanging data with the core product. Mark Shuttleworth captures this point nicely in Choose lossless VCS tools if you have that luxury. Truly caring about integration goes even deeper in my opinion: it means explicitly making it easier to carry/manage attributes from external systems that you may not have nor want in your core model. Round-tripping between systems, particularly central <-> distributed VCS tools, is and will remain a fact of life.

Once again, I think the leading distributed VCS systems are all off to a great start in terms on enabling integration. All are open source. All have a layered architecture. On the ease of management front, I think Bazaar and Mercurial are ahead of Git thanks to their better plug-in architectures. On the rich domain model front, Bazaar leads the pack. As Robert Collins commented in a discussion on Bazaar vs Git, facts plus heuristics is a better direction than just heuristics alone. As part of the Bazaar team, of course I think it’s best! But the things that are common across Bazaar, Mercurial and Git are arguably more important than their differences. Can we share integration designs and code better across the various distributed VCS communities so all of us get to where we need to be faster?

So what are the really important integration projects we need to get behind? In the Bazaar community, I believe these are:

• GUI integration projects, particularly bzr-Eclipse and TortoiseBZR
• VCS migration/round-tripping projects, particularly bzr-svn.

To some people, projects like these are uninteresting sugar. If that’s you, please help with our performance and documentation drives! To me though, these projects are some of the most important things going on in the Bazaar community today. Many developers, let alone less technical project contributors, want version control to be largely transparent. After all, in great development tools right now, the VCS interface is unified so whether the central repository is hosted in CVS, SVN, Perforce, etc. is largely irrelevant in day to day programming. Mass adoption of distributed VCS technology will need that same level of ubiquitous integration. If you’re looking to get more involved in Bazaar development, integration projects are a great place to help. In true open source tradition, a small number of passionate people can really make a really big difference.

Technically, trying to ship a tool which is all things to all people creates all sorts of problems particularly w.r.t. reliability, my #1 evaluation criteria. Bloatware takes longer to ship each time, quality typically drops regardless, and the extra features don’t necessarily hit the mark anyway. As Mozilla has shown with Firefox and Thunderbird in recent years, it is far better to produce a rock solid core product that supports plug-ins in a documented way. Done right, the result is higher quality, better performance, and extensions that better meet the needs of the user base anyway.

Socially, a plug-in architecture increases the engagement of the community that grows around successful products. Whether open source or commercial, it takes a community to raise great software and plug-ins let that community scratch their itches in a sanctioned way.

There are different ways of tackling the extensibility challenge but the best way in my experience is by explicitly supporting plug-ins as Bazaar and Mercurial do. Other tools like Git have gone down the well worn toolkit path, and while that’s much better than having a monolithic application, I believe the plug-in path is a wiser one for a host of reasons.

Firstly, toolkit products tend to get extensions and enhancements developed as Unix shell scripts or Windows batch files, reducing their portability across platforms and hence usefulness to the user community. Secondly, scripting needs tend to put a negative pressure on the UI in that a large amount of options begin appearing just for scripts. That only gets scripts so far though, so then the raw files tend to be parsed, making it increasingly difficult over time for those formats to evolve. (Encapsulation isn’t just good theory – it’s good practice.) Thirdly, a plug-in architecture puts a positive pressure on the product designers to have a rich, clean API for plug-in developers. Tools without that pressure are more likely to have internal APIs that expose physical design details instead of logical design ones. Once again, that leads to a fragile product where few are prepared to make desirable changes after a while in case something breaks. Compatibility dominates as a consideration – innovation grinds to a halt.

Of the new breed of VCS tools around, I think Bazaar leads the pack w.r.t. its plug-in architecture. http://bazaar-vcs.org/BzrPlugins lists over 60 plug-ins! And it is easy to write your own as explained here: http://bazaar-vcs.org/WritingPlugins. Mercurial also has a good plug-in architecture with 30 plug-ins listed on http://www.selenic.com/mercurial/wiki/index.cgi/UsingExtensions. Both Bazaar and Mercurial plug-ins can add new commands while Bazaar plug-ins also let you add options to existing commands. Useful stuff.

As well as plug-ins, pre and post command hooks are really useful for integrating VCS tools into your toolset. While Git is lacking in a plug-in architecture, its hook capability looks good and similar in scope to what Mercurial is offering. Bazaar also has a useful set of hooks both in the core and scattered across some commonly used plug-ins. The documentation on these needs some work though, an area sure to receive lots of attention as we push hard towards Bazaar 1.0.

Bazaar 0.18 is now available. Please download it and give the Bazaar team your feedback so we can make Bazaar 1.0 as good as we possibly can. Here is a sample of the Bazaar plug-ins you may wish to try with it:

• BzrTools -a collection of useful extensions
• BzrGtk – a set of GUI interfaces
• BzrSvn – round-tripping to SVN repositories
• difftools -display differences using the tool of your choice
• extmerge – resolve conflicts after a merge using a GUI tool
• dbus – automatically notify team members on your LAN re branch changes.
• and many, many more.

Do you have a favorite Bazaar plug-in? If so, please add a comment below explaining why.

In terms of impact on team success, tools are pretty low in the pecking order of things when compared to clear leadership, motivated people and just right processes. However, the new breed of version control tools are exciting precisely because they change the software development game: they enable new ways of communicating and that in turn enables new ways of thinking about software development. But outside the AlphaGeeks, Distributed Version Control will only reach criteria mass as a technology across the open source world and in commercial software development shops if we can show the technology is ultra-reliable. To me, that means 3 things: Design for Reliability, extensive test suites and strong auditing, e.g. cryptographic signing of commits.

Design for Reliability is all about taking a conservative approach – valuing correctness above other things. As a young software engineer back in the early 90s, 10s of thousands of people woke up to cold showers in Sydney one morning because of a corner-case bug in my code controlling the off peak hot water system. That sort of event tends to have a life long impact on how one designs software! I found it really interesting watching Linus talking about Git’s design and how someone trying to corrupt the Linux code archives had a similar impact on how he designed Git. I really like how Linus made reliability such an important design consideration for Git yet it hardly got a mention in the almost 400 slashdot comments on the video. By offending so many central VCS users by assuming that what’s best for him must be best for everyone else – a point I’ve addressed elsewhere, I fear a really important point of his has got lost in the follow-up noise.

As a member of the Bazaar community, I want to see the various DVCS communities working together and learning off each other so we can collectively move the IT industry to a better place. While it might generate more press, I really want to minimize my criticism of other DVCS tools, particularly Mercurial which is my preferred tool after Bazaar. But I do feel it is important to mention that some operations in Bazaar may never be as fast as they are in Mercurial because of reliability-driven design decisions we’ve explicitly made. For example, Mercurial uses hard links when branching locally making that operation pretty damn quick. In contrast, Bazaar not only copies content because of our concerns with hard links but cleans up any junk in the archives as it goes. When you branch in Bazaar, you know your new branch is as clean as clean can be. If you don’t care about that, no problem – a simple cp -a will get you a new branch real quick instead (without the integrity checks). Robert has reported other areas where Bazaar is deliberately more conservative in the name of correctness. Bazaar undoubtedly has its own reliability quirks I’m not yet aware of. If you like another DVCS tool because it’s more reliable than Bazaar, please speak up so the Bazaar team can do something about it! Like I said, I think it is very important the various DVCS communities learn from each other and Design for Reliability is something all of us need to be putting at the top of our lists and actively addressing.

An extensive test suite ought to be a mandatory feature of every serious VCS tool released these days. VCS tools need to be reliable across umpteen different operating systems and every one of those has umpteen different versions. In a world where security updates for all major operating systems are released at a high frequency, exactly what an ‘operating system version’ means can be very nebulous indeed. If you’ve never had an application mysteriously break after a Windows hot-fix, my guess is that you’re in the minority. The same thing will happen on OS X and every major Linux distro at some point – it’s just a matter of time. The only safety net you have for verifying that such an important tool is working correctly in your particular environment is an extensive test suite. Bazaar’s reliability and robustness is no accident: its built-in regression test suite for 0.18 currently has over 7000 tests!

While reliable technologies and algorithms are critical for ensuring repository robustness, completely trusting the integrity of a VCS repository is a far larger topic as explained by David Wheeler. Cryptographic signing of commits really matters when it matters. All the leading distributed VCS tools have this while many of the central VCS tools don’t. In 5 years time, I suspect this feature will be available in all the popular central VCS tools as well, partly driven by the Sarbanes-Oxley craze/disease sweeping the US. (Those Enron guys have a lot to answer for.) In the meantime, this is another great example of how DVCS technology is leading the way.

As I mentioned earlier, Bazaar 0.18rc1 is now available. Please help us test it by downloading it and giving it a go. Just right processes are an important part of delivering reliable software and there is no better time than now to tell us what you think and how we can make it better!

Software technologies – operating systems, applications, programming languages – rapidly evolve, so I find the Web a better choice than buying book after book on something with such a short half life. As Grady Booch says though in his interesting look 50 years into the future (warning: 12.6 MB), software engineering fundamentals never go out of style. The fundamentals are the values, principles and best practices of what we do: Requirements, Architecture, Design, Construction and so on. I keep a list of what I believe are the best books in each of these areas. I hope you find it useful reading as much for what I’ve left out as included. I welcome input on evolving this list as we learn more as a profession on the numerous challenges of software development, operations and support.