Keep Your Development Focus Sharp

Brian Button recently suggested, for XPSTL, “a series of presentations where we discuss the biggest challenges we, as team members and developers for the most part, face in our day-to-day jobs with respect to being agile. The challenges can be in technical areas, organizational change issues, or whatever else people think is hard.”

I’ll bite.

First, some background: the context here is an agile but not XP project, with a widely distributed team. The software is sold in an “application service provider” model; we usually deploy updated software weekly, though our planning is not particularly crisp around iteration boundaries. We use source control, a build server, an issue tracking system (paper cards aren’t of much use in a distributed team), unit tests, a few acceptance tests, a mailing list, Campfire, Java, and Eclipse. We build and sell an “enterprise” software product, so our product management is driven both by meeting current customer-specific needs and general target market needs.

We have listed a great number of desired features (and tasks, and bug fixes), and grouped them in to a few large buckets, which are roughly “releases” in the XP sense of the word. A small subset of those work items are considered “To Do” items, i.e. items that should be worked on now, to be deployed it the next week or two.

The ongoing challenge we face is in disciplining our own desires – in limiting the list of what we will do next, to what we can reasonably expect to accomplish soon, and then to work vigorously on getting those things out the door. Thus our goal is to, at all times, keep the team focused on a relative handful (at most a couple dozen) of key things to work on now, to bring those items to end-to-end closure.

Our challenge is in the perpetual tendency to derail the process, in any number of ways:

  • As new issues come up, there is a tendency to throw these in the “do now” list, without regard to their importance in the backlog.
  • Items get stuck waiting for requirements feedback. Either get the feedback quickly, or kick the items off your “to do” list in to the backlog, and go work on something you can finish.
  • Items get stuck waiting for integration in to core code.Items get stuck waiting for a production / deployment task, because your team is hesitant to touch a production system. The solution is to test, test, and test again, then deploy (using whatever the local process is, for your team / project / organization).
  • Items grow in scope, as they are discussed and implemented. The fix for this is to split off the additional scope in to another item (XP card) and put it in the backlog to be prioritized; trim the work item in front of you to a reasonable size and get it shipped.

All of these cause an overall derailment not because they add “bad” items, but because they reduce the focus, and thus reduce the tendency to get items finished and out the door. My advice is to keep your (team’s) focus sharp.

Aiming for Mainstream

Over on defmacro today, a new article appeared: defmacro – Why Exotic Languages Are Not Mainstream in which the author laments that while there appear to be various choices to use Haskell on Windows, it turns out that all of them are, in some way, not ready for prime time… or even for effective hobbiest use.

I’ve noticed this myself, in my last few forays in to esoteric languages: the illusion of plenty of choices, runs in the the reality of no good choices.  This is not a universal problem; I’ve had great results with Ruby, Python, and Lua, all of which are to some extent esoteric.  The thing that those languages have in common is that there is at least one (and generally, only one) robust, production grade implementation with a community actively supporting it.

If you want to see your favorite language gain acceptance, spend your time creating / maintaining / vigorously supporting a production-ready implementation.

DreamHost Disasters

You might have noticed that kylecordes.com and my other sites have had several long downtimes recently.  I certainly have. The cause, in every case, has been DreamHost, the formerly excellent hostiing firm I use.  DreamHost has excellent features and configurability, but two large problems:

  1. Lots of downtime incidents
  2. Slow performance of DB-based web apps, including WordPress

I’ve started the process of moving a site off of DreamHost, so that at worst I won’t have everything down at once.  After some live evaluation, I’ll decide whether to move the rest.

I am much impressed, though, by a long explatation of the recent problems over on the Dreamhost Blog:
DreamHost Blog » Anatomy of a(n ongoing) Disaster..

This contains far more detail and honesty than I am accustomed to receiving from vendors, and I appreciate it very much.  I still recommend DreamHost, if you are looking for a bargain, rich features, and can tolerate (until they demonstrate otherwise) some “issues”.

Long Rails Stack Trace

I was looking at the RailsConf schedule, and saw the Rails web app that runs it fail with this longish stack trace. A few Rubyists sitting nearby commented that the application was not well configured, that usually Rails apps are configured in deployment to store such traces to a log, rather than spit them at the user. Their mistake is my gain though, as it provided fodder for this post.

This stack trace is considerably shorter than a typical Java web app stack trace. But it is not so good either, and it is an ominous sign that excessive complexity is making its way in to Rails.

How To Do Deployment (Dave Thomas RailsConf Keynote)

I just attended Dave Thomas’s keynote at RailsConf. He had many interesting things to say, most notably that 60%+ of his Java-centric conference-circuit friends, mostly people who have written books on Java, speak regularly on Java, and have lots of experience as Java “architects”, are now making a living with Ruby / Rails.

Dave talked about 3 key areas where Rails should improve. The one I want to focus on is deployment. Rails has Capistrano, a slick tools for deployment automation. Capistrano is great, but it misses the boat in one key way: it is push-based. It assumes a world where the developer sets up, controls, and performs deployment to production servers. That is of course the case at the start of a small operation, but it doesn’t scale; in large organization, or even in small growing ones, sooner or later there is staff dedicated to production deployment and monitoring, separate from development.

One way, to me right way, to handle deployment is push / pull:

For the “push”, a developer uploads a new version to a storage site somewhere. It travels in the form of an archive file (ZIP, tgz, whatever) which contains all of the needed artifacts, as well as metadata about the required libraries, pre- and post-install steps, etc. for the new version. It has a unique name; it lands at a URL. Normal, off the shelf technology (FTP, web servers, etc.) can be used to serve and secure this storage, implement policies about who can upload new versions, etc. This upload happens with some simple one-line command (or as part of an auto-build); it does not actually put a new version in production, but only makes it available for pulling.

Then, on a production machine, someone with software installation rights on that machine (administrated by the normal tools – not by any special feature of the deployment mechanism) runs a command (something like “deploy staging http://ourserver/project1/project1-build-3453.tgz”) which downloads the new build, runs various sanity checks, deploys it to a staging area, and makes it available to try out. Once the staging is verified, a similar command brings it in to live production. These builds refer as needed to configuration data on the deployment machines (such as production DB access credentials); the build archives are generic, not specific to any particular production machine.

The main idea behind this is that each person has the rights they need for the work they need to do, and these rights don’t need any special help or support from the deployment mechanism. Developers don’t need any special rights on deployment machines, nor to deployers need any special rights on development machine

I wrote this in the context of a Rails / Ruby talk, but it’s at least as relevant in Java and Delphi world; in fact at Oasis Digital we need something like this on a Java project and a Delphi project right now.

At RailsConf, Sitting Around Ignoring Each Other

I’m at RailsConf. In a flagrant attempt to be one of the cool kids, I’ve set up a Flickr account to share my RailsConf Photos.

Of the few dozen conference I’ve attented, this one has the highest incidence so far of people sitting around, in large groups, working alone on their notebook computers; some of the groups are having conversations, but most of the attendees (including me, as I type this) are staring intently at their own PCs. It makes me wonder why we are all here – are we all so accustomed to interacting online that we don’t bother to interact offline, even when sitting in the same room (and at considerable expense?)