When Will It Ship? Estimates and Promises

I’m trying something new with this post: a short video presentation of approximately the same content.


Here is an area of confusion that has come up both at Oasis Digital, and at every other firm I’ve worked:

estimate ≠ promise

Background: Unpredictability

Around half of my software development and leadership experience has been in enterprise/internal software development, and that is the world I am thinking of as I write this.

Software development, like other endeavors with a significant creative component, is inherently unpredictable. With a good, deep understanding of the development process, you can build a model of the probability distribution of the cost, effort, and elapsed time for software development work. In the large, on average this can be made to work: small and large projects can succeed, within some broad range of predictability.

But notice also how common it is for large complex projects (in software and elsewhere) to be farcically over budget and late. This is not (usually) due to incompetence or fraud. It is because of the inherent unpredictability of the work.

If someone claims that they (or you) can exactly predict software development work, they are:

  • mistaken, or
  • lying, or
  • padding their estimates very substantially, stating a date or cost much later/higher than a neutral median estimate would suggest

As a customer of software development services, and as a provider of such services, I don’t want any of those things.

Blame the Service Trades

I place some of the blame for the confusion of these two wildly different things, on the service trades: it is common for auto repair shops, roof installers, landscapers, and the like to offer something they call an estimate, but which is actually a fixed price quote (a promise).

Sadly, while there are plenty of common good synonyms for promise, there aren’t many for estimate. We’re stuck with using the word estimate, and explaining that we really mean it as defined. Perhaps in a few more decades we will lose the word entirely, much like the word “literally” has come to mean its antonym, “figuratively”, which renders it mostly useless.

Estimates

An estimate is an approximation of an unknown quantity. Typically in the world of software development, it is a prediction of the cost, working hours, or delivery date of a project or milestone. It is not in any sense a commitment, any more than estimating the temperature outside this afternoon is a commitment.

As the word implies, a customer reasonably expects the actual value to vary somewhat, in either direction, from the estimate. In fact, if an estimate turns out exactly match the actual result, there is a good chance the books have been cooked. Moreover, if the work is completed at-or-before the estimate most of the time, this means the estimates (on average) are too high.

An estimate “costs” nothing, other than the time/effort required to create it, which consists of analyzing the work at hand, decomposing it in to parts, and comparing those parts to past work.

Promises

A promise, also called a commitment, deadline, quote, fixed price, etc. is a different beast entirely.

With a promise in hand, a customer should expect with high confidence that the actual value (for cost, hours of work, delivery date) will be less than (before), or equal to, the promised value/date.

Be wary of a promise easily made and freely given: it probably doesn’t mean anything at all. A wise customer (and I aim to count myself in this category) should expect that a casually made commitment will probably be broken; not because the maker is morally defective, but simply because meeting a commitment for complex work requires considerable effort and thought. Without evidence that happens, it would be mere wishful thinking to expect the results delivered as promised.

Likewise, keeping promises often has a cost. If the work underway gets behind the schedule needed to meet the promise, something will have to give:

  • Other work may fall behind, as time and effort are diverted to meet the promise.
  • Weekends, evenings, and overtime work may be needed. These might appear free, but are not.
  • Staff may need to be reassigned, or added
  • Additional hardware and software may be needed.

These risks cost real money; thus a wise promise-maker will find that, on average, it costs more to promise feature X by date D, than to delivery feature X by date D without such a promise.

Estimates are Cheaper, so Prefer Estimates

At Oasis Digital, we provide many estimates, but few promises. Most of the time, an estimate is what our customers need; and we can provide at estimate with very little cost. Typically we estimate reasonably well:

  • small features usually arrive with a day or two (plus or minus) of the estimated delivery date (and likewise for cost)
  • medium items within a week or two of the estimate, likewise
  • large items (major new features or subsystems with complex interdependencoes) within a month or so, likewise

The key here is is that with good estimate, commitments (promises) aren’t needed very often, and therefore the cost of promises can be avoided.

But Learn How to Promise Well, Also

Yet occasionally, a customer needs a commitment, most often because a software version needs to be available to match an important business event with a fixed date, such as a presentation, a legal filing, etc. I’ll follow up later (no promise or estimate, as to when) with thoughts on:

  • how to credibly make promises (as a service provider)
  • how to evaluate promises (as a customer)

Data Center (Cloud) Cost Efficiency

A few months ago I mentioned James Hamilton’s comments on the micro-server trend. Today I came across a talk he gave at MIX10 in which he presented excellent real-world large-scale data, with insightful analysis, about the cost efficiency of data centers. (Here is a direct MP4 download, suitable for viewing across more platforms.)

I had an intuitive feel for many of his conclusions already, and had numbers to back that up on a small scale (as a customer of cloud services, and provider of SaaS services, and employer of people who operate systems). But I am very pleased whenever an opportunity comes along to replace intuition with data.

I won’t attempt to repeat his ideas here. I will simply recommend that you watch this (and other similar analyses) and get a decent understanding, before purchasing or deploying any in-house, self-hosted, or self-managed servers. The latter still makes sense in some situations, but in 2010 the cloud is the default right answer.

A number of the ideas he presents are iconoclastic; some popular trends, especially in enterprise data centers, turn out to be misguided.

We’re hiring a NON-developer

Over at Oasis Digital, we are in the (job) market for our first non-developer, non-project-manager, non-service-delivery-focussed full-time team member. The job post is on the Oasis Digital site; I thought I’d write about a bit more about the story here.

Part of the challenge of running a software development consulting firm, is the tension between:

1) Focus on Customers You Have Now

Some firms are so service-delivery centric (#1 above), that they don’t bother to address the rest of the world, i.e. all of the not-customers-yet. Small consulting firms are especially vulnerable to this, and as a result they stay small.

Historically, Oasis Digital started in this category, with long-term projects, satisfied customers, and slow/organic growth. The shape of our organization reflects this: everyone, including me, works mostly on creating software for customers.

2) Focus on Customers You Want to Have Later

Some other software firms are highly marketing-driven, to the extent that their service delivery value proposition (in terms of design, features, and quality per customer dollar spent) suffers. Large technical service firms are especially vulnerable to this. Anyone who has been around the block a few times, has experienced the unique joy of a very large sum spent with a very large vendor to deliver sufficient, but disproportionately expensive, results.

Turning Up the Heat

Our goal and challenge now, is add a bit of focus on future customers (which is to say, marketing, community participation, etc.), without losing our service delivery mojo. To move things in that direction, we need at least one person who is not occupied by service delivery… hence our upcoming hire.

We will also quite likely start another software product or SaaS business along the way. (I’ve been there before, of course, and did plenty of hiring and business development work along the way.)

Some Things Won’t Change

Since inception, our sales process has been decidedly non-traditonal:

  • We encourage potential customers to meet their needs with off-the-shelf software, not custom software, if possible.
  • We don’t have commissioned salespeople. Nor commissioned pretend-they-aren’t-salespeople.
  • Most of our team is, and will always be, all about delivering great results for our customers.

I’d love to receive feedback on this, from those of you with experience in this particular transition. You know how to reach me.

Update: We’ve filled the need for that job post, with a combination of a few people, rather than a single jack-of-all-trades as I had planned.

The Much-Discussed Apple iPhone 4.0 beta TOS

As I write this in April 2010, Apple has just released a beta iPhone 4.0 SDK with the following rather alarming addition to the legal terms (at least according to many web sites; I haven’t seen it directly from Apple yet):

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

The is an avalanche of commentary out there on this, most of it negative, centered around the word “originally”. John Gruber, in his second comment about section 3.3.1, gives some reasons why this is sensible from Apple’s point of view. The essence is that it will lead to higher quality app, on average.

I will try to add a bit of insight from my experience. I am of two minds on this.

Why It Makes No Sense

From the point of view of using / creating the best tools, it strikes me as ridiculous. Great developers, or developers creating great things, generally use layered software designs to do so; and indeed layering, the increase in abstraction over time, is absolutely crucial to the software industry’s ability to make good use of ever-growing hardware capabilities. One obvious and vital kind of layering/abstraction is the use of higher level languages. Apple’s new insistence that only its provided languages/runtimes be used, ignores this fundamental insight.

More ominously, if this notion of “originally written” turns out to be enforceable. Imagine if earlier platforms had required that all software be written “originally” in the provided languages, with any attempt to build a higher level on top, banned; we could never have gotten to the amazing variety of technology available today, under such a regime. If such restrictions grow popular beyond Apple, that could lead our industry in to a kind of dark age.

I don’t think that’s likely though, and for that matter I think it is most likely that Apple won’t be able to make this new term stick. There are many successful apps already using a variety of languages, frameworks, and layered designs. Enforcing it as written, universally, would impact too many of those. Yet if Apple enforces it selectively (just to squash Flash, as many commentators suggest), they invite a long and painful legal mess. Speaking of that, such a legal mess would be an enormous windfall for Android.

Why It Makes a Bit of Sense

Setting aside the questions of whether it can possibly work, Apple’s notion does make a some sense, at least for the immediate future for this specific (iPhone etc.) platform. Most iPhone users/customers are probably better served by a market consisting mostly of native apps.

The difference between native and non-native apps for iPhone OS, is mostly analogous to the same difference a decade ago in PalmOS software. The native (and tedious to use) C toolset was the most popular, but there were many other (higher level) tools to choose from, and with frank honesty I can say that the results were generally poor, and almost always worse that write the same app with the native tools.

In fact, my firm Oasis Digital worked for at least two different customers on projects to replace applications written using such tools, with code we wrote in C or C++. Yes, it took longer; yes, it was more lines of code. Yes, it took more testing and was more prone to the kinds of problems low level code has, with memory leaks and the like.

But most critically, from the point of view of the users, our results with low level tedious implementation were obviously better. The high level tools available today for iPhone are much better than the Palm stuff a decade ago; but still, I suspect the same applies to a lesser extent on the iPhone-OS platform today.

Apple, Cut That Cord

As I think back to opening up my new iPad, two things stand out. First, the product packaging is delightfully minimal. It does not even contain media with the iTunes software, instead explaining that the first step is to download iTunes.

Second, a new iPad does not work at all until syncing it with a PC (Mac or Windows) via the included USB cable. I find this surprising because, in so many other ways, the iPad is a great device for someone who wants access to basic computing capabilities (web, email, casual games) without caring about the complexity of a PC. Yet such a user must already have a computer, with all its (potentially very-un-Apple) complexity, to use an iPad. The polished experience is potentially debased by being plugged in to a $200 closeout PC, just to get it running.

The need for iTunes seems quite justifiable in smaller Apple iDevices; but for the iPad, I can’t help but notice its specs are ample to be self-contained.

Even for those of us who will sync to a PC, why the cord? The iPad has WiFi and Bluetooth, as do most PCs nowadays. Plugging in that wire to sync seems utterly antiquated. Apple could could scrap that many-pinned proprietary port, and replace with a much simpler power plug, which would also free it from the pitiful rate of charge provided by USB. An iPad could have a MagSafe charger, and charge up more safely and in 1/2 the time.

Apple, cut that cord!

My iPad is (unfortunately, mostly) a toy

Some pundits have declared the iPad a “toy”, or suited only for content consumption. I disagreed with the latter a few days ago, and I think it will have some very interesting business uses. If I come up with a sufficiently novel one, Oasis Digital might implement it.

But sadly I must mostly agree, for at least my personal use of iPad v1, with the “toy” assessment. Here’s why.

I like to hand this iPad (I wrote the first draft of this post on it) to other people to try out, to my kids to play with, to sit it out as a novelty for guests, etc. But if I set it up as a personal tool, with my email, calendar, Twitter, Facebook, other accounts, work related files, etc., then I’d be handing access to all that personal and business information to everyone I offer the iPad to. Clearing out and re-entering my accounts from a whole pile of apps (and Safari, and Mail, and ???) is tedious and error-prone. My personal accounts include occasional bits of my (and our customers’) proprietary information, so leaving them present for guests is clearly unacceptable. Thus, for now this iPad can be only a toy to me. I occasionally set up email, then remove it; set up Twitter, then remove it, etc. I type some notes, then email them to myself, and remove them.

The key missing feature of the iPad, for me, isn’t a camera, a USB port, or an SD slot – it is the lack of a user account/profile capability. My Ubuntu netbook, at half the cost, has this capability in the box.

This isn’t much of an issue with an iPhone, which by nature of being a phone, is inherently almost always a personal (not shared / guest) device.

This isn’t much of an issue with an iPod Touch, at least for me, because I already use it as a guest/toy only anyway.

I suspect that Apple won’t add an account/profile mechanism anytime soon; it is easier to therefore ignore a mixed personal / guest use case like mine. If Oasis Digital ends up with a line of iPad software, we’ll work around the problem by buying more iPads.

But in the meantime, I like the share the one I have – so I must configure it primarily as a toy.