I have often given the advice to start each project with an end to end working first draft, also known as an evolutionary prototype, and sometimes I have even followed this advice myself. My own projects have been most successful when following it, such as in three projects that come to mind:
- A 6+ year project, with a team of developers. In month 3 it was deployed to production, meeting real needs.
- A 4+ year project, with a team of developers. In month 2 it was deployed to production, meeting real needs.
- A 10+ year project, initially just myself but later with a small team of developers. In month 1 it was deployed to production, meeting real needs.
On the other hand, most failed projects that I am aware of had been slated from the beginning to not deliver a working system until late in the schedule, and ended up not delivering at all.
The traditional software development plan for a typical enterprise forms-database-rules-reports application looks like this:
- Planning and design
- Implement schema
- Implement infrastructure
- Implement admin/setup features
- Implement core features (transactional data)
- Implement reports
This waterfall model is a great way to think about software. Think through your requirements in depth at the beginning, and write them down in a coherent form. However, this is generally a bad manner (specifically, a very poor order) in which to write software.
Thanks to the agility movement, many teams are now using iterative, incremental approaches. Unfortunately, it is common nonetheless to lose some of the benefits of agility by laying out a set of releases with features/work in approximately the traditional order.
The solution, espoused by the Extreme Programming movement but rarely executed, is to start at the very beginning of a project, with building an end to end, first draft, working prototype system. This then grows slowly over the life of the project in to the finished product (i.e., it is not a throwaway prototype). We take this approach at Oasis Digital, and generally require it of our subcontractors also.
Development proceeds in a way that we create a working, end-to-end system with minimal features, very early in the work. For example, on a typical forms-database-rules-reports project, after a few days of work, we have:
- A little bit of schema, just enough to support a small subset of desired features
- A few screens, with very few features
- A report or two, again with very limited features
- Placeholders for some other screens and report, to allow the beginnings of the desired final application flow to appear
Then as the work proceeds, every week (or less) we can deliver a working system with a few more features present.
But this is impossible on my project!
The question is not whether this approach is possible; rather it is to what extent it can be done on each project, and what can be done to make it possible? The primary tool is imagination, in selecting a small subset of the desired feature set which plausibly works end-to-end.
For example, perhaps you are building a CRM (customer relationship management) system, one with thousands of features, 200 database tables, 450 reports, 85 screens, etc. How could you build a working system in the first week? Start by reducing the system to be its bare essence of companies, contacts, and interaction history.
A second example is a time tracking system with elaborate rules, reports, approval mechanisms. For the first draft, discard most of that and create only the ability to add a user (forgot edits and deletes, forget most of the fields), to enter hours for that user (forget approcals), to see a screen with those hours (with the simplest possible layout), and export a data file with those hours (any data file format, forget XML for now).
A third example is a trouble ticketing system. For the first week, make a way to add a ticket, a way to add notes to a ticket, to see a list of tickets, and to close a ticket. This could readily all occur on one screen, with only a few buttons. Its schema could consist of three tables: user, ticket, note.
A fourth example is a system which will write checks. In the very first week of the project, make it write a check. Dummy-out as much as needed to make it achieve this core goal.
Just Do It
Certainly there are plenty of problem domains where this advise doesn’t readily fit. But don’t be too eager to make that claim. By finding a way to deliver a working system early, we eliminate the risk that the project will never deliver anything, and we make progress and value visible (continuously) to whoever is paying for the project.
Kyle, you make great points for projects that are in-house or for projects where the scaling needs are known. What would you say about a service like Twitter, though, which took this approach (‘just get it out there’) and then has had numerous embarrassing failures over the past few months? Note that these problems are due to inherent flaws within the system design and can’t easily be fixed without scrapping everything… and turning on a “2.0” that has nothing to do with the original from an engineering standpoint.
What I describe here is getting end-to-end working functionality… I said nothing at all about making it scale, or about deploying to millions of users. Twitter is radically outside the scope of what my post it about. Twitter is also so simple that it seems quite likely they may have had the basic functionality working (for a small number of users) almost immediately.
I have a lot to say about scalability, which I will write someday. The short version is that you need to test under load, spending time as needed to make a load simulator.
Regarding Twitter, I don’t think it makes any sense for me to snipe at anything they have done. There have probably been 10 other similar services that never got any traction or popularity, and anything Twitter did differently (including making better decisions on scalable design!) might easily have also put then in the enormous category of web startups that noone has ever heard of.