Hire a RAIT: Redundant Array of Independent Teams

Life is Risk

Whenever you hire out work, either to a person, to a team, or to a company, there are risks. These risks can easily prevent the work from being completed, and even more easily prevent it from being completed on time. (I’m thinking mostly of software development work as I write this, but most of this applies to other domains as well.)

What could go wrong with the person/team/company you hire?

  • They get distracted by family or personal issues.
  • They turn out to not be as qualified or capable as they appeared.
  • They leave for better work. Sure, you might have a contract requiring them to finish, but your lawsuit won’t get the work done on time.
  • They turn out to not be as interested in your work as they first appeared.
  • They start with an approach which, while initially appearing wise, turns out to be poorly suited.
  • Illness or injury.

Of course you should carefully interview and check reputations to avert some of these risks, but you cannot make them all go away. You don’t always truly know who is good, who will produce. You can only estimate, with varying levels of accuracy. The future is unavoidably unknown and uncertain.

But you still want the work done, sufficiently well and sufficently soon. Or at least I do.

Redundancy Reduces Risk

A few years ago I stumbled across a way to attack many of these risks with the same, simple approach: hire N people or teams in parallel to separately attack the same work. I sometimes call this a RAIT, a Redundant Array of Independent Teams. Both the team size (one person or many), and the number of teams (N) can vary. Think of the normal practice of hiring a single person or single team as a degenerate case of RAIT with N=1.

To make RAIT practical, you need a hiring and management approach that uses your time (as the hirer) very efficiently. The key to efficiency here is to avoid doing things N times (once per team); rather, do them once, and broadcast to all N teams. For example, minimize cases where you answer developer questions in a one-off way. If you get asked a question by phone, IM, or email, answer it by adding information to a document or wiki; publish the document or wiki to all N teams. If you don’t have a publishing system or wiki technology in hand, in many cases simply using a shared Google Document is sufficient.

There are plenty of variations on the RAIT theme. For example, you might keep the teams completely isolated in terms of their interim work; this would minimize the risk that one teams’ bad ideas will contaminate the others. Or you might pass their work back and forth from time to time, since this would reduce duplicated effort (and thus cost) and speed up completion.

Another variation is to start with N teams, then incrementally trim back to a single team. For example, consider a project that will take 10 weeks to complete. You could start with three concurrent efforts. After one week, drop one of the efforts – whichever has made the least progress. After three weeks, again drop whichever team has made the least progress, leaving a single team to work all 10 weeks. As you can see in the illustration below, the total cost of this approach is 14 team-weeks of work.

How might you think about that 14 team-weeks of effort/cost?

  1. It is a 40% increase in cost over picking the right team the first time. If you can see the future, you don’t need RAIT.
  2. It is a 50% decrease compared to paying one team for 10 weeks, realizing they won’t produce, then paying another team for 10 more weeks.
  3. If you hired only one team, which doesn’t deliver on time, you might miss a market opportunity.

Still, isn’t this an obvious waste of money?

To understand the motivation here, you must first understand (and accept) that no matter how amazing your management, purchasing, and contracting skills, there remains a significant random element in the results of any non-trivial project. There is a range of possibilities, a probability function describing the likelihood with which the project will be done as a function of time.

RAIT is not about minimizing best-case cost. It is about maximizing the probability of timely, successful delivery:

  • To reduce the risk of whether your project will be delivered.
  • To reduce the risk of whether your project will be delivered on time.
  • To increase your aggregate experience (as you learn from multple teams) faster.
  • To enable bolder exploration of alternative approaches to the work.

What projects are best suited for RAIT?

Smaller projects have a lower absolute cost of duplicate efforts, so for these it is easier to consider some cost duplication. RAIT is especially well suited when hiring out work to be done “out there” by people scattered around the internet and around the world, because the risk of some of the teams/people not engaging effectively in the work is typically higher.

Very important projects justify the higher expense of RAIT. You could think of high-profile, big-dollar government technologies development programs as an example of RAIT: a government will sometimes pay two firms to developing different designs of working prototype aircraft, then choose only one of them for volume production. For a smaller-scale example, consider the notion of producing an iPhone app or Flash game for an upcoming event, where missing the date means getting no value at all for your efforts.

Thanks to David McNeil for reviewing a draft of this.

Helping Our Customers Hire

For today, a “day job” topic:

Oasis Digital (my firm) is a custom software development shop. It is not a staffing or recruiting firm; there are many good firms in those businesses, and I have no desire to join them in that market. Oasis Digital doesn’t offer contract-to-hire, it doesn’t charge a percentage of a hire’s pay, and does not recruit in for customer placement.

Nonetheless, we do occasionally help our customers hire.

Really?

I’ve heard customers and our own team members express surprise at this. Isn’t it against our own interest to help a customer with a direct hire, who might end up doing some work instead of Oasis Digital doing it?

In a short-term, trivial sense it is perhaps against our interest. If we wanted every project to go on forever, using 100% only Oasis Digital staff, then we would make sure to never help any customer with any hire at all. But that is completely unrealistic. There are millions of software developers (and sometimes it seems almost as many software development firms). We are in a competitive market. Our customers have a choice, they can do business with us or have someone else write their software instead, regardless of whether we help them with hiring.

Therefore, the real question is whether to be greedy for the short term, or visionary for the long term. We choose the latter. Our policy is that we are happy to help our customers hire direct staff. We believe that this will, in the long term, lead to success for our customers and for Oasis Digital.

We assist with hiring in 3 ways.

Direct Assistance with Hiring and Onboarding

At Oasis Digital we have a somewhat unusual hiring process: in addition to the usual interviews by phone, in person, and otherwise, we ask (and pay) prospective developers to write some code for us based on a short specification. The resulting code, and conversation about it, provides a great opportunity to get to know someone (and to assess the results they will create) very quickly before hiring them. We assess technical skills as well as teamwork / cultural fit. We have a high bar to hiring and a defined process to reach that bar. At the same time, our process respects potential employees, by not asking for sample work to be done for free.

A good hire, though, is not the finish line. It is the starting line! During the first months of a new developer’s work we have an onboarding process in which the new developer sets up a work environment (mostly by referring to project documentation), then implements tiny changes, then small changes, then medium changes, then finally can begin work on large, important tasks. Throughout these initial months, the new developer works with more frequent collaboration and code/change review than will be needed in the long run. We have found that with our hiring and onboarding processes (described above at a very high level), we have a high success rate.

The first and most direct way we can assist our customers, therefore, is to simply execute these processes for them: assist with interviews, sample projects, and lead the onboarding effort.

Same Standards

When working with a mixed team consisting of Oasis Digital staff as well as customer staff, we hold everyone to the same high standards.

I’ve seen teams that work the other way: accepting a lower standard of work from a customer’s internal staff. It ends badly. We would rather lose a customer, than ship bad software. Our reputation matters more than the next dollar.

Pass It On

Lastly, our processes aren’t a deep secret; the key is not the ideas, it is the execution. We are happy to teach our way of working to customers (and everyone else, in blog posts and talks). Even at the price (free to read, normal billing for customer work) it is a hard sell, though: hiring is often deeply embedded in how companies work.

Stay Tuned

I’ve summarized here at a high level; expect future posts and talks with many more details.

We’re hiring a NON-developer

Over at Oasis Digital, we are in the (job) market for our first non-developer, non-project-manager, non-service-delivery-focussed full-time team member. The job post is on the Oasis Digital site; I thought I’d write about a bit more about the story here.

Part of the challenge of running a software development consulting firm, is the tension between:

1) Focus on Customers You Have Now

Some firms are so service-delivery centric (#1 above), that they don’t bother to address the rest of the world, i.e. all of the not-customers-yet. Small consulting firms are especially vulnerable to this, and as a result they stay small.

Historically, Oasis Digital started in this category, with long-term projects, satisfied customers, and slow/organic growth. The shape of our organization reflects this: everyone, including me, works mostly on creating software for customers.

2) Focus on Customers You Want to Have Later

Some other software firms are highly marketing-driven, to the extent that their service delivery value proposition (in terms of design, features, and quality per customer dollar spent) suffers. Large technical service firms are especially vulnerable to this. Anyone who has been around the block a few times, has experienced the unique joy of a very large sum spent with a very large vendor to deliver sufficient, but disproportionately expensive, results.

Turning Up the Heat

Our goal and challenge now, is add a bit of focus on future customers (which is to say, marketing, community participation, etc.), without losing our service delivery mojo. To move things in that direction, we need at least one person who is not occupied by service delivery… hence our upcoming hire.

We will also quite likely start another software product or SaaS business along the way. (I’ve been there before, of course, and did plenty of hiring and business development work along the way.)

Some Things Won’t Change

Since inception, our sales process has been decidedly non-traditonal:

  • We encourage potential customers to meet their needs with off-the-shelf software, not custom software, if possible.
  • We don’t have commissioned salespeople. Nor commissioned pretend-they-aren’t-salespeople.
  • Most of our team is, and will always be, all about delivering great results for our customers.

I’d love to receive feedback on this, from those of you with experience in this particular transition. You know how to reach me.

Update: We’ve filled the need for that job post, with a combination of a few people, rather than a single jack-of-all-trades as I had planned.