Maximum productivity when you are the bottleneck

Scenarios that make you scarce

Software development productivity is the ratio of desirable high-quality software to money spent. With this meaning, productivity is aligned with quality and effectiveness: it only counts creating “the right software” and encompasses creating “the software right”. Productivity is more important than efficiency (the lack of waste), as occasionally a bit of inefficiency pays off.

In the quest for some combination of these values, project management methodologies or practitioners generally assume that members of a team have approximately similar scarcity/availability/cost. But sometimes, you may find yourself more scarce than a group you are working with:

  • You are leading at a high “fan-out” – one person leading a team of many.
  • You are leading a team expected (for good reasons or bad) to expand or turn over significantly.
  • You are much more senior than the rest of your team.
  • You are in an expensive city or country, other team members are in a less expensive locale.
  • You are a “hired gun” consultant, brought in at great cost, expected to “move the needle”.
  • You are a professional developer responsible for mentoring, teaching, and getting results from a group of interns.
  • You have a communication advantage with the customers of a product or project; perhaps you speak the customer’s language more fluently than others on your team.
  • You are the only team member with extensive and relevant experience to the problem at hand.

Productivity wins

In a competitive world, productivity wins. These techniques can help optimize productivity to get the greatest possible results when working at a scarcity gradient. Such a gradient is often accompanied by variable or unpredictable availability of some or all team members; and many of these techniques optimize results for both constraints.

Every idea here comes from experiences at Oasis Digital, on those projects which had the highest efficiency and effectiveness.

Key principles

As a scarce developer:

  • Act as a leader, as an expert, as a coach, and as a pseudo-customer.
  • Facilitate, support, and guide less scarce developers.
  • Work in a scalable way; each unit of your work enables many units of others’ work.
  • Scale across space (multiple other developers) and time (present and future developers).
  • Each hour of your work should enable N hours of others’ work.

Techniques

Multicast, mostly

Avoid 1-1 email; avoid 1-1 IM; avoid 1-1 Skype or phone. Avoiding writing text for only one person; avoid creating a screen video useful only once for one person. Instead, spend as much of your communication effort as possible, communicating to a group of current and future developers.

Avoiding one-on-one can be very challenging. Depending on the culture of an organization or community, you might find that the less scarce developers strongly prefer one-on-one communication. You must resist this to work efficiently.

Generally skip one-on-one communication unless it is for a personal or HR matter… but for those critical conversations, spend the time you’ve saved up and speak, live. In-person or (for remote) with video+audio conferencing.

It is key to remember the “mostly” part. Have occasional one-on-one meetings with each team member. Because of the scarcity factor, do so much less often than the typical recommendation of once per week per team member.

Write documents

When you write, prefer writing in documents with ongoing usefulness. Prefer writing in shared documents, not in email. Prefer writing text anyone could read, not notes made only for yourself. Text written in this way is useful for many developers, both at the present and in the future. Name the documents well – not just “Notes” or other nondescript name. If you type slowly, consider voice dictation. (Reasonably good and immediately available voice dictation is available in Google Docs!)

Record screen videos

Screen videos are are especially useful to explain to many developers, across space and time, how something works. They are also very fast to produce.

Screen videos made for fellow developers have a lower quality bar than those made for customers. It is still important to have clear audio (neither too loud nor too quiet, and not full of background/fan noise), but the pace and flow can be somewhat rough, and a “first cut” is almost always good enough. Still, always name screen videos well – make sure people can find them with a reasonable search, and can understand the topic by reading the name.

Small, topical pieces

Resist the urge to ramble for 30 minutes explaining everything in one video, or to spend 4 hours writing up everything in one document. Divide your effort into separate topics, and make a separate document or video for each. Pay attention to the names, and name each piece in a way that reflects the topics in it.

By doing this, the material you make is much more likely to be useful beyond the need at hand. For example, an explanation of a database schema should not be in the same video/doc as an explanation of a report that uses the schema; if is, then most likely a developer working on something other than reports, would not watch/read it. By splitting that to two assets, each can be reused independently.

Meetings

Minimize the need for schedule synchronization – either work entirely asynchronously, or with perhaps one set meeting time per day. To maximize use of your scarce availability, decline requests to add more meetings or to set meetings for specific topics. Instead meet on a regular schedule and cover whatever topics are most beneficial to keep the group moving, each time.

Even though they are called meetings, don’t just meet. Work together, as a group, on the critical issues of the day. This is the most powerful mechanism to get a team in sync and pulling together.

Make room for social time

Whether your meetings are in-person or online, you are scarce and have limited availability… but meetings don’t need you the whole time. Leave meeting when you have to, and encourage the rest of the team to keep talking and working afterward.

Record meetings

It is unfeasible to ensure every relevant person is at each meeting. Specifically, some relevant people may not even be on the team yet, or maybe on vacation, or maybe unavailable for random reasons.

Therefore many development sessions and meetings should be recorded. This is typically inexpensive and easy to do with modern technology. The recordings can then land in a suitable place for developers who missed the meeting a need to keep up to review. Many will find they can do so in much less time than the meeting originally took.

Fast answers to small questions

Developers who are first learning about a project (or a part of a project they haven’t touched before) are prone to get “stuck”. It is thus helpful to answer small questions fast, if you are available to do so without major disruption.

Keep the “ball rolling”. Respond quickly, keep things moving. This can be tough to get used to, for anyone who has mostly worked alone in the past. Be careful to balance between good responsiveness and your own focus. For example, you might work steadily on their own direct tasks and pause once per hour to “come up for air” and respond to any pending quick needs.

Delegate this quick answering process as much as possible.

Batch larger questions and discussion

On the other hand, resist the urge to answer a larger question right away. You might accidentally:

Write a detailed explanation in email or IM, where it will get lost.
Write an explanation as a 1-1 answer to one person, where it won’t benefit anyone else.
Write an explanation where a screen video would have been better.
Combine an explanation of many parts of a system, making it not-very-reusable.

Instead, attack larger questions en masse, at the next meeting already scheduled for a convenient time.

Here is a rubric to route various sizes of questions and issues:

SizeHow to Handle
SmallAnswer or do immediately
MediumBatch up, prioritize, execute every day
LargeBreak up, analyze, find ways to make bits of progress to allow others to move forward; prioritize to attack big things with enough intensity to complete them.

Delegate

Delegate most of the small or easy questions. After you have answered or explained a certain area once, expect and enable new or junior team members to first get help from the less junior team members before seeking your scarce availability.

Delegate in the right direction

There is a surprising tendency for junior developers on a team to delegate work “upward”. Respectfully decline. Only for narrow reasons (such as requiring access to sensitive systems or data) should a task be delegated upward, toward a person in scarce supply. Most of them time, the real need is for more knowledge transfer, guidance, or key decisions – so do that, but don’t let actual work be delegated upward.

Aggregate, stochastic project management

Take an aggregate, stochastic approach to project management. Don’t attempt to micro-manage; don’t worry much about who works on what, when. Instead:

  • Periodically write/groom a set of items that ready be done. These items should have enough explanation that a developer can begin work. This is the to-do bucket.
  • Developers can grab items from the to-do bucket and work on them.
  • No developer “owns” an item, or reserves it – if a task isn’t getting done, someone else can grab it and do it.
  • Sort items between the to-do bucket and backlog.
  • Work ahead: elaborate on items that will be worked on soon, don’t wait and make anyone “pull” the elaboration from you.
  • Depending on the reason for the scarcity gradient (if it is based on a globally distributed team), there might be no common “weekend” across a team, no common working hours, no single idea of day and night, no common holiday schedule, and wildly unpredictable availability for each developer. Accept this.
  • Embrace chaos, and work in a way that relies on the aggregate productivity. Let it all average out over time.

Place two bets

This is an extreme technique, suitable only for those able to think abstractly in terms of bets rather than concretely in terms of estimates, commitments, and dates. If the scarcity gradient is sufficient, and a piece of work is vital, consider placing two bets instead of one.

To place two bets, assign two different team members, or even two subteams, to take on the same work in parallel. If either one succeeds, work can be merged/shipped in your project wins. The only way to fail is for both independent efforts to fail.

This idea most sharply separates the ideas of productivity and efficiency. Sometimes the most productive thing is to intentionally be inefficient, to intentionally create a bit of waste, if the scarcity gradient is sufficient that the benefit is high and the waste is bearable.

Retrospect

Most agile processes have a notion of a retrospective meeting from time to time to discuss what is working, and what areas are struggle.

Don’t skip these just because your time is scarce. Make time to do retrospectives, and in particular you may be called on to explain why things are done a certain way. Don’t be shy, be willing to say forthrightly: we are doing X instead of Y because my scarce availability makes this the most productive overall solution.

Project Setup

For the whole team to work efficiently, the least scarce developers must be able to operate their computers and development tooling effectively with little assistance. Assistance would require 1-on-1 help, which isn’t workable as explained earlier.

Therefore, do not provide one-on-one assistance for project set up. Instead, create or update documents about how to perform project set up. New developers join, receive the documents, and work on their project set up. If developers have difficulties or questions for things not covered in the documents, bring up the document and edit it together live to answer the person’s question. Don’t answer their questions by providing extra info that doesn’t land in the document.

The project set up process can also verify that a hiring or project assignment process has produced workable results. Developers well-qualified to work on a project should be able to perform project set up without handholding. When you are on the scarce side of the project, keep this in mind; you can demonstrate your suitability for a project by working hard, getting through project set up, asking the questions that have to be asked, but figuring out most of it yourself.

Daily / Continuous Process

Combine many of the techniques above into a daily (or really, continuous) process to get the most overall value out of your’s relatively scarce availability:

Source Control

  • Review any new commits. Review them in a time efficient way; quick comments can be typed, detailed reviews are best done via video or as part of an en-masse live discussion.
  • Fix any source control problems. If someone did something wrong (like pushing unreviewed or incomplete work to master), fix this right away, do not let it sit.
  • Merge anything that is ready to be merged – so every open branch/commit should be either merged, or have at least some comment pointing its developer in the right direction.

Issue Tracker

  • Look over the current milestone.
  • Notice anything not up to date – update it, or have the person who did the work update it. It should be possible for an observer to know the status of a project by looking at the contents of the issue tracker.
  • Look over any “triage” items and place them in a milestone.
  • Check that the current in-process and to-do work is still a reasonable set of priorities, adjust if needed.

Chat / Email

  • Discuss project matters in batched meetings or chat, rather than in painful, expensive, tedious email threads. If it’s too big for chat, it probably should be discussed in the next regular meeting.
  • Look for opportunities to share the load of this answering; if one team member could have answered another’s questions, remind them to do so.
  • Answer small questions quickly. Letting them sit greatly slows down a project. At the same time, many questions should be pushed out of email.
  • If someone in email triggers a need for longer term work, it should probably be in the issue tracker.
  • If technical discussions wander in to direct email, move them immediately back to shared discussion.
  • Look past the immediate questions (which can be poorly written, especially with developers who don’t know a lot about a project yet) and answer the underlying questions.
  • If some information appears in chat or email that should be in a document, copy and paste it there, or delegate someone to do so.

Transparency builds trust

The common theme running through most of the advice here is to bias strongly toward transparency. Team meetings, where you can see each other, and everyone’s work is in the open. Issue trackers visible to everyone, where everyone can see who has grabbed what work, and what they have done so far. Code review during group working sessions, where things that need improvement are candidly discussed. Nearly all communication dragged away from 1-on-1 chat email etc. and instead in the open.

This transparency builds trust, much more rapidly than would be expected without those practices. This way of working maximizes productivity not only toward producing working software, but also toward improving candid human relationships.

That’s enough words for today’s summary. Some of these topics warrant more elaboration; stay tuned.

Popular your issue tracker at scale

Sometimes when working on a project at work, we find out about a pile of features or changes needed. This can happen at the beginning of a project, at the start of the major initiative, after deploying a project (which triggers much user feedback), etc. Sometimes we have so much to absorb and divvy up into issue tracker items, that the logistics of doing so are painful.

Just thinking through and writing down 50-100 issues (or more!) is too tedious for one person to get through quickly. To divide up this work (of describing a bunch of issues in enough depth someone could work on them), I’ve come up with the following approach.

First, I jot down a list of all the areas of the system where there are new issues to enter. This forms an outline of areas that have issues, I don’t even attempt to make an entry for every likely issue.

Second, I record one or more videos, showing the screen of the system I want to add issues for, alternating back and forth with code. As I go, I describe each problem/opportunity/fix, that should become an issue. Depending on whether the new issues are closely related to existing ones, sometimes this includes bringing up the issue tracker (Jira, etc) also, talking through existing items about work remaining on them.  Sometimes something that first seems like a new issue, is really just a refinement of the success criteria of an already known issue.

Having spent potentially quite a while just describing issues (there have been times when this goes on for over an hour), I hand over the recording(s) to a relatively new person on the team, who will go through and translate this rapid-fire description into a set of items. Typically it’s fastest for the person to do that not by directly entering the items, but by just typing the candidate issues into a document. (If the list is big enough, it can pay off to have a transcriber handle the first pass – turn the words from the video/audio, into text.)

Finally, that initial rough list of candidate issue, goes to the project leader(s) of the project in question, to clean up, refine, review, approve. Then someone copies the approved text into the issue tracker.

Admittedly this is not a complex process, hardly worthy of a blog post. But someone once asked me how we successfully enter so much detail into so many items on complex projects – and here is the answer. Entering all that really does pay off. It is much more plausible to delegate work if you have described it as thoroughly as you can.

Apply Enough Force

I have written and spoken many times about the importance of getting started. This is a key idea in agile software development (or for that matter, to many other kinds of work). If you refuse to start projects until you fully understand how they can be completed, you will miss many valuable opportunities.

When in doubt, start. But start with a modest budget, because there is an opposite and equally unavoidable fact of life. To complete a project of any size (one person-hour, or 1000 person-years), a sufficient effort must be mustered. Sometimes some of my fellow agile lists talk about working on a huge project one bite at a time, planning just the next few iterations, just the next scrum, whatever. This is all good, this is all agile. But it is important not to ignore the size of the thing you’re trying to accomplish.

To accomplish something of size (fast enough for it to be valuable) requires an approximate overall understanding of its size, then committing sufficient “resources” of all kinds:

  • Money
  • Time
  • Developers
  • Project managers
  • Marketers
  • Salespeople
  • Physical facilities
  • Lawyers
  • The list goes on

If it is not possible to commit enough firepower to the job at hand, then the right time to cancel or rescope an effort is at the beginning, not halfway through (the infamous point at which you are 90% of the way done but have 90% of the work still remaining). Keep that in mind next time a project is proposed. Understand the resources that can be committed, the overall scope of the project, and if they don’t fit, pick a different project.

 

Martin Fowler defines Software Architecture

Yesterday I saw the following video of a brief talk by Martin Fowler, in which he defines software architecture. I have grumbled about that term myself, in that firstly it is often ill-defined, and secondly it can be pretentious. I have sometimes defined software architecture as “high level design”, or as the design of systems complex enough to warrant substantial input from someone who is been around the block many times.

Fowler’s definition is crisp: Software architecture is those decisions which are both important and hard to change. This means it includes things like the choice of programming language, something architects sometimes gloss over or dismiss. Both aspects land squarely on the economics of software development. Said another way, software architecture is those decisions which, if made poorly, will make a project either succeed or fail, in a needlessly expensive way.

This connection resonates deeply with me. I have often talked about the economics of software development, the economic impacts of tool choices, the economic impacts of process selection, platform selection, etc. But Fowler’s talk made a connection I had never set out loud at least: the economic concerns, not only the technical ones, are software architecture.

 

Record Development Meetings with Hangouts On Air

At Oasis Digital, our project teams coordinate in numerous ways, sometimes by meeting “live”. At these meetings we discuss, we plan, we model, we code, we review. For teams/projects where everyone is physically present at headquarters, this is readily done by plugging in to a large TV re-purposed as a computer monitor. (Proposed bumper sticker: “My other monitor is a 60 inch Samsung”.)

For some teams and projects, though, not everyone is at our St. Louis office. Some of our developers are around the US, and occasionally projects include developers around the world. Therefore, some of our development meetings take place online, using a variety of meeting tools. Mostly these meetings are useful only at the moment they occur, but it is useful to record them for developers who are unable to attend “live”.

There are countless tools to choose from. We have had particularly good results with Google Hangouts. Unfortunately normal Hangouts has no recording feature – the recording capability is available only in the form of “Hangsout On Air”, which is more typically used to “broadcast” an online meeting. Google Apps customers, though, can use Hangouts on Air, decline to broadcast to anyone at all, then share the recording privately.

The process is not obvious. Here is how to use Hangouts On Air to privately record and share online meetings:

  1. Consider performing all of these steps in a dedicated browser. For example, use Chrome for Hangouts, while using Firefox for other browsing. This reduces the opportunity for Hangouts (which is irritatingly bound to the browser) to interfere with other use of your computer.
  2. Log in to YouTube. Use the Google account selector in the upper right to select your company Google Apps account, if you are logged in an more than one account.
  3. If you don’t already have a YouTube “Channel”, which you probably don’t, click to create one. It is harmless, since for this Hangouts On Air purpose you don’t need to every actually publish anything to your “channel”.
  4. Click “Video Manager”. The button for this is not in the most obvious place (the left navigation), rather it is along the top, and small.
  5. Click Live Events.
  6. Click to create a new live event.
  7. Give your meeting event a name.
  8. The description, keywords, etc. do not matter, since you won’t be publishing.
  9. Adjust the setting to make it Private. This is a vital step, don’t miss it.
  10. Don’t share it with anyone yet. That comes later.
  11. Choose the default Quick settings, there is nothing important to adjust in the Custom settings.
  12. Click to start the Hangout On Air. This will launch the same Hangouts app as any other Hangout – almost. As of December 2014, the features for Hangouts on Air are somewhat behind the normals Hangouts app in features and screen layout.
  13. Click the invitation/add button in the Hangouts controls at the top of the Hangouts window.
  14. Copy the link. Paste it in group chat so that other developers (meeting attendees) can join.
  15. Wait for everyone to get in the meeting. Adjust microphones, webcams, speakere, etc.
  16. Remind everyone to turn on their webcam and keep it on if possible. This provides a much better experience for anyone watching the video later – they get more of a sense of having “been there” for the meeting.
  17. Discuss anything that isn’t part of the project in question, etc. while waiting for everyone to be ready.
  18. Click Start Broadcast. This will start the “broadcast” to noone (because this HoA is private and not shared with anyone), and more importantly, start the recording.
  19. During the meeting, remember that whatever is visible in the Hangouts app window, will be recorded. Adjust this regularly to keep it relevant to the discussion. Often the default behavior of switching to whoever is speaking, works well.
  20. Remember that Hangouts On Air recordings are limited to pseudo-high-def, “720p”. Keep screen share window sizes moderate, so that text can be easily read by others in the meeting and on the recording. Sharing a full-screen at 1920×1080 or higher resolution will yield unreadable results.
  21. When the meeting is done, Click Stop Broadcast.
  22. Back in the Youtube Video manager, paste in the email addresses of those with whom the recording should be shared.
  23. Don’t trust Google/YouTube to maintain privacy settings over the long term – leave the recording there until it has outlived its usefulness (a week or so in a typical project, maybe longer in a slow-paced project) then delete it.

Ouch, a 22-step process. That is the price of using Hangouts On Air in for an atypical purpose. The payoffs:

  • HoA works well across all major platforms, including Windows, Mac, Linux, Android, Chromebook, and more. This is the hardest thing to replace, many other meeting tools don’t come close.
  • The recording is immediately available privately on YouTube, all transcoding and hosting is seamless and effortless.
  • Google / YouTube is infinitely scalable, for all practical purposes.
  • No cost, for either the tools or the hosting.
  • Part of software (Chrome) that most developers have already installed.

 

Bits are Free, People are Valuable

A few days ago, I caught myself thinking about whether to save some images and video; whether the likely future value of those megabytes would be greater or lesser than the cost of storage. This is a sort of thought that was important and valuable… a couple of decades ago.

Bits are Free

Today, and at least for the last decade, bits are so absurdly cheap that they can be considered free, compared to the time and energy of people. Storing gigabytes or terabytes because they might come in handy is not just for government agencies, it makes sense for all of us in our daily work.

Waste bits to save time.

Waste bits to help people.

People matter, bits are free.

Bits are Free at Work

Here are some ways I waste bits at work:

  • We often gather a few people working on project to meet or code together; it is very easy to start a video or screen video recording of the work. That recording can then be shared with anyone on the project who wasn’t present.
  • We record screenshots or videos of the future in progress, and send it to a customer. Yes, we could wait and present to them “live” using WebEx or whatever. But it is cheaper to waste the bits and conserve human coordination effort.
  • If I have something complex to explain to someone, I can do it in person, and I can do it on the phone, and I can write lots of text. But if I already know them well enough to partly anticipate their questions, I will start a video recording and explain something using voice and white board. The bits are cheaper than the coordination to work “live”. The bits are cheaper than asking them to figure it out themselves.
  • Driving down the road, it is unsafe to text, or read, or write, or (I am looking at you, morning commuters…) apply makeup. But while the numbers are unclear, we have a broad assumption that merely talking with someone while driving down the road is OK. I sometimes make use of traffic time, and burn some bits, by recording audio about some problem to be solved or other matter. The bits are free, who cares if this uses 1000x as much data as sending an email?

Wasting bits can grow somewhat extreme. In the first example above, I described a screen video of developers working together, recorded for the benefit from other developers. You might imagine this as a high-ceremony process, done on important occasions about important code. But bits are free – so we sometimes record such video even if no developer will ever pay attention to it – the value is there even if just one developer maybe lets it play in the background while they work on something else – much like by working in the same room as other people, it is common to pick up some important bit in the background. If one developer learns one small thing to make their work better, that is more valuable than 400 MB of video storage space.

Bits are Free at Home

Here are some ways I waste bits at home:

  • When I take photographs, I take a lot of photographs. One might come in handy. Who cares if 95% of them are bad and 90% of them are useless?
  • Why do cameras have settings other than maximum resolution? Bits are free.
  • Nearly 100% of paperwork I get in the mail, other than marketing, I scan, OCR, and keep in a searchable archive. The disk space for this costs almost nothing. The time saved deciding whether to keep each item, and how to file each item, is irreplaceable. I probably only ever look at 10% of what is scanned, but who cares?
  • If my kids are doing something even vaguely interesting, I try hard to remember to take photos and record video. Looking back at when they were very young, snippets of video we have (from before every phone had a decent video camera) are priceless. I can’t reach back and record more of those, but I can easily record things now that might be fun in the future. Who cares if 95% of that video no-one ever looks at? If I ever need to go through it, I can do it then. The storage space in the meantime doesn’t matter.
  • If I need to look at a model number, serial number, etc. of anything around the house, I snap a photo of it. Then I can look back at the photo anytime from anywhere. Yes, it is absurd to store 3,000,000,000 bytes of JPG rather than 20 bytes of serial number. But they both round to zero.

How Free Can Bits Get?

I expect this tradeoff will shift even more exponentially in the future. In a couple of decades, perhaps:

  • We will record 5 petabyte ultra beyond-HD holographic recordings of insignificant activities just in case someone wants to watch them.
  • We will run video and audio recording of our lives 24 x 7, to be indexed just in case anyone ever wants to look back at it.
  • Future cameras won’t even come with an on-off switch, and will instead record continuously for the lifetime of the camera.