Maximum productivity when you are the bottleneck

Scenarios that make you scarce

Software development productivity is the ratio of desirable high-quality software to money spent. With this meaning, productivity is aligned with quality and effectiveness: it only counts creating “the right software” and encompasses creating “the software right”. Productivity is more important than efficiency (the lack of waste), as occasionally a bit of inefficiency pays off.

In the quest for some combination of these values, project management methodologies or practitioners generally assume that members of a team have approximately similar scarcity/availability/cost. But sometimes, you may find yourself more scarce than a group you are working with:

  • You are leading at a high “fan-out” – one person leading a team of many.
  • You are leading a team expected (for good reasons or bad) to expand or turn over significantly.
  • You are much more senior than the rest of your team.
  • You are in an expensive city or country, other team members are in a less expensive locale.
  • You are a “hired gun” consultant, brought in at great cost, expected to “move the needle”.
  • You are a professional developer responsible for mentoring, teaching, and getting results from a group of interns.
  • You have a communication advantage with the customers of a product or project; perhaps you speak the customer’s language more fluently than others on your team.
  • You are the only team member with extensive and relevant experience to the problem at hand.

Productivity wins

In a competitive world, productivity wins. These techniques can help optimize productivity to get the greatest possible results when working at a scarcity gradient. Such a gradient is often accompanied by variable or unpredictable availability of some or all team members; and many of these techniques optimize results for both constraints.

Every idea here comes from experiences at Oasis Digital, on those projects which had the highest efficiency and effectiveness.

Key principles

As a scarce developer:

  • Act as a leader, as an expert, as a coach, and as a pseudo-customer.
  • Facilitate, support, and guide less scarce developers.
  • Work in a scalable way; each unit of your work enables many units of others’ work.
  • Scale across space (multiple other developers) and time (present and future developers).
  • Each hour of your work should enable N hours of others’ work.

Techniques

Multicast, mostly

Avoid 1-1 email; avoid 1-1 IM; avoid 1-1 Skype or phone. Avoiding writing text for only one person; avoid creating a screen video useful only once for one person. Instead, spend as much of your communication effort as possible, communicating to a group of current and future developers.

Avoiding one-on-one can be very challenging. Depending on the culture of an organization or community, you might find that the less scarce developers strongly prefer one-on-one communication. You must resist this to work efficiently.

Generally skip one-on-one communication unless it is for a personal or HR matter… but for those critical conversations, spend the time you’ve saved up and speak, live. In-person or (for remote) with video+audio conferencing.

It is key to remember the “mostly” part. Have occasional one-on-one meetings with each team member. Because of the scarcity factor, do so much less often than the typical recommendation of once per week per team member.

Write documents

When you write, prefer writing in documents with ongoing usefulness. Prefer writing in shared documents, not in email. Prefer writing text anyone could read, not notes made only for yourself. Text written in this way is useful for many developers, both at the present and in the future. Name the documents well – not just “Notes” or other nondescript name. If you type slowly, consider voice dictation. (Reasonably good and immediately available voice dictation is available in Google Docs!)

Record screen videos

Screen videos are are especially useful to explain to many developers, across space and time, how something works. They are also very fast to produce.

Screen videos made for fellow developers have a lower quality bar than those made for customers. It is still important to have clear audio (neither too loud nor too quiet, and not full of background/fan noise), but the pace and flow can be somewhat rough, and a “first cut” is almost always good enough. Still, always name screen videos well – make sure people can find them with a reasonable search, and can understand the topic by reading the name.

Small, topical pieces

Resist the urge to ramble for 30 minutes explaining everything in one video, or to spend 4 hours writing up everything in one document. Divide your effort into separate topics, and make a separate document or video for each. Pay attention to the names, and name each piece in a way that reflects the topics in it.

By doing this, the material you make is much more likely to be useful beyond the need at hand. For example, an explanation of a database schema should not be in the same video/doc as an explanation of a report that uses the schema; if is, then most likely a developer working on something other than reports, would not watch/read it. By splitting that to two assets, each can be reused independently.

Meetings

Minimize the need for schedule synchronization – either work entirely asynchronously, or with perhaps one set meeting time per day. To maximize use of your scarce availability, decline requests to add more meetings or to set meetings for specific topics. Instead meet on a regular schedule and cover whatever topics are most beneficial to keep the group moving, each time.

Even though they are called meetings, don’t just meet. Work together, as a group, on the critical issues of the day. This is the most powerful mechanism to get a team in sync and pulling together.

Make room for social time

Whether your meetings are in-person or online, you are scarce and have limited availability… but meetings don’t need you the whole time. Leave meeting when you have to, and encourage the rest of the team to keep talking and working afterward.

Record meetings

It is unfeasible to ensure every relevant person is at each meeting. Specifically, some relevant people may not even be on the team yet, or maybe on vacation, or maybe unavailable for random reasons.

Therefore many development sessions and meetings should be recorded. This is typically inexpensive and easy to do with modern technology. The recordings can then land in a suitable place for developers who missed the meeting a need to keep up to review. Many will find they can do so in much less time than the meeting originally took.

Fast answers to small questions

Developers who are first learning about a project (or a part of a project they haven’t touched before) are prone to get “stuck”. It is thus helpful to answer small questions fast, if you are available to do so without major disruption.

Keep the “ball rolling”. Respond quickly, keep things moving. This can be tough to get used to, for anyone who has mostly worked alone in the past. Be careful to balance between good responsiveness and your own focus. For example, you might work steadily on their own direct tasks and pause once per hour to “come up for air” and respond to any pending quick needs.

Delegate this quick answering process as much as possible.

Batch larger questions and discussion

On the other hand, resist the urge to answer a larger question right away. You might accidentally:

Write a detailed explanation in email or IM, where it will get lost.
Write an explanation as a 1-1 answer to one person, where it won’t benefit anyone else.
Write an explanation where a screen video would have been better.
Combine an explanation of many parts of a system, making it not-very-reusable.

Instead, attack larger questions en masse, at the next meeting already scheduled for a convenient time.

Here is a rubric to route various sizes of questions and issues:

SizeHow to Handle
SmallAnswer or do immediately
MediumBatch up, prioritize, execute every day
LargeBreak up, analyze, find ways to make bits of progress to allow others to move forward; prioritize to attack big things with enough intensity to complete them.

Delegate

Delegate most of the small or easy questions. After you have answered or explained a certain area once, expect and enable new or junior team members to first get help from the less junior team members before seeking your scarce availability.

Delegate in the right direction

There is a surprising tendency for junior developers on a team to delegate work “upward”. Respectfully decline. Only for narrow reasons (such as requiring access to sensitive systems or data) should a task be delegated upward, toward a person in scarce supply. Most of them time, the real need is for more knowledge transfer, guidance, or key decisions – so do that, but don’t let actual work be delegated upward.

Aggregate, stochastic project management

Take an aggregate, stochastic approach to project management. Don’t attempt to micro-manage; don’t worry much about who works on what, when. Instead:

  • Periodically write/groom a set of items that ready be done. These items should have enough explanation that a developer can begin work. This is the to-do bucket.
  • Developers can grab items from the to-do bucket and work on them.
  • No developer “owns” an item, or reserves it – if a task isn’t getting done, someone else can grab it and do it.
  • Sort items between the to-do bucket and backlog.
  • Work ahead: elaborate on items that will be worked on soon, don’t wait and make anyone “pull” the elaboration from you.
  • Depending on the reason for the scarcity gradient (if it is based on a globally distributed team), there might be no common “weekend” across a team, no common working hours, no single idea of day and night, no common holiday schedule, and wildly unpredictable availability for each developer. Accept this.
  • Embrace chaos, and work in a way that relies on the aggregate productivity. Let it all average out over time.

Place two bets

This is an extreme technique, suitable only for those able to think abstractly in terms of bets rather than concretely in terms of estimates, commitments, and dates. If the scarcity gradient is sufficient, and a piece of work is vital, consider placing two bets instead of one.

To place two bets, assign two different team members, or even two subteams, to take on the same work in parallel. If either one succeeds, work can be merged/shipped in your project wins. The only way to fail is for both independent efforts to fail.

This idea most sharply separates the ideas of productivity and efficiency. Sometimes the most productive thing is to intentionally be inefficient, to intentionally create a bit of waste, if the scarcity gradient is sufficient that the benefit is high and the waste is bearable.

Retrospect

Most agile processes have a notion of a retrospective meeting from time to time to discuss what is working, and what areas are struggle.

Don’t skip these just because your time is scarce. Make time to do retrospectives, and in particular you may be called on to explain why things are done a certain way. Don’t be shy, be willing to say forthrightly: we are doing X instead of Y because my scarce availability makes this the most productive overall solution.

Project Setup

For the whole team to work efficiently, the least scarce developers must be able to operate their computers and development tooling effectively with little assistance. Assistance would require 1-on-1 help, which isn’t workable as explained earlier.

Therefore, do not provide one-on-one assistance for project set up. Instead, create or update documents about how to perform project set up. New developers join, receive the documents, and work on their project set up. If developers have difficulties or questions for things not covered in the documents, bring up the document and edit it together live to answer the person’s question. Don’t answer their questions by providing extra info that doesn’t land in the document.

The project set up process can also verify that a hiring or project assignment process has produced workable results. Developers well-qualified to work on a project should be able to perform project set up without handholding. When you are on the scarce side of the project, keep this in mind; you can demonstrate your suitability for a project by working hard, getting through project set up, asking the questions that have to be asked, but figuring out most of it yourself.

Daily / Continuous Process

Combine many of the techniques above into a daily (or really, continuous) process to get the most overall value out of your’s relatively scarce availability:

Source Control

  • Review any new commits. Review them in a time efficient way; quick comments can be typed, detailed reviews are best done via video or as part of an en-masse live discussion.
  • Fix any source control problems. If someone did something wrong (like pushing unreviewed or incomplete work to master), fix this right away, do not let it sit.
  • Merge anything that is ready to be merged – so every open branch/commit should be either merged, or have at least some comment pointing its developer in the right direction.

Issue Tracker

  • Look over the current milestone.
  • Notice anything not up to date – update it, or have the person who did the work update it. It should be possible for an observer to know the status of a project by looking at the contents of the issue tracker.
  • Look over any “triage” items and place them in a milestone.
  • Check that the current in-process and to-do work is still a reasonable set of priorities, adjust if needed.

Chat / Email

  • Discuss project matters in batched meetings or chat, rather than in painful, expensive, tedious email threads. If it’s too big for chat, it probably should be discussed in the next regular meeting.
  • Look for opportunities to share the load of this answering; if one team member could have answered another’s questions, remind them to do so.
  • Answer small questions quickly. Letting them sit greatly slows down a project. At the same time, many questions should be pushed out of email.
  • If someone in email triggers a need for longer term work, it should probably be in the issue tracker.
  • If technical discussions wander in to direct email, move them immediately back to shared discussion.
  • Look past the immediate questions (which can be poorly written, especially with developers who don’t know a lot about a project yet) and answer the underlying questions.
  • If some information appears in chat or email that should be in a document, copy and paste it there, or delegate someone to do so.

Transparency builds trust

The common theme running through most of the advice here is to bias strongly toward transparency. Team meetings, where you can see each other, and everyone’s work is in the open. Issue trackers visible to everyone, where everyone can see who has grabbed what work, and what they have done so far. Code review during group working sessions, where things that need improvement are candidly discussed. Nearly all communication dragged away from 1-on-1 chat email etc. and instead in the open.

This transparency builds trust, much more rapidly than would be expected without those practices. This way of working maximizes productivity not only toward producing working software, but also toward improving candid human relationships.

That’s enough words for today’s summary. Some of these topics warrant more elaboration; stay tuned.

Angular dependency injection: why?

At work we teach and consult on various topics, notably (for this post) Angular. We are often asked why Angular has and heavily uses dependency injection. Here are my answers to this question; I haven’t had a chance to compare notes with the rest of our Angular expert team, so here it is, on my personal blog. (However, special thanks to Paul Spears, who helped clarify these ideas.)

Dependency injection has a considerable complexity cost, so it’s important to have good reasons to add this complexity to a framework or application. There are an unlimited number of potential features and patterns to follow, so the default for any particular feature or pattern must be “no”. Moreover, popular Angular alternatives (like React) thrive without a dependency injection system, constituting an “existence proof” that it is not a necessity, but rather a choice.

There are a variety of potential reasons to say “yes” to DI, categorized here.

General software engineering reasons

Dependency injection far predates Angular, and is in common use among the (fast, long-term) community of Java Spring developers (among many others). Discussion and reasoning around dependency injection (and related concept, Inversion of Control) are widely available, here are a few starting points.

https://en.wikipedia.org/wiki/Dependency_injection

https://en.wikipedia.org/wiki/Inversion_of_control

https://martinfowler.com/articles/injection.html

Angular team reasons

Angular provides access to various capabilities via dependency injection. If you code needs access any of these things, it get access via the dependency injection mechanism. In other words, the Angular team has chosen to use dependency injection therefore your application must use dependency injection. Of course this is something of a tautology. The underlying question is: but why did they do it this way?

To answer that, the Angular team has supplied an explanation of the DI pattern and why they think it is a good idea. Rather than repeat all that here, their reasons are worth a read, and overlap what I have written below.

Polymorphism for testing

Angular developers often talks in terms of services; injecting services, using services, coding services. Although there are additional meanings to the word service beyond the word class, service is often used as a synonym for class, or even as a synonym for “file with some code in it” among the Angular community.

It is common for a unit (or integration, or even E2E) test to need a different implementation of a class, then that which is used during normal operation. DI provides a convenient and flexible way to provide such alternative testing implementations.

Of course, the React community gets by without this, and still is able to perform testing. Typically this is done by treating the JavaScript module system (CommonJS “require”, or the newer “import”) as if it were a configurable DI system, with special machinery to override the meaning of an import during testing. Conveniently, this is done without any syntactical or semantic overhead during normal non-testing operation – a fairly compelling feature. Still, DI provides additional flexibility, configurability, and a more explicit means of achieving polymorphism for testing.

Polymorphism for environments

Sometimes an application needs a different implementation of the same interface (or “service”) depending on the environment in which it is deployed. Dependency injection handles this nicely and explicitly. For example:

  • An application might be compiled for both web use and Electron use, with certain service implementations swapped out.
  • An application might be hosted locally versus in the cloud service, and use a different data backend implementation (of the same Angular service interface presented to the UI).
  • An application compiled for Universal (server-side rendering) use, commonly needs certain bits swapped out during server compilation/execution.

Transitive injection

Code which itself relies on a service available via dependency injection (for example, HTTP) will need to be injected so that it can receive its own injectables.

Instance control

Without a dependency injection system, it is still straightforward to implement either a set of functions (that is, not a class at all, no state), or a singleton class instance, or a class instantiated for each use. But the Angular dependency injection system can achieve much more sophisticated instance control. For example, by configuring providers, a service could be shared by portions of a component tree, but not by the entire application. This sort of clever DI configuration can replace extensive discussion and code with just a couple of lines of configuration.

Reminder: don’t always use services

As we teach classes, and discuss code “in the wild”, developers innocently ask:

“I have some code that doesn’t go in a component, where do I put it?”

Unfortunately, when this question comes up in a Angular context, the obvious answer is usually “put it in an Angular service”. This answer is convenient, especially for teaching how to use Angular features. Unfortunately, this answer is also a lousy default.

The default answer of where to put code should be: in a source code file, in a function or class etc., which are not in any particular way connected to a web application framework by Angular. Only move the code to a Angular-related construct when the need to do so comes up. In a mature, complex application you may find that a sizable portion of the overall code lives outside any Angular construct, outside a Component or Service or NgModule, simply sitting as TypeScript code in source files.

Code should be moved from an ordinary source code file to a Angular service only to meet these needs:

  1. The code relies on some other service that will be injected to it (for example HTTP).
  2. The code will be swapped out for testing.
  3. The code will be swapped out for different environments.
  4. The code has state, in the instance of that state should be tied to the lifecycle of a Angular construct (module, component, etc.).
  5. The code is example code, for the purpose of teaching how Angular service or DI works.

 

Popular your issue tracker at scale

Sometimes when working on a project at work, we find out about a pile of features or changes needed. This can happen at the beginning of a project, at the start of the major initiative, after deploying a project (which triggers much user feedback), etc. Sometimes we have so much to absorb and divvy up into issue tracker items, that the logistics of doing so are painful.

Just thinking through and writing down 50-100 issues (or more!) is too tedious for one person to get through quickly. To divide up this work (of describing a bunch of issues in enough depth someone could work on them), I’ve come up with the following approach.

First, I jot down a list of all the areas of the system where there are new issues to enter. This forms an outline of areas that have issues, I don’t even attempt to make an entry for every likely issue.

Second, I record one or more videos, showing the screen of the system I want to add issues for, alternating back and forth with code. As I go, I describe each problem/opportunity/fix, that should become an issue. Depending on whether the new issues are closely related to existing ones, sometimes this includes bringing up the issue tracker (Jira, etc) also, talking through existing items about work remaining on them.  Sometimes something that first seems like a new issue, is really just a refinement of the success criteria of an already known issue.

Having spent potentially quite a while just describing issues (there have been times when this goes on for over an hour), I hand over the recording(s) to a relatively new person on the team, who will go through and translate this rapid-fire description into a set of items. Typically it’s fastest for the person to do that not by directly entering the items, but by just typing the candidate issues into a document. (If the list is big enough, it can pay off to have a transcriber handle the first pass – turn the words from the video/audio, into text.)

Finally, that initial rough list of candidate issue, goes to the project leader(s) of the project in question, to clean up, refine, review, approve. Then someone copies the approved text into the issue tracker.

Admittedly this is not a complex process, hardly worthy of a blog post. But someone once asked me how we successfully enter so much detail into so many items on complex projects – and here is the answer. Entering all that really does pay off. It is much more plausible to delegate work if you have described it as thoroughly as you can.

Adjust base href via a NPM package.json scripts

Suppose you wanted to adjust the base HREF in an index.html file using a regular expression, as part of a build process you are cobbling together using NPM scripts. How could you do that? Here’s one answer, though this only works if the sed command line tool is available (as it will be almost always on Linux or Mac, and only if you install it on Windows):

Unfortunately, I have begged a question. (In the traditional sense and that I assumed an answer to a question, not the modern repurposed thing of the phrase). Is it actually a good idea to use the combination of a regular expression and a script written in a JSON file, to do anything at all?

Looking at the multiple levels of \\/\/\/\\\///\// escaping above, I think the answer to this question is clearly no. It would be better to use some other means for this adjustment. On other projects we have done the same manipulation using a tool that manipulates the HTML structure. It is more verbose but vastly easier to understand.

(Incidentally, I used to do things somewhat like the above to adjust the base URLs for Angular applications.   This is not necessary anymore if using Angular CLI  – it has an option built in, –base-href.)

 

 

TypeScript as ES2015/ES2016+ compiler

I frequently encounter packages on NPM which bring in Babel to compile ES2015/2016+ down to older JavaScript suitable for a wider range of browsers. Babel is a very effective tool for this, it has been around quite a long time, and is highly configurable and flexible in future ready. It has an amazing community and reputation.

But – it is also surprisingly “heavy” in its impact on your tools installation. A typical Babel installation with a typical set of plug-ins can easily bring in 20,000 or more files, spread over hundreds of transitive dependency packages. On a huge project this might not be noticed, but on a small or medium project it can easily be the largest dependency.

Are large NPM dependency trees a problem? Theoretically no, but practically they have some downsides. Dependencies bring risk, and lots of dependencies inevitably bring more risk. If you depend transitively on hundreds more packages, your project is more vulnerable to the sort of NPM vagaries that happen from time to time (the most famous being left-pad), your project is more inclined to fall behind current versions, and most annoyingly in corporate settings, that long list of plugins and transitive dependencies is more likely to cause heartburn among lawyers who evaluate licenses.

There is an easy way to ease this problem, but it involves a minor update to one’s thinkinh. Find the spot in your mind where you may think:

“the TypeScript compiler is a tool for compiling TypeScript code to JavaScript and type checking it”.

The actual truth in 2017 is:

The TypeScript compiler is an excellent, well supported, frequently-updated general-purpose tool very suitable for non-TypeScript users. It does a great job compiling a broad array of standards-path ES2015/16/+ features down to ES2015 or ES5. It fits in well with overall project build tooling. It is completely suitable to use the TypeScript compiler on a non-TypeScript project for this purpose.

The TypeScript compiler has several substantial practical benefits over Babel:

  • It is a single dependency with no transitive dependencies.
  • Usually faster compilation.
  • It is developed by a team at Microsoft, who do a great job grinding away at the boring aspects of this work, shipping a long series of updates which keep fixing bugs and adding features.
  • It installs quickly and has far fewer failure modes in installation (because of no transitive dependencies).
  • Just one open source license to review (no transitive dependencies)… from a company that your company probably already buys a lot of stuff from anyway.
  • Just one version to update, and one configuration file to tweak which features you are using.
  • WebPack and other plug-ins available, quite analogous to Babel.

As of 2017, my recommendation is to consider TypeScript as the default solution for compiling ES2015/16+ code in non-TypeScript projects, and use Babel as a fallback if your project needs cutting-edge features that are (intentionally) not yet in TypeScript.

Cycle JS – example application and video tour

I recently learned a lot about CycleJS, which had been recommended to me in the category “you probably won’t use this for your day job right now, but you will learn a lot by looking”. And indeed, it has turned out to be the case. Cycle can be thought of as an answer to the question “what if we only had Observables and virtual DOM ? Would that be enough to build applications?” It appears to me that the answer is yes, though the tooling is not really all that complete yet.

Along the way I wrote a small example application and recorded a series of short videos (below) explaining some of the things I learned along the way, how to use various Cycle features, and especially about interesting points of comparison between Cycle and Angular and React.

These videos are not an introduction to Cycle, nor do they replace the documentation. Rather the idea is to give a flavor of how things work in Cycle from an application developer point of view, compare and contrast with other approaches, and point out why some of the ideas are quite valuable even if initially not obvious.

As I write this, have recorded four videos and made it about 70% of the way through the example application, talking through how it works and why. Questions and suggestions are much appreciated, probably the best place to put those is in the issue tracker of the example application linked above.