How to Measure Productivity of Software Development Teams [15/100]

Many, o so many people in the software industry believe that paying a developer per hour is the only way. Well, for outsourcing, selling hours of work is the easiest way to go about it. This one simple assumption breeds a huge number of real-life practices that spread to software development companies, penetrating people’s souls and sucking them dry of life.

Talking about it must be centered around fair pay for software engineering, though. Why does John earn $2,000 a month while Jane makes $5,000? The obvious answer: because Jane is more productive. She brings more value to her team and company. And here we run into the most exciting, very tricky question. How do we measure this value? How to measure developer productivity?

Cool, you’d say, humanity exists for quite some time now, we’ve probably solved similar questions somehow. We’ve managed to salary before, didn’t we? So let’s look into the history and find an answer to our questions on software developer productivity there!

Measuring productivity is rather easy when work is monotonous, uniform. John carries out 200 operations a day, Jane does 500. Jane is just great, we pay her 2.5 times more. But what if the work is more complex? Say, John and Jane are firefighters. They extinguish the same number of fires per month, work the same hours, how do we measure their productivity? Here, we can compare one team to another, yet measuring the individual productivity of a firefighter isn’t a piece of cake. Maybe count the number of people saved? Sounds like a dubious productivity metric.

Let’s bring this to knowledge work. What if John and Jane are scientists at a research institute? How should we assess the individual productivity of John, who for a month hasn’t been able to come up with a model to describe the interaction of rhinovirus proteins with a new drug, and Jane, who wrote a program a long time ago and is waiting impatiently for John’s model to come up? I simply can’t answer this.

Just as John has no model at hand, we don’t have a straightforward and fair way to measure the productivity of a software developer, a scientist, a teacher. Therefore, we often try to get rid of uncertainty by using a very simple, very understandable productivity metric that happens to be fundamentally wrong. This metric is working hours. Huh, why wrong? Let’s use two variables:

• Rate, assessing person’s abilities (or skills), and
• Number of working hours (because these we at least can measure)

We’ll then keep multiplying these two, hoping to measure the productivity of a developer, their value to the team and company, and accordingly, their pay.

Being great in its simplicity, this model has one flaw: it doesn’t work for software development. Even if measuring skills with some reasonable accuracy was possible (which it isn’t), we absolutely can’t estimate the time needed for a software development task. We can’t, because the complexity of tasks is sometimes unpredictable, and we can err (greatly) in estimating the time needed to complete them.

Say, John spends 40 hours on one feature. How complicated is it? We don’t know. Perhaps Jane would’ve written this code in 20 hours simply because she’s done something similar before. Or she’s in a better mood this week. Or she’s just back from a vacation and is very energetic. Or she knows that one trick of the language that allows her to avoid a bug that John spent 8 hours on. Or her team is very supportive. Now, what is the quality of the software delivered? And how useful is this feature? This is the perfect moment to stop philosophizing because finding out the usefulness of the feature will take us a long, long time. Perhaps, we’ll never know how useful it is.

The obvious conclusion is that you can fairly comfortably use hours of work to measure developer productivity for relatively simple tasks. I do think that this naturally affects the nature of the tasks that most software engineering outsourcers face. It works as follows: we measure developer productivity with hours → this only works for easy and medium software development tasks → we only take up easy and medium tasks.

In my opinion, this is one of the main reasons why software development outsourcers have such difficulties creating their own products. Their model can’t stand uncertainty that is an intrinsic property of product development. Can’t live with uncertainty? Pack your stuff and leave your software project, forget about technologically complex products altogether. Alright, but this model works for many projects, right? Not every software engineer is launching rockets into space! Some just make websites, transform data, “create” forms. Sure thing it works. But let’s see what practices measuring hours as a metric of software development productivity requires.

• Strict office time (John can’t come in at 12 because then his manager will have a hard time controlling him)
• Control systems for entering/leaving the office (how many times was John late for work?)
• Software development team accounting for how they spent working hours (let John and Jane explain what they were doing and why so long)
• Web application monitoring systems (doesn’t Jane visit Facebook too often?)
• Video surveillance (how often does every individual developer leave for a coffee or a bathroom?)
• Individual KPIs (whatever happens, we need to know how many tasks a programmer completed this last month).

This bullshit is just there to strip the job of all the fun. It messes up teamwork, fosters selfishness, feeds a culture of mutual distrust between management and employees, stimulates manipulative behavior on both sides, and screws up all creativity. And the most annoying is that it doesn’t even solve the problem!

How to consider the quality of work? How to take into account the impact of the task on the entire system? What if there’s a hole in the system, and all the user data will drain through it? How can we pay respect to the time a developer spends solving a problem in their sleep or on a walk? (Though when tasks are simple, this won’t happen). How do we salary code maintainability and evolution? How to account for productivity losses from working in the lockdown? There’s no way to, so all of this is usually ignored.

Working within such a model hinders personal development. I’ve worked with it for a couple of years, and damn it’s hard to get this rubbish out of your head! Without noticing, you somehow get used to it, get involved, and cease realizing that things can work differently. The hourly measurement of developer productivity changes your mind (to the worst), and getting somewhere else becomes harder with every billed hour. Tactically, you’re making money. Strategically, you’re on a highway to nowhere.

Measuring developer productivity by hours creates an illusion of control. It’s an attempt to reduce uncertainty by squeezing it into a simple model. Yet in our industry, we must accept uncertainty, learn to thrive with it.