DORA metrics don't measure productivity
Leaders often use DORA metrics to measure developer productivity, but this isn’t correct.
The DORA metrics are not a measure of developer productivity. They are a measure of software delivery performance. Conflating productivity and software delivery performance is like conflating cholesterol and blood sugar—while both are important factors, they are fundamentally distinct things.
The definition of software delivery performance finds its origins in the first annual State of DevOps report in 2013. This report predates the birth of DevOps Research and Assessment—the company founded by Dr. Nicole Forsgren that was dedicated to research and assessment solutions—as well as the book Accelerate which popularized the DORA metrics.
The 2013 State of DevOps report referred to IT performance as the “ability to ship business-critical applications quickly without disrupting service”. More recent reports refer to software delivery performance as the “ability to deliver software with speed and stability.” Note that software delivery performance has never been characterized as or linked to developer productivity in any of DORA’s research or reports.
The DORA metrics have been popularized to the point that they are often taken out of context. As a result, leaders often overlook a critical point: the DORA metrics measure the ability to produce not how much is produced or the quality of production. Yet it is these latter factors that people most often are referring to when they talk about wanting to measure developer productivity.
Measuring or even defining developer productivity has remained an elusive problem. SPACE—also authored by Dr. Nicole Forsgren—lays out the complex multi-dimensional nature of developer productivity, a concept still lacking a single authoritative definition or measure. To further underscore the problem: a more recent study by Dr. Margaret-Anne Storey found that developers and managers not only diverge in their perceptions of productivity, but also in their misconceptions about how it is perceived by others.
While there’s no clear solution for how to measure developer productivity today, there is growing consensus amongst researchers and leaders that measurement approaches need to evolve beyond mechanical views reliant on measures of developer output or task cycle time.
Software development is knowledge work, not factory work. The way we think about measuring developer productivity should reflect this difference. Personally, I’m hopeful about recent research into the complexities of measuring knowledge work, and human-centered approaches such as focusing on developer experience.