Combining qualitative and quantitative metrics

Qualitative metrics and quantitative metrics are complementary approaches to measuring developer productivity.

Qualitative metrics, which are typically derived from surveys, provide a holistic view of productivity that includes both subjective and objective measurements.

Quantitative metrics, on the other hand, provide an objective view of development activity based on data from your engineering systems.

So, how do you use qualitative and quantitative metrics together?

Our recommendation – and the approach being built into DX – is to start with qualitative metrics to figure out where to focus. Then to use quantitative metrics to drill in deeper into specific areas.

There are a couple of reasons for this.

First, qualitative metrics provide a much more holistic view from which you are able to identify opportunities, whereas quantitative metrics are typically only available for a narrow set of areas of the development process.

In our recent paper, we explain:

Second, quantitative metrics by themselves, lack the context needed to assess whether something is good or not. Google similarly advises its engineering leaders to go to survey data first before looking at logs data for this reason.

We can focus our attention and efforts by using each type of measurement based on its strengths. To break it down:

1. Start with qualitative data to identify your top opportunities
2. Once you know what you want to improve, use quantitative metrics to drill in further
3. Track your progress using both qualitative and quantitative metric

There are exceptions to this approach, of course. For example, if you are looking for an answer to a specific question, it makes sense to go directly to a quantitative metric if one is available.