Combining qualitative and quantitative metrics

Qualitative metrics and quantitative metrics are complementary approaches to measuring developer productivity.

Qualitative metrics, which are typically derived from surveys, provide a holistic view of productivity that includes both subjective and objective measurements.

Quantitative metrics, on the other hand, provide an objective view of development activity based on data from your engineering systems.

So, how do you use qualitative and quantitative metrics together?

Our recommendation – and the approach being built into DX – is to start with qualitative metrics to figure out where to focus. Then to use quantitative metrics to drill in deeper into specific areas.

There are a couple of reasons for this.

First, qualitative metrics provide a much more holistic view from which you are able to identify opportunities, whereas quantitative metrics are typically only available for a narrow set of areas of the development process.

In our recent paper, we explain:

Survey data can be collected quickly and, when designed correctly, provide fast and accurate measurements to establish baselines and guide improvement efforts. To augment surveys, organizations should also collect data from systems.”
- Abi Noda

Second, quantitative metrics by themselves, lack the context needed to assess whether something is good or not. Google similarly advises its engineering leaders to go to survey data first before looking at logs data for this reason.

We encourage leaders go to the survey data first, because if you just go look at logs data, logs data doesn’t really tell you whether it’s good or bad.”
- Ciera Jaspan, Engineering Productivity Research, Google

We can focus our attention and efforts by using each type of measurement based on its strengths. To break it down:

  1. Start with qualitative data to identify your top opportunities
  2. Once you know what you want to improve, use quantitative metrics to drill in further
  3. Track your progress using both qualitative and quantitative metric

There are exceptions to this approach, of course. For example, if you are looking for an answer to a specific question, it makes sense to go directly to a quantitative metric if one is available.

Published
July 19, 2023

Get started

Want to explore more?

See the DX platform in action.

Get a demo