Measuring DORA, SPACE, and DevEx at Docebo

With an ever-expanding team of engineers, Docebo, the leading AI-powered learning platform, needed a clearer picture of how to manage developer productivity and improve the work environment. Engineering leadership was interested in tracking DORA and SPACE metrics as a way to get a comprehensive view of how effectively the organization was operating. At the same time, the HR department wanted to use data to proactively spot and address any process, cultural, or tooling issues affecting developers.

“In my view, we were missing a forum for addressing DevEx problems,” said Marco Vadori, who leads Employee Experience at Docebo. “Our CTO was also looking for a better way to capture quantitative metrics like the DORA metrics.” In this regard, the CTO said, “The data collected is invaluable for our People Managers in the Product & Engineering team, aiding them in making well-informed decisions as part of our continuous improvement process."

The company has tried several ways to gather these insights. They were using in-house surveys, but these surveys fell short in delivering actionable insights specifically about developer productivity. HR was also collecting feedback by conducting interviews with developers; however, these weren’t practical for large-scale applications.

Docebo was aware of tools like Jellyfish, which offer quantitative data on developer activities and resource allocation. These tools are able to capture parts of DORA and SPACE but not aspects such as satisfaction, collaboration, efficiency, and flow. They are also often rigid in the exact metrics they track, which is difficult for teams looking to integrate with multiple developer systems or define metrics in unique ways that fit their workflows. For these reasons, as well as because of Docebo’s need to capture perceptual data, these tools were not considered in the company’s evaluation.

“When we saw DX, it was exactly what we had in mind,” said Vadori. “Our CTO was very aligned with DX because of the way it puts together quantitative metrics with qualitative feedback, which we believe is the best way to measure productivity holistically.”

In their evaluation of DX during their proof of concept, Docebo’s leaders had specific criteria that were important to them:

• The ability to view both qualitative feedback and quantitative metrics together. Vadori noted, “It is useful for us to compare developers’ perceptions with quantitative metrics. If there’s a difference, we’ll want to dig into that. DX is the only metrics solution that provides both perspectives, qualitative and quantitative.”
• Customization of how DORA metrics are tracked. Metrics like Lead Time for Changes can be measured in many different ways. It was important to engineering leaders that they could define and track these metrics in a way that suited their specific needs, however other solutions on the market are limited in this regard.
• Survey participation: Past experiences left some leaders concerned about whether DX could get sufficient engagement from developers. This concern was addressed quickly when they saw a 100% participation rate in their first DevEx 360 snapshot.

After meeting these criteria, Docebo chose DX to improve their understanding and management of developer productivity and experience. DX’s products will be used by Docebo’s engineering leaders.