Six tools to track DORA metrics

The DORA metrics, consisting of four key indicators, are crucial for gauging developer productivity and enhancing the software delivery process. Originating from the influential book Accelerate by Nicole Forsgren, Jez Humble, and Gene Kim, these metrics are pivotal for assessing the velocity, frequency, and quality of software deployments at both team and organizational levels.

Implementing DORA metrics empowers DevOps teams to pinpoint and resolve bottlenecks, facilitating swifter and higher-quality releases. Teams that already excel in cadence and quality can leverage these metrics to refine the DevOps practices, enhancing the engineering team’s experience, shortening the mean time to market, and boosting developer productivity.

You’ll need the right DORA metrics tools to track and kickstart your journey toward DevOps performance improvement. Here are six top tools we recommend for accurate DORA metrics tracking, which is essential for any development team aiming to optimize its software engineering and delivery processes.

What are DORA metrics?

The four DORA metrics measure software delivery performance. They are:

Deployment frequency measures how often a development team releases updates, such as new features, bug fixes, security enhancements, or reductions in technical debt. This metric is a fundamental aspect of the software delivery process, reflecting the operational performance of the DevOps team.

Lead time for changes refers to the mean lead time from when changes are initiated (e.g., a pull request in GitLab CI) to when they are deployed to all users in production. This metric is crucial for assessing the delivery process’s efficiency and the engineering team’s responsiveness.

Change failure rate calculates the percentage of deployments that fail in production due to undiscovered issues during testing, highlighting the importance of code quality and robust DevOps practices in minimizing operational disruptions.

Time to restore service is the average duration needed to restore full operational status after a production failure. It is a critical measure of a team’s ability to quickly rectify issues, ensuring reliability and continuous improvement in service delivery.

These metrics constitute the essential toolkit for tracking the speed and quality of software development, ensuring that teams maintain a balance between rapid delivery and accurate, reliable outputs. For instance, increasing deployment frequency can lead to higher productivity but must be managed carefully to avoid introducing defects that could lead to customer dissatisfaction and increased costs.

Why measure DORA metrics?

Tracking DORA metrics comprehensively gauges the health of a team’s DevOps processes. These metrics enable teams and organizations to implement continuous improvement initiatives that shorten time to market and enhance overall system quality. This improvement can lead to increased revenue and better customer satisfaction.

Additionally, DORA metrics offer valuable insights into development and operational efficiencies that might otherwise be overlooked.

For instance, if you have two teams where one deploys updates five times a week and the other only once, without specific metrics, you can only guess at the reasons for this discrepancy. By implementing DORA metrics, you can accurately identify and analyze the differences between the two teams, helping you pinpoint exactly where and why they diverge in performance.

Six tools to measure DORA metrics

The DORA metrics are recognized and well-regarded enough that several existing DevOps tools can support a version of them that is out of the box. You may even use one of them in your CI/CD pipelines. Below, we’ll cover a few of these tools, the benefits of using them to capture DORA metrics, and where each tool falls short.

Jenkins

One of the most utilized tools in the industry, Jenkins is an open-source CI/CD automation server that enables teams to build DevOps promotion pipelines. Jenkins is open-ended and unopinionated, meaning you can build a pipeline for any software system to any level of complexity using a combination of Groovy (the Jenkins language for defining pipelines) and a development language of your choice.

What it supports: Jenkins has the data to measure deployment frequency and lead time for changes.

What’s missing: You don’t get these measurements out of the box using Jenkins; you have to have a way to calculate them—e.g., by licensing CloudBees CD/RO, which is built on Jenkins and offers a DORA metrics dashboard.

GitLab

GitLab is a hosted Git server that enables source code control for development teams. It also offers numerous other features for managing software development, including project planning, CI/CD pipeline support, monitoring, and security.

What it supports: GitLab supports DORA metrics out of the box via its Value Streams Dashboard. Gitlab’s dashboards enable you to have your core metrics in one place, making it simple to start with DORA.

What’s missing: GitLab’s DORA metrics assume you’re using GitLab exclusively for project management. In particular, the change failure rate and time to restore service metrics assume you use GitLab incidents as your incident management system. GitLab DORA metrics also require an Ultimate tier subscription to the service.

Prometheus

Prometheus is an open-source metrics, monitoring, and alerting system that many teams use as their overall monitoring solution. Many cloud services support leveraging Prometheus for cloud-based apps.

What it supports: As a generalized metrics and monitoring system, you can track all four DORA metrics by exporting data from your CI/CD build and incident management systems to Prometheus. So long as you can export the relevant data, you can use your choice of CI/CD and incident management tools to avoid vendor lock-in.

What’s missing: There’s no out-of-the-box support for DORA in Prometheus, which means you must build it yourself.

Spinnaker

Spinnaker is an open-source Continuous Delivery platform that supports multi-cloud deployment. Like Jenkins, it’s the tool of choice for many teams that must deploy frequently and at high velocity.

What it supports: Using Spinnaker, you can gather data on deployment frequency and lead time to changes. You can also leverage features such as Spinnaker’s canary analysis support to track defects in production, which you can incorporate into your change failure rate metric.

What’s missing: Spinnaker doesn’t support a DORA dashboard out of the box. You’ll need to build one using a tool like Prometheus or use a commercial service built on Spinnaker, such as OpsMx, that adds it.

Grafana

Grafana is a graphical user interface for analyzing and visually displaying data. Devs can use Grafana with Prometheus to turn data from CI/CD and incident management systems into metrics that track developer productivity.

What it supports: As with Prometheus, you can use Grafana to track any metric you need - including the DORA metrics - so long as you have access to the correct data. Some in the Grafana community have also published reusable dashboards you can leverage for specific combinations of tools. (E.g., here’s one that relies on data from GitHub and Jira.)

What’s missing: As with Prometheus, unless you can find a pre-built dashboard that leverages your toolset, you may have to build out your Grafana solution yourself.

DX

DX is a superior tool for understanding DORA metrics and overall developer productivity by integrating a unique combination of qualitative and quantitative data insights. Traditional tools often fall short by focusing solely on numerical metrics or relying heavily on qualitative feedback without synthesizing the two. DX, however, bridges this gap with its comprehensive platform, offering a holistic view of developer performance and experience. By leveraging tools like DevEx 360 and Data Cloud, DX captures various metrics, including deployment frequency, lead time for changes, change failure rate, and time to restore service, providing a detailed understanding of software delivery performance.

Here’s what DX supports for customers looking to understand developer experience and productivity:

Real-time feedback and continuous improvement

DX’s ability to gather real-time feedback through PlatformX and accelerate onboarding processes ensures that development teams are efficient and continuously improving. This real-time feedback loop is crucial for identifying and addressing productivity blockers promptly. As a result, teams can quickly adapt and optimize their workflows, leading to sustained improvements in productivity and a better developer experience.

Qual and quant approach to capturing developer experience

The platform’s unique approach to capturing hard metrics and the softer, more subjective aspects of developer experience sets it apart from competitors. DX provides a more nuanced perspective by quantifying internal developer experiences and unifying these insights with traditional performance metrics. DX empowers organizations to make data-driven decisions that address the root causes of inefficiencies, leading to more than surface-level improvements and promoting a culture of continuous improvement.

Empowering data-driven decisions and demonstrating impact

DX’s superiority lies in its comprehensive and integrated approach to understanding developer productivity and DORA metrics. It empowers teams to measure, analyze, and improve every facet of their development processes, ensuring they can deliver high-quality software efficiently and effectively.

How to choose the right DORA metrics tool

When evaluating a DORA metrics tool, make sure to consider the following factors:

• Ease of use. Does it offer a clean, easy-to-use UX?
• Setup and integration. Is it easy to integrate 3rd party systems? Does it support the tools you use?
• Scalability. Will it scale to the number of teams and the building activity you need to support? Can it make new data available quickly at low latency?
• Security. Can you secure it appropriately (e.g., using SSO)? Does it support access controls at your required level of granularity?
• Reporting. Does it provide out-of-the-box reports that you can use immediately?

Limitations of DORA metrics

While DORA metrics provide valuable insights into certain fundamental aspects of your software development process, they do not encompass several crucial elements of the developer experience. For instance, DORA metrics cannot assess cross-team collaboration, the presence or quality of local builds for development teams, the amount of existing technical debt and efforts to reduce it, or whether developers have uninterrupted work time to achieve deep focus (“flow”). These factors are essential for a comprehensive understanding of and improvements in the developer experience, which directly influence productivity and job satisfaction.

Additional frameworks for a comprehensive view

Implementing other frameworks can provide some of these additional metrics.

For example, the SPACE framework emphasizes gathering metrics in five quadrants: Developer Satisfaction and Well-being, Performance, Activity, Communication and Collaboration, and Efficiency and Flow. Using SPACE, you can identify new metrics in focus areas where your teams want to capture data and make improvements. For example, you can use developer surveys and calendar data to track how much dedicated flow time engineers feel they have or gather data from internal messaging tools like Slack or Teams to gauge communication and collaboration.

Enhancing developer experience with DevEx and DX25

DX goes beyond traditional metrics by focusing on Developer Experience (DevEx) through its unique tools like DevEx 360 and the DX25 framework. DevEx 360 quantifies internal developer experience using surveys and feedback mechanisms, clearly showing how developers feel about their tools, processes, and work environment.

The DX25 framework identifies 25 key areas impacting developer productivity and satisfaction, such as tooling efficiency, support systems, and team dynamics. This comprehensive approach ensures engineering managers and leaders can notice every aspect of the developer experience, leading to a more engaged and productive workforce.