DORA metrics explained: The four key DevOps measurements

As engineering leaders continue to prioritize developer productivity, organizations are constantly seeking ways to measure and improve their performance. Enter DORA metrics—a set of key performance indicators that have revolutionized how we evaluate and enhance software delivery processes. Developed by the DevOps Research and Assessment (DORA) team, these metrics provide a comprehensive framework for assessing software development pipelines’ efficiency, reliability, and overall health.

The DORA metrics consist of four primary measurements: deployment frequency, lead time for changes, change failure rate, and mean time to recover (MTTR). Recently, a fifth metric, reliability, has been introduced to refine the assessment further. These metrics offer a holistic view of an organization’s software delivery performance, enabling teams to identify bottlenecks, streamline processes, and deliver higher-quality software more rapidly and consistently.

This article will delve into the intricacies of DORA metrics, exploring their definitions, importance, and practical applications in modern software development. We’ll examine how these metrics can be effectively measured and interpreted and discuss their impact on developer productivity and overall business outcomes. By understanding and leveraging DORA metrics, development teams can gain valuable insights into their performance, drive continuous improvement, and align their efforts with broader organizational goals.

What are DORA metrics?

The DORA metrics are deployment frequency, lead time for changes, change failure rate, and mean time to recover (MTTR). In recent years, leaders have introduced a fifth DORA capability: reliability.

Deployment frequency and lead time for changes measure the pace at which engineers release new software changes, while change failure rate and time to restore service gauge the software delivery systems’ reliability and resilience. DevOps teams can significantly improve their engineering performance and business outcomes by continuously monitoring and enhancing these metrics.

The DORA framework leverages these metrics to classify teams as Elite, High, Medium, or Low performers. Their research shows that elite performers who excel in these metrics are twice as likely to meet or exceed organizational performance targets.

The four key metrics, foundational to evaluating software delivery performance, originate from the influential book “Accelerate” by Nicole Forsgren, Jez Humble, and Gene Kim. The DORA team synthesizes years of research to identify the ‘Accelerate’ metrics that drive high performance and elite value stream management in technology organizations.

Here are the definitions of the DORA metrics, using definitions from Google Cloud’s 2023 State of DevOps Report.

Deployment frequency

Deployment frequency is how often a development team releases new features, bug fixes, or improvements to a live production environment.

Different DevOps teams may have different definitions of high frequency. For some engineering teams, the high frequency might involve deploying changes several times per day, while for others, a short cycle time might mean deploying changes a few times a week. A solid continuous delivery (CD) pipeline allows elite performers to make frequent minor updates or enhancements to address user needs and improve the overall user experience.

Lead time for changes

The lead time for changes metric measures how long it takes for a code modification to progress from the development stage to successful deployment in the production environment.

Agile methodologies, automated code review processes, and continuous improvement can significantly improve change lead time. DevOps teams can rapidly translate user stories or feature requests into actual code that can be deployed by adopting agile processes, alongside shipping often and in small batches. These strategies reduce commitment and deployment time, resulting in a more efficient workflow and faster turnaround times.

Quality code and frequent deployments contribute to the overall efficiency of high-performing DevOps teams. By focusing on the entire software delivery process and utilizing tools like Code Climate Velocity, teams can make data-driven decisions to enhance their deployment pipeline. Tracking deployment frequency measures and the failure rate metric helps operations teams identify opportunities for improvement and align with business goals.

Change failure rate

The change failure rate (CFR) is the frequency with which a deployment results in a failure in production that requires immediate attention.

For instance, if an engineering team implements automated testing and thorough quality assurance practices, it will likely have a low change failure rate. In contrast, a team with a higher change failure rate may experience frequent post-deployment issues requiring urgent fixes.

Mean time to recover (MTTR)

Mean time to recover, or failed deployment recovery time, is illustrated by the average time it takes for a team to restore service after a deployment failure.

For example, a software development team with efficient incident response procedures may have a low MTTR. In a scenario where a deployment leads to system failures, a shorter MTTR indicates a quick and effective recovery process, minimizing downtime and ensuring a more resilient system.

Reliability

Reliability is a service consistently meeting or exceeding its availability, operational performance, and accuracy goals.

A reliable cloud service with 99.99% uptime responds quickly to user requests and consistently provides accurate results, demonstrating high reliability and DevOps maturity. Software teams can measure reliability against industry benchmarks to ensure their software provides a stable and dependable user experience, reducing degraded service and meeting standards set by development and operational teams.

Tracking DORA metrics

Every engineering leader will have different ways of measuring their DevOps performance with DORA metrics. As you set your benchmarks and expectations, team members should adapt and become familiar with how their work will compare to the established DORA capabilities and other software development metrics.

Here’s a simple way to measure each DORA metric:

How to measure deployment frequency

Count the number of deployments over a specific period, like deployments per week, to measure deployment frequency.

Track the number of pull requests, commits, and releases using version control systems like Git. Continuous integration tools such as Jenkins or GitLab CI can automate the software delivery process and provide deployment frequency metrics.

How to measure lead time for changes

To calculate lead time, record the timestamp when someone makes a code commit and when they deploy the corresponding change.

This metric helps identify bottlenecks and delays in the development pipeline. Tools like Jira or Git can track timestamps and calculate lead time.

How to measure change failure rate

To measure the change failure rate, calculate the percentage of deployments that fail by dividing the number of failed changes by the total.

Version control systems, continuous integration tools, and observability solutions like Prometheus or Grafana can provide data on the frequency and percentage of successful deployments.

How to measure MTTR

To calculate the mean time to recovery, record when an unplanned outage occurs and when an engineer fully restores the service.

Monitoring tools and incident management platforms, like PagerDuty or OpsGenie, can provide timestamps and data on incident resolution times.

How to measure reliability

Measure reliability by tracking the application’s overall uptime and performance.

Assess the system’s reliability using monitoring tools, log analysis, and user feedback. Calculate metrics such as Mean Time Between Failures (MTBF) to quantify reliability.

Understanding the impact of DORA metrics

Once you’ve started tracking the DORA metrics, compare your current data to industry benchmarks, which provide context for your operational performance. Continuously improve by setting incremental goals based on these comparisons, analyzing trends, and learning from any downturns in metrics. This approach helps assess your performance and drives ongoing enhancements in your DevOps practices.

Improving or decreasing DORA metrics can provide insights into your software delivery performance. Here are some examples:

Incorporating these DORA metrics in evaluations enables leaders to comprehensively understand factors influencing developer productivity. This knowledge empowers them to make informed decisions, prioritize improvements, and create an environment conducive to efficient and effective software development.

Dos and don’ts for using DORA metrics

Leaders should follow guidelines when implementing DORA metrics to understand their relation to developer productivity and production output.

Do use DORA to improve at the team level.

DORA metrics are most effective in measuring a team’s overall software delivery performance as opposed to an individual—the metrics aid in comparing a team’s present performance to its past performance. By assessing a team’s ability to produce and deliver work with stability, teams are better equipped to evaluate their progress over time. This evaluation helps them identify areas for improvement and implement strategies to enhance efficiency.

Don’t compare teams against each other.

Using DORA metrics to compare teams is not advisable because these metrics are context-specific, reflecting each team’s unique challenges, workflows, and goals. Such comparisons can lead to misleading conclusions and potentially undermine the collaboration and morale of multidisciplinary teams by not accounting for the diverse nature of projects and team dynamics.

Do use DORA to reduce software delivery friction.

DORA metrics can reduce software delivery friction by identifying bottlenecks and inefficiencies in the deployment process. This allows teams to target specific areas for improvement. By continuously monitoring this software delivery throughput, organizations can streamline operations, enhance collaboration, and accelerate time to market.

Don’t use DORA as a way to measure productivity or quality.

While DORA metrics highlight outcomes, they do not identify specific organizational issues directly. Organizations need a deep understanding of their developer experience to improve engineering performance substantially and must actively develop necessary capabilities through targeted initiatives. DORA metrics alone are insufficient for driving meaningful progress over time, particularly in high-performing teams.

DORA metrics and developer productivity

DORA metrics alone do not directly measure developer productivity. While they offer valuable insights into developers’ performance and the reasons for their varying outcomes, these metrics are historical and not predictive, making them lagging indicators of DevOps performance. Using DORA metrics alongside other methods like SPACE and DevEx is recommended for a more complete view of developer productivity.

Integrating DORA metrics effectively

Effective use of engineering metrics such as DORA involves integrating them into comprehensive developer experience programs that reduce friction and build capabilities to improve team performance. With these insights, organizations can identify challenges and develop targeted strategies to boost engineering productivity among their developers or ‘internal customers.’

Many off-the-shelf developer tools and developer productivity dashboards include observability of DORA metrics as a standard feature. These tools collect workflow data from your developer tool stacks, such as GitHub, GitLab, Jira, orLinear. Using workflow data from these tools, you can see measurements for all four DORA metrics.

How DX works alongside DORA metrics for better software delivery

Are you looking for more insights beyond DORA metrics? DX, the developer intelligence platform DORA and SPACE researchers created, offers a comprehensive solution. As the only platform combining qualitative and quantitative measures, DX empowers you to identify critical opportunities and quantify your impact in financial terms.

DX provides advanced DORA metrics dashboards and tools for real-time and periodic data capture from surveys and systems. Trusted by developer productivity leaders at startups like Vercel and Brex and Fortune 500 companies like Pfizer and eBay, DX caters to organizations of all sizes.

Beyond its product suite, DX offers expertise in implementation, program design, and transformation. This holistic approach enables leaders to adapt to changing environments and drive sustained impact year after year.

With DX, you can maximize developer productivity, translate insights into action, and stay ahead in the evolving tech landscape.