DORA metrics explained: The four key DevOps measurements
Taylor Bruneaux
Analyst
Many software engineering leaders face challenges in effectively measuring performance to drive improvement. Pinpointing areas to enhance software delivery is essential, but navigating the practical complexities of measuring developer performance can be daunting.
In 2018, Google acquired DORA, which stands for DevOps Research and Assessment. DORA is a pioneering organization known for identifying key metrics that significantly enhance software delivery performance. These metrics include deployment frequency, lead time for changes, change failure rate, mean time to recover (MTTR), and reliability.
While these metrics are widely recognized and adopted in the industry, the true challenge lies in how organizations effectively implement them. Our guide offers a practical exploration of DORA metrics—detailing what they entail and how to leverage them to achieve development excellence. The critical question is whether relying solely on DORA’s capabilities is sufficient or how these four metrics should integrate into a broader strategic framework.
What are DORA metrics?
The DORA metrics are deployment frequency, lead time for changes, change failure rate, and mean time to recover (MTTR). In recent years, leaders have introduced a fifth DORA capability: reliability.
Deployment frequency and lead time for changes measure the pace at which engineers release new software changes, while change failure rate and time to restore service gauge the software delivery systems’ reliability and resilience. DevOps teams can significantly improve their engineering performance and business outcomes by continuously monitoring and enhancing these metrics.
The DORA framework leverages these metrics to classify teams as Elite, High, Medium, or Low performers. Their research shows Elite performers excelling in these metrics are twice as likely to meet or exceed organizational performance targets.
The four key metrics, foundational to evaluating software delivery performance, originate from the influential book “Accelerate” by Nicole Forsgren, Jez Humble, and Gene Kim. The DORA team synthesizes years of research to identify the ‘Accelerate’ metrics that drive high performance and elite value stream management in technology organizations.
Here are the definitions of the DORA metrics, using definitions fromGoogle Cloud’s 2023 State of DevOps Report.
Deployment frequency
Deployment frequency is how often a development team releases new features, bug fixes, or improvements to a live production environment.
Different DevOps teams may have different definitions of high frequency. For some engineering teams, the high frequency might involve deploying changes several times a day, while for others, a short cycle time might mean deploying changes a few times a week. A strong continuous delivery (CD) pipeline allows elite performers to make frequent minor updates or enhancements to address user needs and improve the overall user experience.
Lead time for changes
The lead time for changes metric measures the amount of time it takes for a code change to move from the development stage to being successfully deployed in the production environment.
Agile methodologies, automated code review processes, and continuous improvement can significantly improve change lead time. DevOps teams can rapidly translate user stories or feature requests into actual code that can be deployed by adopting agile processes, alongside shipping often and in small batches. These strategies reduce commitment and deployment time, resulting in a more efficient workflow and faster turnaround times.
Change failure rate
The change failure rate (CFR) is how often a deployment results in failure in production requiring immediate attention.
For instance, if an engineering team implements automated testing and thorough quality assurance practices, they will likely have a low change failure rate. In contrast, a team with a higher change failure rate may experience frequent post-deployment issues requiring urgent fixes.
Mean time to recover (MTTR)
Mean time to recover, or failed deployment recovery time, is illustrated by the average time it takes for a team to restore service after a deployment failure.
For example, a software development team with efficient incident response procedures may have a low MTTR. In a scenario where a deployment leads to system failures, a shorter MTTR indicates a quick and effective recovery process, minimizing downtime and ensuring a more resilient system.
Reliability
Reliability is a service consistently meeting or exceeding its availability, operational performance, and accuracy goals.
A reliable cloud service with 99.99% uptime responds quickly to user requests and consistently provides accurate results, demonstrating high reliability and DevOps maturity. Software teams can measure reliability against industry benchmarks to ensure their software provides a stable and dependable user experience, reducing degraded service and meeting standards set by development and operational teams.
Tracking DORA metrics
Every engineering leader will have different ways of measuring their DevOps performance with DORA metrics. As you set your benchmarks and expectations, team members should adapt and become familiar with how their work will compare to the established DORA capabilities and other software development metrics.
Here’s a simple way to measure each DORA metric:
How to measure deployment frequency
Count the number of deployments over a specific period, like deployments per week, to measure deployment frequency.
Track the number of pull requests, commits and releases using version control systems like Git. Continuous integration tools such as Jenkins or GitLab CI can automate the software delivery process and provide deployment frequency metrics.
How to measure lead time for changes
To calculate lead time, record the timestamp when someone makes a code commit and when they deploy the corresponding change.
This metric helps identify bottlenecks and delays in the development pipeline. Tools like Jira or Git can track timestamps and calculate lead time.
How to measure change failure rate
To measure the change failure rate, calculate the percentage of deployments that fail by dividing the number of failed changes by the total.
Version control systems, continuous integration tools, and observability solutions like Prometheus or Grafana can provide data on the frequency and percentage of successful deployments.
How to measure MTTR
To calculate mean time to recovery, record when an unplanned outage occurs and when an engineer fully restores the service.
Monitoring tools and incident management platforms like PagerDuty or OpsGenie can provide timestamps and data on incident resolution times.
How to measure reliability
Measure reliability by tracking the application’s overall uptime and performance.
Utilize monitoring tools, log analysis, and user feedback to assess the system’s reliability. To quantify reliability, calculate metrics such as Mean Time Between Failures (MTBF).
Understanding the impact of DORA metrics
Once you’ve started tracking the DORA metrics, compare your current data to industry benchmarks, which provide context for your operational performance. Continuously improve by setting incremental goals based on these comparisons, analyzing trends, and learning from any downturns in metrics. This approach helps assess your performance and drives ongoing enhancements in your DevOps practices.
Improving or decreasing DORA metrics can provide insights into your software delivery performance. Here are some examples:
Metric | High or Increasing | Low or Decreasing |
Deployment Frequency | Sign of efficient development processes | Possible bottlenecks or inefficiencies |
Lead Time for Changes | Speedy movement of code changes from development to production | Indicates areas for improvement and optimization in the development pipeline |
Mean Time to Recover (MTTR) | Lower MTTR signifies a resilient and responsive engineering team | Higher MTTR may indicate challenges in promptly identifying and resolving failures in production |
Change Failure Rate | Reflects a reliable and well-tested codebase | Indicates potential testing or quality control issues |
Reliability | Demonstrates a stable and dependable software system | Indicates frequent failures or disruptions |
Incorporating these DORA metrics in evaluations enables leaders to comprehensively understand factors influencing developer productivity. This knowledge empowers them to make informed decisions, prioritize improvements, and create an environment conducive to efficient and effective software development.
Dos and don’ts for using DORA metrics
Leaders should follow guidelines when implementing DORA metrics to understand their relation to developer productivity and production output.
Do use DORA to improve at the team level.
DORA metrics are most effective in measuring a team’s overall software delivery performance as opposed to an individual—the metrics aid in comparing a team’s present performance to its past performance. By assessing a team’s ability to produce and deliver work with stability, teams are better equipped to evaluate their progress over time. This evaluation helps them identify areas for improvement and implement strategies to enhance efficiency.
Don’t compare teams against each other.
Using DORA metrics to compare teams is not advisable because these metrics are context-specific, reflecting each team’s unique challenges, workflows, and goals. Such comparisons can lead to misleading conclusions and potentially undermine collaboration and morale of multidisciplinary teams by not accounting for the diverse nature of projects and team dynamics.
Do use DORA to reduce software delivery friction.
DORA metrics can reduce software delivery friction by identifying bottlenecks and inefficiencies in the deployment process, allowing teams to target specific areas for improvement. By continuously monitoring these software delivery throughput, organizations can streamline operations, enhance collaboration, and accelerate time to market.
Don’t use DORA as a way to measure productivity or quality.
While DORA metrics highlight outcomes, they do not identify specific organizational issues directly. Organizations need a deep understanding of their developer experience to improve engineering performance substantially and must actively develop necessary capabilities through targeted initiatives. DORA metrics alone are insufficient for driving meaningful progress over time, particularly in high-performing teams.
DORA metrics and developer productivity
DORA metrics alone do not directly measure developer productivity. While they offer valuable insights into developers’ performance and reasons for their varying outcomes, these metrics are historical and not predictive, making them lagging indicators of DevOps performance. Using DORA metrics alongside other methods like SPACE and DevEx is recommended for a more complete view of developer productivity.
Integrating DORA metrics effectively
Effective use of engineering metrics such as DORA involves integrating them into comprehensive developer experience programs that reduce friction and build capabilities to improve team performance. With these insights, organizations can identify challenges and develop targeted strategies to boost their engineering productivity.
Many off-the-shelf developer tools and developer productivity dashboards include observability of DORA metrics as a standard feature. These tools collect workflow data from your developer tool stacks, such asGitHub, GitLab, Jira, or Linear. Using workflow data from these tools, you can see measurements for all four DORA metrics.
How DX works alongside DORA metrics for better software delivery
Many developer productivity teams have implemented DORA metrics but have realized they only provide high-level signals, leaving them wondering what to do next.
DX is a developer insights platform designed by the researchers behind DORA and SPACE. DX is the only solution providing qualitative and quantitative measures, enabling developer productivity leaders to identify their most significant opportunities and translate their impact into dollars.
DX offers tools, including DORA metrics dashboards, that help capture data from surveys and systems periodically and in real-time. These tools are trusted by developer productivity leaders at all stages, ranging from startups like Vercel and Brex to Fortune 500 companies like Pfizer and eBay. In addition to its products, DX provides expertise in implementation, program design, andtransformation to help leaders adapt to changing environments and drive continued impact year after year.