DORA metrics explained: Key insights for DevOps success

Many software engineering leaders face challenges in effectively measuring performance to drive improvement. It is essential to pinpoint areas for enhancing software delivery, but navigating the practical complexities of measuring developer performance can be daunting.

In 2018, Google acquired DORA, which stands forDevOps Research and Assessment, a pioneering organization known for identifying key metrics that significantly enhance software delivery performance. These metrics include deployment frequency, lead time for changes, change failure rate, mean time to recover (MTTR), and reliability.

Although these metrics are widely recognized and adopted in the industry, the true challenge lies in how organizations implement them effectively. Our guide offers a practical exploration of DORA metrics—detailing what they entail and how to leverage them to achieve development excellence. The critical question we address is whether relying solely on DORA’s capabilities is sufficient or if these four metrics should be integrated into a broader strategic framework.

What are DORA metrics?

The DORA metrics are deployment frequency, lead time for change, change failure rate, and mean time to recover (MTTR). In recent years, leaders have introduced a fifth DORA capability: reliability.

Thefour key metrics, foundational to evaluating software delivery performance, originate from the influential book “Accelerate” byNicole Forsgren, Jez Humble, and Gene Kim. The DORA team synthesizes years of research to identify the‘Accelerate’ metrics that drive high performance in technology organizations.

Here are the definitions of the DORA metrics, using definitions from Google’s2023 State of DevOps Report.

Deployment frequency

Deployment frequency is how often a development team releases new features, bug fixes, or improvements to a live production environment.

Different DevOps teams may have different definitions of high frequency. For example, for some engineering teams, a high frequency might involve deploying changes several times a day, while for others, it might mean deploying changes a few times a week. Continuous delivery allows engineering teams to make frequent minor updates or enhancements to address user needs and improve the overall user experience.

Lead time for changes

Lead time for changes is when a code change moves from the development stage to being successfully deployed in the production environment.

Implementingagile methodologies and continuous improvement can greatly improve DORA metrics for a DevOps team. By adoptingagile processes, the team can rapidly translate user stories or feature requests into actual code that can be deployed. This reduces the time between commitment and deployment, resulting in a more efficient workflow and faster turnaround times.

Change failure rate

Thechange failure rate is how often a deployment results in failure requiring immediate attention.

For instance, if an engineering team implementsautomated testing and thorough quality assurance practices, they will likely have a low change failure rate. In contrast, a team with a higher change failure rate may experience frequent post-deployment issues requiring urgent fixes. This metric reflects the reliability of the production deployment process.

Mean time to recover (MTTR)

Mean time to recover, or failed deployment recovery time, can be illustrated by the average time it takes for a team to restore service after a deployment failure.

For example, a software development team with efficientincident response procedures may have a low MTTR. In a scenario where a deployment leads to system failures, a shorter MTTR indicates a quick and effective recovery process, minimizing downtime and ensuring a more resilient system.

Reliability

Reliability is a service consistently meeting or exceeding its availability, operational performance, and accuracy goals.

A cloud service that maintains a 99.99% uptime, responds promptly to user requests, and consistently delivers accurate results, showcasing high reliability andDevOps maturity. Software teams can measure reliabilityagainst industry benchmarks to ensure that their software provides a stable and dependable user experience, meeting the standards set by both development and operational teams.

Tracking DORA metrics

Every engineering leader will have different ways of measuring itsDevOps performance with DORA metrics. As you set your benchmarks and expectations, team members should adapt and become familiar with how their work will be compared to the established DORA capabilities and othersoftware development metrics.

Here’s how to measure each DORA metric.

How to measure deployment frequency

To measure deployment frequency, count the number of deployments over a specific period. Useversion control systems like Git to track the number of commits and releases. Continuous integration tools such asJenkins orGitLab CI can automate the process and provide deployment frequency metrics.

How to measure lead time for changes

To calculate lead time, record the timestamp of when someone makes a code commit and when they deploy the corresponding change. This metric helps identify bottlenecks and delays in the development pipeline. Tools likeJira orGit can track timestamps and calculate lead time.

How to measure change failure rate

To measure the change failure rate, divide the number of failed changes by the total number of changes. Version control systems, continuous integration tools, and monitoring solutions likePrometheus orGrafana can provide data on successful and failed deployments.

How to measure MTTR

To calculate this metric, record when an incident occurs and when an engineer fully restores the service. Monitoring tools and incident management platforms likePagerDuty orOpsGenie can provide timestamps and data on incident resolution times.

How to measure reliability

Measure reliability by tracking the application’s overall uptime and performance. Utilizemonitoring tools,log analysis, anduser feedback to assess the system’s reliability. To quantify reliability, calculate metrics such asMean Time Between Failures (MTBF).

Understanding the impact of DORA metrics

Once you’ve started tracking the DORA metrics, compare your current data toindustry benchmarks, which provide a context for your performance. Continuously improve by setting incremental goals based on these comparisons, analyzing trends, and learning from any downturns in metrics. This approach not only helps assess your performance but also drives ongoing enhancements in your development practices.

Improving or decreasing DORA metrics can provide insights into your software delivery performance. Here are some examples:

Incorporating these DORA metrics in evaluations enables leaders to comprehensively understand factors influencing developer productivity. This knowledge empowers them to make informed decisions, prioritize improvements, and create an environment conducive to efficient and effective software development.

Dos and don’ts for using DORA metrics

Leaders should follow guidelines when implementing DORA metrics to understand their relation to developer productivity and production output.

Do use DORA to improve at the team level.

DORA metrics are most effective in measuring a team’s overallsoftware delivery performance as opposed to an individual—the metrics aid in comparing a team’s present performance to its past performance. By assessing a team’s ability to produce and deliver work with stability, teams are better equipped to evaluate their progress over time. This evaluation helps them identify areas for improvement and implement strategies to enhance efficiency.

Don’t compare teams against each other.

Using DORA metrics to compare teams against one another is not advisable because these metrics are context-specific, reflecting the unique challenges, workflows, and goals of each team. Such comparisons can lead to misleading conclusions and potentially undermine collaboration and morale by not accounting for the diverse nature of projects and team dynamics.

Do use DORA to reduce software delivery friction.

DORA metrics can reduce software delivery friction by identifying bottlenecks and inefficiencies in the development process, allowing teams to target specific areas for improvement. By continuously monitoring these metrics, organizations can streamline operations, enhance collaboration, and accelerate time to market.

Don’t use DORA as a way to measure productivity or quality.

While DORA metrics highlight outcomes, they do not identify specific organizational issues directly. For substantial engineering performance improvement, organizations need a deep understanding of their developer experience and must actively develop necessary capabilities through targeted initiatives, as DORA metrics alone are insufficient for driving meaningful progress, particularly in high-performing teams.

DORA metrics and developer productivity

DORA metrics alone do not directly measure developer productivity. While they offer valuable insights into developers’ performance and reasons for their varying outcomes, these metrics are historical and not predictive, making them lagging indicators. For a more complete view of developer productivity, it’s recommended to use DORA metrics alongside other methods like SPACE and DevEx.

Integrating DORA metrics effectively

Effective use of engineering metrics such as DORA involves integrating them into comprehensive developer experience programs that reduce friction and build capabilities to improve performance. By utilizing these insights, organizations can identify challenges developers face and develop targeted strategies to boost their engineering productivity.

Google’s approach to measuring engineering productivity

At Google, where DORA metrics were developed, various methods are employed to assess developer effectiveness. On the Engineering Enablement podcast, Google researchers Ciera Jaspan and Collin Green explain that Google’s leaders establish broad goals and balanced objectives across speed, ease, and quality. The team’s use of a mixed-methods approach, incorporating both qualitative and quantitative data, offers a robust understanding of engineering productivity to all stakeholders.

How DX works alongside DORA metrics

Many developer productivity teams have implemented DORA metrics but have realized they only provide high-level signals, leaving them wondering what to do next.

DX is adeveloper insights platform designed by the researchers behind DORA and SPACE. DX is the only solution providing qualitative and quantitative measures, enabling developer productivity leaders to identify their most significant opportunities and translate their impact into dollars.

DX offers tools, including DORA metrics dashboards, that help capture data from surveys and systems periodically and in real-time. These tools are trusted by developer productivity leaders at all stages, ranging from startups like Vercel and Brex to Fortune 500 companies like Pfizer and eBay. In addition to its products, DX provides expertise in implementation, program design, and transformation to help leaders adapt to changing environments and drive continued impact year after year.