An introduction to DORA metrics

Taylor Bruneaux

Analyst

Measuring performance to drive improvement is a challenge for many software engineering leaders. Although it is crucial to identify areas for enhancing software delivery, the practical complexities of measuring what truly boosts developer performance can be difficult to solve.

In 2018, Google acquired DORA, the leader in identifying metrics that have a significant impact on software delivery performance. These metrics include deployment frequency, lead time for change, change failure rate, mean time to recover (MTTR), and reliability.

Though widely embraced by the industry, the real test lies in how organizations implement these metrics effectively. Our guide provides a practical exploration of DORA metrics – what they entail and how to leverage them. 

The critical question we address: Are DORA metrics enough on their own, or should they be part of a more comprehensive strategy? Join us as we navigate these software delivery metrics, offering actionable insights for leaders steering the course of development excellence.

Table of Contents:

What are the DORA metrics?

DORA metrics are crucial for evaluating software delivery performance, previously known as IT performance. These metrics assess two key elements: throughput (delivery speed) and stability (reliability in production). Balancing speed and stability optimizes software development and deployment.

The DORA metrics are deployment frequencylead time for changechange failure rate, and mean time to recover (MTTR). In more recent years, leaders have introduced a fifth metric: reliability.

DORA metrics measure an organization’s ability to produce— they do not measure output or production quality.

The five DORA metrics

Here are the definitions of the five DORA metrics, using definitions from Google’s 2023 State of DevOps Report.

Deployment frequency:

Deployment frequency is how often a development team releases new features, bug fixes, or improvements to a live production environment. 

Different DevOps teams may have different definitions of what a high frequency means to them. For example, for some engineering teams, a high frequency might involve deploying changes several times a day, while for others, it might mean deploying changes a few times a week. One use case for a high deployment frequency is with a web application team that follows a continuous deployment approach. This allows them to make frequent small updates or enhancements to address user needs and improve the overall user experience.

Change lead time

Change lead time is then a code change moves from the development stage to being successfully deployed in the production environment. 

A DevOps team that embraces agile methodologies and incorporates continuous integration can significantly improve this DORA metric. This allows them to quickly transform user stories or feature requests into actual code that can be implemented and deployed, thereby shrinking the time between commitment and deployment.

Change failure rate

The change failure rate is how often a deployment results in failure that necessitates immediate attention. 

For instance, if an engineering team implements automated testing and thorough quality assurance practices, they will likely have a low change failure rate. In contrast, a team with a higher change failure rate may experience frequent post-deployment issues requiring urgent fixes. This metric reflects the reliability of the production deployment process.

Mean time to recover (MTTR)

Mean time to recover, or failed deployment recovery time, can be illustrated by the average time it takes for a team to restore service after a deployment failure. 

For example, a software development team with efficient incident response procedures may have a low MTTR. In a scenario where a deployment leads to system failures, a shorter MTTR indicates a quick and effective recovery process, minimizing downtime and ensuring a more resilient system.

Reliability

Reliability is a service that consistently meets or exceeds its availability, performance, and accuracy goals. 

A cloud service that maintains a 99.99% uptime, responds promptly to user requests, and consistently delivers accurate results showcases high reliability and DevOps performance. Software teams can measure reliability against industry benchmarks to ensure that their software provides a stable and dependable user experience, meeting the standards set by both development and operational teams.

It’s important to note that different teams might use different definitions for these DevOps performance metrics. We recommend that teams agree on clear definitions for each metric tailored to their workflow.

Tracking DORA metrics

For every team, measuring your DevOps performance with DORA metrics will be different. As you set your own benchmarks and expectations, team members should adapt and become familiar with how their work will be benchmarked against the established DORA metrics and other DevOps success metrics.

Here’s how to get started with assessing DORA metrics:

  • Deployment frequency: To measure deployment frequency, count the number of deployments over a specific time period. Use version control systems like Git to track the number of commits and releases. Continuous integration tools such as Jenkins or GitLab CI can automate the process and provide deployment frequency metrics.
  • Change lead time: To calculate lead time, record the timestamp of when a code commit is made and when the corresponding change is deployed. This metric helps identify bottlenecks and delays in the development pipeline. Tools like Jira or Git can be used to track timestamps and calculate lead time.
  • Change failure rate: To measure the change failure rate, divide the number of failed changes by the total number of changes. Version control systems, continuous integration tools, and monitoring solutions such as Prometheus or Grafana can provide data on successful and failed deployments.
  • MTTR: To calculate this metric, record the time from when an incident occurs to when the service is fully restored. Monitoring tools and incident management platforms like PagerDuty or OpsGenie can provide timestamps and data on incident resolution times.
  • Reliability: Measure reliability by tracking the overall uptime and performance of the application. Utilize monitoring tools, log analysis, and user feedback to assess the system’s reliability. Calculate metrics such as Mean Time Between Failures (MTBF) and Mean Time to Recover (MTTR) to quantify reliability.

Dos and don’ts for using DORA metrics

Many leaders confuse DORA metrics with developer productivity and production output. Here are some guidelines for leaders to follow when implementing DORA metrics

Do use DORA to improve at the team level.

DORA metrics are most effective in measuring a team’s overall software delivery performance as opposed to an individual—the metrics aid in comparing a team’s present performance to their past performance. By assessing a team’s ability to produce and deliver work with stability, teams are better equipped to evaluate their progress over time. This evaluation helps them identify areas needing improvement and implement strategies to enhance efficiency.

Don’t compare teams against each other.

Different teams have varying workflows, tech stacks, and objectives. 

As you introduce DORA metrics, assess what is achievable for each team and which metrics are relevant. Teams are not necessarily ‘high performers’ or ‘low performers’ based on their DORA metrics

When evaluating a team’s progress against their DORA metrics, consider all the internal and external factors affecting each team’s ability to deliver software performance. Universal metrics may not capture these factors.

Do use DORA to reduce software delivery friction.

Prioritize the development of capabilities that positively impact DORA metrics. 

Identify and address areas of friction that hinder teams from building high-quality software quickly. By focusing on these areas and making necessary improvements, you can use the DORA metrics to measure the success of your efforts. 

Ultimately, this will lead to more efficient and effective software development processes, reflected in improved DORA metrics, benefiting your team members and your organization.

Don’t use DORA as a way to measure productivity or quality.

While DORA metrics emphasize outcomes, they need to pinpoint specific organizational problems. 

To foster genuine engineering performance improvement, organizations must thoroughly understand their developer experience and actively cultivate the required capabilities to improve software delivery. DORA metrics aren’t the solution; more profound analysis and targeted capability development through thoughtful initiatives are essential for meaningful progress, especially for high-performing teams.

DORA metrics are lagging indicators

DORA metrics are a set of DevOps metrics that engineering leaders can use to track their organization’s software delivery performance. Remember that an organization should not solely rely on these metrics as they are lagging indicators of the ability to deliver software. They are historical metrics, not predictive of future performance.

Engineering metrics like DORA are most effective with holistic developer experience programs that reduce friction and develop capabilities that drive performance. Then, use these metrics to track progress.

How DORA metrics impact developer effectiveness

To evaluate the effectiveness of your developers, DORA metrics provide critical insights. 

These metrics help you understand why your developers may perform above or below expectations regarding software delivery.

By using these insights, you can better understand where your developers may be struggling and take the necessary steps to improve their engineering performance.

Here are some examples of what the DORA metrics can tell you about developer effectiveness:

Deployment frequency:

  • High frequency: Sign of efficient development processes.
  • Low frequency: Possible bottlenecks or inefficiencies.

Lead time for changes:

  • Shorter lead time: Speedy movement of code changes from development to production.
  • Longer lead time: Indicates areas for improvement and optimization in the development pipeline.

Mean time to recover (MTTR):

  • Lower MTTR: This signifies a resilient and responsive engineering team.
  • Higher MTTR: May indicate challenges in promptly identifying and resolving issues.

Change failure rate:

  • Low rate: Reflects a reliable and well-tested codebase.
  • High rate: Indicates potential testing or quality control issues.

Reliability:

  • High reliability: Demonstrates a stable and dependable software system.
  • Low reliability: Indicates frequent failures or disruptions.

Incorporating these DORA metrics in evaluations enables leaders to understand factors influencing developer productivity comprehensively. This knowledge empowers them to make informed decisions, prioritize improvements, and create an environment conducive to efficient and effective software development.

DORA metrics are not enough

DORA metrics offer a valuable means to quantify and monitor software delivery performance. Yet, they alone do not provide a comprehensive insight into developer experience, productivity, or the nuanced aspects of developer performance. 

These metrics are most effective when integrated into a broader toolkit employed by DevOps leaders to evaluate, measure, track, and enhance team performances. 

Even at Google, the originators of DORA, a combination of methods is utilized to gauge developer effectiveness. In a discussion on the Engineering Enablement podcast, members of Google’s engineering productivity research team, Ciera Jaspan and Collin Green, emphasize key considerations for measuring engineering productivity. Google’s approach involves leaders defining high-level understanding goals and establishing balanced objectives for speed, ease, and quality. The research team employs a diverse set of metrics for each goal, adopting a mixed-methods approach that captures qualitative and quantitative data. This comprehensive strategy provides stakeholders with a more holistic understanding of engineering productivity.

How DX works alongside DORA metrics

Many Developer productivity teams have implemented DORA metrics but have realized that they only provide high-level signals, leaving them wondering what to do next. 

DX is a developer insights platform designed by the researchers behind DORA and SPACE. DX is the only solution providing qualitative and quantitative measures, enabling developer productivity leaders to identify their most significant opportunities and translate their impact into dollars.

DX offers tools that help capture data from surveys and systems, both periodically and in real-time, trusted by developer productivity leaders at all stages, ranging from startups like Vercel and Brex to Fortune 500 companies like Pfizer and eBay. In addition to its products, DX provides expertise in implementation, program design, and transformation to help leaders adapt to changing environments and drive continued impact year after year.

Published
November 1, 2023

Get started

Want to explore more?

See the DX platform in action.

Get a demo