Challenges with metrics and how DX solves them

Metrics alone fall short – unlock the ‘why’ behind developer productivity

Scott Hillman

Product Marketing

One-minute summary

  • DX provides all the pieces you need to improve developer productivity in a single platform: metrics, research-backed guidance, developer insights, and customizability.
  • Metrics show what’s happening but miss the why behind developer productivity.
  • DX gets you to why. How?
    • A research-backed framework to highlight the metrics that matter
    • Survey-based developer insights that guide you to improve those metrics
    • Customizable metrics and reports to suit the way your team works
  • The DX Core 4 combines the strengths of DORA, SPACE, and DevEx to get your organization aligned on the metrics that matter.
  • DX Snapshots provide qualitative insights that reveal the underlying drivers of metrics, showing you how to improve.
  • DX Data Studio enables users to create custom metrics and reports with an AI-powered interface.
  • DX is the only tool in software engineering analytics that offers a complete view of developer productivity.

Full article

Many organizations use metrics like DORA or cycle time to gauge developer productivity. These are good starting points. But metrics alone lack the context for meaningful improvements. DORA metrics might show high deployment frequency, or rising failure rates, but they don’t explain why. Are bottlenecks, team dynamics, or organizational issues to blame? You can’t answer that without deeper insight.

Metrics gave us a starting point, but they didn’t tell the full story. Combining metrics and developer sentiment with DX…that’s how we began uncovering the real pain points.”
- Sheila Wakida, Sr. Director Engineering, Dropbox

At DX, we’ve learned that collecting metrics is just the first step. To actually utilize metrics and drive productivity improvements you need three additional pieces:

  1. Alignment on the metrics that matter
  2. Data that guides you to improve those metrics
  3. The ability to customize metrics and reports when necessary

“We have 40 dashboards. What should we focus on?”

Capturing metrics is straightforward; using them effectively is not. A common challenge is that organizations collect tons of data but don’t know what to do with it. To address this, we designed the DX Core 4 – a unified framework that builds on the best ideas from DORA, SPACE, and DevEx, distilling them into four key dimensions: speed, effectiveness, quality, and business impact. This evolution offers a focused set of metrics that works at any sized organization, and can be augmented with additional metrics for specific goals.

The DX Core 4 is backed by benchmarks from data across 200+ companies in tech, finance, retail, and pharma. Organizations using the DX Core 4 consistently report a 3-12% increase in efficiency, a 14% boost in feature development time, and a 15% improvement in engagement. These benchmarks provide teams with clear alignment on goals and a way to track progress.

“So what do we do to improve?”

A clear set of metrics creates a shared definition of what productivity means, and gets everyone rowing in the same direction. Armed with initial baselines and benchmarks, the next question is: “what should we do to actually improve productivity?”

The challenge with metrics is that they tell us the what but not the why. For example, a high change failure rate might indicate stability issues, but it doesn’t explain whether the underlying problem is a testing gap, unclear requirements, developer burnout, or something else.

This is where DX provides unique insight through its developer surveying capability. DX Snapshots collect feedback on 14 key drivers of engineering performance that provide answers on exactly what is inhibiting (or enhancing) developer productivity. Ciera Jaspan at Google captures the value of this well:

Quantitative metrics give you a number, but you can’t interpret that number unless you have the qualitative side of the world. It doesn’t tell you why. It doesn’t tell you if what they’re doing is good or bad. It doesn’t tell you if there’s room for improvement. Without qualitative data you can’t understand the actual behaviors.”
- Ciera Jaspan, Engineering Productivity Research Lead, Google

A common concern with survey data is that it is unreliable and difficult to collect. Using DX, customers are able to achieve sustained 96% participation rates which provide statistically reliable data points that can be broken down to the team level.

Without these qualitative insights, teams are left feeling lost on what steps to take to improve productivity metrics. By combining qualitative and quantitative metrics in one solution, DX provides leaders with a complete view into productivity.

With DX, we’ve been able to combine quantitative metrics with our qualitative data to get a holistic view into developers’ challenges, sparking meaningful improvement discussions and greatly enhancing our velocity and positive culture.”
- Justin Abrahms, Developer Productivity Lead, eBay

“What if I need to customize metrics and reports?”

DX offers out-of-the-box metrics and reports covering common use cases. But we know that each organization and team is unique. For example, different teams may want to calculate cycle time differently, based on their specific Jira workflows.

A common challenge with engineering metrics tools is that they don’t allow you to go beyond what’s offered out of the box. DX is unique in offering customers access to their own data lake, along with an AI-powered interface for building custom reports and dashboards.

Before DX, we tried a number of different vendors that offered prebuilt metrics and charts. But the way these platforms worked didn’t align with how we wanted to present our metrics. We love how Data Cloud seamlessly integrates into our workflows and gives us flexibility to create custom reports and access our raw data.”
- Jarryd Trokis, Engineering Manager, Attio

DX answers the “what now?” question

Metrics are just the start. The real value is knowing what to focus on and how to make meaningful improvements. DX brings everything together in one platform — metrics, research-backed guidance, developer insights, and customizability. That’s why DX is the only software engineering analytics tool that delivers a complete picture of developer productivity.

To learn more about DX and our full suite of products, request a demo through our website.

 

Published
November 12, 2024