From dashboards to decisions
What three real-world case studies taught us about productivity measurement that works.
This post was originally published in Engineering Enablement, DX’s newsletter dedicated to sharing research and perspectives on developer productivity. Subscribe to be notified when we publish new issues.
I’ve spent a lot of time over the years thinking about developer productivity metrics. The longer I do this work, the more convinced I become that the hardest part isn’t measuring software engineering. It’s improving it.
That’s where I think the best measurement systems distinguish themselves. They don’t just tell you whether things are getting better or worse. They guide you to interventions that actually improve how developers work. If your dashboard isn’t changing decisions, it’s not creating value.
That idea sits at the heart of EngThrive: Make It Fast and Easy to Do Great Work, a paper I recently co-authored with Tim Bozarth, David Liu, and Dean Carignan. The paper describes the measurement and improvement system I helped build at Microsoft, but what has stuck with me most aren’t the dashboards or the framework. They’re the stories.
One team intentionally “gamed” a productivity metric and improved onboarding for an entire year. Another protected developers’ focus time and discovered that the biggest gains came from work they weren’t even trying to improve. A third gave every developer two unexpected days off and found that the lost “productivity” disappeared within weeks while the wellbeing benefits lasted for months.
Three different organizations. Three different problems. Three different interventions. Yet they all taught the same lesson: the best measurement systems don’t just tell you what’s happening. They help you figure out what to do next.
Before we get to those stories, though, it’s worth remembering how easy it is to measure confidently and be wrong.
In the first two months of mandatory remote work at Microsoft in early 2020, pull requests per developer jumped more than 20%, and the company’s stock price rose more than 15%. By those measures, things looked great. During that same period, however, 78% of developers reported feeling burned out. Three signals from the same quarter, pointing in two different directions. Any one of them on its own would have told a confident but completely misleading story.
That is the trap EngThrive was built to avoid. Organizations often measure activities—pull requests, commits, tasks completed—and quietly treat them as proxies for outcomes like delivery speed, software quality, or developer effectiveness. The two are not the same.
EngThrive instead organizes measurement around outcome dimensions like Speed, Ease, and Quality, with Thriving serving as a guardrail: if an intervention makes developers faster but leaves them burned out, we don’t consider it a success. We use a handful of outcome-oriented North Star metrics supported by diagnostic metrics that help explain why those outcomes move.
With that frame in mind, here are three case studies that fundamentally changed how I think about measuring, and improving, developer productivity.
Case one: Improving one outcome changed four
“Too many meetings” is one of the most frequently cited workplace challenges among software engineers. So in late 2025, Microsoft’s CoreAI organization launched an initiative to protect developers’ focus time. Rather than simply banning meetings, leaders set an explicit target: lift the bottom 20% of developers to at least 25 hours of focus time per week.
To achieve this goal, teams did a handful of sensible things. They removed low-quality meetings, they clustered meetings together to form larger uninterrupted blocks of time, and they explicitly blocked time on their calendars to do focus work.
Within eight weeks, the results showed up across multiple dimensions. Focus time increased by 2.1 hours per developer per week, roughly twice the improvement seen in the control group. Bad Developer Days, a composite measure of daily developer friction, fell by 25%. PR velocity increased by 13%, about four times the control group. Taken together, the productivity gains were roughly equivalent to adding the output of 350 developers.
What surprised me wasn’t that focus time improved. It was that improving focus time seemed to improve things we weren’t directly trying to change.
Only about half of the reduction in Bad Developer Days could be explained by the additional focus time itself. Teams appeared to be using their newly protected capacity to pay down technical debt and eliminate other sources of friction that had been generating bad days in the first place. Creating focus time didn’t just help developers concentrate. It gave teams the space to improve the system that had been interrupting them.
That’s exactly why I think outcome-oriented measurement matters. If we had only measured focus time, we would have concluded that developers gained two extra hours each week. Looking across multiple dimensions revealed something much more interesting: the intervention triggered improvements well beyond its original goal.
One team lead summarized the lesson better than I could:
Metrics led to questions, questions led to improvements, improvements reinforced the metric’s value.
Case two: When gaming the metric is the right answer
One of the most common objections to productivity metrics is that people will game them. I worry about that too, but I increasingly think it’s also one of the best tests of whether a metric is well designed. The best metrics are ones where gaming them is indistinguishable from genuine improvement.
Time-to-First-PR is my favorite example.
One organization of roughly 4,000 developers decided to “game” the metric on purpose by assigning every new hire a trivial pull request on their first day. As a gaming exercise, it worked exactly as intended. Time-to-First-PR improved by 30%.
The surprise came later.
Those same developers went on to complete 23% more pull requests over their first year than the control group. Interviews explained why. The first pull request was never really about the code. It was about setting up the development environment, learning the team’s tools and review process, and becoming comfortable contributing. By forcing all of that to happen in the first week, the organization didn’t just improve a metric. It accelerated onboarding.
A separate AI-assisted onboarding tool, FirstMate, arrived at the same conclusion from a different direction. By automating environment setup and helping new hires navigate an unfamiliar codebase, it reduced Time-to-First-PR by 65%.
That’s the lesson I keep coming back to:
A well-designed metric shouldn’t be easy to game. It should be difficult to improve without doing something genuinely valuable. When that happens, gaming the metric and improving the system become the same thing.
Case three: A cost that turned out to be free
During the burnout crisis of 2020, one organization tried something that looked reckless on a Speed dashboard: it gave every developer two unexpected days off, called Health Days.
At first, the metric behaved exactly as you would expect. Pull request output dropped during those two days.
Then something surprising happened.
Within two weeks, the “lost” pull requests had all been made up. The apparent productivity cost disappeared. The burnout relief, however, lasted another 14 weeks.
To me, this is the clearest example of why we treat Thriving as a guardrail rather than just another metric. If we had only looked at Speed, Health Days would have appeared to be a costly intervention and might never have been attempted again. Looking across multiple dimensions told a completely different story. The intervention was effectively free from a Speed perspective while delivering a sustained improvement in developer wellbeing.
I’ll also acknowledge an important limitation. This wasn’t a controlled experiment; it was one organization’s experience. But it’s consistent with a pattern we’ve seen repeatedly: leaders often assume wellbeing and productivity exist in tension, when in practice the trade-off is frequently much smaller than expected (or doesn’t exist at all).
The lesson isn’t that every organization should schedule Health Days. It’s that looking across outcome dimensions gives leaders permission to try interventions that a single productivity metric would immediately reject.
Why this matters for engineering leaders
The common thread across all three stories isn’t focus time, onboarding, or Health Days. It’s that none of those interventions came from optimizing a single activity metric. They came from measuring outcomes, looking across dimensions, and using supporting metrics to understand why those outcomes changed.
That’s the distinction I hope readers take away from the EngThrive work. A good measurement system isn’t just a reporting system. It’s a learning system. It doesn’t simply tell leaders whether things are getting better or worse. It helps them discover interventions they wouldn’t have tried otherwise, understand why they worked, and build confidence in repeating them.
Activity metrics still have an important role to play, but not as the destination. They’re clues. They help explain why an outcome changed, not whether it mattered in the first place.
Ultimately, I think that’s the shift engineering organizations need to make. Stop asking, “What should we measure?” Start asking, “What decisions are we trying to make, and what measurements would help us make them better?” The metrics should serve the intervention, not become the intervention.
None of this is the work of one person, or even four. EngThrive is the product of a large team that has spent years building the platform, the research, the surveys, and the discipline behind these results. If these stories are useful, the credit belongs to them.