How 18 top companies measure the impact of AI in engineering

AI coding assistants are now ubiquitous—used by more than 85% of engineers according to The Pragmatic Engineer’s 2025 survey—and companies are spending millions on licenses and tokens. Yet most executives still struggle to answer a basic question: Are these investments actually improving engineering performance?

At DX, we’ve spent the past year deeply engaged in this challenge, partnering with engineering leaders, analyzing longitudinal data from across our customer base, and studying how teams are rolling out and measuring AI tools. Those insights, including input from many of the companies featured in this report, shaped the AI Measurement Framework we released earlier this year. The AI Measurement Framework is a recommended set of metrics to track AI adoption and impact across engineering teams.

This article shares some of the real-world context behind the framework, including the metrics that engineering leaders at 15+ top companies are using today, the patterns we’re seeing across teams, and the lessons that are helping organizations make smarter AI investment decisions. You’ll also find guidance on measuring AI within your own organization, including both what to measure and how to measure it, so you can apply these learnings to your own AI strategy.

In this report, you’ll see:

• How 18 tech companies measure AI impact. Details from Google, GitHub, Microsoft, Dropbox, Monzo, and many others. Details on the metrics measured; for example, how GitHub measures AI time savings, pull request (PR) throughput, change failure rate, engaged users, innovation rate, and more.
• Why solid foundations matter for measuring AI impact. You can and should use existing “core” metrics, together with AI-specific ones like customer satisfaction (CSAT) scores.
• Unique metrics, interesting trends. Microsoft measures “bad developer days,” while Glassdoor measures experimentation outcomes from AI tools. It’s likely that more teams will measure data on how autonomous AI agents perform.
• How to measure AI impact. An overview of the AI Measurement Framework and tips for collecting this data.

Many leaders are still working through how to identify which AI tools are delivering value, how tools are being used, and what productivity gains they’re achieving. Our hope is that this report sets you up to address those questions.