How to measure GenAI adoption and impact

The organizations with the most insight into how their engineers use GenAI will reap the most significant rewards from this powerful technology. Drawing from DX’s experience with more than 200 companies measuring GenAI adoption, this guide outlines a straightforward approach to measuring the adoption and business impact of tools like Copilot, so you can drive adoption where needed and prove its value to leadership.

The measurement conundrum

Despite widespread enthusiasm and a promising outlook, many organizations struggle to demonstrate tangible results from GenAI investments.

Traditional metrics often fail to tell a compelling story, while basic utilization data provides limited insights. Without proper measurement strategies, leaders can’t promote adoption or validate returns on their AI investments.

Beyond basic metrics

The guide introduces a measurement framework, combining three complementary approaches to create a picture of GenAI’s impact:

• Telemetry metrics: System-generated data that provides objective insights into developer output patterns, though with essential limitations when used alone
• Experience sampling: A robust methodology that captures in-the-moment feedback as developers work, revealing insights unavailable through other approaches
• Strategic surveys: Periodic assessments that establish baselines and track changes in adoption, satisfaction, and developer perceptions over time