What we learned from Netflix, Pinterest, and Dropbox about measuring developer productivity

Last month, we brought together engineering leaders from some of the most innovative companies in tech to discuss a question we hear constantly: “How do we actually measure developer productivity in ways that drive meaningful improvement?”

The insights were remarkable. Leaders from Netflix, Indeed, Twilio, Pinterest, and Dropbox shared their approaches, which go far beyond the typical vanity metrics most organizations rely on.

Here, I’ll share some of my notes and takeaways from the panel. You can also listen to the full discussion here:

The metrics that actually matter

One theme emerged quickly from our panel: if you’re only tracking lines of code or commit volume, you’re not only measuring the wrong things, you are potentially harming your culture.

Ash Jones, our panelist from Pinterest, described how his team initially went down this path but quickly discovered these metrics were easy to measure but not meaningful indicators of true productivity or developer experience. Indeed found something similar—when they focused heavily on metrics like JIRA-based delivery lead time, teams simply automated ticket closures rather than improving actual productivity.

As our Indeed panelist Jeff Kemp put it: “People can get that pending closed state done immediately if they really have to care about it. But ultimately it never actually translates into the value to the business.”

So what works? The consensus was clear:

• Balance quantitative and qualitative data. The most successful organizations combine telemetry metrics with self-reported data to create a comprehensive view.
• Expand beyond engineering-only metrics. Jeff from Indeed emphasized that software delivery is a team sport including product managers, UX designers, and data scientists. Measuring cross-functional collaboration leads to better outcomes.
• Use industry benchmarks for context. Jyothi Nookula, our panelist from Netflix, highlighted how the DX platform provides valuable cross-industry comparisons that were impossible with their homegrown surveys.

Creating accountability that drives results

Here’s a hard truth: data without accountability is just numbers on a screen. The panel revealed dramatic differences in outcomes based on how organizations distributed responsibility.

Executive visibility created “night and day” differences at Dropbox, according to Uma Namasivayam, when CEO Drew Houston began reviewing productivity data regularly (he logs in to DX almost every day). Jyothi from Netflix described how CTO Elizabeth Stone reviews all tech staff metrics, driving accountability across the organization.

But executive involvement is only part of the equation. As we learned from our Twilio panelist Jesse Adametz, Twilio was able to achieve a staggering 97% engineering survey participation rate because developers saw their feedback leading to actual improvements. They’ve been able to create a culture where engineering managers are expected to triage survey results with their teams and set specific improvement goals. As a result, they’ve seen that when teams set specific improvement targets, 80-90% see measurable progress in those areas.

Measuring AI tool ROI, beyond the hype

AI tooling was a hot topic among all panelists, but with a focus on real-world impact rather than marketing claims. The approaches here were particularly insightful:

• Netflix measures not just GitHub Copilot acceptance rates but also tracks the number of AI-generated lines of code that make it to production—a significantly more meaningful measure of value.
• Pinterest monitors incident-per-commit rates when deploying AI tools to ensure they aren’t sacrificing reliability for raw speed.
• Dropbox increased AI tool adoption from 10% to over 80% in a single quarter by providing targeted training on high-value use cases.

As an example of measurable impact, Pinterest reported saving approximately 1,000 engineering hours weekly through their AI tooling initiatives, with potential for significant additional gains.

Breaking down productivity barriers

Perhaps most interesting was the panel’s insights on cultural framing of developer productivity. Developers respond better to initiatives focused on “eliminating friction and toil” rather than “driving productivity.” This aligns with our own view, that optimization of developer experience is really about improving the platforms and systems that developers use in their work. This subtle shift in language leads to dramatically different engagement levels.

Other key strategies that emerged:

• Netflix implemented “Deep Work Weeks” with no meetings after survey data revealed developers felt they lacked time for concentrated effort. The success was so evident that the entire infrastructure engineering organization has now adopted this practice.
• Indeed discovered that a major pain point was engineers not knowing how to access user research resources. By implementing office hours and better resource discoverability, they measurably improved developer sentiment surrounding cross-team collaboration.
• Pinterest emphasized that developer productivity is a sociotechnical problem requiring balance across three pillars: tooling, processes, and culture—with culture being the most challenging but impactful area.

What’s next?

As I reflected on these conversations, one thing became abundantly clear: the organizations seeing the most dramatic improvements aren’t just measuring developer productivity—they’re creating systems of accountability and improvement that span from the C-suite to individual contributors.

Is your organization ready to move beyond vanity metrics and create a data-driven approach to developer productivity? Here are three questions to consider:

1. Are you measuring what truly matters to your developers and business outcomes?
2. Have you created clear accountability for improvement at all levels of your organization?
3. Are you tracking the real impact of your tooling investments, particularly AI?

I’d love to hear your thoughts and experiences. What’s working in your organization? What challenges are you facing? Reach out to me on LinkedIn to let me know.