Building better software faster: Leading in the age of AI

How much faster can we go with AI?

This question is dominating every conversation I have with VPs, CTOs, and other engineering leaders. Your board wants the numbers, your CEO is asking you for timelines, and your engineers just want tools that work and don’t feel gimmicky. But here’s the problem with this question: we’re focused on the faster part, and a lot of us haven’t stopped to think where we’re even going.

There’s so much space for innovation with AI. We can use it in new and novel ways, but at the end of the day, AI is a tool that needs to help us improve the system. We need to think about using AI on an organizational level if we really want to see the organizational benefits that are being promised.

We agreed a long time ago that lines of code are not equal to developer productivity. We’ve established this already. But when we look at what’s in the headlines about AI, there’s a huge focus on the percentage of code written or lines of code written, focusing on the output.

AI can have tremendous impacts on output, for better or for worse, but really, AI is a tool that improves developer experience. It’s not about more lines of code. We need good developer experience, reliability, and long-term stability and maintainability if we want this tool to help our organizations win.

The headlines aren’t helping us.

The headlines these days aren’t helping us as leaders stuck in this disappointment gap. If you open LinkedIn right now, it doesn’t take you very long to come across some sensational headline about 30% of code being written by AI, or 60% of code, or that in three to six months it’s going to be 90% of code. We’re hearing that AI will simultaneously replace all junior engineers, as well as all senior engineers, and eventually all engineers.

This is the reality of the situation. These headlines are headlines for a reason. It’s like looking at a glossy fashion magazine filled with Photoshopped models or looking at food advertisements that’s been artificially enhanced. It just doesn’t hold up to reality.

I’m not saying that there aren’t great things happening with AI. It’s just that we have to have realistic expectations; otherwise, we’re going to stay stuck in that disappointment gap.

The antidote to all of this hype is data. Data beats hype every single time, and as a leader during this time of change, you need the right data to explain both the capabilities and the limitations. You need to explain what a realistic expectation looks like to yourself, most importantly, to your leaders, your organization, and also to your development teams.

Understanding how to turn developer productivity metrics into actionable improvements is crucial for moving beyond the hype.

The reality on the ground

AI can be transformative. I’ve been working with organizations that are very far along their AI journey that have seen some tremendous results, but that’s not what we’re seeing in the average organization right now. Among industry peers, people like me who have access to large amounts of data across the industry, it’s well understood that those headlines don’t reflect the reality on the ground.

This hype around AI is the most significant barrier to its adoption because what happens is that developers see these headlines, and they think it’s promising them the world. They go to use the tool, and no tool can live up to the expectations in those headlines because they’re sensational headlines, and so then developers abandon the tool, or they perceive it just as a gimmick.

On the other hand, we have leaders who may not have a background in software engineering, seeing those headlines and having inflated expectations of what can happen in reality.

This quote is from Brian Hauck, who is a co-author of the Space Framework of Developer Productivity. He’s researching AI and its impact at Microsoft right now. He’s incredibly smart, and his insights on this topic are well worth exploring further.

Real statistics from real organizations

Here are some statistics from real organizations. Currently, the organizations that are performing best—in the top quartile, the top 25% of organizations—have about 60% of developers using AI tools on a daily or weekly basis. This is not necessarily in line with the 30% or 100% code written by AI claims. So there’s quite a gap.

But these organizations and every organization that is investing in training, support, and enablement to help engineers adopt AI tools and identify the use cases that are working are seeing this number rise every day. We’re getting the tools in the hands of the developers. They’re liking using them, and they’re seeing good organizational results and good individual results.

On average, developers who use AI tools are saving about three hours and 45 minutes per week by using AI to assist or augment their development workflows. This is data across 38,880 developers at 184 companies, data that we gathered in Q1 and Q2 of 2025. This is what we’re seeing in reality on the ground right now at organizations. These are very promising results, but not necessarily near what we see in the sensational headlines.

For context on how to measure AI’s impact on your engineering team, this data represents a significant shift from theoretical potential to practical gains.

Your responsibility as a leader

You have a challenging role right now as an organizational leader. If you don’t think of yourself as a business leader, I encourage you to shift that perspective. The reality is that you have subject matter expertise in engineering, and that’s what makes you great at your job. But if you are in a leadership position, you are an organizational leader. You need to be leading on the business side.

Whether you like to hear this or not, it is your responsibility to educate around the hype. It’s your responsibility to educate others in your company about what to expect from AI, how it’s impacting your organizational performance.

To do that, you need two things. First, we have to go back to the fundamentals of what great engineering performance looks like. It’s easy to get distracted by a shiny new tool when we don’t have a firm foundation of what it is we’re chasing after.

The first thing that you need to do is have a perfect definition of engineering performance and excellence. This will serve as a common language across your business, ensuring everyone speaks the same language and has a solid foundation of principles and a definition of performance to go back to.

Then you can enhance that definition, build on top of it, and look at some precise AI measurements that are going to help you measure AI ROI in enterprise software projects. You need to know what’s working with AI, what’s not working with AI, and how it’s improving or not improving some of those foundational measurements of performance to make the right choice of what to do next.

I work with many engineering leaders every day, and the truth is, many of us still feel like we’re showing up unprepared, especially when it comes to AI, because it’s all so new and changing rapidly. If that describes you, it’s not because you’re a poor leader or bad at your job—it’s because this is genuinely difficult. It’s changing rapidly, and it can be challenging to keep up with. This is all new territory.

Show up with confidence

You should be able to show up to every meeting with your exec team and leadership team when they’re asking, “What are we doing with AI?” with the ability to tell them three things:

1. How you perform as an organization in general
2. How AI is helping you or not helping you
3. What are you going to do next

Again, it is your job to educate others around you, especially your business counterparts and cross-functional stakeholders. I know this can feel unfair that we have to bear the burden of educating around AI, but it’s better to understand this responsibility now than realize it months later when it’s too late and you have to undo decisions made with incomplete information.

What does it actually take to build better software faster?

Can AI do this for us? Well, AI is accelerating the good, solid fundamentals of software delivery and software performance. It’s changing the way we work, absolutely, but it’s not changing those fundamentals. We need quality and reliability. We need a great developer experience. We need teams that can collaborate effectively. We want a fast time to market so we can get feedback from our customers and adjust our course accordingly.

You also need to protect your org from long-term damage. What I mean by that is not sacrificing long-term velocity, stability, and maintainability for short-term gains when it comes to AI, because that is the reality for a lot of companies. We’re producing more code because AI makes it easier to produce more code. If we’re not taking care to ensure that code is high quality, that our pipelines are ready for that code, that our SRE operations can support that amount of code going into production, we can do lasting damage.

Understanding software quality metrics that actually matter becomes even more critical when AI is generating more code. We need to have a complete picture.

The DX Core 4 framework

The first thing I’m going to share with you is the way to align your organization on a definition of excellence. The DX Core 4 is a framework that I co-authored with Abi Noda, who is the co-author of the DevEx Framework. This framework is a collaboration between the two of us, and also Nicole Forsgren, who founded DORA, Dr. Margaret-Anne Storey, who’s the co-author of the SPACE Framework, and many other researchers.

The idea here was to bring together DORA, SPACE, and DevEx and put them together in one framework that is easy to deploy and easy for you to use. This framework measures across four different categories: speed, effectiveness, quality, and impact. These are all holding each other in tension, so you have to look at all of them together.

Just like a spider web is holding itself up, and if one of the dimensions goes down, the whole structural integrity is compromised. The same thing goes for the Core 4. We can’t optimize for speed at the expense of developer experience. We can’t invest so much in quality and pay so much attention to it that our innovation and business impact grind to a halt.

Dropbox, Etsy, Pfizer, and hundreds of other companies are already using Core 4. Dropbox was a very early adopter. Drew Houston, who is an engineer but also co-founder and CEO of Dropbox, says that “Core 4 gives you a much more cohesive picture of what’s happening in your organization. It answers the question, ‘what does performance look like?’”

Drew, being an engineer, knows that developer experience is at the heart of great engineering performance. Developer experience is the biggest lever that you can pull as an engineering leader to improve your organizational performance, to improve your team performance, and individual performance. This is not just my opinion. This is what the data says.

The developer experience impact

Over 20% of time is lost due to friction across the whole developer population that we’ve been studying as we’ve developed the Core 4. 20% of time is lost due to inefficiencies and poor tooling.

When we study developer experience and its correlation to time loss, there is such a strong correlation that we were able to make a model out of it. The DXI is how we measure developer experience. It’s a research-backed, evidence-based metric based on 14 different drivers of developer experience. The DXI is a composite metric that reflects those drivers.

For every increase in DXI, developers save 13 minutes a week or about 10 hours annually. That might not sound like a lot, but if you think about your developer population—50, 100, 200 developers—and how much opportunity you have to increase developer experience, that time adds up so quickly. It is so valuable to your organization. Time is a very scarce commodity, and the fact that 20% of it is being lost due to poor developer experience is a critical business problem.

Real results from Block

Block, which is the company behind Cash App, Square, and TIDAL, has been using metrics in the Core 4. They used DXI to identify 500,000 hours that they were losing annually due to inefficiencies and frictions. What they did then was they took the data around that developer experience and decided what to invest in next. Azra, who leads the developer experience team at Block, said, “Those improvements then helped us move faster without sacrificing quality or focus.” It’s so important to have that “without” clause in there for you as well in your organization.

You don’t need to be a large company like Block and have a dedicated DevEx team to use Core 4 to make better decisions about where to invest and to assess the performance of your engineering organization. We designed it to be easy to deploy, even on your own.

How to implement the Core 4

You can use this approach with Google Forms, Typeform, or any form tool you prefer. Make sure that when you’re choosing a tool to run this survey, you can export the data into Google Sheets or Excel because you’ll need to do some computation. This way, you can get all of the core metrics from Core 4, all the primary metrics from self-reported data, and make better decisions in days or weeks, not months.

The other thing that we designed the Core 4 with is industry benchmarks. Going into important meetings, sometimes you need more ammunition to make a business case for something, and industry benchmarks have been highly valuable for that. It’s one of the reasons that DORA became so popular—they have industry benchmarks—and we wanted Core 4 to be benchmarkable.

These benchmarks are openly available. You can check out the benchmarks, use the survey template, and then compare your results. You’ll get 75th percentile median values as well, so you can see where you’re doing well, where you might need more investment.

The Core 4 is designed to be a comprehensive, robust, future-proof way to measure engineering performance. This is the whole point of software. The whole point of developing software is to do it in a way that’s sustainable, high quality, and high business impact, and that’s what the Core 4 is designed to measure. When you introduce any new tool, whether it’s AI or a new framework or a new process, you’re going to see the impact in the Core 4 metrics.

The AI Measurement Framework

With AI, we want to accompany the Core 4 with some specific measurements that measure the effect that AI is having on our organization. I’m excited to share this framework more widely—we published this recently.

The AI Measurement Framework is the result of a year’s worth of research at top companies out in the field. We also collaborated with Cursor, DORA, Sourcegraph, and numerous other individuals at AI companies and large corporations. The AI Measurement Framework gives you hard metrics and clear guidance on what to capture across three different categories: utilization, impact, and cost.

Utilization

Utilization is the first part of the story here. Tracking AI tool usage is important because, according to our data, we see the biggest uplift from AI from people who go from being non-users to people who become persistent, consistent, daily, or weekly users. Even occasional usage weekly or monthly can bring significant benefits, going from non-usage to consistent usage. We want to get the tools into more developers’ hands because that’s where we’re seeing the biggest gains.

For teams evaluating different options, understanding AI coding assistant pricing across different tools helps inform both adoption strategy and budget planning.

Impact

On impact, it’s so important to keep in mind that AI is a tool propelling you to some outcome. It’s not just there for the sake of being there. We need to look at its impact on Core 4 metrics and developer satisfaction with the tool. Our top metric that we recommend tracking in terms of impact is time savings per week. That’s where the industry has aligned. Google published an article outlining how they do their measurements, and AI-driven time savings is also at the top of their list, so that’s a good place to standardize your measurement.

Understanding AI code generation best practices for enterprise adoption is crucial for maximizing these time savings.

Cost

Cost is an important part of this conversation because we are in the business of business. If you want to have better conversations about the ROI of these tools, understanding where to invest and where not to invest is important. It’s not just about the cost of licenses, though. It’s also about the cost of training, enablement, and support. You want to make sure you’re not underspending in those categories.

For a comprehensive analysis of costs involved, consider reviewing the total cost of ownership of AI coding tools.

Again, like the Core 4, we want to look at all of these dimensions together and not overemphasize just one or the other because overemphasizing, for example, on utilization gives us that tunnel vision where we might see some short-term gains, but we’re forgetting about all of the other fundamentals in our foundation of what software engineering excellence looks like.

Success story from Booking.com

Booking.com has used this measurement framework to inform its strategy of rolling out AI to its developer population of 3,500. Zane, who’s on the DevEx team at Booking, said, “It showed us where to focus and help us get way more impact out of the tools, both in how deeply and how widely they’re being used.” They have 3,500 engineers. They saw 65% higher adoption and saved an additional 150K hours by looking at metrics like the ones in the AI Measurement Framework and then determining the next steps.

Bringing it all together

When you combine these frameworks, you have Core 4 answering the question, “What does great organizational performance look like?” and then you have the AI Measurement Framework telling you, “What is the impact of AI in my organization?” You can make confident decisions and ensure that you’re operating with the right data to know what to do next.

Leaders are going to win who have the right data. We don’t want to walk into a meeting and make the wrong decision based on the wrong information, and so you need to have the right data to adapt quickly. Using the Core 4 together with the AI Measurement Framework will help you lead during this time of rapid change.

AI can help us build better software faster, but it is up to you to determine the best way to leverage it for your particular organization. Looking at industry trends, such as knowing that average developers are saving 3.75 hours, is very important for contextualization.

Still, you must know what it’s doing on the ground in your organization in order to make better decisions. I don’t want you to stay stuck in that disappointment gap between inflated expectations and reality. Data beats hype every time, so it’s on you to get the right data.

For engineering leaders looking to make informed decisions about AI tools, understanding how GitHub Copilot compares to alternatives can be a valuable starting point.

Your action plan

If you are an engineering leader who needs to communicate engineering performance out to the rest of your leadership team, maybe to your board, if you are worried about AI adoption and don’t know how to measure it, you don’t know what to do next, you have some work to do when you get back to your desk.

You can start using the Core 4 to align your business on a solid definition of what engineering performance looks like. Use the template and benchmarks—they’re free and available. You can identify points of friction, fix them, and improve your developer experience.

Then you can use the AI Measurement Framework to start tracking how AI is having an impact on the development population at your company. Think about it as a research project. What’s working, what’s not working, how can we replicate some patterns, and where might we need to try something different?

To support this research approach, consider exploring developer productivity metrics that actually matter alongside AI-specific measurements. You might also find value in using an AI ROI calculator to quantify the productivity gains you’re seeing.

You should approach every meeting with confidence and the data and frameworks to back it up. You should be able to answer three questions:

1. What is organizational performance? What’s the definition, and how are we doing?
2. What is AI doing in our organization? What’s the impact?
3. What are we going to do next?

Better software, faster is possible with AI, but AI is not a magic button that’s going to do it for us. I don’t want you to be stuck in that disappointment gap. Data beats hype every time, and it’s up to you to get the good data.