Is GitHub Copilot worth it? An analysis for engineering leaders

Engineering leaders continually evaluate tools that promise to enhance developer productivity. GitHub Copilot, along with other AI code assistants, has generated significant buzz, but does it deliver measurable value? Drawing from data from DX’s engineering intelligence platform and our comprehensive research on AI-assisted engineering, here’s what the numbers tell us.

The hype vs. reality challenge

Before diving into the data, it’s crucial to address the elephant in the room: the massive gap between AI hype and on-the-ground reality. As Laura Tacho, CTO of DX, notes:

“You don’t have to go very far to see sensationalist headlines. Microsoft claims that 20 to 30% of code is being written by AI, while Google asserts that it’s 30%. The CEO of Anthropic predicts that in three to six months, 90% of code will be written by AI. It’s really not hard to find these sorts of statements, but in my opinion, they just don’t match up with the on-the-ground experience we’re seeing from real organizations.”

This disconnect creates unrealistic expectations that can actually hinder adoption. “When you get those lackluster results, then we set it to the side, and that hype cycle is kind of working against AI adoption,” Laura explains. Understanding this dynamic is crucial for setting realistic expectations when evaluating the value of GitHub Copilot.

The current state of AI code assistant adoption

The adoption landscape for AI code assistants reveals a clear performance divide. Among top-quartile engineering organizations, 60-70% of developers use AI code assistants on a weekly or daily basis. For the rest of the companies, this number drops to around 50% or significantly lower.

Abi Noda, CEO of DX, puts it bluntly: “We certainly don’t see the level of impact that’s in the headlines. So, 30% of the code is being written by AI, resulting in 2x, 50%, and 100% productivity improvements? We’re not seeing that anywhere in the data on any sort of consistent basis right now.”

This developer productivity disparity isn’t just about tool availability—it’s also about implementation strategy. Our research indicates that successful AI adoption necessitates more than just purchasing licenses; it requires thoughtful integration, proper training, and leadership support.

Quantifying the impact: time savings and productivity

When we examine reported time savings, developers using AI code assistants save an average of 2 hours per week, although there is considerable variation in this metric. The high end reaches 6+ hours per week, particularly among developers who have mastered advanced techniques like meta-prompting and workflow chaining.

Looking at the actual data, Abi explains: “In aggregate, what we’re seeing is definitely a strong signal around self-reported time savings from developers. So, on average, we’re seeing around two to three hours per week of time savings from developers who are using AI code assistants. That’s meaningful.”

Interestingly, we also observe a small but positive correlation between AI usage and PR throughput, though the relationship isn’t as strong as many would expect.

"We’re not seeing as much correlation to PR throughput as we would expect,” Abi explains.

“This aligns with what we hear in the industry. We hear a lot of organizations saying, ‘Wait, why aren’t we seeing the lift in PR throughput that we would expect, especially given the self-reported time savings?’"

This correlation aligns with broader industry observations, suggesting that while AI tools provide clear benefits, their impact on traditional productivity metrics is more nuanced than simply improving speed.

The top value-adding use cases

Our research identified the highest-impact use cases for AI code assistants, ranked by perceived time savings:

1. Stack trace analysis: Rapidly identifying root causes of errors
2. Refactoring existing code: Large-scale, consistent code improvements
3. Mid-loop code generation: Completing partially written functions
4. Test case generation: Automating comprehensive test coverage
5. Learning new techniques: Accelerating onboarding to unfamiliar frameworks

“Using mid-loop code generation for code authoring wasn’t a primary use case,” Laura notes. “Top cases include stack trace analysis, which can be very time-consuming. AI can really help save time and provide direction when determining the meaning of an error.”

Beyond individual productivity: strategic advantages

The real value of AI code assistants extends beyond individual time savings. We’re seeing three key strategic benefits:

Skill amplification

AI tools enable developers to work effectively outside their primary areas of expertise. A backend developer can now handle frontend tasks with AI assistance, reducing bottlenecks and cross-team dependencies.

Quality consistency

AI-generated code tends to follow established patterns and best practices, resulting in more consistent codebases, which is especially valuable for junior developers or teams working in unfamiliar domains.

Reduced context switching

Features like automated documentation generation and code explanation reduce the cognitive overhead of understanding legacy systems or onboarding to new projects.

Implementation success factors

Our analysis shows that organizations getting the best ROI from AI code assistants share some essential traits.:

• Executive sponsorship: Successful deployments start with clear leadership support and systematic adoption strategies.
• Structured training: Organizations that provide training on advanced techniques, such as prompt chaining and meta-prompting, see significantly higher adoption rates and impact.
• Measurement and iteration: Top-performing teams actively measure AI adoption and impact, using these metrics to refine their approach and identify best practices.
• Reduced barriers: The most successful implementations proactively remove obstacles to adoption, including running models on-premise when necessary and ensuring seamless integration with existing workflows.

Abi emphasizes the importance of active enablement:

“With the organizations we’ve been working closely with on really driving these impact and adoption numbers up dramatically, it’s a hands-on effort. We’re seeing a lot of investment in workshops and trainings, we’re seeing office hours, we’re seeing champions programs, we’re seeing a lot of educational content being created.”

The cultural barriers to adoption

A key insight from our research is that the obstacles to adoption are not primarily technical. The main barriers are cultural. The way humans interact with these tools and the need to change established working methods are what hinder widespread adoption.

Laura emphasizes the importance of leadership modeling: “We need to remember that not only do we need to remove the fear or stigma away from using these tools, like it’s okay, it’s not cheating, but we really need to give people time and space and we need to encourage that kind of experimentation from the top.”

One surprising finding highlights the importance of clear policies: companies with acceptable use policies see a 451% increase in AI tool adoption compared to those without such policies.

Want to learn more about overcoming these cultural barriers? Our Guide to AI-Assisted Engineering provides tactical frameworks for driving adoption, while ourwebinar on GenAI adoption obstacles dives deep into proven strategies from 180+ successful companies.

Advanced techniques drive higher impact

The developers achieving 6+ hours of weekly time savings aren’t just using AI for basic autocomplete. They’re employing sophisticated workflows:

• Prompt-chaining: Creating multi-step workflows where one AI output becomes input for another
• Meta-prompting: Structuring prompts with specific instructions for format and approach
• Multi-model engineering: Using different AI models for different aspects of the same problem

These techniques enable what some call “vibe coding,” moving from conversational requirements gathering to complete code outlines with minimal intervention.

The ROI calculation

When evaluating whether GitHub Copilot (or similar tools) is worth the investment, consider this framework:

Direct costs:

• License fees (typically $10-20 per developer per month)
• Training and onboarding time
• Initial productivity dip during adoption

Quantifiable benefits:

• Time savings: 2-6 hours per week per developer
• Reduced debugging time through better error analysis
• Faster onboarding to new technologies and codebases

Strategic benefits:

• Improved code consistency and quality
• Enhanced cross-functional capabilities
• Reduced knowledge silos and bus factor risks

For most organizations, the math is compelling. Even at the conservative end of 2 hours weekly savings per developer, the ROI exceeds the tool cost by a significant margin.

You can estimate the ROI of Copilot with our simple AI ROI calculator or get ananalysis from the DX team.

Rising adoption and impact

Both adoption rates and impact metrics are steadily rising as the AI coding assistant space advances. Models are becoming more capable, integration is improving, and developers are mastering more sophisticated usage patterns.

Organizations that invest in thoughtful AI code assistant adoption today are positioning themselves for sustained competitive advantage. Those who delay risk falling behind not just in productivity, but also in their ability to attract and retain top talent, who increasingly expect these tools as part of their development environment.

Recommendations for engineering leaders investing in Copilot

1. Start with pilot programs: Begin with willing early adopters and high-value use cases.
   
2. Invest in training: Provide structured education on advanced AI prompting techniques.
   
   As Laura notes: “Companies that recognize AI as a tool requiring enablement and support—similar to any other tool—are the ones poised to succeed. Offering webinars and training programs that explicitly teach developers the necessary skills will help them maximize the benefits of using these tools.”
   
3. Measure and iterate: Track both adoption metrics and impact on productivity (such as speed, effectiveness, quality, and impact). You can do this right in DX.
   
4. Remove barriers: Proactively address security, compliance, and integration concerns to ensure seamless operations.
   
5. Lead by example: Ensure engineering leadership actively uses and advocates for these tools.
   
6. Focus on the right audience: Target developers who have tried AI tools but aren’t using them regularly. As Laura explains: “If you have 50% of users using AI every week, every day, those users, that’s great, keep supporting them. But what you want to do is focus on those other 50%—did they use AI once and then stop using it? What can you do to bring them into the fold? Because that is where you’re going to see the most gain.”

–

The most important question isn’t only whether AI code assistants like GitHub Copilot are worth it, though our data clearly shows they provide measurable value. The real question is how you can implement them effectively to maximize their impact on your team’s productivity and strategic capabilities.