What is developer experience?

How can engineering leaders measure developer productivity and, in turn, improve it?

Traditional ways of judging developer output, like looking at how much code they write or how many PRs they submit, only capture a thin slice of developers’ work… These types of metrics don’t take into account the complex and diverse activities that developers perform.  

Many organizations find that focusing on a good developer experience (DevEx) is a more predictable approach to improving productivity. They’re learning that a developer-centric approach to understanding productivity offers valuable insights. That’s why more companies are forming dedicated DevEx and platform engineering teams focused on improving the developer experience.

Developer experience concentrates on developers’ actual experiences and daily issues. DevEx improves business performance by boosting efficiency, quality, job satisfaction, and developer productivity.

Today, we delve deeper into how to measure and improve  DevEx by collecting feedback from developers and combining it with data from the systems they work with.  

Table of Contents

What is developer experience?

Developer experience, or DevEx, refers to how developers feel about, think about, and value their work. DevEx extends beyond just the tools used; factors such as clear project goals, having enough time for deep work, and codebase experience play significant roles in enhancing developers’ performance. These elements contribute to developer productivity, satisfaction, engagement, and employee retention, underscoring the importance of a positive DevEx.

Interruptions, unrealistic deadlines, and lack of reliability in development tools are examples of issues that can create a bad developer experience.

Developer experience varies depending on the organization and the team’s area of focus and processes. Understanding DevEx requires attention to both specific personas and broader organizational patterns.

Developer experience has three core dimensions: feedback loops, cognitive load, and flow state.

Feedback loops

Feedback loops enable continuous improvement through rapid learning and adjustments. The process reuses its output as an input, feeding it back into itself and making improvements.

In software development, feedback loops are pivotal for optimizing value streams by minimizing delivery delays, enabling quick iteration, and allowing swift course corrections. 

Short feedback loops enhance the developer experience. Developers encounter numerous feedback-dependent tasks daily, such as waiting for code recompilation or test completions and navigating through approval processes for code reviews. Conversely, long feedback leads to frustration and delays, interrupting workflows and compelling developers to wait or switch tasks, disrupting focus and productivity.

Cognitive load

Cognitive load is the mental processing a developer needs to perform a task. This load usually increases when a developer works on a complex or difficult task or tries to learn an unfamiliar framework. The presentation of information can also impact cognitive load, which increases when individuals need to process information to transform it into longer-term knowledge.

A heavy cognitive load impedes developers’ most important responsibility: delivering customer value. When cognitive load remains high due to poorly documented code or systems, developers must devote extra time and effort to complete tasks and avoid mistakes.

To improve the developer experience, teams reduce cognitive load by eliminating unnecessary hurdles in the development process. A manageable cognitive load relies on clear and organized code and helpful technical documentation.

Flow state

Developers frequently discuss “getting into the flow” or “being in the zone.” These phrases casually refer to the flow state, a mindset in which someone engaged in activity experiences deep immersion, energized concentration, complete involvement, and enjoyment.

When engineers enter a flow state, they tend to be more productive and grow in their roles. Developers who enjoy their work also tend to create better products. However, disruptions and delays can make it difficult for developers to get into that flow state. Managing the way they work, setting clear goals for the team and projects, and taking on engaging and challenging tasks can all play a significant role in helping them achieve flow.

For a good developer experience, teams and organizations can focus on creating an optimal flow state environment. A great environment involves reducing interruptions by grouping meetings, limiting unexpected tasks, and consolidating requests for assistance. Additionally, it’s essential to recognize that a positive team culture, which provides developers with independence and challenging projects, significantly facilitates the flow state.

Can you measure developer experience?

Yes, it is possible to measure developer experience. It starts with identifying what aspects to measure and how to measure them. 

You cannot measure developer experience and productivity with just one metric. To effectively measure DevEx, metrics should focus on the developers themselves and their firsthand experiences in the software development process. The best DevEx metrics encompass the performance of engineering systems and the developers’ perceptions of interacting with these systems – measuring perceptions, workflows, and software development KPIs

To get a complete picture of how developers experience their work, measure perceptions (how developers feel about their work), workflows (the systems and processes they use), and KPIs (Key Performance Indicators that gauge the effectiveness of software delivery).

Measuring DevEx involves understanding developers’ views, feelings, and data about their work systems and processes. Comparing developers’ opinions with actual workflow data is the best way to understand the entire developer experience, as one factor alone doesn’t show the whole situation. For instance, quick code reviews might still bother developers if they interrupt too much, and even if developers are happy with their build processes, actual build times show the process could be faster.

Focusing only on individual factors may lead organizations to overlook overall goals and misallocate resources. Therefore, track key performance indicators (KPIs) that reflect broader DevEx goals, like developer productivity, satisfaction, engagement, and retention. These KPIs guide strategic decisions and show the broader changes.

Here are some examples of each of these styles of measurement across the core DevEx dimensions:

Developer experience surveys

To analyze their software delivery processes effectively, organizations need to collect data from both human sources and software systems through appropriate infrastructure. Once the data is collected, teams can analyze both quantitative and qualitative results by using key performance indicators (KPIs). These engineering KPIs provide a comprehensive view of the software delivery process, which can help organizations improve their overall efficiency and effectiveness.

Developer surveys

Developer surveys are essential for assessing developer experience and gathering feedback on software delivery challenges. When properly designed, they can rapidly collect data, offering timely and precise insights to set benchmarks and drive improvements for your engineering team.

Here are some tips for designing great DevEx surveys:

• Define your objectives clearly to guide your survey design and ensure it meets your goals.
• Keep your survey concise to increase completion rates and obtain more reliable data.
• Use simple language to make your questions understandable for all respondents.
• Break down results by team and persona to improve experiences for specific groups within your organization.
• Avoid leading questions that suggest a particular answer, as they can bias your results.
• Refrain from overloading your survey with open-ended questions to prevent respondent fatigue.
• Avoid double-barreled questions to ensure clarity and reliability of the responses.
• Don’t focus solely on aggregate results, which can obscure specific issues within smaller groups.

Read our guide for more on designing effective developer surveys.

Incorporating quantitative data

Organizations should collect quantitative data from systems and surveys to understand all the factors of a good developer experience. System data helps understand infrastructure and application performance, offering insights that augment self-reported survey information. This process involves integrating data collection into systems and normalizing data across tools and teams.

To assess developer experience quantitatively, consider collecting system data on:

• Build success rate: the percentage of successful versus failed builds, indicating the health of development processes.
• Application performance metrics: includes response times and error rates, showing how well the applications meet performance expectations.
• Deployment frequency: the rate at which your team successfully deploys new releases, demonstrating agility and efficiency in delivering updates.
• Issue resolution time: time taken to resolve bugs or issues, reflecting the effectiveness of the support processes.

Looking to learn how top tech companies measure developer productivity? Check out our analysis on the most popular developer productivity metrics.

While quantitative data provides measurable and objective insights into developer experience, remember that it tells only part of the story. Quantitative data must be paired with qualitative data to understand the developer’s experience fully. Qualitative data offer context and depth to the numbers, revealing the reasons behind trends and anomalies in quantitative data. Incorporating quantitative and qualitative data ensures a holistic approach to assessing and creating a great developer experience, balancing the “what” with the “why” to inform strategic decisions and foster a more productive, positive development environment.

How eBay accelerated velocity by creating a great developer experience 

eBay has prioritized improving how quickly and efficiently it delivers software. The company has started the “Velocity Initiative” project, which focuses on better feedback and reducing unnecessary work for developers. The internal DevEx team looks into problems and finds ways to fix them by asking developers for their opinions every three months and checking how systems work in real time.

Using this information, eBay has fixed issues with developer tools and instructions, simplified development and release processes, and made it easier for different teams to work together. They work closely with other teams to introduce these changes smoothly, including giving training and feedback sessions.

Thanks to these efforts from the developer experience team, eBay’s developers can release updates faster, significantly improving the speed of software delivery and making eBay more competitive. The company also ensures that developers are happy and have everything they need to do their jobs well, aiming for better customer results.

How Pfizer empowers developers to deliver at ‘lightspeed’ 

Pfizer has been enhancing the Developer Experience (DevEx) to enable developers to produce innovations at ‘lightspeed.’ 

The company’s software engineering division expanded significantly between 2018 and 2022, updating its software development practices with open-source and cloud technologies. Pfizer’s focus is now on improving the developer experience to accelerate team productivity. 

Despite regulatory challenges, Pfizer supports small teams and encourages developer creativity. The DevEx team prioritizes feedback and maintains a productive workflow, using surveys to gather key performance indicators and providing tailored advice to improve team performance. Pfizer allows teams to focus on local improvements, which speeds up organizational changes faster than centralized efforts. This strategy includes significant projects like creating a unified developer portal, minimizing redundant code across teams, and enhancing version control tools.

How to start improving developer experience

If you’re beginning to invest in developer experience, here are some simple ways to get started:

• Choose a leader passionate about a positive developer experience to take on the role of developer experience engineer. This individual should be acknowledged for their strong advocacy to invest in DevEx improvements.
• Create a survey to gather feedback on developer tool usage, process satisfaction, and suggestions for improvement.
• Distribute the survey to developers, encourage honest feedback, set a deadline, and send reminders.
• Analyze the survey results to identify common problems and areas for enhancement. Determine impactful changes to make. If this is your first survey, use this as a baseline for DevEx or update existing metrics with new data.
• Implement changes based on the feedback and DevEx data, focusing on small, targeted projects that align with current business priorities. 

Improving the developer experience goes beyond upgrading tools—it involves a comprehensive understanding and support of software developers’ needs. 

By implementing best practices, engineering managers can address bad developer experiences, leading to a more efficient and happier development team. These initiatives make daily tasks more enjoyable for developers and contribute to better business performance through increased productivity and quality. 

Effective DevEx measurement combines people’s insights and data, focusing on critical areas like feedback loops, cognitive load, and flow state. Prioritizing DevEx is essential for fostering a productive, innovative, and successful organization, underscoring the vital connection between the dev team’s happiness and the company’s overall success.