Resources

Today, many organizations aiming to improve developer productivity are finding that focusing on developer experience (DevEx) unlocks valuable improvements. According to a recent Gartner study, 78% of organizations have a formal developer experience initiative established or planned.

One common challenge organizations face is determining the optimal level of investment in developer experience. Some leaders, such as former Shopify and Atlassian CTO Jean-Michel Lemieux, have suggested investing 50% of R&D headcount towards this type of work, while others are more conservative.

To gain insight into the process for headcount decisions for DevEx investments, we interviewed Mike Fisher, former CTO of Etsy. Etsy dedicated 20% of their engineering headcount toward a long-term DevEx initiative. This article describes the rationale and process behind their headcount investment decision. 

How Etsy justified their investment 

Etsy’s DevEx journey started when the company saw historic growth during the pandemic. Leadership put in place an aggressive hiring plan to keep pace with demand, but also recognized the importance of reducing friction for developers to maintain engineering velocity.  

Several factors influenced Etsy‘s decision to invest 20% of their capacity towards the initiative: 

They saw the need to invest in DevEx improvements in order to scale

“No matter where you’re scaling from — from five to 50, 50 to 500, or 500 to 5,000 — you’re going to have scaling challenges,” explains Fisher, who started and led Etsy’s DevEx initiative while CTO. “If you just add more engineers, you’ll be paying for more engineers but they’re not going to be as productive. So really, the thought was that if we don’t do this work, hiring more engineers is not a good investment for the business.” 

Companies tend to see productivity slow after hiring more engineers when they don’t take into account how their processes and technology need to scale to support the growing organization. At Etsy, leadership understood the need to evaluate how people, process, and technology would scale together: scaling one without the others would result in waste. 

They examined the ROI of DevEx improvements

Engineering leadership effectively presented the ROI of select projects they intended to focus on as part of the DevEx initiative, including the opportunity of improving deployment times as well as reducing disrupted pages for on-call developers. 

In general, when discussing engineering projects with business partners, Fisher recommends framing the conversation around incremental revenue or reduced costs. He says, “Instead of explaining that you want to change the way you deploy code using packages, tell the CEO that the work will save 50% of engineering hours spent waiting for deployment, translating to 500 hours of saved engineering time or roughly a quarter of a full-time engineer.”

They understood the impact of DevEx improvements on developer satisfaction

Etsy’s mission is to “keep commerce human,” and internally they’ve always cared about supporting and helping humans. The DevEx initiative was a natural fit for their culture and how the company approaches problems. 

Fisher adds, “One of my colleagues used to say, ‘developers are happy when they’re shipping product.’ We decided that if we can make sure people are able to ship things, we’re not only making them more productive but we’re also making their lives better, which helps them feel more engaged and committed as well.” 

As for focusing on engagement, attrition was also a factor that influenced Etsy’s decision to invest in the DevEx initiative. By focusing on reducing toil and providing an environment where developers can be highly productive, the company reduces attrition risk. “If the industry average is 10% attrition per year, and we can cut that number in half, we save many months of productivity that would otherwise go into hiring and ramping up new engineers,” Fisher says. “These numbers really start to matter as you have a large engineering organization.”  

They reduced risk of a substantial investment by committing to frequent check-ins

With any major engineering initiative, executives will naturally be concerned that engineering will go heads-down and then surface right at the end of the project with an update. To alleviate that concern, Etsy’s engineering leadership committed to showing progress twice a quarter. These progress updates started out as status emails and meetings, and were shared even when the progress was simply “lessons learned.” Eventually, they also started sharing progress with the entire engineering organization during their all hands meetings. 

How Etsy defined success for their investment in DevEx

Measuring the success of DevEx initiatives can be a challenge, but it’s still important to try. Etsy's leadership recognized this and spent considerable time initially brainstorming and debating the right metrics to gauge success. They opted to use multiple metrics, including deploy times and an NPS-like developer score. 

Another core measurement Fisher monitored was experimentation velocity, which represented the number of experiments launched per team per time period (usually a month). “Maintaining the same velocity as your team grows should be viewed as a success, considering the increased communication channels needed for coordination,” Fisher adds, referring to the book Mythical Man Month. “Another indicator of success is when an initiative like Developer Experience becomes ‘evergreen’ or simply part of the organization's standard workflow.”

Organizations seeking to establish developer experience initiatives or teams will need to determine an appropriate level of investment. Etsy's headcount investment and their reasoning may serve as a helpful benchmark for other organizations planning their own investments.

At some point, engineering leaders inevitably face the task of figuring out what metrics to use to measure and understand developer productivity.

There are many different approaches and solutions for this problem, one of which is “engineering intelligence” tools that provide metrics based on data pulled from tools like GitHub and Jira. There are numerous vendors marketing these types of solutions, leaving some leaders wondering how effective these tools really are.

This article covers a real-world story shared by Matthew Schrepel, who leads the Developer Experience team at Extend. Over the past couple of years, Extend has iterated on multiple approaches to measuring software delivery, including initially rolling out a Git and Jira metrics tool. Matthew shares his experiences with rolling out the tool across several hundred engineers, and the outcomes after using the tool for a little over a year.

With that, it’s over to Matthew. 

Executives want information about how the company is doing. How are our teams? What are our issues? They want this information to be able to make choices that improve the situation at their company. Unfortunately, getting that signal in engineering is hard. 

When one of our leadership teams brought up the idea of using one of the many DORA and GitHub metrics-y tools, I was opposed to the idea. In particular, I thought it would be too easy to consume the metrics from those tools and all of sudden start creating weird targets. For example, if the data said “our Jira turnover was slower this week,” and we felt like we needed to do something about it.  

Nevertheless, the leaders that wanted the tool had a clear idea of what they wanted to do with it: they wanted to use it for gut checks. As a manager or leader, you‘re thinking “something is going on with this team,” so you dive into the data to see if your intuition is correct. 

Once we rolled out the tool, exactly what I had feared would happen, did. Engineers saw the metrics and changed their behavior — which is unavoidable, because it’s human nature to shift your behavior when something is being scored or graded or tracked. At the end of the day, this distorted our metrics and made them less useful. 

Another challenge we faced with the tool is that it felt like an endless sea of dashboards, where you have to determine how to get the signal out of the noise. Every time you’d ask a question, you would have to come up with a new way to determine what you’re doing is right. 

Ultimately we felt we weren’t getting what we needed. We ended up instead connecting with a tool that captured developer sentiment, and the signal started flowing much more consistently. We found a lot of value in what we were getting, and our executives did too: after using the new tool, our VP of Platform very quickly said, “This is the data that I trust” — more than the one that had all the dashboards in it. He found the tool capturing sentiment to show what the actual problems were more accurately than did the tool showing us how many tickets developers were pushing each week. 

Thanks to Matthew for sharing his team’s experience at Extend. His team’s story is helpful for leaders considering these tools for their own organizations. For more from Matthew and his team, listen to their interview on the Engineering Enablement podcast. 

‍

Developer experience surveys can unlock powerful insights on what is affecting developer productivity and how to improve. However, analyzing and interpreting results from surveys isn’t always a simple task, particularly in large engineering organizations with diverse teams and roles.

We interviewed Max Kanat-Alexander, who leads the Developer Insights team at LinkedIn, to learn about how his team approaches survey analysis and follow-up. LinkedIn has been conducting a quarterly DevEx survey for several years. In this article, Max explains how his team utilizes developer personas to better interpret and act upon survey results. 

With that, it’s over to Max. 

Infrastructure teams often make the mistake of building one-size-fits-all tooling for all software engineers. This occurs in part because they lack a strong understanding of their customers and inadvertently build what they think their customers need. 

To prevent this at LinkedIn, we implemented a segmentation approach that categorizes the company’s developers into distinct groups. We call these groups "personas." This method has proven highly valuable in our data analysis and decision-making processes.

Developing developer personas 

At LinkedIn there are many kinds of developers, and each have different needs, usage patterns, and productivity issues. To segment our data, we started by developing an understanding of the largest, distinct cohorts of developers at the company. Then we segmented developers into cohorts based on their workflow. These were the groups we identified:

• Backend 
• Web
• iOS
• Android
• Data Scientists
• Machine Learning (ML) engineers
• SREs

Looking closer at the groups, the differences stood out. For example, our Backend developers wrote a lot of Java and mostly used IntelliJ. Our Web developers wrote mostly JavaScript or TypeScript and mostly used VS Code as their IDE. 

We then developed “personas” for each of these groups, which listed the tools they use for each phase of the development cycle.

Assigning “persona owners” to analyze survey results

We look for a developer from each of the personas to be a representative of that group. We call them the “persona owner”; for example, if you are the backend owner, you are actually a backend developer at LinkedIn. Persona owners are responsible for performing the analysis on their group from our quarterly DevEx surveys. 

A persona owner’s job is to advocate to tool and infrastructure owners on behalf of their cohort. Here’s how this works: 

• Persona owners are volunteers and it is worth noting that their role requires a fair amount of effort. We collaborate with the relevant managers to identify persona owners. Nearly all persona owners are senior engineers. In cases where a persona owner is already overloaded with work, we work closely with the relevant managers to reassign the role to someone else before the next survey. A dedicated Technical Program Manager (TPM) assists us in managing this process. 
• Every quarter the persona owners will get the survey data. They will do an analysis on the satisfaction scores and the free text feedback, and then produce an analysis. This analysis document is then presented to various executives and tech leads in charge of the relevant infrastructure for that persona. 

Part of the reason we want persona owners to be members of their cohort is because when reviewing the data, they will have a better sense of what the feedback from developers means and which areas are especially painful. They also operate as the point-of-contact for engineers building the tools when they want more insights about the cohort.  

Why developer personas have been revolutionary at LinkedIn

This system — the Developer Personas and Persona Owners — has been extremely successful. With this model, we transformed the way infrastructure leaders think about internal users. 

Without Developer Personas, it’d be all too tempting to take the largest group of developers and serve only them, typically backend engineers. However, smaller groups of developers are also key to the business. For example, the number of engineers doing Backend Java work is greater than the number of developers doing iOS work, but obviously our iOS platforms are very important for LinkedIn’s members and customers. We need to think about the priorities of serving iOS engineers separately from how many of them we have. We need to look at the pain points of all Developer Personas and consider how we’re going to prioritize our efforts, taking the business context into consideration as well.

Thanks to Max for sharing his experience. Infrastructure teams can fall into the trap of building tools before gaining a complete understanding of their customers. LinkedIn’s strategy of using personas and assigning persona owners can help leaders better analyze and act upon the results from their developer experience surveys. 

Search “engineering metrics” on Google and you’ll find countless vendors offering metrics dashboards using data from tools like GitHub, Jira, and other developer tools. While these solutions may seem compelling on the surface, an increasing number of leaders are opening up about their actual experiences using these types of tools. 

Laura Tacho has been an engineering Director and VP at several high-growth companies including Codeship, CloudBees, Aula, and Nova Credit. While at Cloudbees, she led the roll-out of a Git metrics tool in the hopes of diagnosing and improving bottlenecks across her organization. In this article, Laura shares her experience in detail, beginning with her motivations behind rolling out the product, the challenges she ran into, and the lessons she learned from the process. 

Now, let’s hear from Laura.

Editor’s note: Laura’s story was originally published in The Pragmatic Engineer. Read the full story as well as how Laura believes leaders should measure developer productivity today in the article Measuring software engineering productivity. 

When I was a Senior Director of Engineering at CloudBees several years ago, I helped roll out GitPrime (aka Pluralsight Flow) to 300+ developers. Like so many of you, I sought quantitative data to help me diagnose bottlenecks and free up the flow of work. On top of that, I had pressure from my senior leadership team to quantify performance and report on it. So, I worked with a few other senior leaders to pilot and eventually roll out GitPrime to our engineering organization.

A spoiler: it did not go as I’d hoped. I made three mistakes: I had a blind spot when it came to the importance of qualitative data, I thought activity data would easily showcase areas for improvement, and I let built-in definitions of productivity and performance overshadow the specific needs of my team.

Here’s what I’d hoped would happen: using quantitative data would finally allow us to objectively determine how we were performing. It would be easy to blow past certain topics in retrospectives as we’d have data to show where the real problems lay. I’d share tidy reports on performance, and I could use the data to defend my budget and strategic investments in tooling. I’d finally be able to see performance issues before they became bigger problems.

This wish list is what you’ll find on most marketing websites for developer productivity tools. They promise to identify bottlenecks and help your team, before it becomes a bigger problem. Keep product delivery on track. Get valuable insights you simply can’t get any other way. And so on.

What really happened? My team felt like I didn’t trust them, and that this data was going to be weaponized against them if they had a slow week. Other managers were curious, but lost interest after a while, when it was clear the insights about bottlenecks simply weren’t obvious, even after numerous helpful sessions with our account partner. The heavy involvement in open-source projects made workflow insights almost unusable, pushing the boundaries of the UI, since not everything was self-contained within our own company. No one patted me on the back and congratulated me for finding the key to productivity and performance.

During this time, the most useful insights into our development workflows – and specifically, what we could do to reduce friction and ship more code – came from 1-1s and retrospectives. My team knew where the pain was; they experienced it every day. 

At a team onsite, one of the engineers said the single biggest thing holding us back was the time it took to review PRs. So, we made it a focus for the next sprint. Suddenly, friction evaporated. 

He was right; because we were working on so many single-threaded projects, it was tough to get a code review because context switching is so expensive. A dashboard will give you stats about PR turnaround time. But I didn’t need a tool for this; my team already knew it. And this was mistake #1, overvaluing quantitative data. It was really the qualitative data that steered us toward making the most impactful changes. 

My teams were already practicing continuous delivery, and we were proud of the investments we’d made into our automated testing, build tooling, alerting, and monitoring. We were deploying on-demand, sometimes as often as 20 times a day, with a very high quality of service. My hope the data from our build systems could easily pinpoint bottlenecks was completely misguided. We’d already addressed the biggest bottlenecks. Sure, there was still potential for optimization, but nothing obvious. 

In the absence of obvious bottlenecks to fix, I fell into the trap of relying on other metrics in the tool; either things I hadn’t thought to measure yet, or proprietary calculated metrics like “impact,” to give me signals about my team’s performance. But when I did this, I let a group of people who didn’t know anything about my team’s goals make a judgment call about how these developers should work. I felt bad that our “weekly impact” stat wasn’t going up week after week. 

Some of the out-of-the-box metrics directly contradicted our culture. We spent a lot of time prototyping and used feature flags heavily. This meant our code churn was high, which negatively influenced some metrics like “impact.” And since we started staffing projects with 2 or more engineers whenever possible, it was true – and intentional – that the same groups of people were reviewing each other’s PRs. That often showed up as a red flag in these tools.

Knowing what your metrics incentivize is so important, and I learned the hard way not to let someone else make that decision. I was caught in a trap; DORA metrics had shown us there were very clear trends and patterns evident in high-performing teams, meaning I assumed all other parts of productivity and performance could be similarly quantified and standardized. 

“When a measure becomes a target, it ceases to be a good measure” is Goodhart’s Law, and I hadn’t fully wrapped my head around how this would manifest in my teams. Did I want to penalize a staff engineer for mentoring other engineers, because her commit count was lower on days when she’d spent a lot of time helping others? This wasn’t the type of team I wanted to lead, and honestly not the kind of team I’d expect any of my team members to want to be a part of, either.

Still, for so many of us, coming up with a dashboard of KPIs and developer productivity metrics is not optional. Whether you’re just starting to implement performance metrics, or if you’re already reporting on them, I’ll share what I’ve learned so you can avoid my missteps: 

Don’t let tools make decisions about what’s important for your team. Some metrics in productivity tools are calculated in ways very difficult to defend. For example, Pluralsight Flow’s definition of a thoroughly reviewed PR is calculated on the length of the comments. Of course, you don’t have to look at this metric. But for teams that pair-program or invest a lot of time in group architectural planning, it’s quite a stretch to tell them their PRs aren’t being reviewed thoroughly because there isn’t enough back and forth.

Use metrics to measure system efficiency, not individual performance. Metrics like DORA were never intended to measure individuals’ performance. If your organization is wielding DORA metrics on an individual level – like counting the number of deploys per engineer – these metrics are being misused. This has likely already caused your team to lose trust in leadership, and it incentivizes the wrong things and causes your teams to adapt their ways of working to align with the incentives. Rolling out a tool like Pluralsight Flow, Code Climate Velocity, or similar, in order to get an objective view of individual performance is very misguided. 

If there is one thing to learn about performance metrics, it’s this: development teams want to be consulted about their productivity and performance, not just informed of it. They are highly motivated to improve the efficiency of the systems they work with, because they feel the pain of inefficiencies and friction every day. Include them in this conversation, both by aligning with them on definitions of productivity, and also by listening when they share perspectives on what should be improved.  

‍Thanks to Laura for sharing her experiences. There are many solutions on the market today which provide Git metrics and advertise themselves as effective ways for measuring and improving productivity. Laura’s story is helpful for leaders who are considering these tools for their own organizations. To hear more from Laura, follow her on Twitter and check out her Maven course, Measuring Development Team Performance, which guides leaders on how to adopt engineering metrics.

For leaders championing investments in developer experience, getting executive buy-in can be a big challenge. Executives aren’t the ones to blame: topics like tech debt and release infrastructure are pressing concerns for developers, but are easily drowned out at higher levels of an organization in the face of other pressures and demands.

To capture advice on how to successfully advocate for DevEx investments, we interviewed four leaders from Peloton, Notion, GitHub, and Etsy. This article contains the strategies they use to effectively frame and communicate DevEx investments in ways that lead to executive buy-in. 

Categorize projects into themes that resonate with leadership

Jim Wang, VP of Internal Developer Experience at GitHub, shares how organizing projects into coherent themes helps leaders grasp the bigger picture of projects and enables them to more easily assess their potential value. ‍

GitHub is the home of all developers, and we know how a smooth developer experience makes for happy, productive developers that produce high quality software. We look at the following factors when determining where best to allocate resources and attention to improve our internal developer experience.      

1. Developer friction. We regularly survey our developers to understand the biggest points of friction and determine how to address them. For example, we recently invested in cutting CI times by almost 50% to ensure faster development feedback and speed of hotfixes. 

2. Streamlining costs. We look for areas where we can cut costs without impacting quality of service delivered. If needed, we will pay more for a better experience but also want to be efficient in our use of funds. For example, we parallelize CI to make sure we have a speedy experience, but also are looking for ways to save money - see below for an example where we were able to do just that.

3. Improving the platform. We strive to use our own services wherever possible, which we call “GitHub on GitHub”. This includes everything from Codespaces for development to Projects and Issues for managing work. This allows us to provide feedback to product teams directly and quickly. We are fortunate in this regard that we get to use this lever and be one of our own first users. 

We then take all of these factors into consideration when we prioritize initiatives and work with leadership on alignment. Sometimes all of these categories come together at the same time, making it easy to advocate for improvements. An example of this is the recent initiative where we moved GitHub CI for our biggest internal project from a custom system to GitHub Actions: our developers expressed desire to use Actions, which helped us save money on our CI costs while also delivering a better developer experience. And as we were using GitHub Actions at scale, we were able to provide continuous feedback to the product team leading the work on larger runners for GitHub Actions. 

Being able to advocate for your developer experience initiatives from the perspective of developer friction, streamlined costs, and driving product improvements is very powerful. I’ve personally found it very helpful to use these categories to get leadership buy-in among numerous other competing priorities.

Play the long game: avoid having a single-minded agenda

Willie Yao, Head of Infrastructure at Notion, recommends building trust with executives by making reasonable requests, aligning investments with leadership’s priorities, and being thoughtful with timing.

When seeking buy-in, avoid coming across as being too attached to any particular outcome or agenda. Remember that trust is at the center of all human decision-making. The single most impactful variable of whether you are trustworthy is your "self-orientation"—what others believe your motives to be (see the "Trusted Advisor" or Google "Trust Equation"). Being a broken record of the same predictable agenda ("We need to invest more in developer experience!") is a surefire way to guarantee a loss of attention. Instead, try to represent the company's needs. For example, "If we don't invest more in continuous delivery today, I am concerned that our deployments will grind to a halt in 6 months given our current rate of growth."

Because resource negotiation is a repeated game (preferably an "infinite game"), it pays to build a reputation for being a team player who makes conservative asks. If people trust you and know that you're looking out for the company, they may be more likely to grant you what you ask for. Ask for only what you need and hold yourself to a high bar. Like any negotiation, you have to give something in return for your ask. Don't ask for a blank check, and don't promise something you can't deliver. Be concrete about what problem your leader is having that would be solved with this investment, and be realistic about how long it would take for that impact to be realized.

Focus on what your leaders want and speak in their language. Think about what they care about most and frame your perspective in those terms. For example, if you have product-obsessed founders (as most good founders are), highlight how improving developer experience can lead to a faster or more reliable pace of feature development that will ultimately help them achieve their vision for the product.

Timing matters. You can't fix every area of underinvestment, but you can certainly burn yourself out trying. Conserve your energy on the most important investments that are on the top of a hill today (and maybe just one that you need to push up the hill). You will find that you get a higher expected value this way.

Finally, remember that you are a member of the technical staff for the company first, and advocate for developer experience second. Ask yourself, would you still make this case if you were assigned to a different scope? The best way to get leadership bought in on developer experience initiatives is to advocate for the most impactful possible initiatives across the portfolio of time horizons that matter for the company.

‍Start by identifying “the thorn” in your leadership’s side

Thansha Sadacharam, Tech Insights Lead at Peloton, recommends starting small by tackling a specific problem that is known to the engineering leadership team but that they do not have the bandwidth to solve. By addressing the problem in a data-driven manner, leaders can establish trust and credibility, paving the way for larger initiatives in the future.

We use a Developer Experience survey to identify areas of friction, then we present the results to leadership along with our roadmap. 

I have two pieces of advice for other leaders getting buy-in for DevEx investments. First, if you’re just getting started, identify one problem that is a thorn in your engineering leadership’s side. This is a problem they know exists, wish was solved, but don’t have the bandwidth to fix themselves. For us early on, our “thorn” was our developer onboarding process. 

We gathered data about that problem, compared to industry benchmarks, and presented the results to leadership. This included qualitative data from developers about what parts of the onboarding process weren’t working, and quantitative data about how long different parts of the process took. We presented those findings along with the solution we could implement. This gave leadership a teaser of the things we could accomplish. 

Another piece of advice: try to think about how you can get invited to present to leadership. Here’s what we’ve done in the past: after each survey, we’ll distribute the results to the entire organization, while also sending a one-pager to leadership that contains the most critical findings. This has been successful in getting the Tech Insights team invited to present an overview of the results to the executive leadership team. In these presentations, the focus has been covering one or two key metrics, for example overall developer satisfaction, along with the top three factors driving that score and the top three solutions the team is exploring. In addition, these presentations have included proposals for specific projects, and recommended focus areas for directors and managers across the company.                 

Quantify the business value of projects

Mike Fisher, former CTO at Etsy, recommends focusing on the potential business impact of projects rather than the technical details. By translating project outcomes into business terms, such as time savings or reduced attrition risk, leaders can justify investments to executives.

Most of the time you’ll be the most technical person on the executive team. As soon as you start talking about something technical or getting into the nitty gritty of why something needs to be built, their eyes are going to roll back in their head. It doesn’t make sense to them. 

You have to translate what you’re saying into a language they can understand, which is typically finance. Money. I’m not a huge fan of monetarily-driven metrics, but if you use them to translate the impact of an initiative on the business, they help. 

At Etsy one way we were able to do that was with deploys. If you can take a deploy from 15 minutes down to 7 minutes, we can see how substantial that is by taking into account the average salary for engineers, how frequently they deploy, and the time saved that could be every week. Now, this calculation doesn’t account for things like the cost of disrupting flow. It also is harder to calculate the impact of problems like pages that disrupt on-call developers’ evenings, so they’re less productive the next day. But that’s why I call these calculations “back of the envelope": they don’t have to be precise to communicate the opportunity to the executive team. 

You can also look at longer-term problems like attrition. If someone leaves my company, my estimate is you lose at least 6 months of productivity, or 12 if they’re more senior. So you can start telling your leadership team: “the industry average is 10% attrition per year. If we can cut that in half to 5%, we save X months of productivity.” These numbers really start to matter as you have more engineers. 

The important point is to translate the impact of these problems into numbers that everyone can understand. And know that it doesn't have to be precise; it can be rough math. 

Thanks to Jim, Willie, Thansha, and Mike for sharing how they frame DevEx investments with their leadership teams. Developer experience investments don’t always neatly translate into terms that resonate with executives. The strategies shared in this article can help leaders frame these conversations in a way that builds trust and support from their executive team.

One of the most common problems engineering leaders face is deciding what metrics to track and report to stakeholders. I receive questions about this frequently:

• I was asked by leadership to consider how we measure "FTE productivity" in our engineering organization. How do you handle this kind of thing?
• My CEO came to me and asked me: "how do we know if our engineering org is actually good?"

I personally experienced this challenge in my first job as a CTO. Our executive team had started meeting monthly to review each department’s progress. Our CEO wanted more than just the anecdotal release updates I had been providing, and asked me to come up with a hard metric to report.

The problem: I couldn’t think of a good metric to use. Our CEO suggested the number of features we released, but I resisted because I knew it didn’t accurately account for the size of each feature. Alternatives like the number of lines of code or pull requests felt even worse, and I feared losing the respect of our developers if they found out I was representing our work in this way. In the end, I decided to give in and report the number of features released in order to save face with my boss and other executives.

In hindsight, I wish I’d approached this problem differently and today I am able to give concrete advice to leaders that find them themselves in a similar situation. This advice takes the form of reframing the problem, and a simple classification framework for metrics that leaders should be reporting on. I’ve developed this approach based on years of research and experience advising leaders.

If your boss asks for engineering metrics, start by reframing what they’re really asking for. Leaders often fall into a rabbit hole of worrying about whether to measure individuals or teams, the legitimacy of engineering metrics products, or whether engineering can be measured at all. But this misses the big picture: CEOs don’t know or care about the technicalities of engineering measurement; what they really want is a way to have confidence that you’re accountable for the millions of dollars they are spending on engineering.

Once you reframe the problem in such a way where you’re thinking about how to demonstrate strong stewardship, choosing metrics becomes much easier. For example, your CEO is likely under constant pressure around engineering headcount costs, so a clear accounting of projects being worked on along with their business ROI can reassure them and keep them looped in. If stakeholders are worried about whether the “house is in order” in regards to the performance of certain systems or teams, metrics can help assuage concerns.

Ultimately, your goal should be to proactively address these types of concerns by providing a full picture of engineering. This be accomplished by reporting metrics in these three buckets:

1. Business impact. Your company spends millions of dollars on engineers, which naturally begets the question: “where are we spending this money, and why?”. You should report on current or planned projects alongside data that addresses key questions like: Why are these the right things to build now? How does this project make the business money or otherwise support its goals? Is this project on track or delayed? This type of reporting is often seen as the responsibility of the product team, but only engineering can represent the full set of projects being worked on (e.g. a database migration project is unlikely to be on product’s roadmap).
2. System performance. Engineering organizations produce software, so stakeholders will want to know about the health and performance of these systems. Are they fast and reliable? Secure and well-maintained? Are users satisfied with them? Useful metrics to report here include things like uptime, number of incidents, and product NPS scores. If you have a dedicated Infra or UXR team, they likely are capturing metrics that fall under this bucket.
3. Developer effectiveness. Developers cost a lot of money, so stakeholders will naturally be interested in knowing how productive developers are, and how developer effectiveness is being improved. If you have a dedicated DevProd or DevEx team, they likely are capturing metrics that fall under this bucket. Models such as SPACE and DORA and the more recent DevEx framework are good to check out.

The following table summarizes the three buckets and examples of metrics to use for each one. Metrics can sometimes fall into more than one bucket.

You shouldn’t need fancy tools to capture these metrics–most of the information should already exist through your existing tools and processes, or else manually captured with simple spreadsheets or surveys. It doesn’t matter which specific metrics you use within each bucket or how perfectly things are measured. The important thing is to show intention and effort to your stakeholders, and then continue to iterate on your approach.

There are many metrics being advertised across the internet, which can make the process of choosing metrics overwhelming. I’ve found that categorizing metrics into these buckets forces you to be intellectually honest about why you are using a certain metric: Does the metric actually convey something useful? Or is it just something that was mentioned somewhere?

I see two common mistakes occur when leaders try to come up with metrics without this three-bucket framework:

1. Some leaders try to count things like lines of code, releases, or merge requests. This feels like a good solution because the data is typically right at our fingertips. But there’s strong consensus amongst researchers and experts that output measures are ineffective and harmful.
2. Another mistake I see is that leaders sometimes try to use metrics like lead time or cycle time to convey the impact of their organization to other stakeholders. Although these metrics are useful for measuring organizational effectiveness, they don’t tie directly to business impact.

Take a moment to examine your current engineering metrics, and I think you’ll find that you already have metrics for these buckets represented. Consider reorganizing your reports around this framework to create more alignment around the purpose behind each of your metrics. If you’ve recently joined an organization as an engineering executive, this approach can help you strengthen your relationship with stakeholders from the start.

While many are familiar with the fact that Google, Amazon, and Shopify conduct regular internal developer productivity surveys, there is not a lot of public information about what goes into actually designing and running such surveys. As a result, teams often struggle with low participation rates or harbor doubts regarding the reliability of the results.

Several years ago, Peloton’s Tech Insights team began a developer survey program to help augment the metrics and insights they were collecting through their systems. “Just looking at certain key metrics won’t tell the whole story,” says Thansha. “Having a comprehensive survey to help understand the entire developer experience was really important for us, not only to shape tech learning programs, but also to feed into the platform team and for our engineering managers to understand where they could focus to drive developer efficiency and engagement.”

Today the survey’s results are reviewed across engineering and have become central to how leadership understands developer productivity. This article covers Peloton’s process for designing, conducting, and distributing the results from their survey. For leaders considering building their own DevEx survey, Peloton’s story serves as a helpful guide for understanding what steps are involved. 

‍This article is based on an interview and follow-on conversations with Thansha Sadacharam, who led the design and roll-out of Peloton’s survey program.

Getting executive buy-in

Before starting to design or formalize their survey program, Peloton’s Tech Insights team focused on getting buy-in from leadership to conduct a developer experience survey. To do this, the team zeroed in on a problem they knew was top-of-mind for their executive leaders: the developer onboarding process.

To demonstrate the potential value of a survey, the team gathered qualitative data manually by interviewing developers, then presented the analyzed results to executive leaders. The findings offered a rich understanding of the company’s onboarding process and how it could be improved, and acted as a teaser of the type of insights that could be captured by running a broader developer experience survey. 

As a result, leaders saw the value of moving forward with a formal survey, and stayed engaged in the process throughout the way. Thansha Sadacharam, Peloton’s head of Tech Insights, shared: “You can start this process by identifying one problem that is a thorn in your engineering leadership team’s side. Approach the problem in a data-driven way, present the results to leadership, and solve the problem. That gives them a taste of how a broader developer experience survey and problem will work.”

Designing the survey

Designing a survey that delivers reliable insights is an extensive process. Peloton’s Tech Insights team spent months carefully considering what topics to measure, how questions were crafted, and the overall length of the survey.   

Selecting topics to measure. To determine which topics to measure, the platform team first interviewed developers internally to learn about the tools and processes they interacted with at work. The team then created a map delineating the “day in the life” of a developer at Peloton, which was organized by developer segment (e.g., Android, iOS, backend, frontend engineers).

From this map, the team created a list of topics that could be measured. Then, borrowing the five dimensions from the SPACE framework, the team mapped each topic to its relevant category. Finally, the team added questions to help measure higher-level outcomes, such as satisfaction. 

Crafting questions. The platform team leveraged the expertise of Peloton’s talent team to help write questions and structure the survey. “We have people on our talent team with strong backgrounds in psychology and organizational design,” Thansha says. “They helped us by asking questions such as, ‘what is the purpose of this question’ and ‘how would you take action on the result of this question?’ This helped us refine the topic list and write questions that would yield accurate results.” 

Trimming survey length. As the team refined questions, they also removed many from the list when questions were repetitive or wouldn’t lead to action. Thansha notes that this process of refining the topic list is never-ending: the team reevaluates questions with each survey. Today Peloton’s survey includes ~50 questions and takes developers between 15 to 25 minutes to complete. 

“One thing I learned through this process is the importance of roping in experts. Thankfully we have those at Peloton and we were able to build this with them, because our original version had poorly worded questions, and we had a lot to learn about scales and open text questions and so on. I don’t think we could have done this on our own without those experts.”

In total, designing the developer experience survey was a several month process. Thansha explains: “It was months of design and a lot of human hours considering everyone who was involved. That’s before the launching of the survey, which needed a communication plan and administering and analyzing the results afterwards.” 

Conducting the survey

To conduct its survey, Peloton’s Tech Insights team uses dedicated survey software and proactively engages leaders and developers across the organization. Two specific strategies have helped Peloton achieve higher participation rates: intentional communication plan, and optimizing survey sampling method and frequency.

Communication planning. For years, Peloton has run a company-wide engagement survey, so Thansha’s team created a communication plan to make sure developers understood that the developer experience survey was distinct as well as needed. Additionally, they wanted to be sure developers were aware that the results would lead to real action. 

“We focused on communicating to developers, ‘our team wants to build the right processes, tools, and systems that will drive efficiency, effectiveness, and engagement for you. There are a million things we could work on, but we want to work on the things that are most important to you, and we need your feedback to know what those problems are.’” 

After each survey, Thansha’s team puts together a “state of engineering” report that includes a summary of the results, actions the organization is planning in response, and what actions have taken place since the previous survey. Developers can access this report at any time through an internal landing page which provides access to previous survey results and commitments made by the company. 

‍Iterating on the survey. One way the platform group increases participation rates is through the use of sampling. In earlier versions of their survey, the developer population was split into three groups that were surveyed at multiple points throughout the year. More recently, Peloton switched to surveying half of their developer population twice per year, thereby asking each individual developer to fill out the survey just once per year. As a result of this change, Peloton's average survey participation rate has increased from 52% to 60%. In addition to improved participation rates, reducing their survey has lessened the amount of work involved in conducting and analyzing results.

Sharing the results 

After each survey, the Tech Insights team shares a “state of engineering” report with the entire software organization, which sparks discussion in their Slack channel and captures attention from leaders across the organization.

The team also sends a short one-pager to leadership, containing only the most critical findings. In the past, the Tech Insights team has been asked to present an overview of the results to the executive leadership team. In these presentations, the focus has been covering one or two key metrics, for example overall developer satisfaction, along with the top three factors driving that score and the top three solutions the team is exploring. In addition, these presentations have included proposals for specific projects, and recommended focus areas for directors and managers across the company.

Peloton’s Tech Insight team has invested a lot of time and work into the design and execution of their developer experience survey, in order to achieve success. For others looking to build their own DevEx survey in order to inform developer productivity improvements, Peloton’s story should serve as a helpful guide for understanding the effort and process involved. 

The practice of tracking developer activity metrics is as common as it is controversial. Many organizations track metrics such as pull requests merged, commits, or story points in hopes of measuring and managing team performance. At the same time, there is continued outcry against the use of metrics amongst many leaders, developers, and researchers.

To capture a helpful perspective on this topic, we interviewed Bryan Finster, who previously founded Walmart’s DevOps center focused on scaling engineering excellence across the enterprise. While at Walmart, Bryan witnesses the organization adopt developer activity metrics — an approach he predicted would negatively influence the way developers work. Here, he describes why these metrics were adopted and how they impacted teams. 

With that, it’s over to Bryan.

Metrics are the siren song of so many organizations. “If only we had a simple way to track how we are doing!” The problem is that metrics can be either very useful or very destructive to our goals. It all depends on what we measure and why. If we are measuring things that give us insights into waste, friction, and delivery pain that negatively impact our ability to deliver value, then they are very helpful. When the goal is holding people accountable, they are less than useless. 

For several years, I’ve used delivery metrics to identify problems and help teams correct them. If you’d like to read more about those experiences, you can find my thoughts in  How to Misuse and Abuse DORA Metrics. In that time, I’ve also seen misguided attempts by some to manage through metrics. One common and incredibly bad approach is measuring individual activity metrics. Let me share some real examples.

“What gets measured gets managed.”

An engineering manager I worked with wanted a way to manage team performance. The team was a mixture of experience, from barely out of college to 15–20 years in the industry. The manager, naturally, wanted to ensure everyone was pulling their weight. He also needed a way to stack rank resources for their annual review. Jira dashboards were the obvious choice. By dashboarding the number of tasks each person completed during the sprint, the manager could easily discover who the top developers were and who were candidates for “trimming the bottom of the bell curve.” This lightened the manager’s workload, focused the developers on increasing output, and helped the senior engineers by eliminating the distraction of helping others to grow. Strangely, it didn’t improve the team’s ability to deliver value to the end users.

The outcome of this method of management was predictable:

• Team members would cherry-pick the easiest tasks that could be completed quickly rather than working in a value-priority sequence.
• Staff Engineers focused on the tactical work of closing issues rather than the work they should have been focused on: helping set technical direction, growing their replacements, looking for ways to improve the overall operability and maintainability of the application, and ensuring general technical excellence on the team.
• Work in progress was high because reviewing someone else’s code didn’t help you make your numbers. 
• Collaborating with others on the team reflected negatively on developer activity reporting, so the best thing for each developer to do was to be an individual hero.

This wasn’t a team. This was a co-located group of people working on the same code base. Poorly. Because the goal we had was to help this team improve delivery, the first priority was to educate the manager on how destructive his management practices were to value delivery. “What gets measured gets managed” isn’t a recommendation, it’s a warning.

Code Harder

Our enterprise delivery platform included delivery metrics as a first-class citizen. As an organization, we were using continuous delivery as a forcing function for improving engineering and value delivery across the enterprise. We’d gamified team dashboards to encourage frequent code integration, low tech debt, fast and stable pipelines, and frequent delivery. Something we deliberately avoided in our dashboards was individual developer activity. Teams deliver solutions, not individuals. One area decided to fix this “defect” in our platform by purchasing GitPrime, now Pluralsight Flow. 

GitPrime, and other similar “developer productivity “tools, collect metrics from version control to track individual development metrics such as which developer is contributing the most to which part of the code, how many commits each developer is making per week, how many comments developers make during code review, etc. When I received a demo of this tool, my response was, “Under no circumstances should this be used.” It focused on volume instead of value and would incentivize exactly the same behaviors we’d seen in the previous example. I predicted that it would inspire fear, reduce collaboration, and cause micro-managing. None of those helps the end-user. My prediction was accurate. One team even reported that their director was reviewing the number of code review comments and challenging why they were not exceeding the GitPrime recommended minimum. Their response? They added more words to the review comments. Leaving aside the fact that the best code review process will not result in written review comments at all, no customer ever received value from the volume of review comments. In fact, the opposite is true.

The end result of using GitPrime was the impacted teams focused on individual activity and not value delivery. If I am focused on how many lines of code I’m shipping every day, then I’m NOT focused on the problem we are trying to solve. Solving problems and delivering better value is the job, not typing. The goal of purchasing this tool wasn’t to help teams identify flow constraints. The goal was to hold developers accountable for pushing code.

Are tools that report individual activity totally useless? Not entirely, no. They are far more likely to be used in destructive ways, but there are some things they report that are useful if used correctly. Having a way to see hot spots in the code that indicate that only one or two people on the team modify that part of the code is useful. That shows siloing in the team that puts application stability at risk. However, the same thing can be accomplished without needing any metrics at all. If everyone is pulling the next priority from the backlog, then silos are automatically disrupted.

I’ve been watching more vendors release solutions for tracking individual activity over the years. I don’t blame them. People are throwing money at them to do that. The problem isn’t the tools. The problem is that the people buying the tools don’t understand that software development is not assembly line piece work done by overpaid typists. Software development is a team sport where novel problems are being solved. I do understand the need for understanding individual performance. There needs to be a way to reward high performing people and to give low performing people opportunities to improve or find other employment options. However, there are no metrics for this. Go and see how the team is working. Listen to the people doing the work. Be part of the value stream, not someone who manages with a dashboard.

Thanks to Bryan for sharing his experiences. Developer activity is easy to measure, thus many leaders fall into the trap of tracking metrics that can cause negative consequences. Bryan’s experiences at Walmart can help leaders better understand the pitfalls of using these metrics to manage teams.

At Shopify, the Developer Infrastructure team exists within a broader Developer Acceleration organization that defines their mission as making developers at the company highly productive. Although this mission statement helps guide direction, it is a broad description that leaves a lot of room for interpretation about what projects to actually focus on.

In this article, we cover the story of how the Developer Infrastructure team built on its parent organization’s mission statement to define its own team charter and guiding principles. This article is based on an interview with Mark Côté, who leads the Developer Infrastructure team.

The need for a charter

Shopify’s Developer Infrastructure team owns a wide range of work, spanning everything from developer environments and CI/CD to frameworks, libraries, and productivity tools. Without a clear charter, the team found that it could easily end up taking on projects that weren’t necessarily the highest priority for the business.

Another reason why the team decided to formalize their charter: to help clearly communicate the reasoning for specific projects with leadership. Côté explains: 

We wanted to better communicate to leadership about why we exist and how we think about our problems. That way if they’re looking at an individual project and wondering ‘How does that fit in?’ they can go to our charter and see, ‘Ok, this ladders up to a particular area of opportunity.’ It helps teams communicate about projects without getting into the nitty gritty of why a system needs a particular new feature in it, for example.

Defining a charter

The Developer Infrastructure team’s charter includes two distinct components: opportunities for impact, and guiding principles. These components both tie back to their parent organization’s mission of making developers at Shopify more productive. The following sections describe each of these components in further detail.

Opportunities for impact

Defining a team’s areas of focus, or “opportunities for impact,” helps answer the question of what problems a team does and does not work on. To identify these areas, the Infrastructure team reviewed projects they’d already worked on as well as future opportunities they saw. They also reviewed notes from blogs, podcasts, and conversations about how similar functions at other companies define their charter.

Once the team had reviewed and organized all of their notes, they came up with three categories under which all opportunities could be organized: tightening feedback loops, reducing cognitive overhead, and scaling up engineering. 

‍

Tightening development feedback loops refers to providing developers with the information they need when they need it. Two examples of projects the team has focused on that map to this category: 

• Building an automation system called "Caution Tape Bot", integrated into their service catalog, that monitors pull requests for specific patterns and posts warnings about potential problems or unexpected system interactions.
• Improving notifications from their deployment system to be more focused and ping the right people at the right time without unactionable noise.

Scaling up engineering is about finding ways to increase the total impact of the organization. For example: 

• Spin was put into place because of how hard it was to keep Shopify’s monolith and related services running locally. Côté says, “Keeping dependencies up to date slowly became a very large problem, frustrating developers and overloading our team with support requests.” By having individual, cloud-based systems that were already configured for use and quickly available, developers were able to stop fighting our increasingly complex app configuration.

Reducing cognitive overhead is about eliminating the number of decisions coders and managers alike have to make. This category has included projects such as: 

• Building a system that uses static analysis to identify and automatically remove dead Ruby code. “We're actually also building this into CI, so it fits with ‘tightening development feedback loops’ as well,” Côté adds.
• Simplifying their system that reports on completion rates for action items and degraded SLOs to “clarify the results and ensure they roll up to leadership so they can easily determine the engineering health of their organizations and make deliberate and informed trade-offs.” 

Today, the team focuses exclusively on projects that fall within these categories. The team identifies and prioritizes projects by surveying and speaking to developers. Their channels for collecting developer feedback include a quarterly survey, office hours sessions, and 1:1 interviews with developers. They also receive additional roadmap ideas from a separate team within their parent organization that provides direct support for their internal tools.

Guiding principles

The second component of the Developer Infrastructure team’s charter is a set of guiding principles. The guiding principles outlined below help define how their team should work and make decisions:

1. Maximize impact with effective two-way communication. “We have to talk to our users all the time,” Côté says, “both listening to their needs and communicating our solutions back to them. 

2. Pursue both incremental and big step-change improvements. Côté explains, “We have many services, and they all have rough edges, bugs, and missing features. While one of these services provides value, we'll continue to iterate on it. However, sometimes we need a significant alteration to the way engineering works; sometimes this is because we can't further scale a system, sometimes because we're getting diminishing returns on a solution, and sometimes because there's a new need or useful technology to leverage.”

3. Enable self-service by providing extensible platforms, information, green paths, and guard rails. The Infrastructure team can’t solve every problem for developers, so they try to build open and extensible systems that developers can build and extend on their own. This allows the Infrastructure group to concentrate on higher-impact work. 

4. Absorb complexity. Accelerating development requires reducing the overall complexity that developers face. “We take on some of this complexity for global simplicity,” says Côté. “Accordingly, we don't push any of the complexity of our systems onto our users. Sometimes this is a lot of work for our teams but saves our users more time and effort as a whole.”

In summary, although internal-facing teams focused on developer productivity may understand the value of their work, leaders of such teams still sometimes struggle to articulate this value to leadership and explain why they are prioritizing certain projects. If you're in a similar situation, you can look to Shopify's approach as a useful path forward: defining a team charter can help frame all discussions with leadership, ensuring that the value of a team's projects is properly conveyed.

One of the biggest challenges DevEx leaders face is advocating for improvements in ways that other business leaders can understand. Problems like tech debt or a poor on-call experience don’t always neatly translate into ROI cases because they impact developers in more ways than just wasted time. 

We reached out to Mike Fisher, who was previously CTO at Etsy, on how he  successfully advocated for the company to invest 20% of their engineering capacity into a multi-year DevEx initiative. Here, Mike shares his advice on how to make the case for developer experience improvements. 

With that, it’s over to Mike.

While I was CTO at Etsy, I led a multi-year DeEx initiative that had around 20% of our engineering capacity allocated towards it. 

We were already familiar with that level of investment because we had recently completed our migration to the cloud, a project in which we had invested about 25% of our capacity. That made it a little easier for us with the DevEx initiative, but still, this was not insignificant. These programs come with an investment, but we’ve proven the importance of them time and again at different companies. 

There are a lot of tech leaders who would find it difficult to convey that the company needs to invest nearly a quarter of their engineering capacity in developer experience improvements. But these are the types of conversations you need to be prepared to have. 

Here’s what worked for me when advocating for these investments: 

Translate estimated results into financial impact

Most of the time you’ll be the most technical person on the executive team. As soon as you start talking about something technical, their eyes are going to roll back in their head. It doesn’t make sense. 

You have to translate what you’re saying into a language they can understand, which is typically finance. Money. I’m not a huge fan of monetarily-driven metrics, but if you use them to translate the impact of an initiative on the business, they help. 

We were able to do that with deploys, for example. If you can take a deploy from 15 minutes down to 7 minutes, we can see how substantial that is by taking into account the average salary for engineers, how frequently they deploy, and the time saved that could be every week. This calculation doesn’t account for the cost of disrupting flow. It also is harder to calculate for things like reducing pages that disrupt developers’ evenings so they’re less productive the next day. But that’s why I call these calculations “back of the envelope": they don’t have to be precise to communicate the opportunity to the executive team. 

You can also look at longer-term problems like attrition. If someone leaves my company, my estimate is you lose at least 6 months of productivity, or 12 if they’re more senior. So you can start telling your leadership team: “the industry average is 10% attrition per year. If we can cut that in half to 5%, we save X months of productivity.” These numbers really start to matter as you have more engineers. 

The important point is to translate the impact of these problems into numbers that everyone can understand. And know that it doesn't have to be precise; it can be rough math. 

Showcase results quickly

We made an upfront commitment to share results on a quarterly basis. These check-ins would be used to show progress and signals of whether our investment was giving the returns we expected. We actually shared these back with the engineering organization and eventually made them part of our engineering all-hands. 

By committing to these check-ins, we helped alleviate the concern from other executives that we’d go off, spend a lot of money on these efforts, and they wouldn’t really do anything for the company. We also alleviated the concern that we’d work on the initiative for 18 months and come back with a report once the project was completed. Instead, we decided we’d take them along for the journey and make sure everyone was aware of what was going on. We shared increments results through status emails and meetings, even when the progress was simply lessons learned. 

(As an aside, there is a story where we rolled out GraphQL and had some setbacks and ultimately pivoted away from it. Showing executives that you’re willing to change plans when things aren’t meeting your needs is important.) 

Get ahead of planning cycles

Many organizations believe there should be a pairing of Product and Engineering from the C-suite all the way down to the line manager. Etsy worked that way too, and it was an important way to build relationships. As CTO, I spoke to my Product partner every day. We were in meetings together, we spoke on Slack or in 1:1s, we were in constant communication. That partnership between product and engineering is an important channel to start socializing plans for how engineering capacity should be allocated. 

The timing here also matters. If you’ve already put together an annual plan for engineering capacity, you’re going to have a difficult time getting an investment like this pushed in. If you’re thoughtful, you’ll start having these conversations and socializing your plan ahead of the organization’s annual or quarterly planning cycle.  

In these conversations, keep selling the benefits of developer experience improvements. If we add another engineer, they’re not going to be as productive but they are going to cost the same. So it’s not a good investment for the business to hire more engineers unless we do this work. 

One additional piece of advice: when socializing this idea, executives will want to see examples from other companies who have led similar initiatives. Those examples can come from other companies who have done this, examples from your own experience, or examples from people you’ve talked to. Show what the investment was and the ways that it paid off.

For more from Mike, listen to his interview on the Engineering Enablement podcast and subscribe to his newsletter. 

At some point along an organization’s journey with measuring developer productivity, there is often a discussion on whether or not to track individual developer activity metrics such as the number of commits, lines of code, or pull requests. Interest in these metrics is typically prompted by leaders wanting objective insights into developer performance, or how much work is getting done.

Although the reasons for tracking these types of metrics make intuitive sense, the industry is fraught with polarized opinions on this practice that sometimes spill out into the public. Fortunately for leaders, there’s extensive research on this topic that can help with navigating through discussions and decisions on what to (or not to) measure. This article summarizes this research.

Activity metrics capture only a thin slice of developers’ work 

Software engineering is difficult to measure in part because it is a highly complex task. Therefore, traditional productivity metrics that focus on output volume do not readily apply. Furthermore, when evaluating these metrics based on individual or team, more is not necessarily better. A recent paper written by Google researchers articulates this point bluntly:

Similarly, in their seminal paper “Defining Productivity in Software Engineering,” authors Stefan Wagner and Florian Deissenboeck state that leaders must consider factors beyond activity when measuring developer productivity: 

Aside from the creative nature of the work, developer activity is also difficult to measure because it involves many different types of tasks that are not easily captured. In the paper titled “The SPACE of Developer Productivity,” the authors caution that “because of the complex and diverse activities that developers perform, their activity is not easy to measure or quantify.”

Google researcher Ciera Jaspan further elaborates on this idea in the paper, “No Single Metric Captures Productivity”:

Still, many leaders today focus solely on measuring individual activity metrics. As described by Dr. Margaret-Anne Storey, a co-author of the SPACE framework, this is something researchers are concerned about:

Next we’ll cover research on the potential consequences of utilizing developer activity metrics. 

Activity metrics create misincentives and can hurt morale

Several research studies have investigated the consequences of tracking individual activity metrics. One key finding is that developers hold skepticism and fear towards these types of metrics, which may result in them “gaming the system” out of self-preservation. This is further discussed in a recent paper from Google: 

In another paper titled “Summarizing and Measuring Development Activity,” the authors caution that the mere presence of activity metrics can warp incentives, even even when those metrics are not explicitly being used to reward or penalize developers:

Google researchers have also written about the morale issues that can arise from use of individual activity metrics. These morale issues negatively impact overall productivity, and can also cause attrition issues.

Activity metrics provide limited value overall

There are several common motivations for why a company may be interested in measuring the activity of their developers. However, in each of these cases, research shows that activity metrics generally do a poor job at providing valuable insights.

Identifying high and low performers. Companies often look to individual metrics to help them assess individual performance. However, as described earlier, activity metrics provide a limited view of activity that often doesn’t provide value. In “No Single Metric Captures Productivity,” Google researchers state:

Helping developers grow their skills. Although many metrics vendors cite this as a use case, there are no research studies which validate the use of activity metrics for helping developers grow their skills. Rather, a recent paper by Microsoft researchers identified the top five attributes of great engineers as: 

1. Writing quality code
2. Anticipating future needs and trade-offs
3. Gathering information to make informed decisions
4. Enabling others to make decisions efficiently
5. The ability to continuously learn and find solutions

For leaders who hire or coach developers, these findings present valuable insights for hiring, upskilling, and supporting developers’ growth. Activity metrics, however, do not help developers or managers assess or boost skills across these attributes. 

Instead of attempting to use individual activity metrics to evaluate developer performance, consider improvements to your formal performance review processes, starting with a clear definition of role expectations and then working backwards to identify potential metrics.  

Improving engineering productivity. Another common motivator for tracking individual activity metrics is to identify inefficiencies and improve engineering productivity. However, activity metrics are heavily influenced by outside forces: when they’re used in isolation, they may provide misleading signals into where productivity issues exist. This is explained by Microsoft researchers in their recent paper:

A recent paper by Thoughtworks further recommends against using activity metrics to try to improve developer effectiveness:

The decision of whether or not to track developer activity metrics is an often debated topic. The findings discussed in this article, which come from researchers at companies like Google and Microsoft, can help leaders better navigate the potential pitfalls of using these types of metrics.

Choosing what to measure is foundational to a successful developer experience program. Here, we’ll cover the topics to measure, ways to measure them, and other important information to capture in your developer experience survey — all while weaving in insights from implementing our DevEx360 product with organizations of all sizes. 

1. Measure a complete set of topics

Most leaders will already have formed assumptions about the areas of friction that developers are facing. These assumptions might have been formed from conversations with developers about specific tools that are difficult to work with or processes that are wasteful. However, choosing what to measure based on such conjectures alone is risky, as it may result in leaders receiving an incomplete understanding and failing to identify the most significant opportunities for improvement. 

DevEx 360 consistently tests and refines its measurement framework, which was originally developed from research that investigated the factors that matter most to the developer experience.

2. Capture two metrics for each topic

There are two types of measures we can use to capture each developer experience factor: perceptual and workflow measures. We recommend both because neither alone can provide the full picture. 

Perceptual measures capture how developers feel about or perceive something, and are a great way to understand pain, friction, and toil. These answer the question, "How do developers feel about this topic?"

Workflow measures balance perceptual measures by capturing an objective view of how systems or processes are behaving. These answer the questions, "How long does this take?" or "How often does a thing happen, interruptions or failed builds?"

Taken together, we balance our view of performance with developer satisfaction and identify problems we may not be able to notice otherwise. For example, our workflow measures may show that we have a fast build process, yet at the same time our perceptual measures are telling us that builds feel disruptive to developers, because they regularly interrupt their work progress. Conversely, we may see that developers feel content with their build processes, however workflow measures reveal that build times are slower than they could be. 

3. Ask developers about the importance of improving each topic

By measuring a comprehensive set of factors as well as capturing workflow and perceptual measures for each factor, we now have a full understanding of the developer experience at our company. However, reaching this stage can be daunting: the data will show that there are many problems the organization could focus on. This is why capturing how important problems are to developers is critical.

The importance of each item can be measured by asking developers which items they would most like to see improved. This adds an important layer of context: developers know what their biggest problems are, so by asking them to rank the problems they’d most like to see improved, this gives us an idea of which areas are either causing the most cognitive load or causing friction frequently. 

4. Ask for comments at the right time

Nearly every company we’ve worked with that previously ran developer experience surveys saw especially low participation within their comments. When leaders treat comments as a “nice to have” or a bonus if a developer responds, they miss an important layer of context that would otherwise guide their decisions. When a survey is designed to strategically capture comments, the result is having the majority of your developer population sharing detailed context about what problems they’re experiencing, when, and how that affects them. This can eliminate the need to run focus groups or interviews to understand what problems developers have. 

There are two main reasons comments get low participation rates: anonymity and capturing comments on too many topics. Comments should be non-anonymous because it allows DevEx teams and managers to follow up with the engineers who provided the feedback to ask clarifying questions, or to get their perspective on how the problem may be solved. We should also only ask for comments on the problems that are deemed as top priorities.

5. Track KPIs to report on progress

While tracking higher-level outcomes will help leaders stay aligned with business goals when prioritizing initiatives, we find that customers find tracking Key Performance Indicators (KPIs) especially useful for communicating the value of the work they’re doing. 

KPIs provide a north star metric that can be reported on and communicated concisely. For example, a DevEx team might say their overarching goal is to “make engineering more efficient”, and use Weekly Time Loss to show leadership the impact of their work. 

The foundation of a successful developer experience program 

As with any metrics program, not being thoughtful about what information is captured can lead to incomplete or misleading results. When organizations start by investing in building a solid foundation of measurements, they can more easily achieve a comprehensive understanding of the state of engineering. 

Platform engineering work is seriously under-resourced in many organizations, in part due to the lack of a universal definition of what platform engineering is and the difficulty in communicating the reasons for different investments to business stakeholders.

Jean-Michel Lemieux, former CTO of Shopify and VP of Engineering at Atlassian, has strong opinions on platform work—and equally strong strategies for elevating it. In this interview he explains how to advocate for serious investment in platform work to leverage long-term business impact.

How do you define platform work?

Jean-Michel: That definition has actually evolved over time—and a lack of shared understanding of what that means, means we probably talk about different things.

Historically, you think of platform companies like Microsoft or Apple that are building APIs and STKs. If you think about what these platform companies have done, they’ve built a long-term high-value feedback cycle of work that people can plug into. 

Every company, I’d say within a year of having a product, has some parts of platform work that could provide outsized long-term advantages. Recognizing what those things are versus the things that give you shorter-term leverage is really important. You work on those things in different ways.

The platform is more than the infrastructure that you run your stuff on. A lot more. The “lot more” is key, and includes everything that helps developers go faster: the development environments, the tests, etc.

That includes your platform decisions, the technology choices across all those layers, your UI framework, your server, so on. Then your core abstractions, and—this is important—your domain model. Where does your code go? Do you have one service or many? How do you actually model your database structure and your business model? That’s part of your platform because it powers a lot of things. It’s probably going to make the difference of your medium- to longer-term velocity.

At least in the context of our chat we’re having, that’s what I consider platform work.

I like that definition: any work that provides leverage to the organization, that you can distinguish from feature work providing direct dollars or direct value to customers. How can we think of platform work organizationally? Is it somehow distinct from feature work?

Jean-Michel: Your organizational playbook of how you get work done is part of your platform. I call that your company operating system. The way you decide to work together as a team gives you long-term leverage.

If you want to apply that to the definition that we just talked about, it’s also important to realize that platform work is actually very customer-centric. Look at your platform work as being some of the most important customer features that you have.

If your performance sucks and you have no scalability, you probably have really low extensibility. It’s probably hard for you to release new features so your customers aren’t unhappy. You have to look at platform work not as a black hole of tech debt. Some of your top customer features are going to be enabled sustainably by work in this area.

What’s your advice for getting buy-in for platform work from less technical stakeholders, who may not intuitively understand what platform work is or why it matters?

Jean-Michel: I advocate for 50% platform work. It’s a bit of a controversial point, but starting a conversation with that creates a bit of “what-the-hell.” Now a conversation has to happen because you just brought out a big number that scares the crap out of everyone. 

So, my biggest advice for folks is to spark the conversation. That 50% is a tool for engineering leaders to start conversations. You have to be an advocate for resources. You need a better plan than any product manager in your company, and you have to be better organized than anyone else.

I dare say it almost puts the burden of proof on executives to answer why not 50%, rather than on you to answer why 50%? But still, most teams are far off from that 50% of R&D spend focused on platform work. How do you arrive at this number?

Jean-Michel: Well, I picked that number partly because it sparks a conversation. But I also pick it because, through the last ten years of jobs that I’ve had, as an engineer, as an executive, I see the realities of what’s under the covers.

We’ve created companies where, organizationally, conversations aren’t actually happening. It’s almost like those long-term issues are dismissed as “just engineering stuff.” But some of the most important decisions you’re going to make as a company are about how the platform is built.

We’ve created companies where, organizationally, conversations aren’t actually happening. It’s almost like those long-term issues are dismissed as “just engineering stuff.” But some of the most important decisions you’re going to make as a company are about how the platform is built.

I think, conceptually, people see strategy as the why but not the how. Execs are going to go, “We want to go tackle these things, these are our objectives. I don’t care how it gets built, just get it built.” I have a very different philosophy. I think the origins of the word strategy are actually in the how.

How then becomes how are we going to implement all these things, and that is an extremely strategic conversation. Leadership has to vet, understand and support the how. But platform work gets ignored because people feel it’s in the weeds. I think some of those weeds are strategic.

I’ll give you a super interesting example. The most strategic decision that Twitch made was, do they build a video streaming server themselves or not? That was an extremely important platform decision in their first 18 months.

The leverage it provided meant that they could stream videos at 10x lower costs than anyone else. Which means they can do more of it. Twitch is alive today because of that one technical decision. They decided to do it and it changed everything. That’s a leverage decision.

In your company, are you making any of those kinds of decisions that get ignored as being an implementation detail, when the how matters a lot? That’s why companies are at 10% or 20% platform work. They call me and it takes us three years to unwind it. That’s the story with every organization I’m working with right now.

Does this 50% number in your mind apply mostly to large organizations? Does it apply to small organizations? What’s your take on how you think about how allocation pertains to stage and size of a company?

Jean-Michel: The rule of thumb to invest 50% of R&D into platform work applies to one-person companies and 10,000-person companies. Extremely strategic decisions for any company in the long term are encompassed in platform work. It has the bad rap of being tech debt or just infrastructure, but I see a lot of problems in core domain models that don’t get updated.

The rule of thumb to invest 50% of R&D into platform work applies to one-person companies and 10,000-person companies. Extremely strategic decisions for any company in the long term are encompassed in platform work.

People piggyback. They want to go fast short-term and they end up with three years of unwinding. That work could have happened in year one of their existence, but companies push it off.

There’s a lean startup way of “just throw it all together, we’re going to have to rewrite it anyway. Throw it at a wall and see if anyone likes it.” You do have to get used to rewriting things. Which means that platform work’s not about getting perfection, it’s about the habits of yearning for perfection but not getting it. That has to start on day one of your company. 

You’ve seen a lot of organizations get blindsided later. Yet we all know on some level that this platform work is essential. So why do you think it is so consistently underinvested in? 

Jean-Michel: Culturally, we split. We have a lot of specialized roles in companies now. There’s a power dynamic difference between who decides what work gets done. There’s a lot of product managers out there that are fantastic, and they run a lot of the roadmaps. At a lot of companies, that’s just how it is.

And I think that gets in the way. I believe that people don’t have a shared understanding what platform is, so you don’t know what to put in a bucket. It’s a lot of work to be a really good product owner of the platform. It’s a part-time job. And I think engineers are shitty at communicating that long-term. They don’t sell it well, and their story’s not good, and they fall behind.

It’s a lot of work to be a really good product owner of the platform.

All that combined is probably one of the reasons why platform work is not an investment as much as it should be.

I want to ask about how developers and leaders can actualize fully supported platform work in their organizations. One approach gaining a lot of attention across the industry is this notion of a dedicated platform team. What are your thoughts on that?

Jean-Michel: Let’s ignore the org structure first. I think having a platform team could be an anti-pattern, because I think there’s a platform component of every team.

Every team that’s building parts of your product has a platform component in what they do. If every team has 50% platform, 40% features and 10% experiments, then you can’t really reorg and create a platform team. It’s actually dangerous to do that.

Now you can have a team that does developer productivity and maybe a specialist on data infrastructure. But saying that the platform team owns your entire platform roadmap is an anti-pattern. So the concept of having a platform roadmap has to transcend your org structure.

Saying that the platform team owns your entire platform roadmap is an anti-pattern. 

And you need a leader, a PM of the platform, who decides what to work on, with a view across the company of what’s going to provide leverage. Platform is so misunderstood, I’d call the other teams what they’re actually doing.

I find it is useful over time to extract some things that you’re doing across the company. At Shopify we had a Rails team. Everyone’s doing Rails at Shopify. But there was a team, super involved in the open-source community, working on the next version of Rail. Their job was always to be an accelerator for all the other teams. When we adopted a new Rails version, every team had to do a bit of work. But these were the experts enabling everyone else to do it, without doing it for everyone else. 

So don’t call that a platform team. Call them what they’re actually doing. Developer acceleration team, hosting team infrastructure, data pipelines team, whatever the mandate is for that team.

‍This interview is part of our Fieldnotes series, where we share how industry leaders are improving different dimensions of the developer experience. In this interview, Bruce Weller, VP of Engineering at Divvy (acquired by Bill.com), describes the practices he coaches teams on to help them set clear and realistic deadlines. 

Missed deadlines are often the result of simple miscommunications. I’ve seen it throughout my career: a team didn’t separate their “estimate” from the “deadline”, didn’t discuss the cost of failure, or didn’t agree on the definition of done. They “missed” the deadline because there wasn’t full clarity around what the deadline meant. 

At Divvy we’ve written these practices down and coach managers to follow these practices. We’ve gotten a lot better as a team about hitting deadlines.

1. Separate “estimates” from “deadlines” 

There’s a difference between an estimate and a deadline:

An estimate is your closest approximation of when something might get completed. If you picture a standard bell curve, where the top of the bell is your estimate, what you see is that most things get completed either a little before or a little after the estimate. The important thing to note is that “a little after” or “a little before” are both perfectly valid outcomes.

But oftentimes when people say “estimate” they’re actually talking about a “deadline”. And as soon as we’re talking about a deadline, we’re talking about a date where the work is ideally completed “ahead” of time rather than behind. 

Picture the bell curve again. Where with an estimate you’d expect 50% of your projects to be completed before and 50% after the date, with a deadline you’d slide that date back to a confidence level you’re comfortable with. (e.g., a 90% chance the work is completed before the deadline, 80% or 70%.) For deadlines, we take the estimate and slide it back to a point where there’s a higher probability of the work being completed “before” that date.

2. Discuss the cost of failure

Teams should decide what level of confidence they’re comfortable working with. Do you want 70% of your projects to be completed before the deadline? 80%? 90%? That’s a decision teams can agree on. They should also reevaluate that level of confidence for “big” projects. 

We can come to a specific level of confidence by considering the cost of failure for a given project. When I explain what the cost of failure is with teams, I use an analogy: Would you rather walk the length of a ten-foot plank that is four inches wide, or that is eight inches wide? Of course, we would all take the eight inches if we are trying to succeed. 

Now let’s place the four-inch plank two feet off the ground and the eight-inch plank twenty feet off the ground. Make your choice again. Same thing with the plank at one hundred feet. You get the idea.

Which height best represents your project? What is the cost of failure? Is this a “bet the company” project? If so, you’ll want more certainty.

3. Agree on what is “complete” at the deadline

Get on the same page about what will be complete at the deadline. Is the project through testing, is it through production, does it mean we’re A/B testing with customers? 

I’ve seen developers believe they are aiming for “code complete”, when stakeholders were expecting deployed features available in production. Make sure everyone knows the definition of “complete.”

4. Communicate the stages of an estimate

One way teams can be more clear in their communication is to use the terms “rough estimate,” and “firm estimate.”

I want a low barrier for Product to ask Engineering for an estimate. I want them to come to us early. Maybe they ask “Hey, what would it take to get this feature?” We want to partner and also to protect against misunderstandings. If they give me a rough spec (which could be just a sentence), we can give them a rough estimate. Dropping the word “rough” can get us into trouble — it might go to the marketing team or the executive team — but using the word “rough” gives us flexibility. 

So Product comes back with more information, “We figured this out, here are some bullet points of what we’re going to need.” We look at it and say, “Oh, we thought you were going to need this and it’s not there, so we can actually reduce the rough estimate a bit.” Product goes off and comes back with a more refined spec, now there’s an integration we weren’t anticipating. We can say, “let’s change the rough estimate to reflect that.” 

What we get through this process of refining specs and estimates is a funneling process. And at a certain point, we’ll be in agreement of what the spec is, and what the estimate is. We’ll be expecting little change from this point. Now we switch to the terms “firm spec” and “firm estimate”. And from the firm estimate we can build a deadline.

The team knows the deadline is important to hit. We all expect to hit it. If we miss a deadline, we hold a post-mortem and learn from the experience.

If missing deadlines is a pattern, then we want to understand why. What’s the failure in the process? Is there a skill that’s missing from the squad so they end up waiting for someone else? Is there miscommunication around the deadline or are the requirements not fully clear? 

If a team has a pattern of delivering and then has a miss, we don’t punish them — why punish someone right at the moment they’ve gotten smarter? But we do want to dig into the root cause and see what we can learn and adjust.

5. Allow pushback 

We don’t hire software developers for their proficiency at public speaking. Some of them are naturally great at it, but not all of them. Product Managers, on the other hand, must be proficient with this skill. They use it to motivate people, get members of a group on the same page, communicate progress, and build support for their initiatives.

Due to this mismatch, an anti-pattern that sometimes emerges is a Product Review where Product speaks for Engineering. This can lead to a misunderstanding about estimates and deadlines and an incomplete commitment to a deadline. Our groups have worked together to keep this from happening.

One trick we’ve discovered is for the Product and Engineers leaders to do a dry run of the Product Review in advance of the actual meeting. This lets everyone see the deck and voice concerns if anything is off. It’s a low-stress environment where engineers feel more comfortable pushing back.

6. Track say:do ratios

As part of our Product Reviews, teams share their say:do ratio. They feel a sense of pride in it, and get to brag, “Here’s our say:do ratio; we hit our last four deadlines.” Time ranges can be flexible. The point is to give teams a reason to track and a forum to brag about making and meeting commitments. (Teams get to brag about quality too, but that might be another post.)

Parting words 

I’m big on autonomy, which means teams get to make their own decisions and they also get to fix things when they go off the rails. Estimates and deadlines are discussed openly and constantly. They are focused on. And, that being the case, our squads and squad leaders naturally get better at them with each trip through the process.

This interview is part of our Fieldnotes series, where we share how industry leaders are improving different dimensions of the developer experience. In this interview, Olya Royall, Senior Engineering Manager at DroneDeploy, shares tactics she uses to set realistic deadlines and help her team meet them.

Olya: I joined DroneDeploy as an individual contributor over five years ago, and moved into management since then. So some of what I focus on in my role now is a result of having experienced different management styles so recently. One of those areas that I focus on is making sure my team has realistic deadlines. 

Today our team consistently rates “realistic deadlines” highly in DX — it’s an area the team is satisfied with. So here are a few things I think about when setting deadlines and making sure our team’s processes aren’t getting in the way of us delivering. 

1. Get to know the team and the codebase 

It’s especially difficult to estimate for the first quarter of starting a new team. You can use your previous experiences as a manager to set estimates, but you still have to audit how the team does for that first period to see what their velocity is. 

It helps to know the codebase too: if there’s a particularly complex part of the codebase, we can add buffers in our estimates to account for the extra time it may take to work in that area. You don’t have to be “in” the codebase to gain that knowledge; managers can gain that knowledge by paying attention to projects that were delayed or slower than usual. 

Getting to know the team and the codebase can help managers check the team’s estimates. We do t-shirt sizing ahead of the quarter for every project, for example, so we set small, medium, large, or extra large for each project and then see what we can do with our capacity. If my colleagues think something is a medium project, but from my experience I know that working in this area of the codebase tends to take longer, I can push back on their estimates. And as a general rule, we try to set an upper limit of an estimate versus being overly optimistic about what we can accomplish. 

2. Limit projects in flight 

Things move slower when the team is working on a bunch of projects in parallel. We try to limit the projects in flight during any given sprint to one or two projects — even three or four projects is too many. And the team really appreciates this. It helps them stay focused and finish projects faster. 

3. Put engineers on a support rotation

It’s easy to fall into a reactive mode with fixing bugs and ultimately miss deadlines. Our engineering team is around 60 people, and we previously only had one engineer fully dedicated to support engineering. We made the decision to put engineers on support rotations — we have several clusters of engineering teams, and each cluster has its own support rotation schedule. This enables each team to be effective at triaging issues in their team's domain and frees up the rest of the group to focus on feature delivery when they're not on rotation.

Each rotation is one week long, and the frequency which engineers go on rotation depends on the size of the team: I have seven engineers on my team, so each engineer goes on rotation once every seven weeks. 

4. Make space for deep work with no-meeting days, short standups, and async status updates

Uninterrupted time is critical for teams to meet deadlines, so we try to be thoughtful about the meetings we schedule. For example: 

1. We have no-meeting Wednesdays. Team members can pair with one another during this day, but otherwise team members don’t have to worry about getting interrupted or making it to meetings.
2. We keep our standups to 15 minutes. These aren’t status updates: we have a chatbot that helps us with status updates (it asks “what are you working on today?” “Anything you’re blocked on?” and “Anything you’ve been particularly proud to work on?”). Standups are for raising concerns, questions, or to discuss blockers. Also worth noting is that we do standups on Tuesdays, Thursdays, and Fridays. 
3. We also have team pairing hours, which are essentially office hours, two to three times a week. Team members block time on their calendar where they know they can pop in and ask each other questions. 
4. Every two weeks we do a Friday fun day, where we hang out for an hour as a team and play games. It’s a way to close out the week and relax with the team. 

Generally, Mondays and Fridays typically have the most meetings and the middle of the week is better for focused time. 

5. Keep a pulse on team sentiment

We keep a pulse on what’s going on with teams using perceptual data so we can know whether deadlines are realistic — and if not, we consider what might be causing teams to miss their deadlines. (Is it the deadline itself, or is the problem other things that are getting in the way?) 

As a leadership group we also meet quarterly to discuss our OKRs, and we always include initiatives to improve areas of the developer experience within our objectives. 

This interview is part of our Fieldnotes series, where we share how industry leaders are improving different dimensions of the developer experience. In this interview, Nahi Ojeil, VP of Engineering at CaptivateIQ, outlines the principles he’s established for working effectively across teams. 

‍Nahi: Cross-team collaboration isn’t something leaders need to think about in the early days. It’s when you start formalizing teams around areas of ownership that this needs to be top of mind. No company can be perfectly split into silos of teams that never interact (nor would you want it that way). 

At that point, leaders will need to give people the tools they need to make good decisions at scale. They don’t necessarily need to establish process, because we want people to understand the reasons for working in specific ways and not dictate the exact solution they take. Instead they need to establish principles that teams can use to know how to work with other teams. Here are the principles I’ve used in different companies: 

1. We optimize for the same company

There are a few principles we follow in our company for cross-team collaboration, but I always start with this: everyone needs to understand that we’re optimizing for the same company. There’s a single company, with a single set of overarching priorities and goals. 

That’s an important principle to ingrain in every single person. Because when they encounter a situation where there are different priorities, there’s still an understanding that those priorities aren’t “competing.” There can be different priorities, but we can always ultimately answer which one is right for the company. 

One tactic here is to encourage internal mobility. If you’re an engineer and have moved between different teams, you’re more likely to look for dependencies, ask questions, and reach out to different teams — because your identity is not as tied to a single team. In practice, we do this by: 

1. Having internal programs that the People team runs that encourage people to apply to available roles.
2. Running employee assessment and talent mapping exercises as an engineering leadership group, and flagging people who may benefit from trying something different. That’s either because they’ve been in a role for a long time or because they’re in a role that’s not a great fit for them. 

2. We identify the core dependencies upfront

This one is a balance: it’s hard to identify every single dependency before starting a project. It can be time consuming to the point where that level of planning may be a worse option than simply finding and working through dependencies during the project. Also, as a leader, you don’t want to push a culture that’s overly focused on de-risking things. That can create an environment where there’s fear around pushing forward without getting all the answers first. 

But what we can do, within each team’s cadence of planning, is identify the biggest dependencies on other teams. 

A couple of tactics have helped us identify dependencies on other teams and better manage them: 

1. We use a template for kicking off a project, and that template explicitly asks you to assess your major dependencies. 
2. We also try to include the teams that are dependencies in that project’s kickoff. It helps those teams to feel more included in that initiative. 

3. We’re comfortable escalating issues 

Being comfortable with escalating issues and asking for help is a skillset that very few teams have. Leaders need to make sure it isn’t taboo, because these issues happen all the time: one team doesn’t prioritize a task for another team, or they’re slow to get it done, because they don’t see it as being the priority. 

There are a couple reasons teams are often not great at escalating: 

• They’re trying to be nice and collaborative. Sometimes they’re collaborative to the point where they’re unwilling to realize that the project’s not going anywhere. So the reluctance towards escalating something is coming from a place of goodwill. 
• Also, people worry that escalating is a negative thing. They worry they’re putting the other person or team in a bad position. Or it’s a sign they personally failed because the expectation is they should be able to figure things out on their own. 

Unless there’s a good understanding of the value of escalating issues and how to effectively use escalating as a tool, teams won’t do it as often as they should. 

Leaders should also be mindful to model the behavior of escalating things well. They shouldn’t act like there’s never friction between them and their peers. I’m not always on the same page as my peers, but that’s not a sign things are bad. It’s a sign that we all have our own perspectives and we need to figure out how to get to a good outcome together. So as a leader, if I can show to the team how I’m escalating things — instead of hiding it — that can go a long way to helping teams become more comfortable doing it too.  

One final note: sometimes there are communication gaps, where a team doesn’t communicate their ask in an effective way. It can go a long way to coach teams on how to communicate their priorities—for example, it’s much more effective to help the other group understand the problem instead of giving them a specific solution. 

4. We plan for unexpected things to come up

I like to make sure every team is expecting unexpected things to come up when they’ve done their planning. Someone is going to have a dependency on you for an important project, and you didn’t plan for it. If teams don’t buffer for these smaller dependencies, they’re setting themselves up for failure. 

Now, if there’s a pattern where either 1) dependencies are extreme — there are so many of them that it’s constantly hard to prioritize, or 2) one team in specific is constantly a dependency, then leaders should consider whether they have the right org design. Instead of figuring out how to improve the collaboration across teams, consider whether the strategy behind how the organization is structured is working. 

5. Every initiative has an owner that’s responsible for driving it forward

There’s always a single person that’s driving an initiative forward. Even if they aren’t currently working on the initiative, and another team is, they’re still responsible for moving it forward. That sense of ownership isn’t passed off to another team. If they get blocked, they need to resolve or escalate it. The owner must drive the project through to the end.

‍This interview is part of our Fieldnotes series, where we research what leaders are doing to improve the effectiveness, efficiency, and happiness of their engineering teams. ‍

We recently interviewed Kaitlyn Tierney, Staff Software Engineer at Farewill. Helpful context for this piece on documentation: before becoming an engineer, Kaitlyn was a librarian. Soon after joining Farewill, Kaitlyn took on the project of revamping their documentation system. Here she shares the steps she took, some templates she created, and some ideas for how others can “nudge” teams into the practice of contributing to docs.

For the majority of my career, I’ve worked in fully remote and distributed software engineering teams. I joined Farewill, a company that wasn’t historically fully remote, right in the middle of the pandemic they were in the early stages of investing in documentation. Knowledge sharing up to that point was mainly happening in the office, although some foundational work with documentation was already in place. 

But documentation wasn’t as far along as it needed to be. What that meant for me personally was that it was hard to find the information I was looking for, so I had to constantly ask people questions over Slack. That was a contrast to previous places where I’d worked that had a long-standing cultures of documentation: I could get questions answered just by reading the wiki or finding something in our GitHub history. 

Tactics that helped revamp our documentation structure

Farewill had big hiring plans, so I decided to go heads-down on building out a logical strategy and structure for our documentation system. I focused on a few core elements: 

1. Define a logical structure. 

There’s a concept in library science that’s focused on grouping things in a useful way. You have to start by coming up with concepts that would be useful first for people trying to find information. 

So I came up with a taxonomy of the high-level concepts an engineer needs to know about, which now serve as headings. The headings include Infrastructure, Testing, People, Codebase, and so on. Each heading has sub-topics that serve as folders for more information. 

‍

‍

Note: In the screenshot above you might notice we use Notion. Before I started this project, the team already had some docs in Notion, so it was a natural place to structure everything in. I do like the tool in general, though, as it makes it easy to organize information and has the ability to keyword search. 

The guiding principle for what lives in this wiki is that it needs to be evergreen content, meaning it’s relevant for a long time. That’s different from an architectural decision record, for example, which is more of a specific decision that we made in the codebase. That type of documentation lives in a searchable database and doesn’t need to live in this wiki. 

One interesting thing I've observed is that it’s hard for some people to naturally understand where things should live. There have been a few situations where someone added something to the wiki and it wasn’t the place that made the most sense. So in situations like that, we talk through it — what’s the purpose of this? Where do you think it should fit and why? Should it be a top-level subject or does it belong nested under something else?  

It’s also been helpful for people to use this system to have a page about “how to use the wiki” that lives in the wiki.

‍

2. Create templates for codebase-specific documentation. 

As a separate investment from initially getting our documentation system together, I spent some time coming up with writing guidelines for engineering practices. 

Engineers work in lots of different ways, but we need to standardize around a few things, like writing commit messages, the effective use of version control and git, and handling pull requests. So we introduced templates for each of these things: see our template for crafting git commits and the development process.

This is the kind of documentation that increases quality as a whole. It makes the codebases themselves more reliable. 

At a previous company where I’d worked, you could go back to the beginning, find a foundational commit, and see in the commit message a thorough explanation on what choices were made and why. So when you had to make changes to that code, you were confident doing it. You understood exactly where the author was coming from and what constraints they were working with. It was a completely different experience. 

So that’s what we were aiming for, at least moving forward, at Farewill. We wanted commit messages that explained what we'd done and why. And really, that should be non-negotiable, but I did have to do a bit of selling the team to get people into that practice. 

3. Introduce the new system and nudge people into the habit of using it. 

I shared the new documentation system first with my manager, and then at her suggestion shared it more widely in our #guild-engineering Slack channel. And people were really excited about it — I was surprised by how overwhelmingly positive the response was. 

In terms of building a culture of documentation excellence, Farewill has come a long way. There are a few things I do regularly to encourage team members to write or update docs: 

• Ask team members to update docs in Slack when conversations are happening. When I see people talking about something on Slack that I know has relevant documentation, I’ll add a note with a link saying “Hey, can you update the doc on this?” It’s always well received. 
• Pair with more junior engineers or newer engineers. Sometimes people don’t contribute to docs because they haven’t developed the skill. I like to pair with people to help develop those skills and their confidence in writing documentation. 
• Ask questions about under-documented areas. Early on in this project, I started talking with the people who were responsible for maintaining the codebases that were not very well-documented. In those conversations, I’d work in a “you know, it’d be great if there were some docs on this. I’ve made an empty page where we can fill it in.” 

So there’s been a cultural shift but, like anything, it takes constant nudging to keep everyone doing the task. 

Measuring success

We do quarterly engineering satisfaction surveys where we ask questions about how people are perceiving the maintainability of the codebases. (We review responses in our bi-weekly engineering guild meetings, and the responses from the questions are used to drive the roadmap for our platform engineering team.) 

Below are some screenshots of our scores on Documentation in 2021 vs 2022. (Special thanks to Helena Thompson, who deserves all credit for running and continuously improving these surveys.)

‍

One of the metrics for maintainability is Documentation, and when I started documentation had low scores. Now, we’re scoring highly on that metric across the organization.

‍

‍This interview is part of our Fieldnotes series, where we research what leaders are doing to improve the effectiveness, efficiency, and happiness of their engineering teams. ‍

We recently interviewed Aaron Suggs who was previously Director of Engineering at Kickstarter and then Principal Software Engineer at Glossier, and is now starting his own software business. He’s written about ‘build vs buy’ decisions for LeadDev, and ‘What a deploy bot taught us about documentation’ for Increment, and continues to write on his personal site. Below you'll find his practical advice for spearheading investments in improving internal documentation. 

Define the business case for investing in documentation

Documentation in and of itself is not an important business outcome. But there’s always a business outcome that improving documentation can help support, and we have to frame the investment in a way that aligns with that goal. 

Take refactoring as an analogy: if an engineer says they want to refactor something, it’s good to ask them “to what end?”. Because there should be some business-focused reason — maybe “I’m refactoring this to make it extensible for new features” or “to make it more scalable” or “to make it more reliable”. 

Documentation is a means to an end, just like refactoring.

Investments to improve documentation are usually about sustaining or improving developer velocity. The business argument is that with better documentation, developers can quickly find the information they need to complete tasks or make better decisions. 

That’s a good start, but we should be more specific. Some examples of reasons for improving documentation might be:

• Getting new hires up to speed faster. You hire a new employee, but how do they get up to speed without having to ask other team members or figure things out on their own?
• Responding to incidents faster and shortening downtime. When things break, how do people fix them quickly? Is there a documented playbook that gives information on how to restart the thing or change some configuration? What’s the customer impact if I make a certain change?
• Protecting the business from vendor disruptions. If a system or vendor suddenly shuts down, do we have documentation around how we use that vendor, where the data goes, and which systems touch it? If we do, and if it’s up-to-date, we’ll be able to come up with a plan for addressing the disruption much more quickly. 

Create a project brief for the proposed change

It’s worth noting that if people don’t feel it’s their responsibility to bring documentation up to date, any investment in improving docs will quickly become irrelevant. For the investment to be durable, it’s important to have an executive sponsor who says “we value this” and then continues to nudge everyone.

A good first step for making a change to your documentation process is to create a project brief. Project briefs outline why we’re doing a new thing and what the new thing is. They cover:

1. The goals of what you’re trying to do
2. “Non-goals”, so what we’re not trying to solve (Let’s say you’re moving to a new vendor. You’d say “our goal is to use a reliable vendor”, and the non-goal might be “to rewrite the system entirely”. You’re just doing a band-aid solution, not a full rethinking of how to solve something.)
3. Possible ways this could be solved
4. Key decisions or tradeoffs (For example, in a build versus buy conversation, the key tradeoffs might center around the different vendors — “the best vendor is super expensive” but “the most affordable vendor is not a complete solution for us”. And “we could build it ourselves but the opportunity costs are high”. Sometimes we go into this with a decision in mind, but still going through the exercise of talking through the tradeoffs of different solutions is a good exercise to clarify thinking.) 

What to consider when choosing a solution

There are two experiences to think through to explore solutions to the problem. 

Consider the authoring experience. Aim to make this as low friction as possible. Think about something like Google Docs or Dropbox Paper: writing in those tools is like writing on a sheet of paper. It’s frictionless. That’s how easy it should be. 

There are other tools where you need to learn markdown, or that require you to click a publish button that creates a pull request and so on. These processes are far too complicated and make it much harder to get people to adopt the habit of contributing to docs. 

Slab, Notion, and Confluence are tools that seem to have relatively low-friction authoring experiences. (Slab’s the one I’ve used a lot.) 

The authoring experience also encompasses the “maintaining” experience. Let’s say there’s a typo, or something that's outdated. How much effort does it take for me to correct it? Because if I need to make a pull request or request permission to edit the document, I’m probably not even going to bother. So ideally people feel empowered to make changes from a technical perspective, like editing typos or keeping things relevant. 

There are certain things you wouldn’t want people going in and changing — for example, a team’s on-call expectations — you wouldn’t want someone to go in and make changes without a broader discussion. But in my experience you don’t see people going in and changing those things very often, if ever, because the social expectations are that you wouldn’t make changes without a discussion. 

Then think through the discovery experience. The north star for a good discovery experience, in my opinion, is when employees can find something without interrupting their colleagues. 

For example, in Slab's federated search, when you search for something it’ll look across GitHub, Slack, Google Docs, or any other places you have documentation stored. It’s like a one-stop-shop for documentation. And the reason having something like that is important is it gives employees the confidence that if there is information on a subject out there, they can reliably find it. 

The north star for a good discovery experience, in my opinion, is when employees can find something without interrupting their colleagues.

Inevitably someone will have to nudge their colleague and ask how something works. But the goal is to reduce the number of times that happens. 

A common solution here is to make the navigation or structure of existing documentation better. And to that, I’d say there’s a spectrum of people: there are those who just want to search for keywords and don’t want to go through a hierarchy of folders or tags, and there are those who love keeping things really organized. In my experience, a system that allows for both will drive the most adoption. 

What that means in practice is putting some thought into how things are organized (for example, by team or by use case). And then making sure there’s 1. a categorization or tagging feature as part of the authoring experience and 2. a way to search. 

One other major factor to consider is the administration experience. These are usually privacy-related things to consider, like making sure only current employees have access to docs. Or only certain employees have access to docs that have confidential information. You want to make sure your IT team is on board with how the tool is going to be used to share information. 

Choose a strategy for implementing the process change

Before making any change, keep in mind what the organization’s appetite is for the change. You’re asking people to do something new and possibly unfamiliar, and it might create some pushback. 

A good approach is to figure out where you can start with documentation that’s going to provide the most value. That way we can start small and prove that this is a good investment of time. For that, two strategies come to mind:

‍1. Follow the pain. What is the thing that employees spend the most time on? Maybe it’s that people are interrupting their colleagues to search for things that could be written down. Start there when creating your list of things to document first. 

And it should be natural for team members to talk about time savings. “I used to spend two hours a week answering slack DMs about how stuff worked. Now I don’t spend that time anymore.” 

From there it’s easier to justify further investments in documentation. 

2. Stop the bleeding. “All new documentation needs to follow our new best practices.”

You might have all these legacy systems that have poor documentation. But you’re not going to pause what you’re doing and add all the docs needed in those areas. Instead you’re saying, “we’re going to make sure that anything new coming out from now on follows our best practices for documentation”. 

If you’re able to look at the new systems and recognize that they’re easy to maintain, it’ll be easier to go back and make a dedicated investment to improve the documentation for the legacy systems. 

A combination of those two strategies will help you realize how much time should be invested in documentation and what the next areas to improve are. 

Managers, or champions of the process change, should think about the moments where it’s easy to nudge for more documentation.

A few other tips for implementing a change to the documentation process: 

• If someone isn’t contributing to docs when they should, it’s probably an alignment problem. This person might not understand what you’re trying to do with documentation.
• Managers, or champions of the process change, should think about the moments where it’s easy to nudge for more documentation. Most teams have a daily standup, for example. When an engineer says they shipped something, someone else can say “is there a doc that I can consult on how it works?” And either they’ll say yes or they’ll go write one. Another example from my experience: I’ve been on teams that did a “demo hour” every other Friday to show off the things people were working on. And a common question was “where can I read more about it?” So these gentle nudges can help get the team into the practice. 
• You can also enforce documentation with things like production-ready checklists. Before you launch a new service, complete a checklist requiring a system diagram or playbook for responding to the top three possible failure modes. 

As a final note: know that there are times when it doesn’t make sense to invest in documentation. An early-stage startup that’s trying to find product-market fit should probably stay focused on shipping product and testing experiments without having to document everything they’re doing. A growth-stage company, on the other hand, is probably focused on improving scalability, reliability, and resilience, so investing in documentation makes a lot of sense.  

We frequently see “documentation” listed as a top problem customers want to improve. It’s a familiar problem for many teams: documentation can easily become outdated or disorganized. Sometimes the issue is that the documentation is poorly written, other times there are specific areas that are under-documented. We’ve started interviewing leaders to understand what exactly they’re doing to improve their documentation process so we can distill their approaches for others to learn from. 

In this post, we're excited to share the insights of Daniele Procida, Director of Engineering at Canonical. Daniele's extensive background in establishing documentation systems including contributing to both Divio and Django’s documentation and speaking at various conferences on this topic. (See his talks “What nobody tells you about documentation” and “Always complete, never finished”.) 

Upon joining Canonical, Daniele was tasked with transforming their documentation with the aim of creating a technical documentation practice that represents a standard of excellence in the industry. That means Daniele has spent a lot of time thinking about this problem—but when we asked if other teams could reasonably apply his strategy, he said “most teams don’t need to ‘work harder’ at documentation, they just need to do it in the right way.” 

Note: while Daniele recommends “making documentation easier” instead of having to work harder, he does think that organizations would benefit from having someone who is the custodian of documentation quality and standards. He says, “Whether that's someone's duty or whether it’s a full-time role depends on scale, but the important thing is to have a unity of vision that's kept active and alive.”

Here are Daniele's five key recommendations for making documentation easier.

1. Introduce standards that define what "good" looks like

Daniele’s Diátaxis framework offers a set of standards that meets a matrix of user needs by distinguishing four distinct types of documentation:

1. Tutorials: What you decide a beginner needs to know
2. How-to guides: An answer to a question that only a user with some experience could even formulate
3. Reference guides: Technical descriptions of what something is and how to use it
4. Explanations: Discussions that clarify or illuminate a particular topic

Note: Here’s a slightly different breakdown of the types of documentation from Niklas Begley at Doctave. 

Daniele points to “not knowing where to go” — for documentation authors as well as users — as a common cause of a failed documentation system. Therefore, documentation standards should specify where to go to find or contribute to different types of documentation.

2. Teach good documentation practices during onboarding

‍An interesting tactic Daniele shared: Running a 4-hour workshop once a quarter with new hires. Daniele’s session is focused on training developers about the types of documentation, where each should go, and what good documentation looks like. This helps instill a culture of good documentation from the beginning of each developer’s journey with the company. 

Only a small part of the workshop is a presentation; most of the time is spent doing exercises that reveal to developers the right way to write different types of docs. In the end there’s a certification that acknowledges the skills and knowledge learned. 

Here’s a summary of Daniele’s workshop agenda: 

Outline:
1. Part 1 - Quality: understanding how documentation works and what good documentation looks like; introducing the Diátaxis documentation framework; applying it to documentation problems; applying it to your work.

2. Part 2 - Execution: getting things done; authoring practices and approaches; getting work started and keeping it going; making life easy for yourself.

Transforming a culture is hard and delicate work that’s all too easy to disrupt. It’s a long task of nurture and education that requires finding ways to bring teams and individuals along, willingly, with the desired change. Still, it’s more sustainable than trying to do the right things in an adverse culture. On the contrary, to establish a good culture is to set virtuous cycles in motion, in which doing the right thing becomes the easy thing to do, and in which practice reinforces values.

3. Partner with teams to set quarterly objectives for improving documentation

‍Daniele partners with teams to set quarterly objectives around areas of documentation to improve. He uses a spreadsheet to track all teams’ objectives and progress so that the results of everyone’s efforts can be clearly tracked.

Here’s an (edited) example of Daniele’s tracking sheet: 

A good culture of documentation is the hardest thing to attain. Everything else depends upon it; in its absence, no amount of effort or wisdom will be sufficient. Canonical’s stated ambition to set standards of excellence in documentation helps define values that inform all our work. 

4. Be careful not to force too much structure

Creating good documentation requires skill and hard work; there’s no need to make it harder still by working in inappropriate ways. Documentation is always tracking a target — the software it refers to — and sometimes that target moves swiftly. Documentation work needs to be nimble and lightweight.

Planning in documentation is often unhelpful. Processes that work well for, say, software development are not always suited to documentation. Instead, adopting a model of organic, iterative improvement can reduce documentation-related anxiety and effort, and allow teams to accomplish useful work faster.

5. Make flexible tooling choices

“Tools exist only to serve the work.” Daniele aims to de-romanticize technology decisions, “understanding that since technology itself never stands still, we should expect to have to revisit our choices in due course anyway.”

His personal preference for tooling: Sphinx for publishing, reStructuredText for markup, Git for version control, ReadtheDocs.org for hosting. But, again, he encourages teams to find something that works best for them.

The following is a transcription of our interview with Paulo André. Paulo is a former VP of Engineering who now coaches CTOs, VPs of Engineering, and Engineering Managers. He also writes a weekly newsletter for technical leaders (sign up for it here).

Paulo: Let me set the stage here. Internal documentation is something I’m passionate about because I’ve seen how important it is. But in most of my roles I’ve been fairly reactive, having created documentation after there was an apparent need for it. (Being a coach now, I don’t have the opportunity to be more intentional about documentation but if I do join another company I will be.)

What is documentation?

“Documentation” can mean different things, so let’s start by defining what documentation is not. To do that, I'd like to dispel what I call the myths of documentation: “documentation is bureaucracy,” “documentation is a nice-to-have,” and “documentation takes too much time.” All of these are myths. When done well, documentation helps the team move faster with less rework  — it’s not a nice-to-have. And if it takes too much time it’s because the process isn’t set up right. 

What documentation is can be boiled down to this: documentation is writing down “what we do right now” or “how something works”. The point of documentation is to enable automatic and decentralized decision-making, which is essential for scaling up.

Probably the best and most familiar example of good documentation comes from GitLab. It’s the best system I’ve come across because it’s ever-evolving, it clearly outlines how documentation can be updated, and it relies on a roadmap for the areas the team plans to dedicate time to addressing.

One of the interesting things about GitLab’s documentation is that it has a broad scope. It’s not just technical explanations. It also organizes team goals, team workflows, and makes transparent their career ladders transparent. These are the types of things that can get locked in someone’s brain but would save the team a lot of time if written down. 

A few quick examples from my experience: At HelloFresh, we had over a hundred developers before I set out to document career ladders. You can probably imagine the impact that codifying the behaviors we wanted to see had on that large of a team. When I was at TourRadar, I realized that big technical decisions were being made somewhat haphazardly — and a lot of rework would have to be done when those decisions were actually implemented.

One of my core principles as a VP of Engineering was to make the implicit explicit. As people work, it’s natural to make assumptions that may or may not be right. So documenting things explicitly is important, especially the things that lead to better decisions.

How do you create a culture where everyone works on improving documentation?

There’s certainly a change management process that’s needed if a team is going to maintain a new documentation process. That’s because the problem with internal documentation isn’t usually that people don’t know it’s important — because they do know it’s important — it’s that they don’t have the time, they don’t have the incentive, and they don’t see other engineers spending time on docs. It’s a culture problem. 

So here’s how I think about shifting culture towards one that values documentation: 

Answer this: Why do we need documentation in the first place?

‍There’s a difference between compliance and commitment, and the goal is to get commitment from the team. What we want from the team is for developers to voluntarily write and contribute to docs, and for that they need to see the value in it. 

In the past, my “whys” have included 1. making better decisions across the organization, 2. helping people get the right information when they need it, and 3. making our onboarding process more self-service (my opinion is that onboarding should be 80% self-serve). 

Choosing reasons like the three I listed can be helpful in the day-to-day when we’re actually writing documentation. If we’re deciding whether to write something, we should ask whether writing it will help us with one of those objectives. Is this actually going to help us make better decisions? Is it actually going to help us onboard people better? Is it actually going to help people get the information they need when they need it? 

There’s a difference between commitment and compliance, and the goal is to get commitment from the team.

Sometimes the team surfaces more specific problems with documentation, like:

• We keep having to redo work because the requirements aren’t clear
• Docs are out of date or poorly written
• We’re missing documentation about certain areas of the codebase or important technical decisions or team processes
• Documentation exists but it’s nearly impossible to find

We pick one area to work on and continue pointing to the higher-level reasons why we’re making the investment. That way we can start making improvements while also building a culture capable of maintaining the process.

Tips for helping teams maintain the practice

Make documentation easy. Lower the barrier to writing docs by providing templates. Confluence, Notion, Almanac, and Coda all have templates you can use as a starting point. 

Treat documentation like a discipline. Just as much as you incentivize shipping code you need to incentivize good documentation. That means the ability to write documentation is considered in the hiring and promotion processes. 

If you’re taking documentation really seriously, put it on the scorecard. Look for good writers. Over time you’ll bring on more people that voluntarily write good documentation but also good code reviews, well-written emails or slack conversations. Then put it in your career ladders too. It’s a competency that individuals need to develop if they’re to move to the next level. 

On this note, I’d also add that when leaders own it as a discipline they themselves practice, they lead by example. The better they write, the better everyone writes.

Reinforce the behavior. One problem I’ve experienced: I demanded good documentation from people, but they didn’t know what good looked like. Junior people particularly struggled, and that was my failure as a leader. It was my job to give examples, and one way I did that was through a “top of mind” email. I actually wrote about this for an issue of in Gergely Orosz’s The Pragmatic Engineer newsletter: 

‍

One part of the "weekly top of mind" email was showcasing what ‘good’ looks like. If someone created or contributed to documentation and it was really well written, I’d highlight it as an example for others to see. 

Parting words

We’re not yet post-covid, but I think we can safely say that any company that’s founded moving forward is founded in a context where remote and distributed work is more widespread. They’re starting from a more asynchronous perspective where documentation is an absolutely necessary practice. So as an industry I think we’ll see companies start taking documentation a lot more seriously.