10 key questions for your developer experience roadmap

Deciding which problems to focus on requires a product mindset. The Developer Experience (DevEx) team should get clear on what their role is in helping the company achieve its goals, then consider the areas of friction as well as the estimated effort in solving those problems, before creating a roadmap. 

Thankfully, product prioritization is a well-covered topic, and DevEx teams can rely on what external-facing product teams have learned (e.g., Lenny Rachitsky is a great resource). 

This article will offer questions that DevEx teams can use to inform their planning. These questions are broken into three areas: the company’s goals, the impact of each problem, and the effort to solve each problem. 

Stay connected with business needs

1. Is the company experiencing existential issues? Will Larson says this is the first place to look for work that matters: “Running out of money, like my experience at Digg, can be the most obvious issue, but not every existential issue is financial, like Twitter’s fail whale stability challenges or adapting to the shifts caused by the Covid-19 pandemic. If something dire is happening at your company, then that’s the place to be engaged. Nothing else will matter if it doesn’t get addressed.” If there are no existing existential issues, we can continue on to the next questions. 

2. What problems are top-of-mind for the VPs and CTO? Understand what problems are top-of-mind for executives. Use that knowledge as a lens for recognizing the projects that would contribute to those business needs. 

Estimate the impact of each problem

3. How important is it to developers that this problem should be prioritized? During the data gathering phase, ask developers to rank the priority of the areas they’d most like to see improved. Just because an area is causing a lot of friction doesn’t mean it is important to solve. Similarly, an area that does cause a lot of friction but is experienced every day, could be important to solve.

‍4. Is the problem isolated to a specific team or systemic across the organization?  DevEx teams can think about their scope as being split into two categories: things they can change first and then influence, and things they can only influence. 

• Things they can change: issues experienced across teams. Think tools, tests, builds, environments, or tech documentation. DevEx teams can directly change these areas but will need to focus on getting adoption of the tool or process. 
• Things they can only influence: any issues that are isolated to specific teams. DevEx teams may provide best practices or guidance for team leads to drive their own improvements. 

5. Which developer persona do we need to serve? If there are engineering teams focused on mission-critical projects, it may be worth prioritizing the issues experienced by those groups first. (To learn how Twitter considers personas as part of their DevEx prioritization, listen to this interview.)

6. What is the cost of the problem? We’ve seen teams take different approaches to calculating the cost:

• Expected gain to the company. What is the context-switch time on top of the wait time (or wasted time) for this task? If we take that number with how frequently a developer moves through that workflow, we can calculate the expected boost in productivity from making the workflow more seamless. 
• Cost of time wasted. Take the average engineer‘s salary, multiplied by ~1.5 to 2X, to calculate their cost per minute or hour. (This is called the ‘fully loaded cost’ of an engineer, and accounts for the non-salary overhead that is spent on engineers, like their laptop, software licenses, amortized benefits, facilities costs, etc.) Multiply the engineering cost by the amount of time spent waiting, redoing work, or context switching within a part of the development process. 
• Cost of a delayed investment. Determine the cost of delaying an investment in resolving the problem for a period of time. If the problem is costing the organization an average of $400,000 per week, we can calculate the cost of not investing in the problem for three more months, six months, a year, and so on. (Note: this argument may resonate more with non-engineering groups like product, helping developer experience leaders more easily justify investments.)

7. How early are we (the DevEx team)? Teams getting started should look for quick wins, ideally projects that have mid- to high impact and are relatively easy to complete. 

Estimate the effort of solving the problem

8. What do we believe will solve these problems, and how much work is required to build the solutions? Develop hypotheses around what the team believes will solve the problem. (These ideas will likely emerge in the data gathering stage, where the team uses surveys or interviews to understand the friction developers are experiencing.) Estimate the effort required to solve the problem by talking with the people who will be implementing it. 

9. What level of ongoing work will this problem require moving forward? An introduction of a new tool will likely require support and maintenance. If the investment is in a process, like documentation, a team may consider whether they need to continue coaching teams on how to write or find documentation moving forward. Or technical debt: is the problem specific to an area of the codebase and fixable within a time period, or should the DevEx team instead focus on preventing high interest tech debt longer-term?

1‍0. Do we currently have the capacity to solve this problem? How much time does the team spend maintaining projects or handling support or feature requests? At a certain stage, DevEx teams may split ‘support work’ and ‘strategic bets’ into two separate teams.

Making prioritization decisions

Borrowing a framework from Lenny Rachitsky (below), DevEx teams can take ideas and rate them across two attributes: estimated impact and estimated effort. Discuss and make decisions as a team.