Managing dev requests and prioritizing personas at Twitter

Abi: It’s awesome to have you on the show today. Thanks so much for your time.

Jasmine: Thank you for having me. Excited to chat with you today.

Well, you’ve spent many years of your career in this developer effectiveness, developer experience space. So I want to start by getting your definition of DX from your perspective.

Yeah, absolutely. So I feel developer experience is how the engagement within the developer workflow makes the individual who’s interacting with it feel. There are a lot of core functions that developers depend on every day to get the work done, to ship great features for customers. I think developer experience is how it all ties together and how seamless, wholistic that experience is. And what sentiments it brings. Is it a positive thing or is it frustrating?

So it sounds like it’s very much the emotional side of the tools and processes that developers interact with. And I’m curious, you’ve been observing and working in this domain for several years. How has developer experience or prior versions of it, how’s the space evolved over the past few years from your perspective?

Yeah. I feel working in this industry for quite a while, from a quality standpoint, and then a DevOps standpoint to streamline the delivery of software, make it easier for engineers to take ideas from code to actual features and production, now is the focus on how fast you can do that. And also how that experience can be positive for the people who are doing it. I feel like now there’s a name and metrics tied to those feelings. Whereas before that wasn’t the case. Most folks were measuring, okay. Time to production or time to resolve issues related to DevOps and then when it come to quality, how many bugs are in production? Things like that but now it’s about sentiment, how quickly engineers can do what they need to do. Is it costly for them to contact switch? Those deep questions are being asked.

I really love that perspective. This journey or transition from the DevOps focus measurements and investments more towards the holistic developer experience. You’ve worked on a number of different teams that have focused on these types of problems. In your view what’s the ideal structure in terms of where a DX team should sit within an organization?

Yeah. I think that the ideal position of the developer experience organization is aligned with two things. I think the developer experience organization should be very close to the teams that own the tools that enable that experience because ultimately as you receive customer feedback, you also need a relationship with the teams who are creating that capability. So you can quickly let them know this isn’t working for the customer and work together to mitigate that for the customer. I also feel that developer experience needs customer representation from a product standpoint. So that would be an ideal alignment because ensuring you have that consistent continuous customer feedback loop is core in ensuring you’re continuously improving that developer experience. Different organizations are structured different ways, but I think those are two key things to think about when positioning it.

Yeah, that’s really interesting. So in your view, developer experience is distinct from the more typical platform teams that own and build the tools themselves it sounds like.

Yeah, it’s very different because we’re thinking about all of the pieces together. Historically when you have platform teams they’re focused on a potential or a small piece of the platform for good reason, you want to optimize those pieces so it’s the best and performance is optimized. Things like that. We’re looking at it from a wholistic perspective. So that’s why it’s meaningful to be close, but not directly embedded into one particular component of it.

I really like that. In your view then you mentioned close partnership with platform teams and a customer-oriented product-oriented approach. What about the people side of it? Is there a partnership with maybe HR or people teams or is there more of a boundary there in your experience?

I definitely think there is a consistent relationship with people teams because as you build out a developer experience function within your organization, you’re going to need to make sure that you’re honing in on those particular skill sets and how they’re different from core platform skills, how they’re different from a standard backend engineer or front end engineer, because ultimately you want to get those people who are passionate about this particular work. It’s a different thing being fulfilled by shipping features that improve performance and optimize the way the platform runs. I personally find fulfillment in making sure the process of development doesn’t suck for engineers. I found fulfillment in that and it’s a different thing. So working with HR to find out and to define first how you interview, how do you get signal on a candidate’s passion about this space is key. So having a partnership with HR and people teams is a must.

That makes sense. Well, I’d love to dive a little bit more into your personal journey and background. You’re currently focused on effectiveness and developer experience. How did you find your way into this role and what’s your prior experience that relates to the work you’re doing today?

Yeah, yeah. So it was not a planned journey actually. I went to school for computer science as most folks who find themselves in the industry did. I started actually not in a standard software engineering role, but in a quality engineering role where it was my responsibility to automate as much of the interactions with the application that I was supporting so that we could make sure that we weren’t shipping features that had bugs within them for our customers. So that was my first exposure to automation and writing cucumber behavioral driven development, things like that. I then shifted into systems engineering, which is a whole different space and infrastructure as code and figuring out how to deploy infrastructure in the most efficient way as possible and how you can optimize for reliability and resilience in the case of disasters, things like that. And I was doing that for core developer tools like GitLab, Jenkins, Nexus, which were enablers to the development workflow.

After I did that, I was responsible for driving the culture and adoption of this new tool chain. And that’s where I really found my passion in creating a pathway for people to work in the most optimal way as possible and have better balance within their life. So by leveraging the tools that my team at the time at Delta delivered, folks were able to not have to worry about the standard things that they worried about. Release engineering calls for four hours. Now we could do blue green deployments and release at a little bit at a time versus the big bang. So that’s where I really found my stride in optimizing the people experience as it relates to tools. And then in this role at Twitter, I’m doing just that by way of creating experiences for documentation, how people consume the services, making sure they’re able to get the help that they need from a support perspective also.

So that was my journey. And I think that it almost build on each other down the line. And I just keep finding out more and more about this recognized developer experience field. I’m talking to you about it today. So I really feel like I found my stride right within this space.

I feel like that’s a similar journey or sentiment a lot of folks in developer experience have shared that this is an emerging field or space or practice. And everyone found an alignment between their passion throughout their careers and this new function within organizations. On that note, how have the functions differed at the different companies and organizations you’ve worked at? How has the focus on developer experience been similar or different between these different organizations?

Yeah, I would say that in my first two roles, one as a quality engineer and one as a DevOps engineer systems engineer, and then a manager on the first two roles, I would say that they weren’t quite defined and recognized or measured in any very formal way. We all knew that this was something that needed to be done and solved those problems as they came, but it was never recognized as a function. Fast forward to now where these are objectives and key results that I establish with my team. They’re established at the VP level. And we work every day to move those KRs and save developers time. So I think that that’s the core difference for me is the recognition, which obviously leads to prioritization and investment on that front. And that’s the way you really improve that experience is by making sure folks know this is the cost, but here’s the value add that we’re bringing to the organization. And I feel like that is what has sustained it for me in the second phase of my career.

‍That makes sense. I’m curious, could you give examples of how developer experience is being measured or how OKRs, some examples of OKRs that you’ve seen created around developer experience?

Yeah, absolutely. So obviously the developer workflow differs from place to place, but one thing that my team measures from a KR perspectives is how many minutes we can save a developer. So for example, one of the objectives that we had this year was okay, how can we optimize the rollout of new Mac OS versions across our fleet of devices that engineers use every day to make sure that one, we were preventing downtime from incompatibility issues, whether it was hardware or software versions and things like that? So that is one of our objectives. How do we reduce and minimize the downtime of engineers as they utilize these tools on their workstation and save them time? So as we find issues, we quantify that to how many people would be using the tool based on adoption history for the Mac OS version. And then that equals the amount of time that we saved.

And how did you actually measure that? I’m curious.

Yeah. So let’s say there’s an incompatibility issue with a library in use. One of the teams that I lead right now actually creates a tool. It’s called tool insights that teams can instrument their service or tool that engineers are using to understand exactly how many people are using it, how they’re calling it. And we take that data and then quantify it and multiply it by how many people were adopting the Mac OS version over time. So that’s how we measured for example, there was an incompatibility issue with Python in this Mac. I forget the Mac OS version that came out, but Python 27 was deprecated. So how do we test that and make sure that everything leveraging Python is compatible with Python 3 and get ahead of any issues before it reaches the masses?

Thanks for that. I’m curious. What other examples do you have of developer experience really being driven at that VP level? What are some other OKRs or other types of measures you’ve seen being focused on?

Yeah. Another measure that my team drives right now is migration acceleration. So migrations are a part of a life cycle of software. There’s vulnerabilities that are introduced. Libraries are deprecated, and you just have to have this posture of continuously moving along with the software versioning and things like that. So my team focuses on how we can accelerate migrations. For example, if we know that we need to deprecate this JDK version, but we know, okay, some teams will not be able to do this customer side of the work. There’s potential for automation of some portions of this migration work. We look at opportunities there and figure out, okay, how much can we ask vendors to do on the customer team’s behalf or automate this portion of the migration? And then that leads to developer minutes saved, because you’re able to quantify how much of the work is getting done by these alternative methods and therefore accelerating the migration and getting done faster because migrations, they can go on forever. Our goal is to get them done and move on to the next one.

Well, those are both great examples and unique solutions. So I appreciate you sharing those. We talk to a lot of leaders on this show who are in earlier stages of standing up a developer experience function. A lot of them have just made that transition from just a platform team to a developer experience posture, and are coming up with charters, figuring out how to advocate for resources and investments. In your view, what does a mature developer experience function look like once you’re past the acceptance and awareness around developer experience, and you have some headcount, you have a clear charter, what does that look like?

Yeah. I think that a mature developer experience organization has consistent and clear communication with customers around what their roadmap is and what they intend to improve for a set period of time. When you’re able to set a strategy and vision, that to me, is a signal that you’re mature or you’re no longer just reacting to the needs of the organization as you go along, you’re actually being proactive about what you want to solve. You’re getting ahead of some of the issues that maybe might be on the horizon for the group, whether it’s from a cloud perspective or other perspectives. So that’s one signal. Another signal is going to be consistency and representation of all personas within the engineering organization.

I know that some developer experience organizations maybe segment themselves to a certain developer persona, but when you encompass and consider all persona types, I think that is a clear signal that your developer experience is mature and you’re solving not only problems for one space, but seeing how they can impact other developers of a certain type and potentially prioritizing those efforts that may impact more than one persona. So that’s just a couple of ways I think that a mature developer experience organization would operate.

Yeah, those are both such great thoughts. The setting a proactive strategy and vision is definitely resonates because I think so many developer experience teams are really formed to solve, put out some fire in the organization. Some things, whether it’s some tool or some monolith, some things just gotten reached a boiling point and a team is formed to tackle that. And that relates to your second point around representation as well. I think so many developer experience teams are really just special squads that are tackling a very specific problem for again, maybe a monolith that is only maybe a 50% of the developers in the organization, even working. So having a posture where the developer experience team is looking across the entire organization makes a lot of sense in terms of maturity. So I’m curious, what have you reached maturity in your own roles in developer experience?

I think in some ways we have, I do think that it’s important to remain consistent. You have to have this sustained engagement with personas and then the sustained proactive efforts being sourced. I think we’re getting there. I do feel that we still have a ways to go on making sure that we are incrementally introducing the value related to the feedback that we’re getting. A lot of times it’s difficult to balance. Okay. Do we do this big bet or do we just invest in these increment improvements that we know will be wins for the team, but maybe not as significantly impactful as the big bet? So it’s making sure you have a healthy balance there and we’re still working on that.

‍So on that note, what advice do you have for going about creating a roadmap? Where do you even begin? I think a lot of developer experience teams, there are usually at least one or two known hotspots in the organization, maybe with a particular tool or particular part of the code base. But beyond that, how, if you’re coming into a new organization and you want to create a year long vision or two year vision, how do you do that?

I think it starts with getting the data. Honestly, it’s very simple. There are lots of untraditional places that may have data. I lurk in Slack, for example, at the company level, there’s persona specific channels where folks will give very transparent feedback on how they’re feeling about certain components within the developer workflow. We, for example, have quarterly surveys that go out to engineers, asking them about their experience. That is a great way to get broad engagement and understanding of what your organization is feeling from a sentiment perspective. That married with actual instrumentation and data from the tooling is a great signal also. One untraditional thing that I think our group does, we have a centralized tier one support team. Most of the issues with developer workflow are actually sourced within one channel, which gives us a really unique opportunity and perspective into what are the real pain points and how consistently are they being surfaced by individuals. And then we’re able to draw understanding on what the trends are. And then that is direct input into our roadmap and things that we need to resolve immediately from a blocker perspective.

That’s really interesting. Can you share more? Where did the tier one support model, what’s the origin story of that?

Yeah, so I can’t take credit. I actually came into the group already established, but essentially because our engineering effectiveness group has so many tools that we support for so many different persona types, the tier two teams, which are the tool teams that actually own the service, they were being weighed down from a bandwidth perspective, by engaging with customers solving those problems. Everyone probably has the JIRAQ. It just continuously grows with customer requests and support tickets. So we needed a centralized way to take some of that pressure off the tier two team. So they could focus on delivering those incremental wins to improve the developer experience. So the tier two team was established. It’s a 24 by five team in between PST times all the way around the world.

So we support our global organization by having a global team. And essentially it’s a place where folks can go in, ask for help and get pointed to documentation that currently exists. Get triaging help from the engineers who are on the ground on our tier one team, or make sure that they’re surfacing the ticket in the right team’s queue. So that was a big issue. So half of the battle was figuring out who owns this part of the developer workflow, so I can open a ticket in the right place. So it really streamlines that engagement. And I think it’s been a fantastic add to improve developer experience.

‍Well, thanks for sharing that. That’s a really interesting strategy and I’m sure a lot of listeners can learn from that. I want to go back to something you mentioned earlier around the consistency and representation. Of course, I can see how using things like surveys, you can hear the voice of developers across the organization, but it does seem like oftentimes a lot of problems can be concentrated in certain areas of the organization. How do you more tactically? And I should add, I’ve also heard a lot of developer experience talk about how there’s just not really enough headcount to do all the things. And so how do you achieve that consistency and representation or how do you posture the developer experience team in a way that is supportive of the entire organization, if you don’t really feasibly have the resources to do things to help everybody?

Yeah. Yeah. It’s a great question. And it’s a constant struggle. It wasn’t always the case where we could focus on large scale efforts that impacted everyone. I do think where you can do that, great, but tactically internal at my current company, what we do is have a priority list for across the company. And if you can’t focus on everyone, you should focus on those who are doing the most important work for the company to optimize their workflow. So for example, if you have a list of 10 projects at the company level that are very high and they’re tied to two to three personas out of 10, you should focus on optimizing for those. So figure out ways and strategies for prioritizing the personas that you can help whether it’s at the company objective level. I feel like that’s the great way to go and just to align the resources to where they would make the most impact and have the greatest business value.

That makes sense. And I want to comment that throughout this conversation, you’ve used the word persona, which is indicative of a refined product or customer-oriented mindset. What advice do you have for engineering folks maybe bridging into a developer experience role who don’t have maybe prior product experience and want to understand what even being customer minded or product minded means, where have you learned to use terminology like personas and what advice do you have for others?

Yeah, it’s definitely a shift. In multiple roles I’ve had to put on product hat and then engineering manager hat. I feel like customer experience is actually in college I was customer experience in pre-graduation. So it’s always been a passion of mine. There is a lot of sales strategy and company product strategy outside of just tech that you can leverage for understanding how to derive meaningful feedback from customers, what they value from an experience in general. So I highly advise folks to Google customer experience, not just developer experience, just customer experience as a topic. There’s a lot of great books that you can read about it.

I just finished one around the Disney customer experience, nothing to do with tech whatsoever, but I love Disney, my Darla Disney. I’m like, let’s see how I can tie this back into my role and what are some of the things I can think about when engaging with customer and creating that magical developer experience because that’s what we want to get to. So yeah, just going back to taking developer experience out of it and just thinking about customer experience, because ultimately they are internal customers.

I love that advice. And one thing I was thinking about earlier, you clearly have this mature understanding of what developer experience and the role it should have in an organization. I feel like what I’ve heard from other folks I’ve spoken to is that there is an aspect of that managers like engineering managers who aren’t in the developer experience organization are also responsible for. So how do you engage leaders outside of your own developer experience organization to also care about and maybe even chip in to helping with developer experience? How have you done that in the past?

I think it starts with alignment on making sure that they understand we have a collective customer. So for example, partnerships with platform or partnerships with IT. These are things that historically they serve their customer base, but they are inputs into the developer experience. So I think it starts with defining our collective customer together and how we can help them and how we can drive whatever their objectives are from a group perspective and improve the experience and just help improve their customer experience. So that way they’re meeting their goals too. So alignment is big for me, creating relationships is big for me and just building together a strategy for improvement. It is very helpful when you have this directive from above, but a lot of time that doesn’t happen. So sometimes you have to do the grind of creating understanding of what you’re working on and what it means for that group. So it just depends on how the environment is I believe.

And do you often come across issues or complaints from developers and you think in your head, man, that team should just fix that or do you see a lot of issues like that and what do you do with those issues?

Absolutely. I think going back to the support topic, there’s lots of issues that folks face that our team does not own, but at the end of the day, we are the face at the moment. We want to take ownership of that engagement with the customer and do what we can to bridge whatever gap that currently exists. So a lot of the work that I do on the backend of the customer engagement is surfacing the issues to those teams and maybe creating partnerships, working agreements so that we can have a warm handoff perhaps, or them providing us documentation and input so that we can triage and mitigate the issue for the customer at the forefront. We take that ownership on because ultimately when it comes to the experience, it is the experience. Doesn’t matter who owns the particular piece. They’re there. Then let’s try to fix it if we can.

Do you feel like it’s possible to keep up with all the issues across the organization? Just seeing the types of ratios of size of developer experience team compared to size of the org and knowing from the conversations and our research, how many issues there are often in the developer experience, it seems like a daunting task for a single team to both triage and own, be the face of, but also actually meaningfully solve. So do you have a large backlog?

Yeah, so we don’t actually take on everything, but if we see consistency in issues that are surfaced in our area and they are easy enough for us to just solve right then and there, we’ll take that on. A lot of the other side of it is making sure we’re creating awareness with the other leadership teams to say we have metrics on how this is coming across our desk, for lack of a better word. How can we work together to create a pathway for customers to get the help that they need in your org? Or perhaps it makes sense for you to dedicate some capacity to inject within our currently running model in order to mitigate this for you. So those are the conversations I have. We don’t take on everything. I believe in having a healthy team. And I don’t think any of them would still be here if we just decided to do it all.

That makes sense. Well, Jasmine, it’s been awesome having you on the show today. I really enjoyed this conversation. Thanks so much for your time.

Thank you, Abi. It has been a pleasure. Thank you.

‍