Abi Noda: Anna, so excited to have you on the show finally. I’ve been such a fan of the work you’ve been doing at Airbnb, so really excited to dive into your journey and story a little bit more today. Thanks so much for your time.
Anna Sulkina: Abi, thank you so much for having me. I was looking forward to our conversation.
Abi Noda: So I want to start by going back, it’s been about two and a half years since you joined Airbnb, and we’ll get into a lot of the things you’ve done. But I understand at the moment when you joined Airbnb, there were a lot of interesting opportunities and challenges surrounding developer productivity. My first question for you is reflecting on the process of being hired for your leadership role and having been successful, what are specific qualities or background you would be looking for in an effective leader and developer productivity?
Anna Sulkina: So first of all, thinking about the qualities for this role in addition to the usual table stakes that you look for in a leader with relevant technical experience and leadership skills. I think it’s really critical, so customer focus and really understanding and having the empathy for developers, for your customers, that’s certainly number one. And to truly have that empathy, it’s ideally you have experienced building products, building products that are sort of the end user products. So you worked on product teams trying to go at a fast speed and where product velocity is important and so on. But also having the experience building products for internal customers. Because that dynamic is slightly different from the end user products.
And really, of course, deep understanding of the full product life cycle, what it takes, having the product mindset from the start, understanding the requirements and so on. And also the full life cycle to actually getting software out of the door and rolling things out and what it means then after the product launched, et cetera. So kind of having a holistic understanding and experience with best practices and all this good stuff. So I think that’s as far as the qualities that are necessary for this role. And of course, because you are in the middle of everything, then communication skills, storytelling, working with stakeholders and partners, that becomes even more important.
Abi Noda: So you took the job, you were hired at Airbnb, and you quickly found there was a lot to do, and over the course of the last two years, you really overhauled the strategy and the direction of the organization. So I want to ask you, before getting into the specifics, which we’ll cover, more broadly speaking, at the time you joined the organization, what were the biggest challenges or opportunities that you felt like the organization was facing?
Anna Sulkina: There were a few. So starting from sort of looking internally within the team developer platform, the team, and by the way, that happened just on the heels of the pandemic. So I felt really fortunate that I had the luxury of gathering my leadership team in my first few months at Airbnb. And I remember people wanting to hug because they haven’t seen each other for three years or however many. But that said, as we spent time together as the leadership team, it became very clear that the team was longing for more clarity around our strategy and direction. So that was one. And then internally also, we had some really, the team has done a great job delivering cool stuff, but there was a sense of siloing and folks felt like there was an opportunity if we were able to join forces around fewer things, we could be even more successful.
And then additionally, we had, again, we had the luxury, and I know not every company has that, we had the luxury of having a few or some product managers. We had one TPM, we had great tech leads and engineering managers, and there was a bit of a confusion of the roles and responsibilities. And so that was, again, I saw that as an opportunity to crispen things up and have folks work together more effectively knowing how to work more effectively. So that was internal opportunities. Then of course, the connection and alignment with partner teams and with stakeholders. That was another important piece where there was a great opportunity to build that better alignment and improve stakeholders perception of the team.
Abi Noda: At a higher level, do you think there was a fundamental disconnect or misunderstanding of what developer experience or even developer productivity was and the value of it to the business? Or do you feel like that was pretty clear and aligned with the company?
Anna Sulkina: That’s a great question. And again, this would be my guess at this point. I think there was probably a combination of possibly misunderstanding as well. And I know the team made some attempts to measuring developer productivity, but those efforts were still nascent at the time, and so not quite understood and acknowledged yet.
Anna Sulkina: So there was a desire, but the team was at the very beginning of the journey for how to do that, and also importantly then how to communicate the value. So I think it was a combination of all of that.
Abi Noda: Makes sense. So what are some of the things you implemented and the strategies that have worked for you to improve the perception and reputation of developer experience within Airbnb?
Anna Sulkina: Great question. So really starting with trying to understand the stakeholders, their goals and pain points, and that comes in different flavors. So one-on-ones and all sorts of forums for gathering input and feedback. And that beyond one-on-ones, it could be group forums and customer advisory boards and surveys and into sort of feedback, etc. But importantly, people need to feel they are heard. Not only that we are actually gathering important input, but also people feel heard and we repeat back or we show, “Here is what we heard from you, here is what we are going to do about it.” So that’s number one. And that, the feedback collection can be just once a year. It has to be some regular, in different forms.
Now, as I said, once you hear, once you figure out what kind of feedback, what input is there and synthesize that, then communicating back. And so there’s communication and transparency with stakeholders is extremely important. “That’s what we heard, this is what we are able to or commit to improve.” And then additionally, looking for opportunities to join forces. Sometimes with, well, that’s a specific need for product team on iOS, for example. Or in infrastructure, where it’s valuable to build strategic partnerships and relationships where maybe we work together towards improving something so they feel part of the solution.
And then really, so once you do all of that, then actually showing and delivering on what you committed and iterating, and then you are never done. Basically that is the rinse and repeat cycle. You hear, you try to improve, you hopefully make progress and you communicate, provide transparency. And as you build a track record of doing that, they’ve all started to change their perception. And I can certainly say that’s been the case for us at Airbnb.
Abi Noda: Transitioning from how you’ve tackled improving the perception of developer experience, I want to discuss the internal org alignment and siloing challenges that you’ve also addressed. I know one of the specific things that you did was to shift the individual platform teams focus beyond just the specific tools they owned to more focusing across boundaries. Can you share more about what that meant and what exactly you did?
Anna Sulkina: I can’t claim that that was just me doing that because some of that evolution has started before I joined, we were at the beginning of the journey. So just at a high level, our developer platform is organized into two pillars, platform-agnostic pillar, and then platform specific pillar. And the first one, as the name says, really focuses on the tooling that is shared across platforms. By platform here, I mean iOS, Android, web, et cetera. And that would be the general CI/CD tooling, for example, the foundation for remote ID and so on, development environment, test environment, et cetera. Now on the platform specific pillar, that’s where we have each team focusing on a developer persona. So iOS platform serves iOS developers and so on. And that would be not just owning, “Well, we’re responsible for this source repo and this two tools and that’s it.”
But rather thinking holistically about developer experience and developer persona for that platform and looking at their needs over time and focusing on the specific pain points. As opposed to gold plating just one layer of the stack, so to speak. So rather than, for example, having a team that owns purely, I don’t know, Spinnaker, more thinking holistically, what does it mean full continuous delivery cycle? What do we need to change or improve? So that’s really the idea. And I think generally speaking, and I don’t think there is one particular org chart that is the structure that is ideal forever. I do strongly believe that organizations are living organisms, and that means that the organism has to evolve with the needs of the business, the maturity of the company, the tech stack, the industry trends and so on. But yet, that’s what we’ve seen work, having the team charters to be defined around the problem space.
And then importantly, for example, even for the platform specific pillar, finding opportunities to jointly solve common problems. For example, say integration testing or CI reliability and speed. Those were a couple of examples where we had teams join the efforts to try to build common blocks or that didn’t always translate into the one tool that solved everyone’s problems, but there could be some shared components and then each individual platform team would figure out how to apply and integrate that. But then learning together, learning from observations from each other and applying the best practices in all places in all areas.
Abi Noda: So the shift in org structure impacted the way your teams go about the work in many different ways. Sort of zooming out, what at a high level do you think are the biggest benefits of shifting to a more problem focused versus platform or tool focused org structure?
Anna Sulkina: I think we’ve all observed and experienced times where if a team owns a particular one technology or one to one, I’m oversimplifying here, say Spinnaker, or okay, we own this one technology, GraphQL. Then the teams tend to be narrowly focused and gold plate that one thing as opposed to looking more broader and holistically at the problem. So an example from the past, if people were asked what they want, they would choose a faster horse, while you could build a car. So if you own the horse tool, then you’ll be grooming the horses better, but you wouldn’t be thinking bigger, how to solve really the customer problem as opposed to how to make my tool better.
Abi Noda: Let’s also talk about the clarifying the roles. I know early on there was some confusion around the PM role versus the tech lead and the EM and the group tech lead roles. How did you come to clarity and define that in a better way for the organization?
Anna Sulkina: I think when I just joined and I remember my first conversations with our group product manager, and also talking to people in the teams, there definitely were a lot of questions about what product managers and infrastructure do to begin with. And so that was good to understand that that was the gap there. And then we spent some time thinking through, well, first of all the basics of, okay, well product managers generally speaking are responsible for the why. And then product managers and engineering managers and tech leads together figure out the what. And then the engineering figures out the how, and say the tech lead owns the how technically. Engineering managers look at the how organizationally. Now I acknowledge that what I just said can vary greatly across companies and industry. And so this is just one oversimplified way of looking at it and really the reality that things can be different.
Again, different companies, different organizations, and also it depends, the details depend on the people in the roles as well. So there is the basic, here are the expectations. Here’s how a product manager brings value to the team, here’s how the tech lead brings value to the team. And so anyways, long story short, we came up with, we wrote down this is how we expect this role work together and the expectations from each, and also acknowledging that there is a lot of overlap. And so ultimately these are the best practices. And then say if a team has a product manager, engineering manager, and tech lead, these three folks have to figure out together what that means to them and how they work. So that was, roughly speaking, the process. And then similarly, we looked at the different kinds of tech leads. So there are group tech leads and the team tech leads and the project leads.
So similarly, we went and analyzed, looked at the examples of what we had and where we saw a need to maybe change and crispen things up, wrote that down, had conversations with folks. Ultimately with all of these written artifacts, the artifact of its own, isn’t it. It’s not the panacea, it’s the process of getting together and aligning on what that should look like. And then having that artifact is useful as a reference point as the new people join or we go and have a retrospective and say, “Hey, things are like, here’s the problem,” maybe, so we can look back and say, “Hey, we need to iterate or change something.” So yeah, I think that’s kind of the process that we’ve leveraged.
Abi Noda: I love your point about how the written artifact is r eally just the reference or the souvenir or the reminder of the process and the thinking and alignment you went through to arrive at that artifact rather than the artifact really being the most valuable piece of this work and aligning the organization. Building off of that, I know beyond clarifying and defining how different roles interacted within the organization, you also were tasked with really defining the overall strategy of the organization as well. And you’ve shared some of this with me before. I love the way you approached it and where you arrived, the artifact in the end. So share a little bit about what that process was like and then where you arrived.
Anna Sulkina: I think we went through a few iterations of coming up with a strategy and first iteration was more inward looking to align the team. And then as we matured a bit, we went through building a more sort of holistic strategy that we shared with the organization ultimately. And so to begin with, really starting with why do we exist question, who are our customers, customers and customer personas? And then what’s our vision? What’s our dream? Really the aspiration, which for us it was equipping Airbnb engineers to build the best software of their careers. That’s the vision that we came up with.
Abi Noda: I always love hearing these types of things. What was the why you exist?
Anna Sulkina: I think the answer to this will be obvious to many folks, but I think it’s easy for people in the thick of things day to day, forget about that. We truly, like developer platform Airbnb exists to serve developers so that developers can build the product and really serve the business needs and serve the end user, the customers of Airbnb ultimately.
And so keeping that in mind really, that we don’t exist to have a perfect CI/CD system only so that it’s perfect, right? It’s a means for our business to do well. So that’s the answer to that question. So let’s see. So then going from the vision to, okay, now how do we get there? The mission that we defined was maximizing developer effectiveness by providing a dependable and easy to use platform for building high quality software, building and maintaining high quality software at scale. So that’s the mission. And so then we define the tenets, so the principles that guide how we make decisions, how we prioritize, how do we choose what to focus on. And so these principles, we came up with four principles. First, be developer-centric. I guess it’s pretty self-descriptive. And then second, make common use cases. We call it made common easy and uncommon possible. What that actually means is providing and building paved roads where your typical common workflows, you shouldn’t, as a developer, you shouldn’t have to figure out what to do and how to accomplish your job. You have the right tools at your fingertips and it’s easy. It’s easy, convenient, fast, and so on. Now, importantly, why the second part, uncommon possible? Because for the team of this size, it’s impossible to predict and anticipate and satisfy every individual need. But we aim to provide the platform in a way that if you cannot accomplish what you need trivially, there is at least a way.
So we call this platformization, where okay, if you can do that, you can build a plugin that can do this or you can take this extra steps and you’ll still be able to accomplish what you need. So essentially paved roads, but opportunities to then extend, extensibility. So that was principle number two, make common easy and uncommon possible. And then number three, have laser focus. Again, the team is, relatively speaking, small. And so we need to focus on fewer things, do that better. And then last but not least, deliver incrementally. Start with immediate, not, well, we’re going to go for a year and do this perfect thing and only then you’ll benefit from it. But rather starting to provide value as soon as we can and starting small and building up on that. So yeah, those were four tenets that we came up with.
And then ultimately where we ended up with, we choose the four strategic focus areas for us. And for each of the areas, we defined what they are, and we have folks who are the DRIs or leaders for each of their work streams. So they can look, going back to what you asked before, from siloed teams to let’s look holistically. For this focus area, all the teams and developer platform are contributing in some ways. But back to the specific focus areas that we ended up with, one is hyperfast inner loop. So that’s really your local edit-compile-test loop. And that’s basically up to the point where you’re ready to check in code to submit PR. And so this is where we see really a huge opportunity there. This is your experience in your development environment. This is unit test effectiveness and all the things that can really truly speed up this inner loop, because that’s where developers spend most of their time.
Now, second area was high confidence, continuous delivery, probably self-descriptive. This is your CI/CD really providing awesome tools for fast deployment, fast and safe deployment, and automated testing and safe rollouts and automated recovery if things don’t go right. So that’s number two. Number three, reducing cognitive load. And there is quite a bit in this work stream and some of it is tooling and some of it is not tooling. So some things that would be part of this, reducing technical debt, how we measure and what we do, governance around our monorepos, we have monorepos per platform in most cases. Documentation has been voted as one of the top pain points across all platforms, unsurprisingly.
Developer paved path, so again, so that our developers don’t have to become uber integrators, but actually are able to navigate things easy. And then improving debugging, improving user interfaces, and then code of your tooling. And then as I mentioned, not all of it is tooling. There is also, and this has been a new sort of focus for us, focus on focus time. So this is something that, again, quite a lot being talked about in the industry these days about how focused time and uninterrupted chunks of times for developers to really focus and do deep work, that contributes significantly to developer productivity and satisfaction. And so this is where our team can provide transparency, visibility, some metrics and best practices, but then it’s obviously on the developers and on the teams to do something with that, right?
Abi Noda: Right.
Anna Sulkina: I sidetracked a little bit. This was reducing cognitive load was number three area. And then last one is accelerating development with the latest technologies, including AI and all sorts of stuff.
Abi Noda: Got it. Well, I’m biting my tongue because I know we can’t get into some of the specific metrics you’ve driven and the dramatic changes you’ve been able to achieve, but it’s safe to say that you’ve made a lot of progress. I know dev satisfaction, for example, has grown tremendously in recent snapshots. Share a little bit more about what you think has been the key ingredient to achieving the success
Anna Sulkina: Yep. without getting into the specific metrics, we have seen in the last couple of years, significant improvement in developer experience and satisfaction. Like 10% year over year, which is great. And at this point it’s impossible to get 10 and 10 more, we are at this point trying to maintain and get incrementally better. And some of the contributing factor, so it again predates me frankly here. Developer experience has been a challenge before. And again, that was my experience as I was interviewing with Airbnb as well.
Things were slow and we had numerous environments for testing, for example, and it was really hard, the confidence that you’d get with that, the amount of time and work that you’d have to invest into testing something and then your level of confidence was pretty low in the end. And we are at this point in a so much better place, there is of course always more to do. And we are seeing in the latest developer experience reviews that developers really appreciate, this is one of the biggest improvements that they are noticing. And so a lot of improvements. Of course, again, AI has made a huge difference as well. And all sorts of technical advances. I would say contributing factors, as we started the conversation with, it’s been, our developers feel heard. We have developer experience survey now being very consistent. We do this twice a year, the results are shared transparently, shared and respected. And I get stakeholders looking for results now, whereas that wasn’t the case when I joined. That wasn’t a thing really. So people recognize that developer experience is important and they are acknowledging the improvement. They’re seeing the improvement made so far. It’s the reality of improved experience and also improved perception and trust that things will get better because we are seeing the track record of things getting better.
Abi Noda: I have to say, I view you as such a thoughtful and thorough leader. So I’m not surprised at the results you’ve been able to drive, but nonetheless, congratulations. I want to ask you about that survey. You mentioned you’ve been able to transform it from something where there was not maybe a lot of value placed in it and maybe even skepticism and lack of trust in terms of the data. What have been some of the key things you’ve done to win the hearts and minds of both frontline management but also upper level leaders as well?
Anna Sulkina: That’s a great question, and that’s been a journey for sure. Well, first of all, iterating our product managers who have been definitely a great drivers here, spend a lot of time really researching and figuring out what exists in the industry, what’s been done, what tools exist, experience from our peer companies. So lots of learnings. And also pulling in some of the higher level ICs within our company, within engineering, who had opinions and suggestions, making them partners in this journey. So rather than us trying to convince and report, having them partner with us so that there is… Well first of all, if there are greater ideas, we were receptive to that. But also it’s building the empathy that it’s actually really hard as we know, because everyone, absolutely everyone has an idea and opinion of how to measure productivity easy, until they start trying, as we know.
Anna Sulkina: So that’s been a really journey and kind of education for everyone. And then our survey hasn’t stayed static, but we have been really careful in how we evolved it to try as much as possible to preserve continuity and historical data. So that’s one. And also we changed and evolved how we reported the results, how we reported the results to the whole organization and also to leaders. We started slicing the results by director, by VP, so that they would get an insight on how their teams were doing and what they were experiencing and perceiving. And then we would again customize, “Hey, we’re hearing you’re having problems here. This is what we are doing to help you.”
And then more recently, we added a tech culture section as well. And that’s what I refer to when I mentioned the focus time or do people feel they have clear goals or things of that nature that really affect the tech culture. And so this is the section that we don’t do twice a year, we only do once, but that’s another area that our leaders have been really interested in getting visibility into. Because then they can take these insights and some of what we do, dev platform can build tools. But some of it in the tech culture only the teams can truly affect that. And we found that leaders do care because they can take these insights and act upon them.
Abi Noda: one of the things you shared with me before is that you’ve heard some senior leaders at Airbnb go around and say something to the effect of, “We’ve fixed developer experience. What else?” Right? So I mean, first of all, that must have felt good for you to hear, but say more on how that relationship between you, your organization, and other executive stakeholders has evolved over time and what’s that relationship really like today?
Anna Sulkina: Maybe I’ll start with the fact that I don’t know if there was a lot of relationship before. So I think it’s probably nurturing that, really building that relationship and nurturing that. That’s been a journey, and being intentional about it. I think it’s probably that and also again, figuring out how to… Having the strategy, providing clarity into what we’re focusing on and communication around that, frankly, communication at each level. So developers, targeting developers as the audience, specifically. Targeting leaders of the organizations, directors and managers, and then up-leveling them, targeting the more high-level execs. Basically customizing the message and the narrative to what people in their roles care about and what’s important for them.
Abi Noda: Yeah, that’s amazing advice. Anna, well, thanks for your time today, coming on the show, telling both the early part of your journey and joining Airbnb and some of the challenges you saw, then sharing about the incredible transformation you’ve driven and some of the things that you’re seeing in terms of that investment and journey really paying off. So thanks so much for sharing your insights and advice. Really appreciated having you on the show today.
Anna Sulkina: Thank you, Abi. It’s really been a great conversation and I love your podcast and great questions. Thank you. Thank you for the questions.
Abi Noda: Thank you.