What’s up with internal developer portals?

Abi Noda: This is the first part in a two series where we’re talking about internal developer portals. This is a topic we wanted to cover because this is something Chris, both you and I are constantly getting asked about. The customers we work with here at DX, many of them are building or buying or thinking about internal developer portals. And I know you’re involved with a number of different organizations talking to folks all the time about this topic. So again, this is a two-part series. In today’s session we’re exploring the concept of what is an internal developer portal and what are we seeing, what are we hearing in terms of how different organizations are approaching this, what are the things to look out for.

Then in part two, we’re going to do a more sort of nitty gritty, deep dive into the current market landscape, looking at the different approaches to whether you were trying to build or buy Spotify Backstage, things like that. We’re going to go deep into the different options and what to look for, how to think about making the best decision for you. A couple tactical notes for this webinar. I’m going to be keeping an eye on the Q&A as well as the webinar chat. This session is going to be recorded. We’ll get the recording out on our website and over email pretty shortly after the event, so if you have colleagues that aren’t able to attend, you’ll be able to distribute this.

I think we’re good to go. I think before we dive in, I would love to ask the audience to put in the chat or Q&A, what’s one question you have about internal developer portals and or where are you at in your journey? Are you evaluating? Have you built one and struggling to get traction? Are you evaluating different vendor options? We’d love to hear where people are at in the chat and we’ll try to inform our discussion based on that as we go. So Chris, if you’re ready, let’s dive into this.

Chris Westerhold: Let’s do it.

Abi Noda: So internal developer portals. It’s funny, I first I think encountered this concept when I was back working at GitHub and we had built a service catalog internally. We were actually thinking about… There were some product managers at GitHub talking about this Backstage thing and how GitHub should maybe release or work on a product. Really curious your view on, first of all, why has internal developer portals become an area of interest just in the past two, three years?

Chris Westerhold: Well, I think there’s a bunch of different reasons for it, but the simplest view is how do you reduce complexity? The more you look across engineering and engineering organizations, the more and more complex they’re getting. You have still plenty of people are living, you’ve had to move to cloud over the last handful of, I guess, going on 10 years now or more. But you now see people with multi-cloud, you see all the distributed type architectures of Kubernetes and now you have serverless functions. You have all of these different things that are making everything far, far, far more complex than it ever was in the past. And how do you navigate that ecosystem is really to me, one of the key drivers behind everything. As a software developer inside of that ecosystem, how do I get the things that I need when I need them without having to dig through piles and piles of documentation or all of the things that happened across an ecosystem, that’s hard enough to do for your own set of tools.

Now when you start to think of the addition of microservices and all of the other aspects that come along with that, you now have all of these loose dependencies across the organization. And so how do you work inside of your own ecosystem but also having to work with all of these other teams and handle that level of complexity. That’s really, to me, the main driver behind most of this. You’ve seen it happen with the microservices tooling space, and I think the same thing is really happening inside of the data space as well. You haven’t seen quite as much consolidation yet from a kind of data marketplace, but the same rules really apply in that game as well.

Abi Noda: Chris, I completely agree with you. It seems like sort of the natural progression of a problem, a good problem in some respects that we’ve created with the shift to cloud shift to microservices service-based architectures. I forgot to ask you before the previous question, share with the audience a little bit about what you do and why you have such an eye on what different organizations are thinking about and talking about when it comes to internal developer portals.

Chris Westerhold: Yeah, yeah. So I am the head of developer experience for Thoughtworks. I’ve been working in the platform space for quite a long time. I’ve also have a love for process improvement and really just building things in general, and that’s really led me into the developer experience space, platform engineering space. We work with clients all over the world and helping them drive better. How do you build a better ecosystem for your developers? How do you give them the tools and services they need to really be able to do their jobs? With that knowledge and that breadth of information that we’ve gathered from so many people, we’ve been trying to focus that in on ways of being able to help solve these problems in kind of a clear and concise way.

Abi Noda: I’m already seeing questions that kind of tie into this next question, but Chris, what have you seen as far as organizations that have tackled this problem on their own? Of course there’s Spotify Backstage. We will talk more about backstage in this session and our next session. But outside of Spotify, what are people doing? What are homegrown solutions to the developer portal type problem?

Chris Westerhold: Well, you see everything from across the board and it really depends on when you started. So there’s people that have tried to build their own and as anybody that’s ever really tried to build something, it gets harder and harder as time goes on, as things become commoditized and it becomes even harder than just to keep up with a market, let alone be a market leader. I know of a handful of different large companies that I’m not going to call out specifically that have gone down this road that have really, really struggled with keeping things up. That being said, it’s all about two different things. One, what’s your budget and how big is your organization and what’s the value that you can get from building your own versus kind of buying one? And I know we’re going to get into that later, but the second part is how much of a journey are you willing to go on?

A developer portal is an enablement tool to a developer experience journey. And if you are trying to stand up a developer portal and check that technology box, that’s a whole different thing than wanting to really start and continue a journey around improving developer experience as a whole and being able to really truly make an impact there. So plenty of people are using Backstage, like Netflix is a really big company that’s using backstage. They’ve done a really great job. They put a lot of time and a lot of money into it, but there’s other people that are now using some of the SaaS versions that are out there that are also really great too. But to kind of wrap that back up, it’s really all about understanding what you want to do, where you’re trying to drive towards and the outcome that you’re really hoping to expect at the end. And you’ll find people doing everything across the board and I know we’ll spend some time looking through the market landscape and kind of pro and coning that as we go along as well.

Abi Noda: Another question related to this I saw in the comments–as far as who is building the developer portals at these organizations, where do you see that responsibility sit? From my perspective, the organizations I talk to, these things sometimes start out sort of as a side project within the platform organization. In many cases that becomes a fully funded team that’s responsible for the developer portal, at least for some time. I’ve seen organizations as they reallocate resources kind of turn it back into maintenance mode, sort of side project type of thing. But what type of investment and where in the organization do you see developer portals being born and developed?

Chris Westerhold: Yeah, I see a lot of them alongside of platform engineering and those kinds of things. That’s usually where it starts and how do you then… I look at that as a bottom up approach of it’s very easy to go out and grab something like Backstage, be able to get it up and running in a few days and be able to show some value there. Well, that becomes harder and harder to do as you’re trying to implement things across your organization and being able to change process, be able to improve process and pull those things together. That’s where that journey starts to come in of needing leadership support, needing some level of budget to be able to support things. So one of the anti-patterns that I do see a lot of is there should be a central team that owns some kind of a tool, but you really should be driving an inner sourcing model across your organization to support those kinds of things.

The last thing you want is a 40 person team running a tool like Backstage, and I’ve actually seen that in a couple of different places. But it should only be a handful, maybe six, that are managing that central spot and then having these other teams contribute back to that in one way or another. A lot of the different platforms out there have kind of a plugin model. So say you have a security team or whatever, they can now give you back some kind of code to then implement inside of your portal and it pushes that towards the edge to the people that understand the thing that they’re trying to build and then get that integrated higher quality, closer to the people and provide those feedback loops to be able to improve it over time.

Abi Noda: I’m seeing some questions from listeners about evaluating different use cases and SaaS versus open source considerations. I want to just call out, this is going to be the focus of our second part two session where we really dive into the market landscape and compare and contrast different paths folks can take. I saw another question that I want to cover, which is–Chris, we were going to talk about this–the distinction between an internal developer portal, an internal developer platform, a service catalog. There’s these somewhat interchangeable, at least to the common eye, terms being thrown out in this product category. I think it would be great to break this down. So first of all, what is your definition of an internal developer portal and how is that different than let’s say a service catalog or IDP internal developer platform?

Chris Westerhold: Yeah, so there’s a ton of different words out there right now. So there’s really two things that have come. There’s the developer portal, [inaudible 00:15:09] Backstage, those kinds of things. And then there are the developer platforms that really grew up from more of the platform engineering space saying, okay, well how can I give a layer on top of a platform to be able to help with orchestration, to be able to help with interacting with that platform? I see these two things merging together where you have the catalog, the search features and the ability to do self-service inside of a portal and being able to leverage the power of your engineering platform and then how do you pull these things together into that kind of single pane of glass or an engineer regardless of what it is that they’re trying to do. So having these two things be separate is where we are right now for sure, but as we continue to move forward, these things are going to have to merge together if you’re going to really try to optimize your developer experience across these many different things that they’re trying to do.

Abi Noda: I was going to say, I completely agree with you. I’ve been actually talking to folks at Spotify, some of the SaaS vendors and asking them the same question and I’ve gotten different perspectives, but my point of view in sort of consolidating everything I’ve seen and heard is that we can kind of think of a service catalog as a feature. A service catalog could be a spreadsheet, it could be in Datadog, it could be in New Relic, it could be something you’ve built, it could be Backstage. A service catalog is a feature, and so is really the orchestration, the ID platform, internal developer platform stuff we see in the market. And a lot of organizations who use Backstage are building some of that themselves, as I’m sure you’ve seen.

Where I’ve come to sort of understand what a developer portal is, is really that idea of bringing all those features together into one place. So as I’ve talked to different folks who are running a developer portal companies and products, that’s been what I’ve heard as well is these are all the orchestration, the service catalog are features no different than GitHub or your CICD platform is sort of a unit of functionality and the vision of a developer portal is around bringing that all together into one place to create a more unified experience for the developer. Anyways, that’s kind of the framing I have right now, but what do you think about that?

Chris Westerhold: I think that’s a really great of putting it and having these separate pieces that are out there, it’s just how do you pull it together? You’re 100% correct and whether you call it a developer portal, an internal developer platform, a developer platform, a service catalog, whatever you want to call it, hey, you need to be able to, like you said, pull those features together and enable the developer experience to allow them to do the jobs that they’re trying to do in the easiest way possible. And also realizing that every time a developer comes to the portal, they probably have a different purpose of being there. Whether it’s there’s some kind of incident management, maybe they’re trying to research what’s out there to help them to build some kind of new service or tool.

One of the keys towards building a good portal is understanding that overall user experience and allowing them to be able to find the information they need in the easiest, fastest, quickest way without having to have some kind of internal knowledge that they may not have or they’re under too much stress to be able to understand at the given time.

Abi Noda: To conclude this topic, there was a question around what’s the difference between an IDP and developer platform, and I’ll just share where I’ve landed on this. The abbreviation IDP is being used to refer to both internal developer portals as well as internal developer platforms, and those terms are of course being used interchangeably as well. But in my view, generally speaking, the portal refers to this sort of unified single experience for the developer that we’re really focused on discussing today, whereas the platform IDP ID platform is referring more to the orchestration tools that Chris was discussing earlier. So we have naming challenges and problems in this industry all the time, but that’s at least the distinction I see and hear most often. Chris, I want to shift into a little bit of a different topic. We’ve talked about how developer portals are on that hype curve and really grown in interest in popularity over the last two, three years. Lots of organizations interested in developer portals, funding, pursuing portals. What do you see as some of the biggest misconceptions or mistakes as companies are early in their journeys in pursuing this?

Chris Westerhold: Yeah, I see a lot of them. A lot of them, they just don’t understand that this is a journey and I said this a little bit earlier. As you’re thinking about a developer portal or platform or whatever, this is a part of a broader developer experience journey and it should be looked at that way and how can you go about optimizing your organization and using your developer portal as an enabler for that? If you don’t look at it that way and you look at it as a technology problem, you will see and you will go and you’ll build a developer portal and nobody will want to use it because they’re not seeing real value in it. It is a thing that’s there that is fine, but in the end, you might’ve made your developer experience worse by standing up this portal because you now just have another thing with another set of information that doesn’t give them everything that they need that it makes it messier.

I talk a lot about building a solid data foundation with developer experience, and this leads into that of what are the actual problems that you’re trying to solve? What are the things that people actually care about? And then how do you go make that problem go away? I see lots of people try to start with the easy things, which generally tends to fall down the road of if it’s easy, people probably aren’t going to care too much about it. You talked about the service catalog, that sounds like an easy thing to do, but it can be very difficult if you’re starting to think about building out your taxonomy, understanding how your organization works, how does all of this fit together from an architecture perspective? How do you start to… There’s lots of questions that come from there, and if you’re not willing or able to go through that journey of understanding what good looks like there, you’re not going to get the results from it and you’re not going to be able to start to build feedback loops to help you optimize as you go forward. And those are really some keys there, and there’s some organizations that aren’t willing to do that. They don’t have the money, the time, the expertise or whatever, and they will have spent quite a bit of money to stand something up and get little value out of it.

Abi Noda: Yeah. One thing I see there’s a lot of anti-patterns. I think we both see when it comes to developer portals. Certainly there is a pattern of platform teams or someone within the organization building that portal as a side project, maybe getting some funding and then later running into problems around what’s the value of this? Why aren’t people using this? What’s the point of this? I think to your point, answering some of those questions upfront, what is the problem you’re trying to solve? What is the value you’re trying to drive to the business?

There have been questions in the chat around how do you measure the value of developer portals for yourselves and to your stakeholders? Certainly there are ways to do that. I think we’ll talk about some of that in our second part session to follow onto this. But partly asking that question after you’ve already built a developer portal is doing a little bit in the wrong order because if you can really pinpoint the value that you’re trying to solve upfront, then how to measure that or how to get signal on that and stay close to that is a much simpler problem.

I actually just published a podcast this morning, which was a conversation with Adam Rogal at DoorDash who bootstrapped their developer portal over a number of months. Really inspiring story. It’s been a really big success. One of the things he did was he didn’t start with the service catalog, and I think both of us, we see everyone start with the service catalog. I recently talked to a pretty large organization and they told me, “We thought the service catalog was going to be this killer feature for developers, and it turned out it wasn’t.”

And what Adam at DoorDash did was he sort of identified that at the very beginning. So when he spun up their internal developer portal, he didn’t even do a service catalog. What he did instead is strategically consolidated some of the existing developer tools that were across the company and brought them in to a unified and better experience. That immediately made their developer portal a go-to hub for developers, and then later months later, I think they eventually added a service catalog. So I’m curious, what are the successful and unsuccessful journeys you’ve seen as far as how to approach what is the problem you’re trying to solve?

Chris Westerhold: Well, it comes down to what are the goals of your organization and what are the things that you are trying to do as you start to look forward? Do you have a gigantic monolith that you’re trying to break down into microservices? Are you a hundred-year-old bank that just says things scattered everywhere that no one knows how to use? You’re not looking at a gigantic transformation, but you’re just trying to go faster. There’s a million different varieties of that. And so what is that solid base of where you’re trying to go? Because say you’re doing a monolith to microservice transformation, a catalog is probably going to be a little more important along with some kind of self-service infrastructure around what a good microservice can look like. If you’re more leaning towards the gigantic old bank or DoorDash, like you said, they’re just trying to reign in the chaos that’s happened across the organization there.

Consolidate tools, maybe stand up something like a tech radar, start to align across different technologies and figure out how do you reduce fragmentation inside of your ecosystem. And then the other part is how do you start to measure some of that and how can you think about ways of designing metrics that are going to give you information about how successful you’re going to be? Something simple as this is the cost, your cost value ratio, but then your adoption ratio. Then using really great survey tools like DX to be able to understand is this actually impacting things positively or negatively in ways that you didn’t even think of?

Being able to use that product mindset to think about, here’s the problem space, here’s the potential solution, here’s how I can measure it and here’s how I can gather feedback is critical towards all of these different things, and if you don’t think about it that way and you don’t start to build that cycle, you’re going to constantly just keep building things no one cares about and you’re not going to really focus in on the actual problem. Like in this scenario, there’s tools everywhere, how do you pull them together in an easy way? Great, that’s instant value to your development community. Then I tell people all the time, if you’re struggling with adoption, you are on the complete wrong track and you need to stop, take a long pause and figure out where the actual problems are, then figure out how do you develop a solution and then be able to go forward.

Abi Noda: Yeah. Chris, I think there’s two more tactical questions. I was reading back through some of the questions we’ve gotten that would be interesting for us explore. So one is regardless of where you are in your journey, maybe you’ve already built something and you’re trying to find internal product market fit with your portal or maybe you’re thinking about it, but I think the first question is what are some common use cases where we’ve seen the developer portal be able to map to business value really successfully? That’s the first question. The second question is I think is, is adoption even the right metric in all cases? I had an interesting question with someone from Twilio. They’ve done a lot of investment in developer portals and their team was telling me, “Look, we’ve come to see this problem as you could either approach it from a MAU standpoint or you can look at the problem from a different perspective.” So we’d love to start with the first question, what are clear wins or common wins we’ve seen as far as… What can a developer portal do for you today to drive value?

Chris Westerhold: This kind of leads into the build versus buy because a lot of the buy are going to get you 80% of the way there and they’re going to provide you an awful lot of value, but you’re going to have to fit inside of the box a little bit. The building, especially building using something like Backstage gives you the ability to do anything that you want. It also comes with the cost of maintaining and building and doing all of those kinds of things as well. So there’s a little bit of that, but any good developer portal is going to have four major features. You’re going to have your catalog, search, some kind of documentation tool and some kind of self-service tool. So if you think about what your goals are as an organization, where the problems exist, it might be any one of those four, or it might be a combination of those four plus things that are below it.

If you think about the plugin model that Backstage and others have, yes, you might stand up your service catalog in a very light way, but the value is going to come from some of the plugins and think about something like contract testing where that is very hard to be able to [inaudible 00:30:58]. Well, it depends. But it’s hard to be able to pull some of that stuff together inside of the context around a given service, and maybe that’s the problem people are having. There’s a million dashboard, there’s a million tools and there’s no way to pull it together. Well, you can do a light version of your catalog and then build good plugins to be able to pull things together. I like to look at it as there’s a usefulness hurdle that you have to get over to get people to stop doing whatever mechanism they’re doing now and to start using this new mechanism.

When I think about kind of a product perspective, people are interested in your product, people are willing to use your product, and then people are willing to pay for your product, and those are three different hurdles that you’re going to have to get over on anything that you’re pushing forward. You need to validate at each one of those steps, but figuring out how do you build those feedback loops is almost always the first place that I start, and you can do that through surveys upfront, you can start to use your developer portal as a feedback mechanism.

I truly love your guys’ platform X product where it allows you to hook up to any event and be able to do a quick survey feedback loop. That’s the way that you’re going to have to use the build, measure, learn cycle and be able to identify what those are. I can give you just some basic things that I see people do, but I’m a little hesitant to do that because I might be pointing you in the wrong direction because no two people are the same, no two ecosystems are the same, and if you just go start to build random things, you’re just hoping and wishing that things are going to work out and that almost never does work out.

 

Abi Noda: Well, I think that caution is prudent, Chris. If everyone’s just trying to build the Spotify version of a developer portal, well, you’re going to run into headwinds if you’re not Spotify because of just the culture that’s built around the developer portal there and the funding over the years and years that’s gone into that. For me, I’ll kind of refer back to that conversation I recently with the platform team at Twilio where they mentioned to me, “Look, we’ve kind of moved away from the MAU model of thinking about developer portals and we’re thinking about it in a little bit of a different way.” To me, as I’ve talked with organizations we work with, different vendors, folks at Spotify, I’ve also come to kind of see the developer portal as solving two slightly separate use cases. So the one, and I think this is the one folks are most excited about, talking about and selling is the developer experience, right?

It’s about creating a more unified experience for developers, reducing toil, enabling self-service. And when you’re talking about those types of capabilities, adoption and MAUs really do matter, right? Because if you’re talking about improving the developer experience, you got to be able to actually show that developers are using and getting value out of these types of tools. But then I think there’s the other sort of use case, which is more around governance and scorecarding and top down visibility into service health really built around the scorecard. And for that one, you could argue adoption numbers don’t really matter. It’s really a problem of just getting better visibility into your services architecture and components. I asked earlier, what are some of the slam dunk use cases for developer portals? At least from what I’ve seen, I would focus on either or both of those two paths.

If you’re focused on developer toil, it’s about you hear, I know we get a lot of data on this from developers, but it’s hard to find information, it’s hard to maintain and deploy software and operate software. There’s a lot of sprawl and information and understanding how to build things across the organization. So that’s where that toil value proposition I think can work really effectively if you have that kind of signal in your organization already. If you don’t, maybe you think about the more reliability and quality top down visibility driven use case. That works well in instances where there are reliability issues happening. I know organizations where they’re just trying to get themselves out of sort of a spree of reliability incidents and they look to developer portals as the answer with great urgency. And so curious what else you’ve seen, but for me, if I had to just boil it down to how to think about aligning developer portal, at least as they exist today to the business, it would be in one of those two categories.

 

Chris Westerhold: Yeah, I absolutely agree with you. One other thing that I would add into there is when people start to think about adoption, and you brought that up a handful of times, there’s three different things you need to consider. There’s the technology space and people really focus on the technology, but there’s also the people problem and the process problem. You have to start thinking about all three of those to make sure that you’re aligned and that you go back to solving the problem that then drives adoption. You also need ways of measuring that, and so coming up with kind of leading and lagging indicators to be able to do that is the other piece that really makes sure that you’re hitting the mark. But that said, the engineering knowledge management is a really big problem everywhere. Everyone’s documentation is basically terrible, and using a developer portal as that single pane of glass to be able to provide better search is almost always a place that I would tell people to start.

Let me be super clear. I’m not saying you have to move all of your documentation into your developer portal. That’s not what I’m saying. But you need a way of being able to search across all of it, because when I think of engineering knowledge management, I think of three major pillars. How do I document a service, how do I document a process, and then how do I get a question answered? And you can facilitate that mechanism through a developer portal. You could also pull in information from Confluence or from whatever other tools you are looking at and be able to have a unified search that can look across them. Then you can do things like sticking gen AI on top of it and being able to now summarize, ask questions based off of the context that’s there, and that’s actually something we’re currently working on that we’re probably going to be building a backstage plugin off of. That’s really where a really great value that you can bring if that’s a major problem for your organization, which it probably is.

Abi Noda: Well, we’re coming up on time. I need to wrap up, so I’ll conclude with one final question for both of us, which is what are the most common traps? Why do these projects seem to be oftentimes struggling a little bit? I’ll go first with my answer, but at Thoughtworks, you all have written some great stuff around platform as a product. But in my experience, there’s often a lack of product management discipline and muscle within platform engineering organizations. And of course, due to the technical nature of the developer portal, it is a little bit out of reach oftentimes of the type of product management talent I think we have within our companies. To me, the biggest trap–it sounds so cliche–really is this idea of treating this as a product, treating it as a startup that you are bootstrapping within an organization. How do you show value quickly? How do you get funding? How do you continue to show momentum? All the things you would have to do as a startup I think with being really product minded, that’s where I see a little bit of sometimes lack of attention from platform teams.

Chris Westerhold: If you look at the Spotify story, Backstage started in around 2014, and it took a long time to build into something, the thing that you kind of see today where most developers inside of Spotify use it. The way that it was grown was by solving people’s problems and then iterating on that. There was no mandate from on high to use it, there was not a lot of funding, but they were constantly leveraging the platform to help solve people’s problems, and it slowly started gaining momentum and gaining momentum and gaining momentum. That to me, is the mechanism that is going to drive cultural change, it’s going to drive a continuous improvement across your organization, and it’s going to be enabler to make that happen. If you think that you can do this by tomorrow or in three months or in six months, you’re kind of wrong probably, because it takes that journey and it takes to go back to the people technology and process.

The technology part’s easy, changing the process. You talked a little bit about governance, but changing the process is hard. Getting people to change the way that they work and the way that they operate and the way that they interact with people is hard, and trying to force that down people’s throat is tough. Like I say this about DevEx quite a bit, but do it with your community, not to your community. And the same thing really applies here. You have to solve people’s problems. You don’t have to go slowly, but you have to be methodical, and that’s the way you’re going to drive success overall, and if you don’t go down that road, you’re going to get questionable results.

Abi Noda: Well, Chris, thanks so much for coming on today and everyone for joining. This has been a really fun discussion. If you’re interested in this topic, Chris and I are hosting a part two to this discussion where we’re going to be diving deep into the current market landscape, looking at the open source and SaaS options, and talking about what should guide your decision making and evaluation of how to solve this. Also, as I mentioned earlier, a new podcast dropped this morning with head of developer productivity at DoorDash talking about their journey of building a developer portal. We’ll see everyone next time. Thanks so much for joining. Chris, thanks again. This was awesome.

Chris Westerhold: Yeah, thanks for having me. I appreciate it.

Abi Noda: Awesome.