Bootstrapping a developer portal

Abi Noda: Adam, so great to have you on the show. Thanks for your time today.

Adam Rogal: Thanks. Really happy to be here. It was great following along everything that’s been happening in the community and just really looking forward to chatting.

Abi Noda: Well, when we ran into each other at DPE Summit recently, you showed me some of the work you’ve been doing with your developer portal. And some of the ideas, the journey you shared with me was really inspiring, so I’m really excited to bring that story here today to listeners. So I want to start at the beginning of the story. You’ve been at your current company, DoorDash, for several years now, but as I understand it, the idea for the developer portal you’ve built really started on day one. So share with listeners how you got the idea, the vision for the developer portal in the first place?

Adam Rogal: Sure, yeah, absolutely. Yeah, and you’re right. I think it was day one that I got to DoorDash and was happy to not see a developer console just yet because that meant I get to envision one for the future for everybody. But one of the first things, when I started at DoorDash, I was set to lead the developer platform organization. And there was a couple folks within that org to begin with, it was a ragtag crew. We were working initially on just thinking about developer productivity in general and one of the main things that came up was how do we know who owns what? Because that’s usually the heart of every software engineering organizational problem that we have. On top of that, there was also a growing need for tooling and a lot of questions on where do we want to put that tooling? This all led to us thinking about what does the future of our tooling look like and the first thing that typically pops to anyone’s head is let’s have a website.

So early on I always knew that we were going to have some kind of developer portal. My previous work at Uber, where I was before, we were building out a version of that, a lot of influence from Stripe who had done their Eng Home work that folks can look into. And we knew that we were going to have some kind of place that is the home page for engineers and we also knew that we needed someplace that was going to be able to be a platform to create more contributions over time. So one of the things coming from a platform background, I always know there’s no way that we can build everything ourselves. We always have to have a community of not only users but contributors as well.

Having been friends with many folks at Spotify over the years and always getting glimpses of their amazing internal software, I’ve always been super envious of their version of Backstage that they had for the longest time, whenever I got glimpses, I was immediately taken back and always asking for more previews of what can I see, what can I poke around, what can I look at? It was just so amazing to see wait, they have everything in one place. They have all the software, all the people, all the teams, all the KPIs, everything in one place. And that’s so different than how you typically see this thing at companies where it’s split across so many different websites and so many different tools that you don’t know where to go and there’s so much context switching that happens.

So pretty much I knew from day one and all of our customers did from day one as well, because they were shouting at us from day one to have something like this. And we wanted to build out a developer console. That’s eventually what we ended up lovingly calling it. It had to start though with really understanding what the need for it would be. Where do you start? Because there’s so many different things that you can add to this. The first step was us just looking at the main problems that our engineers have. And as I mentioned before, there were two main problems. One of them was who do I know who does what? And the second one was, there’s just a set number of tools that we need to be able to do our job and we don’t have those tools yet and we want to make those tools and we can’t just have them be command line.

So that actually was the nugget, the seed crystal for where we wanted to go. It was about four to five months into the role that Backstage was starting to really get traction with early adopters in their open source version out in the community. And right around that time, DoorDash had a hackathon and I thought, wouldn’t it be great if I made this a reality by entering the hackathon and going from there? So I entered the hackathon with an additional engineer and we built out the first version of DevConsole and it had just two plugins. It had a test studio, so a place that you could visualize creating a test dasher and a test consumer and how they interact with each other and how you could automate that for any kind of in-development testing. And it also had a dashboard for all of our quality metrics, which was a big thing that we were looking at the time, how do we look at a recent outage, look at the AIs on the postmortem, tie it together to any kind of delivery loss that we may have had. And we want one central place that engineers can go to and leaders can go to at the same time and have a holistic view.

So we started coming at it a bit sideways. We didn’t go with the traditional route of just that service catalog to begin with. We wanted to show immediate value to our customers rather than just going straight from a platform. So building out those two tools, we won the hackathon and ever since then I’ve been nicknamed on the DevConsole side, but that was really the start, that was the birth of it to begin with.

Abi Noda: Really interesting story and different, as you mentioned yourself. I do have some questions about it. You mentioned that one of the things you envisioned was that this would be a place to house developer tools. At the time when you were getting started with this, were there already some tools that existed outside of this? And so in your mind, did you think, okay, we’re going to just migrate these tools over? How did you think about the transition that would perhaps eventually have to take place?

Adam Rogal: We had a couple tools in various places that were geared towards audiences that I would say maybe orthogonal to engineers. Engineers would certainly use those tools, but it wasn’t your traditional developer portal-style interface. A lot of this was because DoorDash’s philosophy has been to adopt off-the-shelf solutions rather than build them our own. So when we think about our CI, we were using Jenkins, so we were able to take a look at the Jenkins website that we have. When we were thinking about all of our experimentation, we have an internal experimentation tool, but our data scientists use it as well as engineers. So it wasn’t just geared towards engineers. And we had several command line tools as well as Slack bots that we were using for deployment for example.

So it wasn’t quite that. We have a place already that has a bunch of web tools, which meant that there was really fertile ground for something like this to exist, which is really rare. Most companies, you’re going to have something that kind of exists in multiple places and then you have to go through the arduous journey of migration or competition. To our luck, I would say that had not existed just yet. So it was the right place, right time types of situation to go forward with it.

Abi Noda: You mentioned Spotify, which, as we both know, it is pretty incredible the internal developer experience they’ve built around Backstage. And as you’ve seen examples from other companies, what is, to you, an attainable vision for a developer portal? Because a lot of folks I talk to have the Spotify vision, that’s where developers live and do everything. I haven’t seen that actualized outside of Spotify. I’m curious what your perspective is on that?

Adam Rogal: Besides us, right? So when you talk about a vision, I think most folks will boil the ocean, so to speak, where they try to think of everything that needs to exist in this portal. And a lot of times that’s either intimidating or, at worst, your customers don’t quite understand that vision because it’s such a paradigm shift from what they’re used to. So anytime you create a vision, you have to start from a shared understanding with your customers and understanding their needs.

So when thinking about even Spotify, they didn’t start backward from one tool to rule them all. They started brick by brick of thinking about what were the main needs of their users. A lot of those were what I mentioned before of who owns what. So I may be speaking out of turn here for Spotify, but they have a matrix organization, as we all know, with guilds and domains and teams and it’s not a very traditional way of operating a company, which has worked very well for them, but also makes it a little bit more difficult to understand what is a squad, what is a grouping of folks. So I can imagine that building Backstage was a way for them to speak to those initial needs from their customers of just understanding what my peer works on or looking at something, who owns it?

From there, you typically have sort of a read-only view in your dev portal to begin with. It’s usually a facet of information in one place. So you can have a bit of a network effect between all the various parts, but that’s one of the first bricks that you lay down. The next brick that you typically lay down on top of that is saying, all right, what if I want my portal to actually affect the system, right? It’s one thing to look at a service, it’s another thing to actually click deploy on the service and watch it do something. And those parts, that transition is typically pretty scary, right? Because your dev portal was sort of a beta up to that point where you could sort of have a lot of fun, cool demos, et cetera, but once you get to the point where it’s now a critical piece of your infrastructure, there’s a lot more pressure and a lot more expectation on that tool to begin with.

So as you’re going about that journey of creating the vision, I would say you want to collect your customer needs. You also want to identify your customers, because you have different personas across your engineers, not just backend or mobile, but I have a backend engineer that is looking to debug an outage. I have a backend engineer who’s looking to create a new service. And you group the needs together and then you have that to start to influence your roadmap. And then a key part just with any product development is you have to go back to your constituents, your customers, and you have to share that roadmap and you have to understand where the prioritization lies in their head and where they want to go from there. And if you get that loop going and you get a really good core piece of folks who get excited about the product, you can then start to really build out that longer-term vision. And you know you’re headed the right direction because the excitement grows monotonically along with the product itself that you’re building.

Abi Noda: That’s really good advice. I want to ask you, we’re going to talk about how you’ve funded the ongoing development of the portal for the past few years, but I want to talk about the beginning again. When you were just getting started, were you thinking about funding a business case? Or were you more in a position where there was just this kind of latent demand and support and you just kind of went for it

Adam Rogal: More the latter. I think that because we got a lot of excitement in the hackathon and because we knew that something like this needed to exist, I was able to, if you want to map metaphors, I was able to get a little bit of angel investment in this and I was able to ring-fence off that investment and nurture that investment, but also had to really dig in myself and write a bunch of the code myself, which I still apologize to this day for several of the people on the team. But you have to treat it like a startup. You have to treat it like going in there and owning the outcome because these things like developer portals are fledgling. They’re the things that drop by the wayside because they’re not going to show dividends for at least six to nine months when you build something.

And that’s why one of the focuses is how can I deliver value as soon as possible? And sometimes that means an orthogonal approach than the traditional one. Instead of going straight to the service catalog, which might not actually show value until you actually do something unique with it, you can start to think about how can I start developing tools that’ll give me a little bit more of a runway for your customers and for your leadership to also still want to invest in this area.

So early on, I think that it required a lot of leaps of faith from my leaders. They saw really great things early on which kept them excited about investing in this area. There were definitely times where it was touch and go of feeling, are we doing the right thing? Are we headed in the right direction? I think it was maybe a year after its birth that we crossed a first big milestone of just daily active users where we had added a really key critical feature of Codelabs to our site. We are hiring a lot of post-pandemic or post-start of pandemic. And just with all the onboarding that was going on all the time, where do you put all this documentation for those folks who are onboarding? And it’s not your traditional documentation set. Onboarding docs are step-by-step. You need to go back, you need to reread.

So we developed a codelabs plugin and that codelabs plugin was critical for our bootcamp and critical for the engineers. And one of the great things about that is also indoctrinated the engineers that DevConsole is a place that I go to because from day one, you get to use this tool, see the value of it, and then like, oh, there’s a bunch of other things here. I better bookmark this. So it’s really about going to where the engineers are already trying to show value as soon as possible, trying to gain some trust with your customers and your leaders that we’re going to show value, it’s just going to take a little while. But also just really focus that you have to treat it like a startup, there’s a runway, there’s a certain amount of capital you can have in this until people start to get impatient and they want to see results.

Abi Noda: Makes sense. What you just mentioned about indoctrinating people early reminds me of when I worked at GitHub, we had a big program at GitHub around giving away GitHub to schools and the whole mentality behind that was to get people indoctrinated with GitHub before they become professional developers. And so it’s a similar strategy.

Adam Rogal: I mean, look at Google with Chromebooks, right?

Abi Noda: Exactly.

Adam Rogal: I mean, it’s a great approach and it’s win-win for everybody.

Abi Noda: Yeah. One of the things I’m taking away, you were kind of touching on this earlier, is that one of the things you’ve done very intentionally with the developer portal is, rather than approach the developer portal in sort of the conventional way, which is a service catalog, instead you approach it from what are really useful developer tools that we need and we’ll just put them in the developer portal. Am I understanding that correctly? Has that been the strategy?

Adam Rogal: Yeah, I think that’s the nail on the head. Yeah, there’s so much more in our DevConsole than just service catalog. I would say that 90% of the use cases in DevConsole right now is not geared around the service catalog. To give a few examples, we have debuggers across every part of our system and these are actually contributions from other teams. So other teams say, “I need to look up an order by its ID. I need to see all the fields, the history of it, I need to write a tool for this.” And they chose DevConsole to write the tool for this because it was easy to onboard. We created CodeLabs for creating Codelabs and created Codelabs to create DevConsole plugins and eat our own dog food across it. It’s standard React, so it’s very easy to get up and running with it. And we also had a central team that was there to act as developer relations and to not only promote the development of it, but also guide the engineers contributing to it.

So we built out a number of debuggers. We also built out a number of just analytic tools to understand the health across, not only just the quality that I mentioned before, but also the velocity in certain areas. And as you know and your listeners know and you’ve stated more times than I can count at this point, there’s no one single metric for velocity, but we do want to see things like what is the lead time from a pull request to production? Not to necessarily goal ourselves on it, but it gives a manager a good insight into should I be releasing more often? So if my customers are complaining internally that it takes a while for them to see their pull requests out in the wild. I have an empirical way of looking at that.

And that was a plugin to begin with of our dev velocity plugin where we could track numbers that sometimes we’re part of our OKRs when we are doing spikes on those areas, but other times just places that EMs would go to. And then GitHub, great at pull requests, not so great at an engineer’s inbox, to look at what do I actually have to do today and look at. So we created also a plugin that allows engineers to see their pull requests that are waiting for their review, pull requests that need their attention to review as well, and then also for managers to look across their entire team to see a pull request to assignment for individuals to triage and to manage this. One of the reasons that came about, we were interviewing managers and we saw these crazy Google Sheets that they created to basically scaffold this entire system for themselves. And it’s cheat mode. If you see someone who created a spreadsheet out there that does something, build a developer console plugin for it, basically them just creating a UI that you could easily recreate and then add functionality to over time

Abi Noda: As you were talking, a random question popped into my mind, which is the naming, is it called DevConsole? How did you come up with that name for this? And have you changed that over time?

Adam Rogal: No, it’s always been DevConsole. I cannot remember for the life of me exactly where that came from. I’ll give a shout-out to the initial folks of Marco and Jessica and Seth for influencing that. But DevConsole, it’s a console for developers. I think a lot of it was just our past experiences maybe at Google with Cloud Console and those areas. But yeah, it’s always been lovingly called DevConsole. The hardest part is whether or not there’s a space between it, and that’s always up for debate whether or not it’s just one single word with capitalization. Was it Camel encoding on it or whatnot or title case encoding.

Abi Noda: I love it. So folks who are listening to this may not have ever yet written a line of code within Spotify Backstage. They may be inspired by hearing the way you’ve approached it where you basically just bootstrapped it as a hackathon side project, et cetera. What advice do you have? What can you share with people who are in a similar position that they want to do what you did? They haven’t written a line of code yet? I mean, how much time energy needs to go into this in your view to make something happen, get something going that you think might be able to have legs and stick?

Adam Rogal: Great question. There are so many different facets to explore here, but the one I would say is that whoever’s listening that wants to create one of these has to be the chief evangelist and has to be the founder and has to be somebody that is willing to invest every second of their free time into something like this. Because as I said, it is a startup within a startup. It is a product and a culture shift to have something like this. And to do so means that you can’t run that from the back line. You have to be a player coach. You have to be somebody that has their hands dirty and is owning every aspect of the problem. Now, that’s not to say that if there’s an eng leader out there who isn’t a fantastic coder, which is very typical, no offense to those out there, partner up with that yin to your yang, that will be that co-founder with you to make this happen because you do need that level of camaraderie and that level of investment to make it a success.

And then on top of that, you have to mention this product and this use case to a sickening degree across the company, such that I get made fun of constantly or poked fun at because of how often I had mentioned the tool in the past. There’s literally a mug with my face on it saying DevConsole with the Ancient Aliens guy on it with my face on it because of how often I’ve sort of put DevConsole out there as the tool. And to a certain point people might be sick of hearing that, but you have to be the one that’s just pushing it all the time because it has this immense amount of activation energy that’s needed to get over the hill. And then you have to continue doing that with all the challenges that come into place.

Talking with Lee from Backstage, they had to do that internally too. It wasn’t a free lunch for them. They had to build it and then force people to use it and to think of it as their home. And I think that says a lot that even when they had an amazing tool already, it still required a ton of energy to make it happen. So it was a big investment. I think my partner at home was a bit sick of looking at VS Code open all the time and me contributing to it, but I think it showed dividends over time of just where it came from.

And I was also just ready to accept failure on it too. I think that’s the other part is when you create something that’s ambitious like this, it’s going to have a lot of points of just reflection and understanding what you could do better or what you can pivot at. You’re not going to be successful at the first time. You’re going to have challenges. You’re going to have people who don’t quite see the value just yet on it. And if you have a strong following of folks who want to see it be successful and you have a strong vision and you’re willing to go on the front lines for it as well, you’ll get to success eventually, but it’s not a paved road to get there.

Abi Noda: Well, inspiring words, I mean I’m a startup founder and what you’re saying definitely reminds me of my own journey. So it’s really interesting that you’ve viewed and been successful by approaching it this way. What advice do you have for people who aren’t really in a similar position? Maybe they can’t just bootstrap this, maybe they need permission to go put together a prototype. I mean, if you had had to start from the beginning with a bit more upfront commitment like having to put together a business case or convincing others, I mean knowing what you know today, how would you approach that if you were put in that position?

Adam Rogal: Yes. First of all, I look very much at the privilege I had of being judge, jury and executioner of making this thing a reality. Also, the downside of that, you’re laying a lot on the line for something that may not pan out. So if you have to prove the business justification of something like this, I think you’re already in a really tough spot because that means that you’re part of some organization that doesn’t immediately understand the intrinsic value of something like this. Which being there, I think you have to take a very different approach than how you would take with maybe you’re in an organization that already considers many of these objectives as part of their charter. So I’ll look at it from both sides.

Let’s say for example, you’re at a very small company. There is no dev prod team. There’s a couple set of engineers who maybe care about this area. You ask, “Hey, can I put a business justification together?” No, just work on it. Just do it. Find the time, hack something together because you never could convince someone of a big paradigm shift by showing them a slide deck. It has to be the proof is in the launch, the proof is in the usage. If you ask for forgiveness rather than permission on these types of things, I think that’s where you can bootstrap it yourself and get it out there. I’m sorry for everyone who is now having going to have waves caused in their wake with these things, but it’s just part of the role that you have to have. I love companies that encourage this 20% time, whether or not you get the full amount of the 20% time or not, it’s encouraging when you see leaders who say, “We’re not going to know everything. We want to see your contributions."

And if you have a hackathon at your company coming up, work on this. Take the time to do it. And if you don’t have a hackathon at your company, start a hackathon program. Really, you have to find the time to invest in this personally to make it a reality. And that’s true about every personal passion project you might have at a company, not just DevConsoles, but anything you do at a company that you care deeply about, that’s just not the norm yet.

Now for a company, just switching gears that maybe you do have a dev prod team, maybe there’s four or five people on that dev prod team, maybe you got one front-end engineer and a bunch of scattered tools across your various parts of the company. I would say that it’s a misstep to propose Backstage as let’s just unify everything into one place. I think there’s a lot of value in doing that, but I do think that it’s a very, very tough pill to swallow for your customers and your leaders to say, “We’re going to pause development on features for three to six months while we build out this new platform that’s not going to show dividends for nine months as we build this out.” And not to mention the people who had built the tools previously, they might not be bought in that they should exist within your playground now. They have their own place. Why are they going to move to this new one that might not even be the same tech stack? They might be Ruby on Rails and now you’re saying we’re going to move to React, right?

So you have to understand the current ecosystem that you’re in and how to navigate that and it’s going to be different for every company. One of the pieces of advice I have always with dev prod is don’t waste your time trying to measure everything to begin with or form baselines or come up with project plans. Just fix the problems and the paper cuts that engineers have to begin with. Talk to your engineers and see what they’re complaining about and go fix it. And if you do that, then you start to build up whatever this new thing, this new paradigm will be brick by brick and people see the value of it. And then it doesn’t become the cart before the horse of we’re going to build this Backstage instance and put everyone on it. It’s we’re going to build all these tools that solve these customer needs, it happens to be in backstage and then people want to be co-located with your tools later on. But it’s not done through a mandate. It’s not done through a slide deck that maybe people just see as territory grabbing or just going to be extra work for me and not seeing the value of it right away.

Abi Noda:

I share your observation you mentioned earlier around does the organization intrinsically see the value in something like this or does it not? That immediately puts you in a totally different camp and lays out a very different path of struggle potentially, or a different approach at least to get something like this off the ground. The timing of our conversation is timely because I know, you mentioned to me just earlier today, you got off a call with a team from another company who’s kind of thinking about doing a developer portal, but maybe seeing it as a little bit daunting. So I’m curious to ask, obviously keeping their identities anonymous, what kind of concerns did they have and what was your advice to them?

Adam Rogal: Yeah, I think there’s lots of companies out there that I chat with all the time, whether it’s at developer conferences or just pings through LinkedIn. I’m always happy to chat with folks mostly because I get to learn from where other people were at. And any piece of advice I give out is typically what I’ve learned from other folks and from the warnings that they gave me. But any kind of entrepreneur, the best thing about an entrepreneur is they don’t listen to warnings. They just go and do it and then they get to warn the next entrepreneur and it’s always a fun cycle to see that play out.

In either case, I think that the advice still stands for everyone who approaches about building up a DevConsole or dev portal is you need to understand the reason why for it. And there’s the intrinsic value where it’s, “Hey, we’re part of dev prod and that’s what we do,” but that only gets you so far. And when you think about the reason why, it’s very easy to fall into the traps around productivity. It’s very easy to fall into how are we going to measure this, how are we going to see the value of this? And to not get hung up on that I think is really important to really just rely on satisfaction from your engineers, as you talk about quite often. And I think that if you can show that your customers really want something and then you can also understand what your customers want, then I think that you can build the right product.

And that product might be slowly over time we’re going to make everything have the same UI because we have a design system, so let’s actually incorporate the design system. And then maybe one team isn’t happy with Ruby on Rails, so move everything over to react and you kind of inch closer and closer until you get really close to what it could look like for something like Backstage. And then you can make actually an informed decision on do we hop over to something like Backstage? Do we just create our own version of Backstage internally? But you can actually have a closer comparison of fruit rather than fruit to vegetable as you’re going through things.

One thing I really like about Backstage that I would just encourage everyone to do is it never hurts just to spin up a Backstage instance and just start playing around with it because I think they have really fantastic onboarding information and it’s very easy to create a Backstage instance and just touch the code. And one thing you’ll see is their plugin architecture I think is actually really great for folks who are creating developer portals because it helps you segment out the code from contributions versus central funding. It also really helps protect the app in terms of stability. It also helps with migrations. So you can build out a dev portal and sort of have one plugin at a time, but Backstage also has a ton of third-party plugins also to pull off the shelf and start using.

So anyone who’s thinking about Backstage, I mean go play around with it over a weekend, go start it up and see what you can make with it. And really that’s true about every tool out there. There’s always the abstract thinking of adopting a new tool. There’s something else to be said about actually putting your hands on it and trying to use it and trying to see where it goes from, where you’re starting at and where it could possibly go to that gives you insight that you’re not going to get by just reading through docs or looking at a video. So get hands on always with everything you have to begin with.

Abi Noda: So we’ve talked about how you were able to get this off the ground and the advice you have for others trying to get a developer portal off the ground. The next biggest challenge I always hear and see from folks doing developer portals is adoption. They built the thing, but people aren’t using it. How do you get people to use it? They’re a little bit stumped. And you’ve shared how one of the ways you’ve approached that is by building things that are actually useful and needed within the developer portal. Shouldn’t come as a shocker, but what are some of the other things you’ve done? Evangelism, for example, or you brought up building the Codelabs for onboarding as really moving the needle. Have there been any other things you’ve built that have really moved the needle in a similar way as that?

Adam Rogal: Yeah, great question. So one, always just go to where the developer is already. So the Codelabs are a good example where we meet them on day one of what they’re looking for. Another good example is just once you have that service catalog, making that a central hub for where other things exist and then back to that service catalog wherever you might want to be, right? So if you have a system that’s referring to a service somewhere, actually sharing a link that, hey, this goes to the service catalog is a way to get there.

The other way that you tend to gain a bigger audience is just always to keep things fresh, to take a fresh perspective and add new things. So up to about the halfway point of the existence of DevConsole, we hadn’t really thought much about the team aspect or the top-down view. Everything was sort of grouped by services and domains within it. So we created a plugin that was called the Team Health Plugin, and this Team Health Plugin gave an EM or any IC, just a top-down view on their team. We added things like interruptions for after hours on call, added things like pull request load, so seeing how many pull requests are people reviewing all the time.

We added things also like PTO, so that as the EM, I can go up to somebody and say, “Hey, you’ve been working for six months without a vacation day, please take one.” The burnout is real and it is really hard to capture that and support load as well. So how many questions are people asking in Slack channels and how can I best support them? So creating tools that allow engineers to maybe automate or improve a different role that they have at the company, not just writing code, but that operations role was one way for us to inch into other areas and improve in other areas.

The other part is at a certain point you get enough momentum that it just starts popping up everywhere. I saw a big shift one time and it’s just kind of gone from there, but every time we have an eng all hands at the company or demos at the company, it’s a lot of times it’s people writing a plugin in DevConsole and showing it off in DevConsole. I think if you reach that echelon, it becomes a flywheel effect where it’s sort of marketing itself as you go. And I think that’s not easy to get to. It takes a long time to get there. You can’t squander that once you get that either, right? You have to keep it being fresh and a great place to go to. But really this is what that evangelism is for.

My background before going into traditional software engineering was developer relations. I started off at that at Google, and I just always like to help engineers and do speaking events or support or developer conferences, etc. So I think I have the pleasure or the burden of understanding what support and evangelism looks like, and it just requires ongoing investment into that. A developer program really requires you to be talking to developers at all times and to really get the word out, evangelize on what you want to do, and also be an advocate for them internally for where you can provide avenues. So listening to their voice and then improving things for the rest of the backend as well for them. So being a channel for them.

So I think eventually you get to this flywheel and it really improves over time, but it goes back to just laying the right path along the road to get there. And the best way to do that is to listen to the quiet parts of what your customers are telling you. So your customer might tell you something like, it’s very hard to do this, and you might think, okay, I’m going to make it easy to do that, or you take a step back and say, why are they doing that in the first place? And then building a solution for that. And this is the whole thing of a faster horse instead of a car type situation. I think there’s a little nuance there that you don’t want to just tell customers what they’re actually asking for. I think that’s a little bit too customersplaining on that front, but it is something to say that you shouldn’t just only build what your customer’s asking for. You should build more holistic solutions for them.

Abi Noda: Well, if I’m a listener, I’m thinking, man, Adam’s a unicorn. He built this thing himself. He’s good at articulating the needs of developers and he has a background in evangelism. So if you’re a listener and you’ve built something and no one’s coming to you yet, right, you’re not getting inbound support requests or interest, what did you do to catapult things at the beginning? I mean, were you doing live demos and all hands? Were you doing a lot of emojis in Slack? How did you build that initial flywheel to even get engagement?

Adam Rogal: And I’m glad we’re not live. You would see me blushing right now on everything. You have to be a broken record. You just have to mention this thing. You have to mention your passion as many times as you can to the point… You kind of know that you’re doing it enough when people roll their eyes jokingly at you or sort of poke fun at you because of how often you’re mentioning this type of thing. And you have to take pride in it. I take a lot of pride in what we’ve built, and I’m so happy for the engineers who feel the success in it as well. It’s one of many examples of projects that we’ve done here that I feel that sense of pride for, right? I don’t want my org to feel like I have a favorite. They already have that complex anyways with how often I mention this.

But anytime that you create something that you have passion in, people need to see that passion. So if you’re creating a brand new framework for service architecture and your passion is reducing boilerplate for engineers, build it, talk about it, demo it. You have to be that broken record. Now, I think there’s something to be said about also understanding the audience reception of it. If you’re being a broken record and your audience is not being receptive, you’re not getting positive feedback, yeah, you probably need to change your strategy a little bit, right? Because you start to tend towards alienating your customers. Imagine pushing for a product all the time when the house is on fire, right? It’s like, I understand it. I think you’re really passionate about your product that you’re talking about, but I just spent the last 14 hours trying to debug an outage and I can’t even look at this right now.

So it’s really around having empathy with your engineers and knowing the right place and right time to pitch these things and to talk about these things. You could frame it sometimes of, “hey, this is meant to make outages easier”, but definitely always come from the fact that there’s way too much stress going around. Everyone’s doing way too much and trying to get things done, and you really need to find the right avenue and the right voice for doing a lot of these things.

But yeah, you mentioned there’s some hacks, right? There’s all-hands, hackathons, demos. One thing to an eng leader though is you don’t want to be the one in the spotlight always. I think having really passionate ICs that can go out there and spread the word and sort of represent the developer always connects better with other engineers at the company. And then likewise, the ICs who want to do this, find that that leader to partner with because they are going to be doing it in conversations and Slack rooms and meetings that you are not necessarily a part of. So this is also why you need to form that team to push for this.

I want to make sure that everyone knows this was not me, this was my team that did all this work. They were the ones who sacrificed and built a lot and really dug in here. And I am so thankful for the folks that contributed to this and there’s no way this would’ve existed without their contributions. And it’s fine to be poked fun at as like I’m the DevConsole guy, but what they are, and they take immense pride in this and they do a great job of talking to their customers and working with them. And that pushes me to serve them well, to make sure that they get the proper avenues, the proper alignment, the proper funding to make this happen. I definitely, I am not the star talent on the team. They are. They are the ones who are making it happen. And yeah, it’s great to see ICs who are passionate in general about these areas.

Abi Noda: On the topic of talking to the customer and developers and ICs, we were talking before the show how it can be really challenging to interview developers. It can be challenging to get them to be willing to give up their time. You mentioned you had some ways that you found work well for overcoming that challenge. What advice would you share?

Adam Rogal: Yeah, it’s surprising really because a lot of engineers have feedback.

Abi Noda: Opinions.

Adam Rogal: Yeah, they have opinions. Who would’ve thought? An engineer having opinions? And the loudest voice typically dominates the conversation a lot of times. Sometimes the loudest voice is correct, sometimes the loudest voice is just a loudest voice. And when you send out surveys, which are helpful, as your listeners know, a lot of times there’s sort of a selection bias that already goes to people who just open up surveys. There’s a lot of people who just don’t even open up the email and take a look at it. We have internal tracking on emails and we could see the open rate on a lot of our emails and it’s not great. So we know that email is more of a archival first go at things, but it really requires us to get to where the people are.

If you’re a Slack company, probably means Slack, right? Probably means bugging people on Slack. If you’re a in-office company, probably means walking up to them at their desk. Obviously if they have their headphones on and they have a frown on their face as they’re looking at code, probably not the best thing to stop what they’re doing. But hallway conversations always really help as well. But you have to take initiative, so you have to find the right way for every single person that you want to interview to have them donate their time, and you have to realize that they are donating their time to you. They are giving you the gift of feedback because it would otherwise be time that they would be spent working on an important deadline or working on an important tool that they have to do.

So It’s always good to have that level of empathy and have that level of connection with those engineers. It’s also really important why you have to have that network effect. You have to get out there and work with the engineer outside of asking them for feedback so that when you do ask for feedback, there’s already a level of trust there of what are they going to use the feedback for. The impersonal survey I think can go only so far, though the interview itself has so much other benefits besides just the collection of feedback. It shows that you care about the engineer, it shows that you are listening to the engineer. It shows that while you’re having that conversation, you’re engaged and not just looking at Reddit while they’re talking to you. That amount of contact and connection I think spreads because they get to tell the other people that they’re working with of the good positive engagement that you had. And that tends to really culture shift impressions of folks.

And in terms of just hacking the conversation, I think trying to keep it as lightweight as possible, trying to take initiative to book, sometimes the calendar invite with folks. I think you have to be a little persistent but not annoying. And then when you take the feedback, you have to come back to them and say how you use the feedback and make them feel that they were constructive here. And then you probably have secured a line of feedback for the future. You can create an insider’s group, if you will. If you just take their feedback and never get back to them again, then I think that you’ve lost an avenue because they’re going to question the next time you ask for feedback on them. So you have to cherish the feedback that they give you, and you actually have to treat it as step one in the closed loop to get back to them.

Abi Noda: Well, so you bootstrapped this developer portal at the start, and what I’ve been wondering, and I’m sure listeners are wondering is did you ever get some real funding? How have you kept this going for three years? So share with listeners, you mentioned to me an interesting concept. I don’t know much about it, decentralized versus decentralized funding. How are you keeping this going?

Adam Rogal: Great question. So at a certain point, yes, you can only angel invest it as much as you can. I am very lucky to have really great bosses that understand the value of platforms and they understand the value of infrastructure. Shout out to Matan and Ryan for all their support up to this point, not just on DevConsole, but on developer platform as a whole. Because again, you, and as your listeners know, it’s a very, very tough field to feel like you can justify your projects. Try to justify building out a monorepo, right? There’s a lot of negativity and a lot of pushback on a lot of projects that have long-term value, but short-term disruption.

So what you can kind of do is over time, as you build out your roadmap and collect the feedback from individuals, you can start to build a backlog of things that customers want. And if you build out that backlog of what customers want, it allows you to really have sort of a menu that you can present to your leaders or your customers and say, “Look, with the meager amount of headcount we have here, we can only do these three things. But if we were able to get two more heads, we can do this and that and this.” And to every eng leader who happens to manage a group of engineers, you’re going to have to be creative. You’re going to have to sort of borrow resources across your teams, and you should always be doing that anyways for any kind of spike on projects that, maybe not like DevConsole, but just anything that’s going to require a boost of engineering.

But going to your leaders and showing the wins that you’ve had so far, getting them to actually value also the tool, so making it a place for your leaders as well, really drives home that their investment in your team or your area will have a return on it. It’s going to be very hard to tie it to actual dollars, time saved or cost efficiency, et cetera. It’s nearly impossible to tie it to velocity, but you can tie it to satisfaction, you can tie it to that backlog of feature requests that people have. And leaders want to invest in winners. They want to invest in the things that are winning. So showing that success rate and showing that resources won’t be squandered is definitely a way of moving forward and getting these things done.

And then you have to rely on your community as well. So anytime you create one of these tools that’s both a tool and a platform, you need contributions from within the company. So you need to sort of garner that almost open source mentality that engineers can come in and create a tool. And you can absolutely not have a gatekeeping culture which pushes or alienates people away. It might not be the best React code ever, but there is no best React code ever, right? That’s not the point. If they can come in and they can build a tool, encourage them, applaud them, help them try to figure out ways of making it better and improving the guardrails in place. But anytime you build out that community, you have to encourage it and you have to be thankful for it and those contributions.

And then you get to the point that Spotify got to, which I think they have 300 plugins or so, and 285 or something are contributors from around the company, not just centrally funded. And forming that community has a lot more longevity than just having a central team that is just burnt out, trying to jump from feature to feature with the constant thrashing that might happen that comes along with that.

Abi Noda: Last question for you, Adam. Reflecting back on all this, I mean, this is going to be inspiring to listeners and many will envy the success you’ve had, but when you reflect back, what are some things you would’ve done differently?

Adam Rogal: I think there are many points along any project where you feel that perhaps you took a misstep, right? And perhaps you invested in the wrong area at the wrong time. And looking back on the project, I think that we took on a lot of technical debt in a lot of areas because we wanted to get features out there a lot faster. And that technical debt built up over time to the point that a good portion of the team was focused on just paying down that technical debt. And most of my contributions in the last four to five months have just been paying off the technical debt that I feel that I put on the ledger. So upgrading the repo to beyond Node 14, I know scary, upgrading to React 17, basically things that I put off because I said we need to build out some features instead.

And as you’re building out these tools, I think just being really cognizant of that balance of technical debt versus feature development on these types of things, it’s a tough balance, but it’s one that you have to pay down because eventually this thing becomes so important that it becomes critical to your systems. And if it’s not 99.999 on all the time, it becomes the weakest link. And you have to find the right folks that can have the right level of quality to make it a quality product. And you can’t sacrifice that for feature development as you go forward with these things, which is true about every product out there. But I think when you have a dev portal, engineers are not an audience that you want to anger. So that’s this sort of doubly true when you have these type of products.

I think the other part is perhaps a little bit more, I think kind of meta, which is as you’re building out dev portals and thinking about the marketing of it and the utility of it and how things get used, a lot of times you don’t want to get too repetitive. And I think one of the things that we’ve done is you pigeonhole yourself into certain areas with your dev portals, like people say, “Oh, that’s just for this, that’s just for that.” And if you get kind of typecast as that, it’s kind of hard to break away from that into other areas.

So one thing that we’re trying to do that we’ve moved to is being more self-serve within the dev portal now. So you can come in and create a brand new service if you want to or create Kafka topics. So we’re trying to ensure that dev portal, or the dev portal we have, DevConsole, it represents the current truth, right? It’s not just a sort of shadow of what exists in art topology. It actually affects the topology itself. It’s real-time. It feels like it’s the reality of what you have in front of you. So it changes that perception.

And then we’re heavily investing also just in AI as well, because one thing that your dev port will turn into which we have to undo, is it’s just a cacophony of data and information. Across every single place that you look, there’s a graph or there’s place that you can look and it’s just too difficult to surface what you should be looking at right now. So we’re looking into having more anomaly-based presentation of places you should look. If you’re sort of a certain persona, hey, these are SOOs that are burning. Go take a look. These are services that haven’t been touched in a while. These are new RFCs that recently have been published, sort of presenting this data to the user.

And then the second part is also thinking about substituting some of the other avenues that the company already with DevConsole. So support is a big thing we have here. We have a ton of Slack channels where we have people just ask support and it gets overwhelming. And we’ve created an AI support bot that uses LLMs and trains on a FAQ as well as Confluence documentation in our Codelabs and can respond to questions and give answers with a certain percentage confidence based on those. One thing we want to do is move that into DevConsole, right? So instead of having all these ask channels, you now have a single place in DevConsole where you ask the question you’re trying to answer, and then from there, you can get the support or direction to where you need to go. But now it sort of changes the flow and changes the entire culture shift, and it’s a way of keeping DevConsole fresh and new and providing that new functionality in those new areas that we’re trying to explore into.

Abi Noda: Well, Adam, this has been a great conversation. I’ve loved hearing about your journey and this will be a must-listen for anyone who’s at those early stages of dreaming and hoping or trying to stand up the developer portal. Thanks so much for your time today.

Adam Rogal: Yeah, thanks a lot for having me here. And I just want to give a special shout-out again to all the folks along the way that have contributed to this. It’s a team effort and it’s a team product that we’ve done and all the engineers that have contributed to it, all the customers that gave feedback on it, it’s been a really, really fun journey. We could not be there without everyone’s contributions. So, thanks to all those who contributed to it, I am forever grateful of their trust and their contributions to this area. And as I said, always happy to chat again and love talking about these areas, and hope to be on again the next time.