Erin:
Yes. Okay. Well, maybe while we’re waiting for Colin, I’ll just kind of explain so everyone can be aware. This is a little bit more of a unique format. So the way that we’re going to do this, I’m going to be reading a series of statements and our panelists are quite literally going to give us a thumbs up, a thumbs down, kind of where they’re at on this topic, just as a gut reaction, and then we’ll get started.
Okay. So our hope is that you all will walk away with some new perspectives. Hopefully that felt pretty on theme for how the whole day will today. Just like the last session with Uma, there’s not going to be any Q&A during this time, but still do the same thing. Feel free to put questions and again, you’ll be able to talk with our panelists during our reception immediately following this.
So let’s get right into it. So our first statement, an AI first SDLC means fewer engineers. Rafe, will you start us off?
Erin:
Give us a take.
Rafe Colburn:
Sure. So I think poorly to tell, I think probably it’s true that a job is a bundle of tasks and for sure AI is a clean substitute for some of those tasks and people might not be doing those things anymore. On the other hand, the demand for software does not seem to be going away. It seems to be going up. The price of building software is going to go down. And so I don’t think that less engineers is likely to be a scenario in the near future. I bet against it.
Brian Houck:
I think if I could just add something to that. I definitely agree, it’s not about fewer engineers, it’s about doing more. But one nuance to be maybe a little controversial is like, in the future, what do we even define as a software engineer? And so the bundle of tasks that we think of today that make up a software engineer, Abby and I talked about it in the opening where it’s writing code is a core part of the identity of a software engineer. And so the people who do that, the things that we think make up software engineers, that may go down, but the total number of makers of builders, like I said-
Rafe Colburn:
Yeah, yeah, no. Yeah. we’re on the same page.
Erin:
Any other takes?
Collin Green:
I think I could add that you’ll be able to do the same amount of stuff with fewer people probably in the future, but that doesn’t mean, as Rare said, that the demand will go down. And I think actually small companies that want to accomplish something will be able to get sort of over that barrier to entry faster. So may actually increase demand for engineering skills.
Erin:
We’re agreeing too much. We’re going to have to make it harder on you all.
Rafe Colburn:
I know.
Erin:
Okay. Next statement. AI is currently creating technical debt faster than it is helping us refactor it. Give us your quick reactions. Eirini, can we hear from you?
Eirini Kalliamvakou:
Well, because you said currently. So I do think that at the moment, things with co-generation are moving at such a pace where the rest of the system hasn’t necessarily evolved enough to match it. So we’re definitely seeing that. I think we’re also seeing the anticipation of technical debt kind of throttling how much developers are using agents. This is certainly something that we saw at GitHub at some point because they’re trying to balance the risk of something with the velocity of something. And it’s kind of like mental math that needs to happen in that moment.
So I do think that there is a trend there, but there are also ways, I think we heard, I think it was the Airbnb talk, there are also ways to have workflows in place. That’s one solution that we’re working with at GitHub where you can try to manage against the accumulation of debt by having agents that are event driven or run on a schedule that try to do a lot of maintenance and health and hygiene for code bases. It’s not a perfect solution and I’m sure eventually things are going to evolve differently, but for now it is something that is working. But yes, the accumulation of technical debt is certainly a big issue.
Jesse Adametz:
I just think it’s too definitive of a statement. I think we look at our engineers in quartiles and our highest performing engineers who are our highest performing AI engineers, we’ll hear this a couple times because it’s on my mind that AI is an amplifier. We’ve said that a few times today. So if you’ve got high performing engineers or if you don’t, it’s kind of garbage in, garbage out. I may get quoted on that now, but our highest performing engineers weren’t putting garbage into the system and there’s still not.
Rafe Colburn:
I was just going to jump in and say, I’ve worked at the same place for 14 years at Etsy and I would say proportionally, are we creating more tech debt now than we were five years ago, 10 years ago? No. The old code is still there. And so of course we’re creating a greater volume of code, but I’m not sure a greater proportion of it is tech debt. And I think automated code reviews I think are helping a bunch and you can ask a coding agent to look at the patterns in the code. There are a lot of things you can do to write better code, I think, with a coding agent. And so just the greater volume of code for sure probably brings in absolute terms more tech debt, but I don’t think it’s an accelerator for tech debt at a greater rate than it’s an accelerator for anything else.
Brian Houck:
I am envious of all of you because my experience is different. I think that you have a couple of issues. You get the outcomes that you’re optimized for. And how many talks that we’ve had today have been about, oh, increased PR velocity, increased PR velocity. We are optimizing for speed, not necessarily cleanliness. And I think with that, you do sort of get, in absolute terms, maybe not in relative terms, greater tech debt.
But a concept that we’ve also talked about a little bit today is this notion of cognitive debt. And I think that is a form of technical debt, whereas if we understand our systems less, then they are less resilient to us being able to modify them and act on them in certain ways. And yes, maybe we’ll be able to kick that can down the road. It’s like, oh, the agents will take care of all of that.
And so I think eventually the statement will be true, but in the short term, I actually say that tech debt is in fact going up, I think, relative to our PR velocity.
Collin Green:
There’s an interesting bit of what you just said that technical debt being incurred is not strictly an engineering decision, right?
Brian Houck:
Yes.
Collin Green:
It’s a business decision about why do we need to go fast now? What do we need to achieve? What are we trading offer later? So if businesses still have the same overall tolerance for that risk, and if they have the same market pressures to incur the debt, to get where they’re going faster so that they can be first or be better or whatever, we’ll incur the same total amount, I think. It may be a little more volatile-
Collin Green:
We’ll incur the same total amount, I think. It may be a little more volatile, especially right now, because we can go faster. And if we view development and agentic development as zero cost or low cost, that increases the risk that will not make good decisions, prudent decisions about technical debt.
Eirini Kalliamvakou:
The choice of what even are you building becomes so important.
Erin:
Okay. In five years, more than 50% of code in most organizations will be written by AI. Gut reactions. Colin, let’s start with you.
Collin Green:
I’m interpreting that as new code, not 50% of a code base. So I think that’s important because the answer might be different otherwise, but I think more than 50% of new code, maybe even quite sooner than five years. I think it’s worth thinking about whether that’s code that would have been written by humans otherwise versus there’s just a larger volume and now the agent generated code is starting to take over a bit. I also think it’s worth talking about how much of that code will be rapidly rewritten after it’s written.
Erin:
Was there a dissenter?
Eirini Kalliamvakou:
Yes.
Jesse Adametz:
I was interpreting the question the other way. I think because I think we’ve seen those headlines too. It’s like, well, all the code now. And it’s like there’s no way. There are too many companies that are too old that have too much legacy software, we’re not rewriting all of it. There’s no advantage to rewriting all of it. So yeah, if you interpret it that way, I think those claims are wild.
Rafe Colburn:
Oh yeah, it’s got to be new code.
Erin:
All right. The future of software engineering is more about managing agents rather than about writing code.
Eirini Kalliamvakou:
I don’t know.
Erin:
Brian, let’s start with you.
Brian Houck:
I think in this idealized world that is true. As it turns out, as humans, we aren’t actually great at multitasking. And I think that we don’t, at least in the short term, have the skillset necessary to do this. I think what we’re finding is people who had prior management experience, as an example, who are used to delegation and used to being able to context switch between lots of different things, they are going to be very successful at this. But we’ve actually seen with some of our internal research at Microsoft that even when under time pressure, developers will often… Even experienced developers will revert back to sort of sequential agentic workflows because our brains just don’t really work very well that way. And I think it’s going to be a really tough learning curve and it’s a different set of skills that we’re going to have to be hiring for that we don’t necessarily have today.
Rafe Colburn:
Yeah. I think… I put my thumb up and even that is getting… And I think, so I guess maybe the plural is the interesting part of it, right? Will you manage an agent? But what’s really funny is that even if you’re using Claude Code of today, you’re managing multiple agents. It’s just doing it for you. It spins off sub-agents and it does things for you and it comes back and reports back and so, I think that’s another thing that’s a little bit getting abstracted away. I think probably this idea now where you have people in their hobby is like building a crazy harness where they’re managing 15 agents at the same time and doing all this stuff, I don’t think that’s the future.
To the extent that you need to do it, AI will help you do that too. You’re not going to have to do that. I think will a really large chunk of your work be mediated by one agent at minimum? Yes, I think that’s true, but I totally agree. Agent orchestration is not a thing for humans to do really as an all day job.
Jesse Adametz:
I think this question goes back to a handful of topics that I think have been coming up throughout the day. So one, as a director, I have a propensity to use AI in the way that we’re suggesting. I’m managing agents and that’s just where my brain goes. But we come back to the identity of a software engineer. The coders, we’re talking about changing job descriptions potentially. And it’s almost like maybe in the future we find that, that role, how we have it today, is actually useful for this type of work and shaping it differently is this. But I think maybe it goes back to the other topic it touches for me is the whole agent experience versus people. It turns out, again, the right thing here is what’s good for the people.
We can’t just assume that, okay, everybody’s now just going to be a manager. There are people that inherently don’t want to be, that are not good at it, don’t want to be good at it. And so, I think, again, at least maybe it’s too early to say, but the blanket of this is what it’ll look like you’ll all have to manage, I think that’s crazy. It doesn’t mean that you won’t have to work on your communication skills, which has always been-
Rafe Colburn:
Always been true.
Jesse Adametz:
The thing.
Rafe Colburn:
I think there’s a lot have always been true here because I think even when I was an IC engineer, I wasn’t managing agents, but would I run a slow running task in one window and a fast task that I had to think about and another one 100%. Now do you run two agents in different tabs? Say this one’s going to take 15 minutes going through the code base and writing some research and this is when I’m in active mode, 100%. And so I think that people have always juggled. I think they’re still going to juggle. The management word is weird. I don’t think it’s managing agents. You’re just communicating through a different kind of UX.
Eirini Kalliamvakou:
I don’t know. I was going to say something to that effect. So the managing, I guess I’m taking it a different way. I’m reading it to management as supervision, and that seems like not as substantial a role as we see it be in practice. So, I’ve been interviewing developers who are really advanced with their AI use for some time now. And over time, they describe their role differently to the point that the ones that are at the frontier, they do see that what they’re doing is orchestrating agents, but not from the multitasking aspect and not from the babysitting aspect, they’re looking at all the work that they have to do to set up an agent for success so that it can do good implementation. So, everything that has to do with defining intent, which probably has to do with the communication skills, yes. So yes, defining intent, setting constraints, setting guardrails, giving context, all of those things are really important.
And then verification after implementation happens. And that’s real engineering work, it’s not just sitting back and checking out the agents, right? It’s real engineering work that is happening at a different level of abstraction, but still engineering, different flavor, different abstraction, but yeah. So that’s what it looks like things are headed in terms of the professional very much. If that’s what we meant with the sentence, that’s how I interpreted in my head and that’s definitely a thumbs up for me.
Collin Green:
Definitely micromanaging agents is failure mode, right? We don’t want to see that because then you run the task switching in the bottleneck stuff.
Eirini Kalliamvakou:
Yeah.
Erin:
That is kind of what Stuart in his talk about how he’s hearing feedback from some of the engineers at Netflix. It is kind of how he characterized it as them being a little bit needy. So it does sound like it could go down that route.
Eirini Kalliamvakou:
Yeah.
Erin:
Okay. Leaders need to mandate AI usage as a way to make sure that adoption is moving along as it should. We all strongly disagree. Jesse, let’s start with you.
Jesse Adametz:
It’s just not what we’ve seen. I think we did a little bit of enablement, we didn’t have a top down mandate. We did a little bit of enablement because there’s no reason to reinvent all the same wheels. So we enabled, here’s how you should install it and [inaudible 00:41:29] defaults and then after that, it took off.
Rafe Colburn:
I do think, maybe the thing I would comment on this one is the people who don’t come to this conference who are in the rest of the parts of the company definitely think we need to mandate AI usage. By and large, not all people, not my boss, she’s wonderful, but it’s a very common sentiment and people are really afraid of falling behind in the AI arms race and they feel like if you don’t really pressure people to do it, they’re not going to do it. And I think one, we’re seeing a lot of crazy good hearts lost stuff with that where people are just optimizing token maxing or whatever people are talking about all the time. And if that’s what you incentivize, that’s what people will do. And I think the other part is you get the shallow, not meaningful adoption. And what I really think is, you can’t run away from the technology adoption cycle where you have the innovators and the early adopters or the majority, late majority.
And I think how quickly can you accelerate people meaningfully on that curve when it’s everywhere? If someone has said we’re going to have the DX conference a year or two ago and they said every single talk is going to be about AI adoption, AI transformation, people would have been like, no way, but is there anything else to talk about? Not in any room I’m ever in anymore and so, I think like, oh, and also we need to tell software engineers, AI is going to be part of their job. It just already feels like you’re fighting last year’s battle this year and people just need to move on to removing the friction and making it easy to create value with it rather than trying to push people on it.
Brian Houck:
Yeah. So you touched on something there that I think is really important is like we have selection bias in this room, right? We collectively have a fairly sophisticated take on developer experience. And so I’m actually running a study right now where I asked about 600 engineers and engineering managers, which metrics do you think are suitable for measuring individual level performance? And so, do you think lines of code is a good measure of individual performance? No, of course not. Do you think feature velocity is? Well, yeah, that seems pretty reasonable.
The place where engineers and managers disagree most strongly is on AI usage. The majority of engineering managers think that AI usage is a reasonable performance metric for individual level performance. And that is, I think, a myth that we need to dispel in our organizations because it is the way that people view the world. It’s like, no, no, no. Account of activity is interesting to understand output patterns, but that’s why. I love looking at tool usage and things of that nature, but only as a way to understand what helps get us to the outcomes we want.
Erin:
What if we reframe this statement and said, most organizations have an unspoken mandate that AI adoption is required for your job?
Brian Houck:
I think we are seeing this in reality. Lots of companies, and I’d actually be interested to hear at the reception afterwards if this is your organization as well, but you have usage dashboards and you are tracking usage. And even if you’re aggregating at the team level, it’s like, “Oh, well, my team has 99% usage and your team only has 50% usage, thus we are better.” And I was like, yeah, they will likely have better outcomes, but that should be sort of measured separately.
Jesse Adametz:
Yeah, maybe it goes back to, there was a talk earlier, right? If we’re not being crystal clear about how we’ll measure and how these things will be looked at, then people are going to come up with the answer from their biggest anxieties. So, the question for us definitely comes out, it’s been asked of me multiple times, for example. And so far, I know we’ve heard a couple different takes today that some people are putting it in their career development framework, some people aren’t, et cetera. I don’t think we’re at a place where we’re putting it in anytime soon. That said, it’s a big asterisk of, as always, you’re compared to your peers, and if your peers are in a different place than you are, that’s going to be the conversation, but it doesn’t start with the AI topic. If you can use no AI and keep up.
Rafe Colburn:
Yeah, I totally agree with that. And I think part of the reason why I haven’t felt the need to mandate it is I think it’s completely inevitable. And I think so, why would I need to mandate something that’s inevitable? Maybe it speeds it up a little bit, but I mean, I think probably the first time I did a Q&A when I got back to Etsy last year, someone asked this question, I get it at every single AMA and I said, “I think there’s a point maybe we’re already there, where if you went to a job interview and you said, I won’t use an AI coding tool or an AI coding agent, it will be like saying you’re a programmer who refuses to use a text editor.” And it’s like, “Well, a text editor is an essential tool of the job of a software engineer, maybe less so now.” But it would just sound strange to people. And I think even now, if you went and interviewed for a job today and you said, “I don’t do that.” It’s going to be tough to move you to the next round.
Jesse Adametz:
That’s definitely the harder part of the conversation that I think we’re not talking about a lot. Maybe it’s behind closed doors kind of things, but I’ve seen examples from friends who are managers elsewhere and they’re having the conversations with the folks that are against it for reasons, like against using AI. And the best response so far is like, “Are you willing to career change?” I don’t know.
Rafe Colburn:
And just disinterest is probably not acceptable. I heard about a lawyer and they interviewed for a job on a legal team and they were like, “Yeah, I’m just completely disinterested in AI.” It’s like, “Oh, you’re probably not a good fit for the job.” As you can imagine for software engineers, for sure.
Brian Houck:
Now, one thing that I do think this whole conversation about AI usage directly as a performance metric highlights is, as an industry, we’ve always sucked at measuring developer experience. And AI is just continuing to bring that to the forefront. It’s like, “Oh, well, this is an easy thing to measure. And so I’ll measure this activity.” And this is just sort of the merry-go-round of that conversation, I think.
Erin:
Okay. We’ll need far fewer junior engineers in five years. Okay, we all disagree. Rafe, can we hear from you first?
Rafe Colburn:
Yeah, sure. I mean, it goes to the same thing as we need far fewer engineers period, I just don’t think that’s the case. I do think beyond that… Again, I like to look back at the past and see what the patterns are rather than always focusing on predicting the future and I think, I can remember when we went through the transition of to the web and away from client/server applications, that’s how old I am. And people who were web native and used the web for fun and knew these things at home could just code circles around the people who were PowerBuilder developers or whatever else people used back then. And so I think the idea that people who this is native to them are not going to have this unique value to provide is crazy. We’re going to have an intern class this summer.
I’m going to be really excited to see the fresh perspective that they bring to our team. And then I think the other thing, far fewer is an interesting one, maybe this is a little bit of an editorial, but really since the pandemic, it’s been a bit tough times, lots of companies have done layoffs. I know there’s been a lot of big public statements about hiring junior engineers, but it feels like junior engineering hiring has dried up anyway. And so it’s not like we’re hiring huge amounts of them now. I’m hoping there’s a resurgence. I mean, I want this industry to have a real future with young people in it. And so, I know so many companies that are like, “We just don’t really hire junior engineers anymore.” And it has nothing to do with AI. So I’m hoping the trend goes the opposite direction.
Collin Green:
I hope we’re not also not so shortsighted that we don’t think we don’t need senior engineers because they don’t just spring fully formed from universities.
Jesse Adametz:
You just graduate, level three.
Collin Green:
And similarly, we don’t need fewer people who just understand engineering from an educational point of view. So it’s just, yeah, you might not need them to do your project today, but if you want long-term value sustainability in your engineering population, you need them and you need to grow them properly and it’s shortsighted to neglect the pipeline, I think.
Rafe Colburn:
No, totally. And I mean, we’ve been really lucky to have people who stay at Etsy for a really long time. And some of the best senior engineers we have started as junior engineers. I’m sure you’ve been at Microsoft 20 years. You know that’s true as well and so, you need to grow people through that whole cycle. It’s really, really valuable.
Brian Houck:
Yeah. I mean, the question is not, do we need fewer junior engineers in five years is do we think we need fewer experienced engineers in 10 years? And we’ve already answered that question.
Rafe Colburn:
Yep.
Erin:
Okay. The bottleneck of software delivery is no longer writing code, it’s now reviewing code.
Okay. Finally, we’re not feeling the exact same. Colin, let’s start with you.
Collin Green:
All right, I think one of the first things that I heard come out of Brian’s mouth this morning was that developers only spend about 14% of their time writing code. So I think the idea that the bottleneck is writing code in the first place is maybe not the right place to start from. I think more broadly, we also heard a fair bit of discussion today about the other parts of the SDLC being essential here to think about productivity. Implementation is probably the place where we’ve made the most progress. So I think the bottlenecks are going to be decision making, they’re going to be prioritization, they’re going to be product design. They’re going to be support and operation, those other things.
Jesse Adametz:
Yeah. Again, amplifier, right? So it’s whatever your organization’s bottleneck used to be, it still is.
Jesse Adametz:
Bottleneck used to be, it still is. Code was never the thing. Fortunately, our data actually showed us this, so it’s a good place for us to start. Turns out we’ve had deep work as an issue for quite a long time. It’s still an issue. I don’t know. Something about the conversation has changed though again. I forget if I mentioned this earlier, but it’s like the, “Okay, well, we need the agents to X, Y, Z. What are we doing about agent experience?” And I was like, “Well, the good news is we know what to do. It’s what we should have been doing the whole time.” And so make a better developer experience, agents would get better, et cetera. But that comes down to decision making, meetings, coordination tax, all that stuff. We’ve known not to do for a long time.
Brian Houck:
So to be just really clear, reviewing code is still a bottleneck, right? It’s just like no one wants to have to wait for a couple of days in order to get your merge in. But I definitely agree that even if it is increasing as a bottleneck, as we are producing more code, and particularly if we don’t have more sophisticated ways to do risk assessments and use that to tailor the depth of review, reviewing is an increasing burden on us. But I know the earlier talk from Uber was talking about feature velocity. And when I go and measure the feature velocity at Microsoft, I can go and look at features that took us two or more years to get from our planning phase out into the hands of customers. And so going from two days to three days on the review process, that’s not the long pole there. It is the planning. It is sort of the prioritization. It’s like all of the things that were always a problem.
Rafe Colburn:
Yeah. I think maybe thinking about this, the perceived bottleneck of code review is what’s really interesting to me. And when it took me two days, let’s say, to write a PR, to generate a PR and to go check it into a repository that requires a code review. And I didn’t know whether someone gets flagged automatically or I need to go find out if I need to select somebody and say, “Hey, can you review my code?” And then are they actually going to do it today? That is all super annoying, but it’s like, boy, I had to spend a lot of time on the code anyway. It doesn’t feel quite as bad if writing something meaningful takes me 10 minutes.
And then there’s not only a delay to get my code review, but I have to go to the person and say, “Will you please do me this favor and review my code? Why can I not just use my agent and not put up with this stuff?” And so I think what I’m seeing from people is the interruptions of these human processes, unless they provide real value that is clear, are just so grading. And I think that is why we hear so much about code review. It’s not just the code review volume, but it’s also like, it was already kind of the worst part of your day to have to get a code review and now it’s really the worst part of your day.
Jesse Adametz:
The perception change I think is the thing. For us, it was like our highest quartile AI developers, they took a major hit in the perception of code turnaround or code review times of something like 14 points or something. But compared to … And that was compared to the first quartile, but their median merge time, the high performers, decreased by multiple, multiple hours. So it’s like pure perception.
Rafe Colburn:
This feels more expensive.
Brian Houck:
Yeah. It’s like, okay, let’s get personal for a minute. The thing that my significant other hates most about me is when we are out at dinner and the check comes and I sign it, once I have signed that check, I am ready to go. And it’s the exact same. There’s something just psychological about it where it’s just like, “No, I’m ready to be finished now until I want to be finished.” So it was like every second you wait is just like, there’s this psychological toll.
Rafe Colburn:
Yeah, 100%.
Brian Houck:
It just feels so much bigger.
Rafe Colburn:
Yeah. I mean, you’re with your partner. You can just go home and watch TV. Why are you still in the restaurant?
Brian Houck:
Yeah, right? Let’s get out of here. I’m ready to be done here. Move on to the next thing.
Erin:
Okay. We have two statements left. The only thing holding engineers back on AI adoption is unnecessary worry about risk. Okay. Eirini, let’s start with you.
Eirini Kalliamvakou:
I don’t know about the unnecessary part, I guess. So I kind of hinted at that a little bit earlier when I was saying how the mental math that our developers were doing, trying to figure out how much to use agents and how the maximum velocity was not necessarily the optimum when something was high risk.
So if I take that statement, this is something that I have seen in interviews, in surveys. It comes up again and again, where there’s something holding people back and I think it has to do, or that’s how we see it in interviews come up is the accountability. The engineers are feeling very accountable for what ships. Perhaps this is something that would change in the future, I don’t know, but today they feel very much accountable.
So if something is high risk, it’s hard to undo. It also has a lot of review burden that sits on them. Then that is something that they pull the brakes a little bit. They do throttle how much they’re going to leverage the agentic rhythm of work and the velocity that comes with that because there is the accountability kind of barrier there. And that’s a very human behavior.
And I think that there’s probably down the line going to be some AI mitigations to that, but the human behavior is hard to get over. So I don’t think that their risk is like anybody would consider it unnecessary at the point of time that we are at. And if they do consider that there is risk, it changes their behavior of how much, not in general they’re going to adopt AI, but how fast they’re going to go with it and how much they’re going to leverage it. It seems like it comes into their decision making.
Brian Houck:
And like only is sort of an important thing.
Eirini Kalliamvakou:
The only.
Brian Houck:
It’s the only reason. It is certainly a big reason. So I published a study at the beginning of last year where I asked developers, “What are your biggest concerns with using AI?” And the number two most frequently cited answer over a fifth of developers say they are concerned about introducing defects, vulnerabilities into their code. And so it is a very real concern that does limit adoption. It is certainly not the only concern that limits it.
Jesse Adametz:
Sorry, was AI increasing that though? Because they were doing that already.
Brian Houck:
It’s good.
Rafe Colburn:
It’s so much faster.
Brian Houck:
Yes, even real or not, it is the fear that it will.
Collin Green:
The bugs they didn’t mean to introduce.
Rafe Colburn:
I mean, I think it’s interesting much like many things are changing. I think what the risks are and how they manifest is just that the landscape has completely shifted. And so on one hand, you can say, “Well, if we had whatever, a butter knife and now you have a chainsaw, the risks are different. The risks that you won’t be able to cut down a tree have gone way, way down. But the risks that you might really make a mess are a lot higher.”
And I think there are probably some reticence. And on both sides, we have a lot of people who are using these tools now who don’t really know how a chainsaw is different than a butter knife. They’re like, “Look, someone just gave me something, cool.” And on the other hand, I think there are engineers who are like, “Yeah, I don’t want to use the chainsaw unless I really understand everything about how it works.” I think that’s-
Collin Green:
I think beyond the risk too, there was a talk by, I think Stewart from Netflix earlier today about just enablement and smoothing the path to adoption. I mean, it’s not just people worrying about risk. It’s not just concerns about quality, helping them choose, helping them configure, helping them get started and they’re not AI specific things. That’s just getting people to adopt a thing, you have to solve those problems.
Jesse Adametz:
Yeah, Stewart had a really good quote, I think about it. It was just like, we just eliminated options because options was another reason to not get started kind of thing.
Erin:
Okay. Our final statement, our final takes. Most AI adoption problems are really culture and management problems, not tooling problems.
Rafe Colburn:
Oh, I’m the man.
Erin:
Oh, hello. Yeah, go ahead, Rafe.
Rafe Colburn:
I mean, I’m in management. It can’t be our fault. No, that’s not true. It can definitely be our fault. I think there are a lot of culture and management problems that make it hard, and I think that other panelists will talk about it, but I do think it’s just a series of working through the impediments. And I think sometimes the impediment turns out to be a tooling problem, or do we say … Or especially I think I’ll cheat, and I’ll say, I think process is also a form of a tool more than it is a culture thing, or even a management thing where there are a lot of processes in a company.
I mean, again, I work at Etsy, we have a 20-year-old code base, we have really tenured engineers, and we have all the stated processes, and then we have all the unstated processes. Maybe that is what culture is. I don’t know. And so it’s like, “Boy, if you don’t jump through the hoops, you can’t do a thing.” And I think that hurts adoption and just the value of it as well. So for sure. And that’s why mid, I think it’s kind of all of the above. And probably in every organization, what is the next blocker that’s really keeping you from getting to where you need to go? Maybe tools, it may be culture or management.
Collin Green:
I think the last question sort of foreshadowed this one a little bit. I think most problems in the end are like human problems with adoption. So, yes, there are tool impediments for sure. And that’s not to be neglected, but helping people have the right incentives, lessening their anxiety, helping them learn, giving them space to learn, those are big barriers.
Rafe Colburn:
100%.
Eirini Kalliamvakou:
And think about how huge the transformation that is happening is, right? How many things did we touch on? We touched on bottlenecks that used to not be bottlenecks are now processes that are not exactly working how we thought they would, metrics that no longer mean the same thing. People are having an identity crisis and for good reason. So there’s so many things happening all at once and the pace of change is brutal. And so it cannot be just a tooling problem, it has all of these other things attached to it. It is culture inside of organizations. It’s like human motivation that comes into the picture as well. And of course, yes, tooling and infrastructure absolutely are things that need to adjust and be smoothed out. But, yeah, it’s all of the above.
Jesse Adametz:
I could imagine there’s a compounding thing going on too where it’s like for folks who haven’t gotten started, it’s harder to get started. Even for myself, I feel like I’m obviously in the conversation every day, but it’s like, I told my VP a couple weeks ago, I was like, “I have to quit to be able to keep up with the industry.” That’s the day job. So, yeah, if folks haven’t gotten started, I can only imagine how much more daunting it’s getting.
Brian Houck:
I mean, even if you have gotten started, everything is moving so quickly. Everything I learned yesterday is now going to be irrelevant tomorrow. And there is this notion of how do we combat long-term burnout and sort of fear of falling behind in all of these things. And those are all sort of very basic human constructs. But at the end of the day, this has always been true in the software engineering industry, in my opinions. It’s like no matter how complex the technology, it is always a human problem at the end of the day, as long as we are still operating all these tools in a human context.
Rafe Colburn:
I do think if I could just reinforce one thing Colin said, it’s that if you’re a leader and you’re not giving people time to learn, you are doing it wrong. That has always been true in this industry. I mean, I can remember asking engineers like, “Are you working as fast as you can?” “I don’t know.” “Are you learning anything from what you’re doing?” “No.” Then you’re working as fast as you can. You’re not taking any time for that.
And I think now more than ever, whether it’s like hiring junior engineers, if you hire junior engineers, give them time to learn, they’re not going to become good senior engineers. If you have senior engineers who have really, really deep expertise, but you’re not giving them time to learn how to adapt that to the AI world, you’re not going to get the value that you want. And so I just think learning is the whole thing almost, and making space for it solves 90% of these problems.
Collin Green:
We persistently see when we survey developers that learning a new technology, learning a new process is one of the barriers to productivity. And that’s not a new thing that’s not about AI, that’s just a thing for engineers. So I think, one, it’s important to set that time aside. And also, I think it demonstrates from a leader that they’re in touch with how the world actually works and they’re not just sort of sitting in a bubble mandating things like, “Oh, this will take time for you to get good at. I understand that here’s the space to do it, because I understand what you do.”
Brian Houck:
Yeah, actually, I can quantify. I can look at the relationship between teams that set aside dedicated time for learning and their long-term coding velocity, which is not necessarily the best metric to use moving forward, but it does show that forward progress doesn’t come at the expense … Learning doesn’t come at the expense of forward progress. It sort of supercharges it.
Eirini Kalliamvakou:
We’re also seeing a difference between doing the learning as a team versus as individuals as well. So in the data that we’re looking at, teams that took two weeks hackathons, they’re not really hackathons, but they decided that they’re going to do everything using agents. Nobody’s allowed to use anything else. This is how we do work for the next two weeks, and they did it as a team, have seen much better results, both in terms of adoption and engagement and actual outcomes than other teams that have dedicated time, but it’s individuals need to do their own. And then they’re sharing and community of practice forming around that. But the other version where teams are doing one, two, three week kind of group learning has been much more effective.
Erin:
I’ve been pleasantly surprised today by how many things came back to the socio part of the sociotechnical world of software engineering. Pleasantly surprised, I would say. I want to say thank you to all of our panelists for taking the time to share all of their takes today. Thank you so much for being here.
Eirini Kalliamvakou:
Thank you. Thank you.
Erin:
So this officially wraps our content today for DX Annual. Our panelists and myself, we’re going to head on out and we’re going to welcome Grayson back to do a final goodbye to close us out before the reception, which will start right after this and be downstairs.
Eirini Kalliamvakou:
Thank you.