Shownotes

Title: Navigating AI's Impact on Software Development and Trust

Discover how AI is transforming software engineering, from speeding up development to redefining trust and skills. Join Anne Currie and Jon Berger as they explore real experiments, strategic implications, and practical steps for leaders in a rapidly evolving tech landscape.

Main Topics:

• The future of AI in software engineering: rapid development and risk management

• Leadership strategies for integrating AI with team dynamics and trust

• Fundamental principles in software and life that AI may reshape

• Practical ways to start using AI safely in development cycles

• The importance of diversified oversight and multiple 'oracles' in AI deployment

In this episode:

• How AI is changing the long-term landscape of human versus machine-built software

• Why understanding your unique business context matters in adopting AI tools

• The significance of experimenting with low-risk AI integrations like code review and testing

• The evolving concept of trust in AI-generated code and processes

• Strategic leadership tips: balancing risks, risks awareness, and fostering innovation

Timestamps: 

00:00 - Welcome and episode overview on AI's influence on software creation

01:22 - Transition from human-only to AI-assisted software engineering

02:48 - The importance of understanding your context as a leader

03:46 - Considerations for teams experimenting with AI in development

04:19 - Risks and opportunities in AI-driven software processes

05:50 - How speed and scale shape business decisions in AI-enabled environments

07:12 - The role of leadership in managing AI adoption and team dynamics

08:37 - Critical thinking about AI's impact on team sizes and productivity

10:30 - Market-driven decisions: layoffs and strategic AI integration

12:15 - Choosing trustworthy sources and multiple perspectives ("oracles")

13:55 - Balancing risk-taking and risk mitigation among teams

15:22 - Protect the future versus change the future in tech strategies

16:02 - Assessing environment and risk: high-stakes vs low-stakes AI applications

17:11 - Supporting existing systems with AI: deployment, testing, and feedback loops

19:22 - Trust, testing, and automation in the software lifecycle

21:23 - The changing nature of AI models and managing their variability

22:26 - The importance of agility and adaptability in a fast-changing AI landscape

23:05 - Valuing diverse team roles, including skeptics and early adopters

27:28 - Embracing AI as a black box: focusing on outcomes rather than process transparency

30:56 - Revisiting core principles of system resilience in an AI world

31:52 - How AI shifts decision-making trade-offs and process scaling

32:24 - Moving forward with trust: aligning business goals with AI capabilities

33:22 - Starting small: low-risk AI applications in testing and review

36:19 - Building confidence with modular, trustable AI-driven processes

39:38 - Actionable strategies for leaders: experiment, assess, and iterate safely

Note: For further insights into managing AI in software development, stay tuned for upcoming episodes on security, testing, and team leadership adaptations.

Transcript

Anne Currie (00:00) hello and welcome to Asynchronous on Unreliable, a new weekly podcast where we discuss the most interesting ideas and concepts in tech. I'm your host, Anne Currie co-author of O'Reilly's Building Green Software.

Cloud Native Attitude and author of the science fiction Panopticon series. And today, Jon Berger and I are still talking about AI and the effect on the software development process in light of episode seven, which was Martin Davidson's experiments with building mission critical software using AI. So, Jon, as well as being my husband and primary technology sounding board, is a veteran engineering leader whose teams

built software at the highest end of reliability and performance for over 30 years. And today I think we both still want to talk about the questions that Martin's experiments have raised in our minds. I particularly want to hear about the questions that they've raised in your mind And then we can get Martin back on to talk about those questions. We saw him in real life for a coffee last week. He's very keen to come back on and talk about what we think and what questions we're asking.

How as a technology leader now, how do you think that this kind of experimentation should be affecting your thinking?

Jon Berger (01:22) Good question. It's very interesting, isn't it?

We're going from the past world where software engineering was a discipline done entirely by humans, to a somewhere in the future world where how long term that is, we don't know. But in the future, software engineering is solved and humans just don't do it anymore unless they particularly enjoy doing it in a sort of craft sense. But I don't know the same market for craft

human-built software will exist as for sort of human-built paintings or other things that are also solved. And between some point in the past and some point in the future, we have this really rapidly changing world.

How do we cope with that? And I think one of the things that Martin's podcast was saying is we might be a little bit further along in that world, in that line between the past and the future than people might be imagining.

So how do we think about that? How would I think about that? The first thing I'd be thinking is, well, what is my context? What's special about me? Because the answers, I mean, most of the questions people should be asking are pretty similar, but the answers will be different for different people in different spaces.

Anne Currie (02:48) obviously your context, which you can talk about very well because you did it for decades, was leading engineering teams. So this is being on the vendor side, not on the enterprise side. This is you were building the software that enterprises would use or individuals would use across the world. networking the software that needed to be

Really, really performant, really, really reliable, really, really secure. Martin Davidson in his, "I've just built this thing using Claude and Codex", one of the examples he gave was a SIP stack, which we talked about last time we spoke, is a classic networking product, which he reckoned he'd been able to do. From your perspective, if Martin was working, had been working on your team, which is not implausible

and he came to you and said, Look, I've been doing all these experiments and this is what I'm finding. What would have run through your mind? What questions would you have asked him?

Jon Berger (03:59) I'd be asking him, I'd be interested to see how he'd done it, because I'd be getting new information here as to what the state of the art is, and I'd want to understand that. I'd be looking at someone who is clearly more advanced in their thinking than most people and saying, Well, what do you think we should be doing here?

and I'd be examining the risks, you know, and that's something that's going to be at the forefront of anyone's mind in this space. It's one thing to take software written by an individual for an individual where essentially there is almost no risk here. You know, if you don't give that SIP stack your bank account permissions and so on, then the worst that's gonna happen is it just doesn't work.

That's not the context that most people are in, right? let's look at it. 

If that is the context you're in, right? You're a startup, you haven't even built your product yet, or you just have it and you're looking to iterate really, really quickly because you're trying to get to product market fit, you're trying to answer questions.

speed really matters to you before you run out of runway or someone else does what you're trying to do, then actually you should be at the forefront of using AI as well. 

If you're someone who's got an existing product, it's in the field, it's relied on, it's supported, it's making you a lot of money, it is paying the way, then you're gonna have a lot of different things that you need to think about before you jump on that bandwagon. 

And there will be other people where it just doesn't matter yet, or maybe at all. Some software engineers will discover they are irrelevant, not because they've been replaced by AI, but because they haven't. 

All of those things exist.

Anne Currie (06:07) Hmm. Where would you start? I mean, you're saying that you would start with just asking Martin what he would start with, asking the person who's doing the experimentation. The interesting thing is obviously in this case, Martin has put himself forward and identified himself as the person who's doing the experimentation. He wrote a sub stack

which you spotted and then you drew my attention to it and said read this chap who you haven't seen him in quite some time but you know Martin very well, read his substack and then I thought gosh I need to get Martin on the podcast so in that case Martin had identified himself by writing his substack but I bet that most engineers who will be playing around with this have probably not identified themselves

they might have done. if The boss is paying a lot of attention, they might have spotted it, but maybe they haven't.

Jon Berger (07:12) Well, I mean, I don't know, I would have thought if you really have no idea what's going on in your team and you don't know whether people are spending thousands of dollars on tokens and what they're building and how they're building it, I'm not sure this is the most important question for you to address. You ought to be able to ask your team what they're doing.

And go to them. Your team will need leadership in this space, right? This is a time of fast change. Anyone reading the papers will be seeing that there are tens of thousands of engineers being laid off by companies that are software engineering companies, saying, Well, maybe we don't need so many software engineers anymore. And there'll be a lot of people in your team thinking, what is my future here? And If they're scared, then they're probably not operating at their best. So you need to lead in this space. 

One of the things that you said you and Martin talked about was, well, if I can let's say at this point I can develop software 10 times faster. So we're not in the far future where I don't need any people, but maybe I can go ten times faster with AI than I could without it.

And it was interesting to me that you then jumped to, or Martin then jumped to, "well, that means my software engineering team can be one tenth of the size". Because that might be true for some people, but for almost my entire career, if someone had come to me or I'd gone to my boss and said, by the way, I can now do ten times as much as I previously could, they wouldn't have just shot everyone in the head. They'd have gone, Well, brilliant, go on then. Do ten times as much.

Because producing software is what makes us money. If we can produce more software, better, faster, then we'll make more money. That's a good thing. So there will be people, this won't apply to everyone, again, context dependent, but some people are out there, you want them to lean into this because this is the business you're in, and you want to be the best at the business you're in.

Anne Currie (09:34) It's a very good point you said. And I've been a head of IT in my past as well. And you always have a backlog. You've got a roadmap that goes on for years. If someone comes to you and says, I can move up ten times faster on your roadmap, you don't immediately go, I'll sack ninety percent of you and we'll move forward on the roadmap at exactly the same glacial pace that we've been moving forward on it for the past.

It's a very good point. It's a somewhat interesting point that the big folk, like Meta last week, announced that well, actually laid off twenty percent of their workforce. And you think, Well, is it because you just didn't have any kind of backlog? That's quite surprising. Their motivation must be different, isn't it?

Jon Berger (10:30) Well,

it is different because if you're one of those companies, you're driven entirely by short-term fluctuations in your stock price. And right now, the market, for whatever reasons, believes that if you sack a load of software engineers, you are a more valuable company. Maybe that will always be true. If essentially you say Every year we will sack 10% of our workforce. That might turn out to be at about the rate at which you can safely remove costs or change costs from being meat sacks to electricity. That's a lot of what's going on. 

The other thing is that there has been a lot of commentary. I mean Martin's chat about oracles made me think of human oracles in this space. And I think there's a big danger here for folks that there's a lot of people out there who have been human oracles for years or decades that we've listened to and done what they've recommended because They used to know how software was made.

But they don't anymore, but it hasn't stopped them. 

And some of those people are correctly and rapidly adjusting to a new world where software is made in a very different way. And some of them aren't. And that's very dangerous. So choose your oracles with care and have more than one. One of the things Martin very sensibly said was have more oracles. One oracle is a really bad state to be in.

Anne Currie (12:15) And that applies to human oracles as well as mechanical oracles. You listen to multiple points of view. It is very interesting. And of course, we are a point of view and you shouldn't just listen to us.

Jon Berger (12:32) No, absolutely not. 

The other thing to say is, you know, as part of that leadership and going to your team and showing them that you're not scared of AI, you think it is a good thing, it's inevitable, it's happening, you want to embrace it, you want them to embrace it. You understand that there are risks as well. It's not an unalloyed good. Therefore, you want them to be thinking about those things 

And a team of any size is going to have a bunch of folks who really want to just get going on this, and they probably already are. And some other folks at the other end who are just seeing all of the risk and going, no, we can't touch this because, because, because. And you have to listen to both of those sets of people. You really want to, which isn't to say you'll just do whatever they say, in either case.

But you have to listen to both sets because those people who are worried about this will come up with a really good list of concerns, of risks that you need to address and mitigate in order to move forward with your business. So think about those and show you're addressing them, as well as looking at the folks who just want to go as fast as possible and don't really care about risks because it will all work out in the long run. Because those people will be the ones that give you the energy and the momentum going forward. So you need to listen to both sets of people and you need to show that you're doing that and show how you're listening to those, turning that into a set of thoughts, processes, strategy for how you're going to move forward in a way that is safe.

Anne Currie (14:20) Which reminds me, I used to do a lot of talking about whether containers were dev technology or ops technology about 10 years ago. And I used to use the Marvel Civil War pictures to illustrate that all the time. And they didn't have this in the Marvel universe movie, but in the original comics, there's quite a good poster of the two sides in the war, which was one side "protect the future," the other side "change the future." And I loved that as a story because it often mapped quite well to if your ops team is objecting to things, they want to protect your systems.

And then maybe the dev team is saying we should do this because they want you to advance your systems. In the end, you can't just have a civil war. Everybody has to come back together to battle Thanos.

Jon Berger (15:22) That "protect the future, change the future" is like all of software engineering since the dawn of time, essentially. It's a constant battle and you are constantly battling that and you do want to be on the same side. What are your competitors doing? Or what are they gonna do? If you do nothing, is that just fine? Or is that gonna cause you a big problem?

Anne Currie (15:42) Yeah. Because time is always your enemy in all businesses. Other things happen, people move forward, you're in competition. So you can't do nothing. There's always a balance between protect the future and change the future.

Martin was very much in the "change the future" there. But I would say the interesting thing I think both of us—

Jon Berger (15:58) Sorry.

Well, because he's in a zero-risk environment.

Right.

His SIP stack isn't supporting all of the world's 911 calls. His codec isn't used to encode the voice of the president telling people to launch nuclear missiles. There's no risk he's running. He's doing really sophisticated, clever stuff and he's doing it very rapidly, but he is not operating a business.

Anne Currie (16:13) It's a very good point.

Jon Berger (16:36) Yet.

And if you look at when I'm looking at those risks,

Almost everything that you talked about was well, how do you build this software? And that is a very important part of the software engineering discipline, but it's actually quite a small part. If you look at most software, not all software by any stretch, but most of it, you're gonna spend less than 10% of your time building it, and more than 90% of the time

Supporting it, maintaining it, evolving it, and making money from it. That's where you have your business. And that requires right now a lot of people in different teams working together, complex processes that have been built up over quite a long time

To protect the downside. Even if you're someone who's really quite good at change, then you'll have systems and processes which allow you to deploy new code thousands of times a day, right? That's considered like good practice, is like deploying new code is really, really, really quick. There are very few barriers to that.

The reason why there are very few barriers to it is not because you've said, I don't care. It's because you've put in place a whole load of systems to either allow you to roll back or roll forward or protect how bad it can be, or roll out in a small amount to start with, and then only when that works do you flush it out to everyone, so that you can make rapid small changes which end up being effective over a period of time.

Anne Currie (18:32) Paula Kennedy, when she was on the podcast to talk about platform engineering, she talks about guardrails, which is a kind of classic thing to have in your operating platform. You can't set up permissions that would allow this application to overwrite everything and destroy everything. But there's also kind of architectural

things like blast radiuses, so that there's a limit to how much damage you can do. And as you say there are things like within your deployment setup, the ability to roll back, the ability to partially roll out, try it and then roll back. All of those techniques become more important with AI in the mix, don't they?

Jon Berger (19:22) Well, they do, and I am confident that even if AI can't do it now, it soon will be able to do all of those bits for you. But the harder thing is the trust. I mean Martin showed that AI can do a really very good job of testing your code. But do you trust it? 

A brand new developer joins your team and says I'm great at everything. Well, great. But how long does it take and what do you have to do before you go, "actually, this person really is great at everything. Wonderful. Good hire. Well done." Versus, "Oh dear. Thanks, but no thanks." And how do you evolve? Because actually, the reality with any developer is they'll be good at some things and not others. They won't be great at everything. No one's great at everything. So how do you learn?

Jon Berger (20:18) What works for you, what doesn't work for you, which bits you can rely on now. And given that the bits that you can rely on will change over time, how do you capitalize on that?

Anne Currie (20:31) I think it's more difficult than that. I don't think you can really use humans as an analogy here for whether or not you can put your trust in someone. Because something else that Martin said—this was not on the podcast, this is when we were having a coffee with him last week—he said one of the interesting things about the models is they change. Their personality changes with every deployment. They're really significantly different.

So it's not just like a person where you can say, "I trust you now, I know what your judgment's like, I know you're not gonna lie to me, I know that you know about this area." It's not the case with AI. It can be very different between models. And then suddenly you can't quite use the same heuristics. Doesn't mean that you can't do things, it just means that you will need processes that are different.

Jon Berger (21:23) Yeah, I don't think that's enormously different from what you have today because at a large company with a large existing code base, you might have tens, hundreds, thousands of people who can change at least some part of your code base. Those people might be making changes. Those people, you don't know their names, you've never met them, you'll never interact with them directly, but you are between you collaborating to deliver a service.

Those people could change all the time. Who knows which country they're in, what they do.

So it's change. It's you can't just say, all right, we know what our new process is. I just need to explain to everyone the process, sell them on it, launch a project to move everyone onto that process and then we follow that and then we're good. Because it's just changing. It's all changing. So that isn't a valid option, which means you really need to identify those people who like change, who are comfortable with a fast-moving environment because those are people you're going to need to rely on to help steer you into this, while at the same time listening to the people who don't like it, who are worried about it, because they'll help build up all those what-ifs and all of that risk protection, which you also need to be investing in as well.

Anne Currie (22:50) It is interesting, isn't it? All of those people are of value and it can sometimes seem like they're not of value, like those people are all naysayers. They're actually all the people you need, you need all the people.

Jon Berger (23:05) Yes.

Just because they're annoying doesn't mean they're not valuable.

Jon Berger (23:11) That's my defense.

Anne Currie (23:14) I do always remember that whenever you were coming up with something new, it was always very difficult because people would say, "we tried that before it didn't work, so we shouldn't try again." And you think, yeah, but have

things changed now so that it could work now when it didn't work before? Or have actually those fundamental things stayed the same and therefore that was just valuable learning that we shouldn't relearn again? You always have to ask the questions and think and think and think.

Jon Berger (23:46) You've got to be interacting with your team to help them to understand that the world is changing and things that you considered highly valuable in the past, that value has changed. Again, Martin discussing the collaboration between him and Claude building software, and he still really enjoyed the productivity he was getting, but he wasn't writing the code himself.

So if you have people on your team who think their value, their prime value, is writing the code, they need to understand what their prime value is in the new world.

Anne Currie (24:25) Well, and it's interesting. I get lots of very good comments I really welcome on the videos, particularly on YouTube. And a lot of the people who comment are saying, "I find that quite interesting because I'm swithering about whether to learn Rust," which is something that's come up over and over again in this podcast so far. I don't know Rust, because I was a C developer, and at this stage I'm not gonna learn Rust, but also I'm aware that Rust is probably the most important language going.

But people didn't learn Rust in the past because it was really hard. It's really, really hard to get your head around Rust, but it is an incredibly useful language with incredibly strong guardrails that eliminate whole classes of bug. But it was really hard to learn. And now you can use it and you don't necessarily need to learn it.

Jon Berger (25:24) I have heard people say that AI writes Rust better than most languages because at the point where the LLMs were trained, there wasn't as large a body of Rust out there and it was higher quality

than the code for languages that have been around for a lot longer and are more popular, where there's a more representative range of code quality out there. But I've not seen anything comparing the quality of AI-generated Rust code, although for sure the more opinionated compiler will help you to trust that code a bit more.

But at the end of the day, that's way low on the trust level.

Anne Currie (26:13) Yeah.

Jon Berger (26:13) I have dealt with a supplier many, many, many, many years ago who would regularly deliver me code which didn't even compile and that got me angry. But I would say that's a pretty low bar for code quality.

Anne Currie (26:33) That's true. Although it's a much higher bar if the Rust compiler compiles it. That's one of the things. It's a much higher bar.

Jon Berger (26:46) It is. But going back to the question we started with, which is how do we move, how do we use, how do we change the way we talk about trust and reason about trust in the processes to enable us to use AI to go faster? The compiler is very low on that list of things that move forward.

If you're worrying too much about that, you're not worrying about the important things, which is I need this thing to work and it's okay to treat this thing more and more as a black box.

not least because

The long-term future, as far as I can see, is why on earth are you using human-readable languages? They're all bad. And we're already getting to the point where people are going, "comments, why do we bother with those?" Some people have been saying that for years, but increasingly with AI-generated code, comments are just a bad thing. Machine code executes faster

and can by definition do everything that you need to do in the best way it is possible to do it.

The only limitation is it's a bit hard for us slow folks to work out what's going on. It's not a problem in the future. So learning a language, your big question isn't "do I pick Go or Rust?" That's a detail now. It's "how do I get good at using this amazing tool to generate some black box code for me in a way that I can trust and support and maintain and evolve and continue to make money out of?" It's a completely different set of questions, isn't it? I think it is.

Anne Currie (28:40) Yeah, that's true. That is very true.

No, I agree. And coming back to something you were saying that's actually been weighing on my mind a lot—mostly because it's something I'm generally quite obsessed with—if you're watching on YouTube, you'll see just above Jon's shoulder is a copy of Building Green Software, my O'Reilly book. That's not because he's a superfan, it's because he's upstairs.

One of the things when Me and Sarah and Sara wrote that, we wanted to be very, very careful about was: what are the principles here? What are the big ideas? Not the implementation details, what are the principles that will last for decades versus the details that might change on a day-to-day basis? Something that has been preying on my mind constantly with

with AI, or what I've been thinking about constantly: What are the fundamentals? Have AI changed things that we would have considered to be fundamental or not? Something that Sara and I have been discussing quite a lot is the idea of doing lifestyle retreats based on distributed systems principles. Because actually what I like about it—it's a joke, but I like it as a way of saying—well, are these principles widely applicable? Preparing things in advance as a means for resilience, having multiple different routes to get somewhere as a piece of resilience, taking one step at a time as a piece of resilience. Those are principles that we use when we're building systems, but they're also principles that we can use in life.

And so in some ways, they're like: is it a useful, general rule or universe principle, or is it just something we've temporarily been using in tech, but we will replace it with something else? Are they essentially human principles that won't apply in the world of AI?

Jon Berger (30:56) I think loads of those change massively in the face of AI. Loads of them. Almost everything about the way we build software today is the way it is because it's humans that do it. And there is a limit to how much you can scale humans vertically. And there are

big challenges with scaling them horizontally. And some people are pretty good at addressing those horizontal scaling challenges, but all of them come with trade-offs. And AI allows us to make very different decisions to capitalize on those trade-offs. I think that's a sort of podcast on its own just looking at that. Because it isn't really the question we

were supposed to be talking about today according to your briefing. But we can talk about it if you like, but we haven't talked about what sort of things can you do to address what I see as the biggest question, which is: how do I move forward with trust, rather than what does code look like?

Jon Berger (32:19) Would you rather talk about what does code look like or would you rather talk about how we move forward with trust?

Anne Currie (32:24) Let's talk about how we move forward with trust, which is also a subject of many of my books.

Jon Berger (32:31) So again, context-specific. You've got to look at what the folks who want to move really fast see as the biggest value coming from. The folks that want to move really slowly, what are the big risks they're worried about? From a business point of view, what actually makes a difference to you? And what doesn't make a difference to you? Where are you looking to optimize not at the bottleneck?

I would think that almost any software engineering organization can do things where you start to use AI as part of the SDLC with very low risk. Essentially anything on the ops front is high risk by definition. So that's not where you start because you need to start building the expertise to get more gung-ho with things. But AI can be really good at reviewing code. And there's no real downside to doing that. If it finds problems, great. If it doesn't find problems, don't use it. People have been using source code scanners for years. And it will just do that better.

Anne Currie (33:53) Yeah, and in fact, I'll be having a person on to talk about security scanning in a future podcast because that's very, very good. It's already good. If you're writing new code now, you should absolutely be doing AI-based security scanning.

Jon Berger (34:10) Yeah, not just security scanning but all sorts. Testing, there's a load that it can do for you in that space. Not in a way that allows you to throw out human testers—I don't think it's there yet—but in a way that those folks can move way faster. I've heard very good things about debugging and its ability to go, "well, here's where your problem is." Not as good things about then suggesting fixes to that problem. So that part of the SDLC maybe isn't yet fully automatable, or maybe it is, it's just that I haven't spoken to the right people yet.

Anne Currie (34:57) apparently, Mythos is better at that than previous. The new version of Claude is better at that, which is why everybody's worried about it, because identifying security holes has been something that AI has been doing for a while, but Mythos is one of the first models that can then exploit them, which is kind of the same kind of thing as coming up with fixes.

Jon Berger (35:07) Well, no, I don't think it is actually. I mean it's related, but as I understand it, Mythos is much better at finding problems than fixing them. Exploiting them isn't the same as fixing them. Once you've found a problem, exploiting it is often not that difficult in comparison, depending on the problem.

Anne Currie (35:33) Okay.

Jon Berger (35:47) Yeah, I mean at the spec level, if you're writing complex software, you probably are separating out spec from some level of design, from some level of code. Each of those are trust boundaries that you can use AI to iterate on and get used to it and see where it's adding value and get good at using it and get comfortable with it, and work out where you still have risks that you want to address and where you can use that to allow you to go faster.

Anne Currie (36:30) Yeah, getting it to write a doc is never dangerous.

Jon Berger (36:38) That's recorded so we'll be able to play that back when all sorts of things go wrong.

Anne Currie (36:43) It's a very interesting take that you've got there, which is on AI and using AI code, try and start with what you can do that's low risk and then do it. So code review, because a lot of people don't do code review anyway. Quite often if you're going to cut something, you cut code review. So do a code review and if you like what it does and it's useful, use it. If it's not useful, well it hasn't harmed anything. Writing better specs or upgrading specs or doing designs gives you something to read, you don't have to use it, you can just throw it away. But that's the same as writing additional tests, unit tests. Martin mentioned that it is very good at writing UTs. Have more UTs. the worst-case scenario is that you're testing stuff that doesn't really matter, but what does it matter? 

There's tons of stuff, isn't there? Before you even get as far as writing functional code. And that all gets you into the mindset of playing around with ai and how you use it as well, particularly the writing test cases.

Jon Berger (38:01) Yeah, and crucially it has to be the case that one of the most important things that you can do trust-wise is have testing that you can rely on, such that you can then make rapid changes and know that the core functionality is still there.

Anne Currie (38:28) We're not saying delete your existing tests. Run your existing tests and also run these tests and then that's very low risk.

Jon Berger (38:34)

Yeah, I mean there's a whole bunch of stuff you can do there that I would categorize as essentially just augmenting what you're doing at the moment. So you and the AI—you're still responsible for the job, but you're getting the AI to help you out. You've not really yet moved into the new world of AI software development, so you're not a long way along that, but

there is a bunch of stuff you can do there which will help you gain confidence and start moving into that space. Ideally, you've got some new project or small project or something which you can use to be the tip of the spear and go a bit faster in terms of

iterating on trust.

Anne Currie (39:38) We've talked for quite a long time and that was very interesting indeed. And we've got some actually actionable stuff for folk there, which is good, which is identify the low-risk stuff and play with AI, with low-risk changes in your systems, things that add but don't hurt, like additional testing or code review. So thank you very much for this talk today. What I'll do is I will point Martin at it and then see if we can entice him to come back and talk about whether or not we're seeing things in the right way given his experimentation or we're still seeing things in the wrong way. Because that's the benefit of a podcast is that you can

literally iterate on the conversation in a way that otherwise you can only do in person, which is nice but doesn't scale in the same way. Martin, we saw him last week, but it did require travelling across the country which you can't do every lunchtime. Thank you very much, Jon, that was an excellent conversation.

Jon Berger (40:42) Thank you.

Anne Currie (40:44) And thank you to our viewers and listeners at home. Hopefully, I will catch you up on the next episode of Asynchronous and Unreliable Podcast. Thank you very much.