Shownotes

How Do We Make AI-Generated Code Good?

Anne and Yanqing Cheng, founder of AI startup Tollens.ai, explore the future of software quality, AI management, and organizational trust. Discover how holistic engineering management, trust, and organizational dynamics influence AI and software quality in today's rapidly evolving tech landscape.

Key Topics

• The evolving definition of quality in AI-generated software

• Tollens.ai's approach as an engineering management consultant

• The importance of holistic management and decision-making

• How AI tools can aid managers and engineers in complex decisions

• Challenges of trust, psychological safety, and organizational dynamics in AI deployment

• The impact of AI on software productivity and the future of coding

• Managing bottlenecks in AI-powered engineering workflows

• Strategies for capturing and leveraging organizational knowledge

• The gradual transition to AI-native companies and the pace of change

• Practical tips for integrating AI into daily workflows to reduce friction

Timestamps

00:00 - Introduction: The focus on quality assurance for AI-generated code

01:00 - Tollens.ai's vision as an engineering management consultant

02:23 - Why quality is a vague term and the holistic approach to software quality

04:16 - The four-step Tollens framework for quality management

05:23 - The origin story: How Anne and Yanqing met and their background in mission-critical software

07:42 - Trust and AI: Can AI be trusted with mission-critical tasks?

09:03 - Defining what good means in software engineering

10:01 - The importance of measuring software quality and gap analysis

11:01 - AI's current limitations in pushing back and autonomy in management tools

12:34 - Management consulting insights and how AI can support decision-making

13:22 - The role of human authority and power dynamics in AI-managed decisions

15:10 - The significance of psychological safety and organizational culture

17:13 - Building trust in AI tools through familiar, low-status personas

18:55 - The role of leadership in enabling AI-driven decision-making

20:00 - Psychological safety lessons from aviation safety and teamwork

22:00 - The importance of trust barriers and cultural factors in AI adoption

23:45 - Designing AI personas for communication and trust within organizations

25:09 - Using informal environments and rituals to surface organizational issues

28:30 - The challenge of adoption variance within organizations and tailoring solutions

31:32 - The new "illities": AI-specific attributes for maintainability, observability, and extensibility

34:45 - The future of software specification, testing, and patching with AI

36:17 - The importance of precise problem definition over solutions

38:25 - Immediate vs long-term AI impacts on software development and management

40:21 - Differing stages of AI adoption across organizations

43:32 - The bottleneck theory: How AI changes what constrains progress

48:40 - Automation's effect on farming and parallels with software development productivity

50:12 - The potential for AI to drastically reduce the need for human developers

52:46 - The importance of focusing on the real constraint and bottleneck

55:30 - Final reflections and future outlook for AI, management, and software quality

Blue Tuxedo

Music licensed through Soundstripe.

Code: 3XRM21XFMMGPO1PF

Transcript

Anne Currie (00:00)

Hello and welcome to Asynchronous and Unreliable, a new weekly podcast where we discuss the most interesting ideas and concepts in tech. I'm your host, Anne Currie, co-author of Building Green Software, the Cloud Native Attitude, and author of the science fiction Panopticon series. And today I'm going to be talking to Yanqing Cheng, founder of Tollens.ai, a startup in the space of quality assurance for AI generated code, which I personally consider to be THE topic moving forward. So I'm really, really happy to have Yanqing on the show. Yanqing is also somebody I've known for over a decade. So yeah, so Yanqing, do you want to introduce yourself?

Yanqing Cheng (00:41)

Yeah, sure. So hi, everyone. I'm Yanqing Cheng. Qing for short to my friends, like the dynasty, I say, to help people pronounce it. And yeah, no, I've known Anne for a long time. It's hard to say how long at this point. But yes, what should I say about myself?

Anne Currie (00:56)

I would tell people what you're up to at the moment. You can correct me on what I've just described your startup as being about. And then we'll dive a little bit into how we met because I still remember that very clearly. You probably don't remember it, but I have a memory like a steel trap. So I know exactly how we met, and I think it's very relevant to this conversation at the moment. So go for it.

Yanqing Cheng (01:07)

Yep. Yeah. I genuinely don't remember that, so I'll be interested to hear what you think of it. But yes, Tollens. Every time I talk to someone about Tollens, I describe it to the next person differently, which I think is interesting because I have a very clear picture to myself since the start of what I wanted Tollens to be. And that picture actually hasn't changed very much, but it's been very hard to communicate it to people. So I've stopped using the word quality.

Anne Currie (01:29)

Yeah.

Yanqing Cheng (01:53)

because when I use the word quality, people think about testing and test suites and test coverage. Or they think about code quality and code reviewing. And those are very important areas. And there are a lot of other vendors in these areas. I was just at LDX3, what Lead Dev is called now these days. 

Anne Currie (01:57)

So this is a conference in London.

Yanqing Cheng (02:23)

Yes. For engineering leaders. 

And so you go around the vendor booths there, who are all various kinds of dev tools mostly, and everybody's basically got the same tagline, which because it's true, right, this is the problem statement: that the AIs are doing a fantastic job of generating prototypes, but these prototypes are not products. And every single one of these very different dev tools are saying, well, that's the problem and we're the solution. 

So how can such different tools be solutions to the same problem? Well, they are because for every different company and every different product, you've got a different gap. 

The AI is taking care of generating code. It's taking care of producing prototypes, but that's not the entire business of software engineering. Where the gap is between that and a product that you think is good enough to be a product, be your product, is different for different companies. 

So in some companies, the code reviewing tools are absolutely the right thing to be investing in. For some companies, the test automation tools, and there's some really cool ones out there doing clever simulation of your environment and different test inputs and so on. For other companies, it's observability tools and instrumentation. And all of these spaces are parts of what move the needle on quality. 

But my approach with Tollens has always kind of been top down rather than bottom up. It's looking holistically at the gap between where your code is now, whether a human has written it or whether the AI has spit out a prototype and where you want it to be and saying holistically, what's the plan that gets you there and how do you define that plan and how do you manage that plan? 

And so what I've started saying to people is, look, what I really want Tollens to be and what I'm building is that Tollens is just an engineering management consultant. It helps you make better software engineering management decisions. And that's the side of it that I'm coming from. How do you prioritize all these things? How do you decide and define such that your entire org, including all of the AI agents working for you, understand what good is for you for right now and understand what the plan is for how to get there and execute on that in such a way that you, the engineering leader, can actually oversee and manage the whole thing?

Anne Currie (05:06)

Now, that is interesting. So you have described that to me before, but now as time's gone on, I understand it a little bit better, I think. So actually at this point I'm gonna step us back a little bit, remind you how we did meet because I think that does provide useful context for why you are on the show and why I think that you're well worth listening to.

Yanqing Cheng (05:23)

Yeah.

Anne Currie (05:35)

So we first met about 10 years ago, just over 10 years ago because I was running a tech meetup. One of my other co-organizers was a person who was on this podcast a few weeks ago. The meetup subject was stack walking, stack trace analysis. And at the time you worked for my husband, who's also on this podcast, Jon, in mission critical network software development. And he'd said, you know, you should really meet this young woman who's in my team, and I think she's really amazing. 

And I thought well actually, come along, you can speak at my meetup and that's how we first met. And you did a very good job, you spoke for me at various things after that because you did a really good job. 

But what was important to me was that you came from a background of knowing what good looks like. It was a company that was producing mission critical software. So in this debate that we're having at the moment about well, can AI take over the kind of jobs that have been entirely the domain of unbelievably experienced developers? The software that the world is built on is mission critical software. So the real question is not just, as you say, not just about building prototypes, which is one kind of software, totally valid kind of software, but also what about the software that has to work all the time, that businesses are built on, that the world is built on. 

And you come from that background. And that's why I am interested in your opinion on this. Because almost everything we're talking about at the moment is trust. You know, AI is writing code. Can you trust it? 

And the question is always, can you trust it to do what? And the first stage in trust is actually seeing what people do, seeing what people are involved in. Human trust. But with AI, we're suddenly gonna have to decide whether or not to trust them with much less information, completely different heuristics. So I'm just really interested in your thoughts.

Yanqing Cheng (08:04)

Yeah, absolutely. And there's a lot to speak to here. So I'm just going to start with the way I think about it, which is, and I've spoken to a lot of people because I started off saying Tollens is about software quality. So I spoke to a lot of people in the software quality sphere who work in testing of various different sorts. 

And I think there's a tendency for people to think of mission critical software and making prototypes as fundamentally different activities. And that's a valid way of thinking about it, because then you can apply different heuristics to these different domains that are suited for the goals of your different domains. But I like to think of it more holistically.

Your challenge is producing software products. I like to think of it as, your quality goals are context dependent, but ultimately what you're trying to achieve is the same. You're defining what good means and you're trying to achieve it. Right. 

And the Tollens framework, I think, boils down to four steps. And I describe it like this. 

Step one, define what good means to you for your context. 

Step two, define how you're going to decide if what you've got is any good or not. How are you going to measure how good what you've got is? And that's where testing fits in, right? That's why testing is core to quality, even if I don't think quality is testing, because a lot of the time how you're going to find out how good your software is, is by testing it. 

Then three, well, three is actually finding out, so, two is how you're going to find out. And there's a reason I separate out two from three. I'll come back to that. Three is actually finding out. So how good is your software for your context compared to what you want? 

And four is if there is a gap between how good it is at the moment and how good you want it to be, then what's your plan for making it as good as you need it to be? 

It's very wordy. I need to trim down on that wording. But fundamentally, what's good? How are you going to measure how good your software is? How good is your software? And how will you make it good?

Anne Currie (10:26)

I like it. I think those four steps make a lot of sense.

Yanqing Cheng (10:32)

And so two and three are kind of logic. Two is logically a prerequisite for three, right? So why do I split that out? One of the things that I noticed when I do a lot of coding with AI coding agents is they're really, really terrible at pushing back on the information and tooling they've been given. They don't have the autonomy to say the tools I have aren't good enough, or I need more information that I haven't been given. 

The environments that they've been trained in don't give them that lever. And actually levers in management is one of the things that Jon taught me way back when he was training me to be a good manager. You got to think about the levers you currently have available and if they're not sufficient to solve the problem, you need to look for a bigger lever. You find someone with more power to change what needs to be changed to achieve your goal. 

And the AI agents haven't been trained to do that at all. I'm sure they will be in the future once they realize that this management skill set is a big gap for them. But right now, whatever tools they have at hand, whatever information they have at hand, they're going to try their very best to solve the problems with that, but they don't step back and think, I can't do the job you've asked me to do, but I could do it really easily if you just gave me these other tools.

Anne Currie (12:05)

Actually, it's very interesting you say that. So I was a management consultant for years and years. Probably unsurprisingly to the listeners, they'll think, Anne sounds like a management consultant. I was a management consultant for many years. And I always thought that I was basically there to talk to all the developers and persuade them to tell me that they were trying to do things that they couldn't realistically do, because nobody likes to say they can't do it. 

And you're saying that the chatbots aren't very well trained to do it, but actually most developers aren't trained to do it very well either. And most managers are not trained to ask the right questions to unearth those problems. So fundamentally you end up in a situation where the manager asks somebody to do something, and it isn't really an ask, it's a demand. And the person says yes, even though they know they can't do it, because they're terrified of saying, I can't do this. So they think, well, I'll give it a go. And the manager doesn't want to hear no anyway. And then that's often 90% of what management consultants do, try and untangle that situation. So actually, your phrasing of the tool is that it helps untangle that situation...

Yanqing Cheng (13:22)

Exactly.

Anne Currie (13:29)

which actually does align with management consultancy. So effectively what you're saying is you're designing an AI to put me out of a job that I don't do anymore. I used to do. My old self out of a job.

Yanqing Cheng (13:42)

I think of it as with a lot of these AI things is people who wouldn't necessarily have decided to pay for the human might decide to pay for the AI. And the AI, at least today's AIs are not going to do as good a job as the human, but they might 80-20 it, right? You might get some of the value for a slightly worse but still fundamentally quite capable management consultant to walk you through a few of the best practices, even if they don't know quite as many as you might be able to come up with on the fly. And I think...

Anne Currie (14:13)

Yeah, I would say that actually management consultants are quite a mixed bag in terms of how good they are. So actually, the result is, I suspect your tool will be used by management consultants. One of the points I constantly make on this, or constantly make in real life, I don't know if I've made it in here before, have you ever seen, you must have seen Galaxy Quest, the movie. It's brilliant.

Yanqing Cheng (14:20

I haven't actually.

Anne Currie (14:42)

Oddly enough, the best Star Trek movie is not one of Star Trek movies, it's a movie about Star Trek movies, it's kind of a meta Star Trek movie. And Sigourney Weaver plays a woman who is an actress playing a kind of Uhura type character in the show. And she said, "What is my job even? All I do is repeat what the computer just said." And I suspect there'll be a lot of people whose job in the future is to repeat what the computer just said. But there is value to that. It's not a valueless job to repeat what the computer just said.

Yanqing Cheng (15:21)

Absolutely. And, you know, one of the challenges that I foresee as I've been speccing out Tollens is that in a lot of organizations actually, when you talk to managers, this becomes really clear. You ask them, what are the challenges you face at work at the moment? And for many of them, it boils down to a politics problem. 

You ask the questions of, well, what's your barrier? Okay. And where do you think the bottleneck is? You know, what would move the needle the most? And it's that there's some team who's been given the wrong remit or the necessary remit doesn't have enough resource and they don't personally have the political power in the organization to say, look, what we need to do is take some engineers off that other team, tell them to drop everything, give them an unlimited token budget and point them at this one problem. 

And if there's a bunch of engineering managers whose orgs would be doing a hell of a lot more if they could just manage to do that one thing, right. And the problem with Tollens is that an AI agent bot, your management consultant bot might not have the authority to make that call. You know, if your management consultant bot comes back and tells you what you need to do is you need quite a lot more resource pointing at this problem, because that seems to be the fundamental top priority thing you need to achieve for your next release to go well. Or all of your internal developer experience and AI agent developer experience quality dimensions are bottlenecked behind this one piece of instrumentation that you don't have. Can that bot convince this manager to make the changes required to actually solve this problem? 

I'm not sure. So maybe sometimes you need another human to just be the source of power in that interaction.

Anne Currie (17:13)

Yeah, it's interesting. So there's a huge question there, isn't there? Who will be the trusted person in the future? Is it going to be people like me? And people trust me, I'm very trustworthy. But a lot of people aren't very trusted, even though they are actually perfectly trustworthy, people don't trust them. 

So there'll be some people for whom the bot is vastly more trusted, trustworthy than anyone that they could bring in. And for other people actually, you know, they have access to somebody who people tend to trust. Having a person who everybody trusts read out the instructions is surprisingly effective.

Yanqing Cheng (17:59)

Yes. So I think, you know, as I'm building Tollens obviously, I need to have an ideal customer in mind. And that for me is some CTO or head of engineering in some relatively small company where everyone still knows that CTO and they can look at what Tollens is giving them, and Tollens is obviously going to give all the reasoning, right, going right back to what you said your goals were and who you were trying to serve and what you were achieving for them. This is why, this is your priority list for this release to achieve these things. And I'm backing that off what you said earlier, and this is how the logic joins up. 

And if Tollens can make that case, this is the why, this is the how, then that CTO needs to have the power and the authority and the trust from their team to be able to then make the changes and say, you know, I need these AI agents using this amount of tokens to do these strands of automated testing. And I need these humans to do this exploratory testing because I and Tollens don't trust the AI agents to do that testing in that area. And I need these humans with an unlimited token budget to work on this piece of tooling because right now we can't measure this thing. And that is one of our top priorities, right? And it needs someone who has the authority to make that plan to work alongside Tollens to make it happen.

Anne Currie (19:33)

Eventually people will trust the AI, and a lot of people do trust the AIs already. Probably at the moment, people overly trust the AIs in some areas and underly trust the AIs in other areas. That's always the difficult thing at the moment, isn't it? Trust is unbelievably important in successful businesses. We talk a lot about... there's a concept called psychological safety, which I know that you're very familiar with. It comes from, oddly enough, air traffic crew management. Planes used to crash because there wasn't the ability within a team that was crewing a plane to go, "Well, hang on a minute, I think we're about to run out of fuel." Even though you as the pilot are saying, "No, everything's fine," it's because you haven't looked at that dial.

Yanqing Cheng (20:03)

Right.

Anne Currie (20:29)

And that used to be a major cause of crashes and deaths because actually people either didn't say it because they didn't like to contradict the pilot, or they did say it, but then said it once and then backed off again instantly. The psychological safety is that you're a crew and you are all allowed to say what you think all the time and be challenging. And it is a way of creating a trust environment even if the folk don't know one another. It is just baking trust-like behavior into the culture. It sounds a little bit like Tollens is a similar kind of trying to bake a trust-like environment into the...

Yanqing Cheng (21:22)

Yeah, I think it's interesting thinking about psychological safety because the sort of the minimal version of Tollens, I'm not really thinking about trust barriers at all because just that decision making is already quite complicated, right? You elicit the information out of your engineering leader about the sort of unspoken assumptions about what good looks like for the product that might not be written down. You get that written down and you share that with everyone. 

But yes, well, if we're talking about organizations, engineering organizations, and what causes them to produce bad quality software, what I'm addressing with that, not describing clearly enough what good is and prioritizing, that's one failure mode. 

But as you say, not surfacing information because of lack of psychological safety is another failure mode, which I do think is something that Tollens could address in the future. But that requires a very different interface to the management consultant coming in and telling you to do stuff. Because for me, psychological safety is a lot about power dynamics. You're not going to speak the whole truth easily to someone who has a lot of power over you. 

I know anecdotally about how in South Korean aircraft cockpits, they're mandated to use English. Do you know this fact?

Anne Currie (22:58)

No, I didn't. That's interesting.

Yanqing Cheng (23:00)

because the Korean language has lots of honorifics baked into it that make it really difficult to disagree with the superior.

Anne Currie (23:07)

That's fascinating. I didn't know that at all.

Yanqing Cheng (23:09)

Yeah, so I think they're mandated to use English. We should fact check this after the call and cut this bit out if I've just made that up. But my understanding is the Koreans...

Anne Currie (23:17)

I will fact check it. Probably even if it turns out not to be true, I'll just put in a little thing saying, "this turned out not to be true, but it's still interesting." It's very truthy. I could believe it.

Yanqing Cheng (23:29)

Yeah. And so I think, you know, in terms of architecting Tollens, like obviously I want an AI agent conversational bot interface, because that does seem to be what people want these days, right? With OpenClaw taking off so much. And you can imagine that just coming in as a trusted colleague. But I think the Tollens bot that goes and talks to the engineers on the ground and sucks up information from them about what they're doing and feeds that back up to the management consultant bot...

Anne Currie (23:45)

Yeah, yeah. Yeah.

Yanqing Cheng (24:02)

that needs to be a different persona. Because that bot needs to be trusted in the psychological safety sense so that people will actually complain at it about the things that are actually going wrong for them on the ground. And so it can't be this high status management consultant walking in with the big hat. It needs to be, you know, a friendly buddy playing extremely low status so that you're not afraid to tell it everything you just complained to your friends about in the pub. 

Like back in the old days when I was working for Jon, the pub was the number one place for surfacing the real bottlenecks and the real impediments in an organization because people let their guard down and there's not so much high status posturing. And when you're playing in that low status way, then all sorts of things come out because the psychological safety is there. 

I remember Jon used to organize pub sessions with a random selection of his employees so that we'd all complain at him and he could hoover up the information about where the actual dysfunctions were in the org that were stopping us from shipping. So there needs to be another Tollens persona where people, you know, Tollens vent bot or whatever, people can actually complain when they might be afraid to.

Anne Currie (25:09)

So it's one of the things you're gonna slightly struggle with. Because I use the same technique myself that if you want people to open up to you, it's really good to share food or whatever. The French do it, they often have lunch with wine every day at work. And apparently that is quite a good way of unearthing things. 

When I was a management consultant, for all my meeting we'd go out to a coffee shop and sit and have a coffee or a cup of tea or whatever. And as soon as people were outside the office in a neutral environment with a cup of tea in front of them, they were just so much more relaxed and they'd tell you stuff. And you know, I'm genuinely quite trustworthy. I did try to do good for them with the information that they gave me, but it did help massively to get them in a different environment, get them relaxed. It's yes, crew resource management. 

Yanqing Cheng (26:36)

And that's quite a hard thing for an AI bot to achieve, right? So even if I could code and construct and make the right skills for that AI bot to do its job, then it's still that CTO's job to work out how to get that information in.

Anne Currie (26:40)

Everybody is fairly trusting of humans, of some humans. People are surprisingly good at immediately spotting whether someone is nice or not nice, and then talking to the nice person and not talking to the not nice person. 

Chatbots... what are the heuristics you use there? How much can you trust what you say to a chatbot?

Yanqing Cheng (27:31)

And from the sort of product design perspective, the difficulty is actually that there's a huge variance in this. So for me, I'm very comfortable with the chatbots and I feel a lot more psychological safety asking questions to my chatbots. I can ask as many stupid questions in a row as I need to and nobody's going to come back and think, that's a lot of stupid questions for one person.

Anne Currie (28:00)

Ha ha ha.

Yanqing Cheng (28:03)

Are we sure they're actually qualified to be here? You know? And so for me, I entrust a lot of stuff to my AI assistants and the more stuff works well, the more rapport you build with them. And you start to get a picture of what they can't do as well. Like the first time it screws something up, you're like, that's really annoying. But then you just know, "this kind of job, this particular AI bot can't do".

But there's a lot of people who are really very wary of the AI bots and they haven't built the rapport, they don't try to build the rapport and they wouldn't trust it. So I think another challenge for something like Tollens is that there's not going to be universal adoption across an engineering org. So how do you make the interfaces into it work such that it still gathers enough information, even if only a subset of the people are going to talk to it?

Anne Currie (29:00)

Yes. Because otherwise you will get loads of information from the kind of people who love to talk to chatbots like you and me. And we have a particular perspective on things, but there's useful stuff to be had from the people who are much more, you know, naysayer type people actually quite often have really good worries about what's gonna go wrong and concerns and things you do want to get that, but they are less likely to trust the chat bot probably.

Yanqing Cheng (29:35)

But that's not a new problem for managers, right? Ultimately, Tollens' job is to help you make decisions and help you do prioritization and do that sort of, I would think of it as like the technical quality management piece of making plans, reporting against plans and deciding what gaps, where the gaps are and what needs to be filled. Gathering the information is still you, the manager's job. 

You could theoretically run Tollens just with you and your code base as the inputs, right? You just have to spend a lot of time talking to it and you would rather delegate some of it out. But if the person that you delegate out to isn't going to report the right information in the right interfaces, that's not a new management problem. We've always had the possibility that you have a particular reporting format that you're expecting to receive back in terms of the information that you need to make your decisions and you delegate that out to someone and you just don't reliably get back the information you asked for. That's not a new management problem and that's kind of what this is. Just the interface is a bit different.

Anne Currie (30:43)

That's true. It's rather than a Google form that people hate filling in, it's a chatbot to chat to. But yeah. You never get the perfect filling in of the Google form.

Yanqing Cheng (30:59)

Exactly. But fundamentally, I still think as long as you're thinking generally enough, I think most people don't think generally enough and therefore they think it's different. But for me, engineering management is still engineering management. 

In my Tollens philosophy, none of the dimensions are hard coded. It's, you think about your maintainability and debug ability and observability and extensibility and all of those illities that are your quality dimensions. Well, now it's, I call it the new world. Old world is before AI coding agents, new world is after. Now you just have your new world illities. You have agent maintainability, agent extensibility, agent debug ability, agent ramp up ability, agent observability... all of those illities of how easily can your team do these jobs and how smooth is it, how successful is it, how painless is it, it's just you ask the same question but with an AI team member. 

And so the actual object level, what good looks like on these axes might look different and therefore people might think that it's different, but the questions you ask are still the same. And so the Tollens approach is if we start from the questions rather than from the answers, then good engineering management is still good engineering management. But because the context has changed and the constraints have changed, your team makeup has changed and the tools available to you have changed, then the answers change, but you're still asking the same questions.

Anne Currie (32:37)

Mm-hmm. So that's quite interesting. It reminds me of something I'm constantly thinking at the moment when I'm looking at the AI stuff, which is there used to be an old thing that used to be said in management, I always thought it was something that really terrible managers said, which was "come to me with solutions, not problems." 

And I always thought, yeah, but the value is in the problem, not the solution. I could think of 10 solutions to a problem. I want to know what the problems are, so that I can then work out what the right solution is for the context. 

And we tend in the past, we've tended to overvalue solutions, whereas in fact the solution is the easy bit. Unearthing the problem is the difficult bit. And a good management consultant is really about unearthing the problems. 

And then you step back and go, okay, this is a problem. Do we care about solving it or not? What's the priority of solving this problem? How much money are we willing to pay to solve this problem? Do we have the skills to do it? Only then do you move on to the solution. The solution is the easy bit. And AI has almost made the solution even easier because quite often if you can do it, almost the whole point of the AI can write the code is you have to define the problem and then it will just produce some code to solve it.

Yanqing Cheng (34:14)

Absolutely. Yeah. You know, there's a lot of people banking on specs are the only thing left. I think they're slightly misguided, because there's always going to be things you can't spec. And your spec is never going to cover everything. I think I wrote a blog post about this at some point in April. But it's still that you have to be able to define the problem for the AI to start tackling it. That's for sure. I think even once you've defined the spec perfectly, there's still a lot of room for testing.

Anne Currie (34:45)

Cause there's something that we've talked a few times about. So Jon, for those who don't listen to this podcast very often, he is a regular guest on this podcast, because he's also my husband. We were talking only the other day about what is the future of patches. 

So at the moment AI is very good at identifying problems with security, it's very good at identifying where your security holes are. It's less good at producing the fixes for the hole, but in the future you can imagine a situation where you just say, okay, actually that problem is now added to what needs to be tested, effectively added to the spec that this is the new problem that we just need to make sure that the tests cover. And then you just regenerate the entire code from scratch. There is no fixing and patching anymore. It's just better definition of the problem set, of the problem environment.

Yanqing Cheng (35:43)

I think this is something where people are only just starting to think about how drastically different the future is going to be. And this is me taking the Tollens hat off because with Tollens I'm trying to make a product for right now and it's like, there's no point in making a product for how it's going to be in a year's time because that's nothing like now. I was reading a blog post by, what's his name? Gary Tan, the Y Combinator guy. And he was saying that he started using prompts rather than code for a lot of his tooling, because why store the code if you know that the AI agent can spin that up perfectly from the prompt. So he just records the skill of generating the code, you know, what it's trying to do, how you're going to check if it's right. And all of that, it's much more compressed than the actual code. And then the AI agent just generates the code when he needs it. And how you check whether it's right is the test case, right? So maybe all we need in the future is a detailed statement of what the requirement is and how you're going to check if it's right. And as long as that's well specified enough, then...

and well specified enough has a big asterisk on it, right? Because, you know, when we talk about mission critical domains, how are you going to make sure that it's right is a very, very long and complex thing to define. But for a simple use case for a simple tool, then all you need is a fairly short prompt to define all of the code.

Anne Currie (37:33)

which is astonishing. And then suddenly code management, your Git repository becomes rather different looking.

Yanqing Cheng (37:40)

just the markdown files.

Anne Currie (37:44)

it is interesting.

Yanqing Cheng (37:48)

But the thing about specifying how the agent is going to check if it's right is kind of the big problem of that then comes back to software quality. But that's always been a hard problem. It's just become more important now.

Anne Currie (38:03)

We probably shouldn't go down the what's the world's gonna look like in three years' time, because as you say, who knows what the world's gonna look like in three years' time. Although I always love to speculate what the world's gonna look like in the future. I am a speculative science fiction writer after all. But for the moment, Tollens is trying to fix a problem that is immediately in front of us.

Yanqing Cheng (38:25)

Yeah, absolutely. You know, one thing that, and this is so I'm being a one person founder team at the moment. So I've got a lot of hats. That was my product design hat. I take that off and put my marketing and business decision hat on for a moment. One thing that is really challenging when you're going into the dev tool space at the moment is that the field has stretched quite far apart in terms of where different people are at. 

Because the tools move so fast, the AI adoption levels of every organization looks different, where they've hit problems are different. And so there's just this massive stretched out field where some people are running really far ahead and some people haven't even started yet. Because, you know, it's only really been six, seven months since the AI coding agents were good enough to rely on. And if you think back in the old days, there's no time at all when it comes to dev tooling, and being a year behind on dev tooling was not this disastrously expensive. Whereas now you can't afford to be a year behind. You're going to be so behind. And so what's difficult as someone going into the dev tool space is where do you pitch your ideal customer? Are you pitching for the companies who are running really far ahead and already working in the AI native way? Because of course, I'm working in an AI native way. Do I pitch for other people who are working like me? Or do I pitch for someone who's not started the transformation? And that's making it very hard to decide who this product should be for. And how far in the future and the past is what I'm pitching, right? But at the moment, my thinking is that I'm aiming for somewhere nearer the middle, maybe the top half of the middle, because actually, AI adoption success is highly correlated with how good you are at engineering management to begin with. Because if you're good at engineering management, you can identify your bottlenecks when you try and get people to adopt this tool, you're able to identify what problems people hit and address them for yourself. And these are going to be novel new world problems. Then you don't need me to come and sell you the playbook. You've got your own playbook. You've worked out how to encode the tacit knowledge that the humans have so that the AIs can act on them. You've worked out how to break down the communication barriers, et cetera, et cetera. It's working for you. So who I want to sell to then is sort of the second quartile, the people who really want to make it work and they're trying really hard but they don't quite have the level of engineering management know-how to ask the right questions and diagnose where it's going wrong. But for those people it looks a lot more like an old-school org rather than the AI native new organizations who are already operating in ways that might be unrecognizable to the people at the back of the...

Anne Currie (41:43)

So it's interesting you say that, because there's an analogy to something that I've been working on for 10 years or so, which is just like 10 years ago I wrote a book called The Cloud Native Attitude, which was about what good looked like in terms of moving into cloud, you know, AWS or Google Cloud or whatever. And that I found good looks amazing. But most people were nowhere. They hadn't even moved. They're still on-prem. And you're thinking, well, this is so, and at the time I thought, my goodness me, there's such advantages to being in cloud native and there's so many skills that you need to learn. The people who haven't moved are going to be left behind and everything, they're gonna fail and that kind of stuff. But ten years on, it's still pretty much the same situation. And the people who were behind have not died. As enterprises, they were actually perfectly resilient to live in the world of on-prem another at least another 10 years and probably for another 10 years to go. The interesting thing about AI for me is, is it like cloud native? Is it something where an AI native company can coexist for years with a company that hasn't even started on the journey yet, in the same way that it could with cloud native? Or is it that AI native is something is going to happen which makes current companies that haven't started down that journey totally dead overnight? I just don't know. Security is the thing that makes me think, I wonder if it's that. 

Yanqing Cheng (43:18)

Mm. So I think it depends on how much of the work of actually producing products gets automated. Because when you look at the cloud native stuff, ultimately the deployment and hosting piece is a relatively small slice of the overall pie when it comes to producing software products. And I think the question everyone is asking now is how many of the components are going to get automated. So I've got a couple of things in mind that are relevant to this. The first is at LDX3 conference, there was a talk by the company DX who gather sort of developer experience metrics on behalf of companies and they publish reports about how much productivity gain different companies are getting out of the AI adoption. And the average was about 10% better. It's well, if you'd said 10% better to someone two years ago, they'd have been astonished, right? Because for any old tooling, 10% extra productivity is a lot. But the interesting thing is that the field is very wide.

Anne Currie (44:29)

Actually very much.

Yanqing Cheng (44:48)

Some, if you actually look at per company metrics, there were companies getting much higher and also companies getting much lower productivity since the AI rollout. It depends a lot on how well these tools are being used. And if you look at Anthropic, their latest blog post suggests that their coding engineering teams that actually produce software, they're getting about 8x the productivity that they used to have. And you would expect a company like Anthropic to be using all of the best practices, right? So that's the bound at the moment. But if people are getting closer towards 8x, 10x, then companies who are not doing that are going to really struggle to keep up because just think about how much they can have to spend to keep up.

Anne Currie (45:37)

Well, interestingly, I'm gonna say that is an odd coincidence because in Cloud Native Attitude, when I was identifying the companies that were doing well versus the ordinary companies who weren't doing well or hadn't even started, it was about a 10x difference. And I really did think that a 10x difference would be enough to move everybody. You know, you'd either fail or you, but it wasn't. 10x is not enough to necessarily, it might, you know.

Yanqing Cheng (45:53)

Mm.

Anne Currie (46:06)

It's not enough to kill off the companies that are not 10x better, interestingly enough. 10x is great, you'd think that 10x would be enough to be a kind of total market mover, but I'm not sure that from my experience with cloud native, and I really thought it would be, it has been. I'm more interested in if is there almost a zero to one thing of... there's gonna be something that you just cannot continue if you can't do that thing.

Yanqing Cheng (46:12)

Mm. That is interesting. Mm.

Yanqing Cheng (46:43)

Yeah, and I think that then comes back to how much of the task can you automate, right? And so something that comes to mind is, I don't know actually how to pronounce his name, but Dario Amodei, the anthropic guy, he writes these enormously long essays on his personal website. I don't know if you've read them. I do recommend them. They're very opinionated because he has to be very opinionated.

Anne Currie (46:59)

I haven't actually.

Yanqing Cheng (47:12)

But there was one recently that came out in about February of this year, and it coincided with when we started Tollens actually. I think it's called The Adolescence of Technology, is the title of that essay, where he gives this analogy of what happened to farmers as farming tools became more automated. So there was a period of time when there wasn't that much difference because the amount being automated was small, it just made the jobs you had to do easier. Then there was a period of time when entire subtasks of farming became automated, but not the whole job. And what happened was each farmer got 10x more productive. And that was amazing. But it didn't fundamentally disrupt how many farmers were being employed. They just used the machines for the bits of their job that could be automated. And they each enjoyed massive productivity gains by focusing on just the 10% that couldn't be automated. And we're not even at that point yet, right? Because it's not the case that 90% of the job has been automated yet. We're a long way from that. But we're sort of beginning every, every model release slightly more of the job can be delegated, right? And so...

Anne Currie (48:22)

Yeah.

Yanqing Cheng (48:40)

as engineers, become higher leveraged on the parts of the job that we still have to do. But then the combine harvester came along and enough of the job got automated that there was really no point to keeping that number of farmers anymore. And so these days there are very few farmers because the vast majority of the farming job is completely automated and there's just no need for that human touch in most of it. And so you expect to see, it was an argument, you expect to see that sort of curve with AI automation, that as more of the job gets automated first, what you see is massive productivity gains for each individual developer before then you automate the developers out of a job. And so I think we're in fairly early stages of that climb. And so you're seeing that a few companies are getting really great productivity gains. But when you imagine what that third stage looks like, then you're basically not going to need any humans at all. And you imagine that one person could do, as long as you could define what you wanted, you could do the job of 100 person engineering orgs, then you really struggle to see how the 100 person engineering orgs are going to be able to afford all their humans.

Anne Currie (50:12)

but it's amazing how slowly those things happen. Cloud Native over the course of ten years, I thought there'd be a complete transformation. I thought that everything would have changed. But it really there was a there was a big leap at the beginning and then an almost imperceptible level of change as far as I can see. You know, very, very slow level of change. And I'm sure more people are kind of using those cloud native techniques now than were 10 years ago. But you think things are going to happen very fast. So the interesting thing is, does AI change everything?

Yanqing Cheng (50:19)

Mm.

Yanqing Cheng (50:54)

I think it is going to be very different. I'm a big believer in what's it called? The theory of constraints where the only thing that moves the needle is your bottleneck. And I think for the cloud native stuff, it just wasn't the bottleneck for a lot of companies. As, yeah, but as AI capabilities get more general, the question fundamentally is how of all the companies that exist and where their bottlenecks are...

Anne Currie (51:00)

Yeah

Yanqing Cheng (51:23)

what proportion of possible bottlenecks can the AI deal with? So there is no company in the world anymore whose bottlenecks should be that code can't get written fast enough to specifications that they understand. There are a lot of companies where code reviewing or testing or observability are the current bottlenecks. Coming back to what I was saying at the start, that's why there's all these vendors saying, well, AI can get you a prototype, but it can't get you a product and we're what you need to get a product. They're all right for somebody. There's a company where that's the bottleneck and they need to buy this product and then the needle will move. But the more general AI's will gradually come and eat those jobs as well. They'll eat those parts of the job as well. So yeah, for me, the fundamental mathematical constant is of the possible bottlenecks, what proportion does the AI handle successfully?

Anne Currie (51:26)

Yeah.

Anne Currie (52:18)

Now, so that's very interesting. Because that's something that's come up on this podcast previously. That in the new world, knowing your situation is vastly more important than it was previously. Because actually, even if you've known the situation previously, you might not have been able to solve it. But these days, if you know what your constraint is and you're focusing on the right constraint, you've identified the problem that's actually the key problem for you to address, you can fix it. But if you focus on the wrong problem... the example that we keep using in this that Jon and I used in a previous thing was a lot of people focus on things like our problem is, or they tell their engineers to focus on five nines rather than three nines as the problem, and you don't need five nines, so you're just focusing on the wrong problem. Everybody is focusing on the wrong problem, and it's not actually a constraint for your business, it's not that important for your business. So, and these days the person who actually successfully identifies their problem will have a great tool for enabling them to fix that problem. But the AI can't do that for you. You have to do that.

Yanqing Cheng (53:01)

No.

Anne Currie (53:27)

Until Tollens helps you do it.

Yanqing Cheng (53:29)

And at the point, well, I mean, the way Tollens is gonna do it is not by AI, right? Because the foundational models can't do it yet. I think even, I would go as far as to say, even with the best harness that I could design, like just pure harness without also adding sort of institutional know-how and the playbooks, I don't think we'd be able to do it with AI, that organizational debugging. The way I'm planning to build Tollens is by baking in a whole bunch of institutional know-how. The playbooks I would use if I was coming in as a management consultant or an engineering manager into the software. So, and having it follow through the structure I would use and pull on the right bits of it at the right time. It's actually extremely deterministic for an AI product. The AI comes in and has the conversations and condenses the information, but the script it's following is still my script for doing engineering management. And there will be a day when just the foundational models will be able to do that. But I don't currently see where they're going to get the data source from to be able to do that successfully.

Anne Currie (54:44)

Yeah, no, that's very interesting.

Yanqing Cheng (54:45)

But maybe Tollens working might give them that data, right? Because if I'm automating the job, then that suddenly means that the data can be generated and then you could use that data to train the foundational models to do good engineering management. But at the moment, if you ask engineering management questions to the foundational models, they do not think critically like that at all.

Anne Currie (55:06)

It is interesting, isn't it? The lack of engineering management is holding back our ability to improve engineering management. Well, the lack of good engineering management is.

so we've been now talking for an hour, and I'm gonna say that was great, and I've got loads of stuff in my head. So and I'm hoping that you will come back and we'll talk about this again in future. But so is there anything you want to add for to leave people to think on, to chew on?

Yanqing Cheng (55:30)

Of course.

Anne Currie (55:36)

Until you are back again to ask questions again. That's I'm throwing a problem throwing a question at you there.

Yanqing Cheng (55:41)

Ooh. Well, I don't really know your audience very well, Anne. What do you think? What do you think they want to hear? What kind of area is most interesting to them?

Anne Currie (55:53)

Well, so interestingly, I would say that the audience is very mixed for this podcast. Especially because on YouTube it's well, it's like I think that the people who listen to it as a podcast and the people who watch it on YouTube are very, very different indeed. And to average, I'm sure this won't be true of everybody who's watching it on YouTube, I know it's not true of everybody who's watching it on YouTube, but statistically a hundred percent of the people who watch these videos on YouTube are young men between the ages of twenty five and thirty five who are very interested in rust. 

Yanqing Cheng (55:59)

Mmm.

Yanqing Cheng (56:32)

Okay. All right. So what I would say then is I think the best bang for buck anyone can have right now out of what the models are currently able to do is to have a much lower bar for problems than you used to. So what do I mean by that? I mean, all the assumptions you ever make in your life about what problems you might have to put up with and what problems you solve, you can attempt to lower that bar quite a lot now because quite a lot more problems have solutions than you're used to. And so, you know, I put a lot of effort into engineering my personal assistants because I just think there's no point having friction in your workflow when it's so easy to engineer it away. So, and that's either by building your own tools or buying in vendors who have come up with new tools, right? I use the voice transcription software, Whisperflow, extensively because it's much faster than typing. And I just transcribe my rough train of thought stream of consciousness into Claude and have Claude structure it for me because I personally always really struggled with getting my thoughts in order. So I just give it my thoughts out, not in order and then have Claude do it. And another thing, for example, that I asked my assistant to do is I have ADHD and I really struggle with processing information that comes in an order. You give me a page of detailed instructions and I just tune out. And so if I have something to process that does make me tune out, for example, if I've got a long design to review for one of my projects, and I do care about getting this design right, I just ask my assistant, okay, write this design up, but make it into really small chunks. And give me review boxes next to each of them so that I can just type my feedback and either lgtm it or type the feedback and you'll receive it.

Anne Currie (58:27)

Yeah.

Yanqing Cheng (58:54)

and just make it a little HTML page and host it so that I can... I have my phone and my workstation on the same local, same private network. So just host it so that I can access it from my private network. And then I just bring it onto my phone. I swipe at it and I go, yes, on each piece or type my feedback or dictate my feedback. You know, ramble about why this doesn't quite look right. I have this iffy feeling, the kind of thing you'd never put in a review comment for a colleague because it would be rude, right? But the AI doesn't care. And so I just make this workflow that works for my brain. I don't feel any friction using it so that it's perfect for me. And it's just unthinkable to accommodate your own comfort in this manner in the old world. And I think right now, now is the time to start switching to everything should be comfortable for me. Everything should be done exactly how I want it. If it's not exactly how I want it, I bet there's a way that's better.

Anne Currie (59:57)

That's really interesting. So that's a very, very interesting perspective and a very good point too. I do massively extensively use the chatbots to chat to. And I love to chat. So in some ways you might say, well, it is already massively well tuned for me. I love to chat. 

Yanqing Cheng (1:00:20)

I think you're a very thinking out loud person as well, so you might enjoy the transcriptions.

Anne Currie (1:00:29)

Yeah, I might do. I might do. Hmm. I will have a think about that. But actually for me as well, because one of the things that I'm worried about at the moment is I think my hands are a massive part of my cognition. And I worry that typing is good, but I used to write everything out longhand and I think that my thinking has reduced because I don't write everything out longhand the way that I used to. So I'm thinking that actually there is a version of me that is not maybe it's like not as good, but it's not the same version of me if I don't use my hands and type things out. And I should actually be writing out longhand more. But...

Yanqing Cheng (1:00:48)

Mmm.

Yanqing Cheng (1:01:14)

Well, I mean, that's the thing, right, about what I'm saying is, that software should become radically personal now. And so if you think what you want to do is write stuff out, get a wad of paper, write it out, take a picture of it and give it to one of the multimodal models, not Claude, because Claude's not so good at the multimodal stuff. But give it to Gemini or ChatGPT. They're both amazing at that. And you don't need to type it up if what you prefer is to write it down or draw a picture.

Anne Currie (1:01:22)

Mm-hmm. Yeah, that's a good idea.

Yanqing Cheng (1:01:45)

You could do it on a whiteboard and just photograph it.

Anne Currie (1:01:45)

That's a really good piece of advice. I will now go and implement. I've already implemented some of your advice because you told me to put all my novels into NotebookLM and that was really good. That was really good. And as a result, when the finale of the Panopticon series comes out, I will release the NotebookLM model of the entire Panopticon series side-by-side.

Yanqing Cheng (1:02:01)

Yes. Well, I mean, since I've advised you to do that, the models from the other companies have got a lot better. I don't know that this will work, but here's what I think you should try. You should give probably GPT 5.5, but Claude over 4.8 could probably do it if you put a high enough thinking level on it and ask it to carefully build you a wiki of everything in your novels.

Anne Currie (1:02:45)

Mm. That's interesting.

Yanqing Cheng (1:02:48)

And then you can just reference whatever you like.

Anne Currie (1:02:50)

I will ponder that. But we have now thought for quite a long time and I hope indeed that you will come back again and talk again. So thank you very much for being on the podcast.

Yanqing Cheng (1:02:59)

Yes, you have quite the editing job to do. Of course! Thank you for having me.

Anne Currie (1:03:13)

And thank you very much to all our listeners and viewers, including but not necessarily limited to twenty-five to thirty-five year old men who are very interested in Rust. But thank you. Thank you very much indeed. And I will catch you again on the next episode of Asynchronous and Unreliable Podcast.