Shownotes

Why even the most popular software platforms have to evolve. Java and the cloud with Holly Cummins

Join us for an engaging discussion with Holly Cummins, Java champion, as we explore the evolution of Java from its initial flexible, runtime-centric design to a modern cloud-native, operationally efficient platform. Holly shares insights into how the shift to immutable, start-up friendly environments like Quarkus is transforming how we build and deploy Java applications in the era of cloud computing, AI, and sustainability.

In this episode:

• The origins of Java's design for runtime flexibility and its implications today

• How Quarkus optimizes Java for cloud native practices by shifting work to build time

• The parallels between software system design and life management, emphasizing efficiency and sustainability

• The influence of AI and automation on software development and operational practices

• The importance of visual communication and storytelling in demystifying complex tech topics

• The evolving role of community and shared knowledge in maintaining secure, high-quality open-source projects

• Challenges and opportunities in balancing flexibility, stability, and efficiency at scale

• The potential future of AI and cloud-native paradigms coexisting and reshaping industries

Timestamps:

00:00 - Welcome and episode overview

02:20 - Java's original design for runtime flexibility and legacy assumptions

05:45 - The evolution toward build-time optimization in Quarkus

09:15 - System design parallels: life, work, and software architectures

13:30 - Visual storytelling and humor in tech communication

17:50 - Zombie servers and light-switch operations for sustainable cloud practices

22:10 - The impact of AI on productivity, costs, and system efficiency

26:40 - Community-driven open source: balancing security, quality, and innovation

31:00 - Changing industry constraints: logistics, logistics, and business constraints

36:10 - AI models, interaction empathy, and resource efficiency in large language models

40:20 - The human aspect: interactions with chatbots, mental models, and feedback loops

44:15 - Boundary-pushing through constraints: rapid code, innovation, and problem-solving

50:05 - Sustainability and energy constraints in AI and cloud-native software

55:00 - Closing thoughts: communication, humor, and future directions

Transcript

Anne Currie (00:24) So hello and welcome to Asynchronous & Unreliable, a new weekly podcast where we discuss the most interesting ideas and concepts in tech. I'm your host, Anne Currie, co-author of Building Green Software, the Cloud Native Attitude, and author of the science fiction Panopticon series. And today I'm going to be talking to Holly Cummins of IBM, who's one of the stars of the new Java platform, Quarkus. Holly is a writer, a speaker, a Java champion, and a Java One rock star. And today we're just going to let the conversation roam all over the place because the whole reason for me doing this podcast is to try and get myself back into what's going on. Well, actually, not just get back into but try and find out what's going on.

 What's everybody doing at the moment? Is everybody 100 % AI or what are they doing? If they are 100 % AI, what are they doing? What else are they doing? Just to try and run a little kind of distributed conference where we just kind of go, well, what's everybody actually working on? And so I'm quite interested in you because I'm very happy to talk about all kinds of quarkus things as well, because I will go over the story that I only found out latterly, when you came and spoke at the QCon green track that we did about why quarkus? Because quarkus was just built to be restartable. That was really interesting.

Holly Cummins (01:32) I think Quarkus is a really interesting example of how you can optimize something and it can be the right decision for like ten years and then eventually the world shifts and it's the wrong decision. And so part of our job as a technologist, I think, is to sort of keep re questioning those assumptions and say, Well, wait a minute, just 'cause we've always done it this way, should should we be doing it this way? 

because Java has sort of a reputation as, you know, well, it uses a lot of memory and it's really slow to start. And that's because Java had been very heavily optimized for environments where it didn't really matter how much memory you used because you had a dedicated server. So it would be wasteful to not use all of that memory and not leverage that memory to give you more performance. And similarly, you know, startup time didn't really matter because in order to start the application, you had to talk to the ops team. And obviously no one ever wants to talk to the ops team. And so startup happened once every six months. 

So it didn't really matter, you know, that it was a little bit slow. But of course now the world is very different. And and with the move to cloud native operation patterns, we ended up being much more elastic and starting and stopping frequently and we were doing much more virtualization or not even virtualization but Kubernetes and that kind of thing where we're having multi tenancy on the same piece of hardware. And then at that point, your memory consumption matters a lot. 

And so then the insight the Quarkus team had is so much of what Java libraries are doing, Java frameworks are doing, is deferring work till runtime. And like, well, wait a minute, actually, if we if we did that work at build time, then instead of doing it every single time we start, which is kind of wasteful, we could do it once. 

And the assumptions that the old JVM designs had was that, it was heavily optimized for dynamism. So you could be changing the engine while the plane was flying and you could go, I don't want to use, you know, I don't want to use the Oracle database. I want to use the IBM database. And I'm gonna change that like mid flight without taking my application down. 

But now you would never do that, right? Because we have CI CD. And so you would never make a major change to an application without running it through the CI CD pipeline. And so then in that sense, we can now much more think of applications as immutable and, in general, immutability is good for sustainability and good for performance. so you have that idea of, well, let's do more in advance. 

A lot of the people who were originally involved in Quarkus were Hibernate maintainers. And they sort of said, What if we made a version of Hibernate that would boot in advance? And so what Quarkus does is it has kind of runtime integration for various libraries, but then it also has these build-time integrations. And so what the integrations are doing is trying to move that work to build time where they can. and get rid of reflection, get rid of that kind of stuff.

Anne Currie (05:21) so one of the themes of this podcast is distributed systems and the kind of high level principles and it's a joke that we always make about how system design and life design are quite similar. You know, you can go, that isn't very efficient in real life if I do that. 

One of the things that you pay a lot for IRL is last minute flexibility. in life it is literally expensive. It costs you a load of money to get flexible tickets, but they're really useful if you need the flexibility. it's like in Java, Java was really designed for flexibility, as you say, runtime flexibility, the ability to run on lots of different hardware, not ever needing to turn off. And then the world changes around you. And then suddenly there's a huge, a bigger cost to that than there was previously.

Holly Cummins (06:22) Mm. I never thought about it like that in terms of the sort of the booking train tickets and booking planes and stuff. But you're you're absolutely right. And and quite often what ends up happening is that, you know, we don't actually need the flexibility, but we're just kind of a bit useless. And so we end up taking that flexibility. And I think that's exactly what had been happening with Java, you know, it it didn't it didn't need the flexibility, but no one could be bothered to to make that optimization.

Anne Currie (06:43) and it's such an excellent example of sometimes the world changes and you need to change. I used to write Java code 25 years ago, same as we all did. And it was a world in which a server would just be on for pretty much forever. And the whole point of the cloud, or cloud native when you're actually adopting the cloud techniques, and you said this in the talk that you gave for us at QCon a couple of years back, I was utterly blown away by it. Of course, why didn't I think that the cloud, all of the good stuff on the cloud, like auto scaling and things requires you to be able to turn things off and on again.

There was a TV show, the IT crowd, in Britain. And all the tech team said to do to fix things was always, have you tried turning it off and on again? But the irony is turning off and on again, it's really useful.

Holly Cummins (08:02) the thing with "have you tried turning it off and on again?" is that we're we're very accustomed to a world where turning something off and on again changes the behavior. And that's why we we tend to have two ways of thinking about it. So if the system is broken, we will turn it off and on again very happily. If the system is functional, we are not touching that system. 

We are not turning it off because it may never actually behave the same way again. And I think one of the things that Cloud Native is trying to go to is getting rid of both of those, right? So you have a system where you can turn it off and on again and that's not a cause for panic. But as well, there's more immutability built into the system. And so it's less likely to fix it as well, turning it off and on again, just because it's you know, probably been up for less long and and that kind of thing. And, you know, hopefully nobody SSH'd into the system and changed the state.

Anne Currie (09:04) that's true. That's not best practice anymore, everybody.

Holly Cummins (09:07) I know.

Anne Currie (09:11) But having said that, there was another thing that we've been talking a lot about on this podcast is it takes a long time for modern best practices to filter through. There are a load of folk who are still running Java in the old way where it stays up and you do SSH into things because you don't want to turn things off. So you do need to change things as they're going.

Holly Cummins (09:33) Yeah.

Anne Currie (09:34) It's surprising how long it takes to make that transition to the modern way of things. Like Quarkus is way of running Java where you can turn it off and on again,

Holly Cummins (09:48) yes. Quarkus allows you to turn it off and on again.

Anne Currie (09:55) But and in fact, you're not only are you allowed to, I think in the early days, it was you were allowed to, so you can use things like auto scaling. But I would say it's now more vital.

So I'm going to roll back a little bit and say, I first ran across you a few years ago when you were doing a lot of speaking about sustainability. I think you're one of the best communicators in tech because you use all kinds of interesting techniques to get a message across, you're very good at spotting descriptive phrases that can get an idea across very, very clearly. So you coined the phrase zombie servers to describe systems that are just sitting there running in your data centers, nobody ever turns them off. And they suck up loads of power. And in fact, I come across loads of examples of zombie servers like that, which aren't doing anything for you, but still running using like half the cost of your hosting

Holly Cummins (11:25) I should say 'cause I I do love making up words and making up phrases, but zombie servers isn't one of mine. That is one where I I got from there was an organization called the Anthesis Institute, which I for many years typed as the Antithesis Institute, 'cause of course why wouldn't you? You know, those two little letters that aren't there, I just didn't notice. But it's the Anthesis Institute and they did a bunch of research on that and they were talking about zombie servers.

Holly Cummins (11:55) the thing I love about the phrase is it's so easy to get the mental image. And it's fairly obvious as well that this maybe isn't something that you want. They took it down, because I think that's what marketing departments tend to do eventually, but Turbonomic had the most fantastic video about zombie servers, which again was before I'd done stuff on it, but they'd taken the Michael Jackson thriller video and they'd done it again with like a little cartoon style and they had these little servers going round and had the music and I mean it was just fantastic.

Anne Currie (12:27) but I would say that you were the person who popularized it in the tech industry because you talk to us so much and rightly so. You don't have to always make things up. Sometimes you just need to spot what's good and amplify that. you don't have to create everything. Sometimes you can just amplify it. But something else, I don't know, I think you did create this, which was another idea, which I love, which was light switch ops. Describe light switch ops

Holly Cummins (13:14) so there's sort of two parts to it. One is it would really be a good idea if you would turn things off and on again the same as you do with lights when you leave a room. But then the other part is what we were talking about before, that fear aspect, which is that there's a reason that in general we don't turn servers off, which is that we did it once and it was the worst day of our life and we had very awkward conversations with our managers. And so we just, you know, we have this fear.

And so as when we're designing systems, we need to design them so that they can tolerate being turned off and on again. And, you know, it's easy to do. And again, going back to what we were talking about cloud native, if you have those sort of cloud native principles in your design, which will give you lots of other good things, you will also then be light switch ops compatible.

Anne Currie (14:02) it's really interesting. I've just thought about this, so I might just say something totally idiotic and you're fine to correct me because what I want to encourage again with this podcast is say things and then find out whether or not you were right or wrong. You don't have to get everything right first time. You are allowed to say things. 

So it comes to me, and I could be wrong, totally wrong about this, is Java in the early days had flexibility, runtime flexibility, like the ability to run it on different hardware and all that kind of stuff built in. And that runtime flexibility can be very good for operational efficiency across your systems. And as you move away from that, there's another thing that's really good for operational efficiency, which is the ability to turn things off and on again, so that you can say, well, actually, it's not doing anything at the moment, turn it off and I'll turn it back on again when I need it, or I'll move it over here or I'll move it to this differently sized machine. Turning off and on again is another operational efficiency tool, isn't it? So it's interesting that Java went from, with Quarkus, it's moving away from one operational efficiency tool, but it's moving towards another operational efficiency tool.

Holly Cummins (15:17) I hadn't really thought about it that way, but you're right, like when we when we talk about sort of immutability versus flexibility and dynamism, it is not at all obvious that one is better than the other. Depending on the context, you know, there's really strong arguments to be made for either. And certainly when I first heard about immutability, this idea of like, well, wait a minute, so I can't change it and that makes it more efficient rather than clearly that makes it less efficient. But it's just that you get these sort of emergent effects. 

And I think, and you're right, that it's the same thing with how you manage your hardware. And do you have it immutable or do you have it patchable? Both have benefits. And I think where we are now, it's that the operational benefits of being cloud native are so significant that that really then pushes us towards the things that enable that, which is the immutability, which is the ability to turn it off and on again and that kind of thing. Twelve factor application principles as well.

Anne Currie (16:20) which is a it's an excellent example. It's all context dependent. the ability to turn off and on again massively amplifies the operational efficiency advantages of multi tenancy. For quite a lot of stuff, if you're just running on one machine, it doesn't really change things all that much, whether you've got the application on or off. But if it's multi-tenant, it's a complete change because somebody else can use that hardware and all those resources while you're not really using them.

Holly Cummins (17:01) And so many workloads are really, really either batchy or bursty in nature. And I think we often don't appreciate that because when we're using the system, the system is required. And so I guess it's like almost like object permanence, isn't it? That we don't really think about when we're not using the system. And like you know, loads of organizations Well, if they're if they're sort of office based and in one time zone, they'll have systems that are used when people are in the office and not outside of those hours. A lot of them will have jobs that run in a batch on the weekend and the systems stay up all the time. Or they'll have the opposites that you know they're not used at the weekend but the systems stay up. 

And then you get the sort of the more extreme examples. So the Netherlands Java user group use Quarkus as the back end for their application, and it's used like twice a year during their conferences. And that's it. So, they are doing the right thing and they have Quarkus and they're scaling up and down and that kind of thing. But there's this app that has so little usage for almost all of the year. And then it has this huge spike.

Anne Currie (18:12) so that's an extreme example of where actually it's probably worth doing even if you're on prem. So, obviously that's kind of your backstory of Quarkus and new cloud native operational efficiency and security as well, because actually things that are sitting on all the time will get hacked. You know, your zombie server is the number one thing that folk are looking for so they can hack your systems.

Holly Cummins (18:38) And it comes back to the sort of the CI CD aspect as well, because quite often the sort of the two kinds of neglect go hand in hand. The operational neglect of am I keeping this up just out of laziness? And the sort of the operational neglect of have I actually updated any of my dependencies in the last five years. 

So if you have that really good discipline of I have a CI CD pipeline, I have tests, I can safely update my dependencies and then I can safely redeploy, then that means that you get the double security benefit of this thing isn't even exposing any surface when I'm not using it. And when I am using it, all my dependencies are up to date. So my surface is, well, safer. It has fewer holes in it.

Anne Currie (19:29) It's really interesting that Java is one of the platforms that is keeping up with actually changing what it's doing as time goes on. lots of the popular platforms are. it is one of the things that comes up a lot when I look into efficiency and operational cloud nativeness is sometimes there's a platform that you start with who seem really switched on, like massively cloud native or they're massively efficient or they're massively green or whatever, but there aren't that many people using them. And then you get an old platform like Python or something that was absolutely terrible. But there are so many people using Python and there's so much investment in it and there's so huge a community around it that it just overtakes 

And so Python now, as long as you upgrade to the latest versions, is super, it's like 10x more efficient than it was, it's really, really improved. I remember chatting to somebody at AWS about this and they were saying, blimey, if only if we could persuade all our customers to upgrade from the version of Python they're on, which is a couple of years, which is maybe three or four years old to the latest versions of Python, then all our problems would be solved but it's really quite hard to migrate. So, it's a really interesting example of the power of a large community and a platform that is evolving. But you still have to be able to evolve along with the platform.

Holly Cummins (21:14) And then if you evolve too radically, you end up with painful upgrades. And if you don't evolve enough, then you don't get the benefit. And so it can be quite difficult to find that sweet spot. And I suspect sometimes there is no sweet spot. Sometimes you just have to choose your poison of are we gonna have the sort of the benefits of change or the benefits of stability.

Anne Currie (21:32) when I wrote Cloud Native Attitude about Cloud Native practices 10 years ago, and I say this quite often, I really thought that the benefits were just so enormous that everybody who hadn't adopted those benefits of CI, CD, and, you know, 10x efficiency improvements and security and resilience and all that kind of stuff would just be killed by all the people who had. 10 years on, nothing has changed. It turns out it's highly possible to coexist old practices and new practices. But my constant question at the moment, which I don't know the answer to, is is AI going to be the same as cloud native? If you're AI native, will a non-AI native company be able to coexist with an AI native company like cloud native could, or is there just no way that that can happen?

Holly Cummins (22:14) Mm. it's a fascinating question because we've we've sort of, you know, we had so much hyperbole about the benefits, and then we had at the same time all this research coming out saying we're not actually seeing that many productivity benefits. Individuals are reporting that they feel more productive, but then when you look at the organization as a whole, it's not. 

But I saw I saw a parallel the other day to older research about productivity and computers in general. And I can't remember the details, but in the in the eighties and the nineties there was all this research that said organizations are not actually any more efficient with the benefit of computers. And I always wonder how they measure efficiency and productivity in that in that context, but it does seem clear in the AI context that you can really make the case that you've fixed the bottleneck in one place, but the bottleneck is elsewhere. So you're not really getting an overall benefit. 

And what we're now of course seeing is that the conversation is shifting a little bit to cost of like, well, wait a minute. If I actually have to pay for my tokens and it's not subsidized by venture capital, this isn't quite as an efficient way of doing development as it as it initially seemed. 

And so that doesn't mean that there won't be organizations that are getting huge benefits from it and that are really sort of oriented around that way of working. But I think they will be able to coexist. I think we will move to that coexistence model where you can really sort of index on AI and optimize for AI, or you can say actually the other ways of doing it have some disadvantages and some advantages and so we find our niche.

Anne Currie (24:30) mean, it's interesting. Some companies are genuinely computing companies and you will need to go faster, whatever. If you're an enterprise with a big brand, if you're a long running enterprise, the value is really in your brand and all your processes around the work that you do, say you're a retailer or something. It will vary from company to company.

Holly Cummins (24:50) Mm. it's a cliche, but the pace of change is so fast that it's just fascinating. I'd love to sort of come back in a year and be like, Okay, what have we learned in the year since? 'Cause it will probably be quite a lot.

Anne Currie (25:11) Well, hopefully you'll able to come back in a year or even earlier on this podcast and we will be able to go, what were we saying a year ago? Is AI at the moment affecting what you're doing at Quarkus? Are you using it? Presumably you are.

Holly Cummins (25:18) We are, and so the debate that we're having internally a lot is how we digest the output of AI. and I think probably where we will end up at is slightly different than some other organizations. And we've been having this debate just in the general context of open source as well, of like, well, if anybody can point an LLM at a problem statement and say, give me a solution, then is there any value in these sort of shared projects? 

Because in the shared project, if you want a feature, you have to talk to people, which is obviously a terrible process. And so if you can, you can just go and you know, you can either fork it or you can just do a clean room re-implementation and then you can have exactly the features you want. Is that is that where we're going in the future? 

And what we eventually decided was I think that will be that now we're seeing that trend short term, but I think then longer term, we will start to reappreciate the value of these shared projects as we start to have more of a security conversation. Because the one of the values that we as maintainers are providing is that kind of I mean, we say gatekeeping and we usually mean it in a negative way, but it is that we aren't just sort of accepting these 20,000 line AI PRs and say no in it goes. We're saying we're quite happy to use AI to assist our development, but we are not happy to use AI to completely short circuit all of the humans in the process, which I think is the right thing in our context. 

And I think that's part of the value we provide. But I think if say we were just like an in-house project, I think maybe the balance would be different. And you'd say, actually, the cost of this review is so enormous that it's not worth it in that context. But because we are used by so many people, I think saying, no, we do want to maintain that architectural governance and we do want to maintain that quality governance. And so we do want to really be reading every line, even though inside we're dying, you know.

Anne Currie (27:53) it's interesting. The whole thing about will you want hyper-personalised? I've got quite a lot of people on the podcast who go, what I really like about this AI is the opportunity for hyper personalisation. I can do my own thing. That's exactly what I want. And you think, you totally love that. But probably most people don't. 

They don't actually want hyper personalisation. They do want somebody to go, do you know, I'll advise you to have this because I think this will probably do more or less what you want. This is what everybody's doing. 

When I was a head of IT in retailing, I learned a lot of interesting things about human behaviour. So when we were advertising something on the website, there were two different ways that we could advertise things that were really successful. The first was "new in" and the second was "bestseller". So, there were some people who really wanted the newest thing, the latest thing, the thing that's most unique to them and like they'll be different to everybody else. And there's another group of people and it turned out that was only really two groups of people when it came to customers. People who wanted something new that nobody else had, and people who wanted something that everybody else had.

Holly Cummins (29:12) Yeah.

Anne Currie (29:16) So, I think that, there's at least those two groups of people, that's not a bad way of dividing everybody up. the way we very successfully divided people, the way that retail as a whole pretty successfully divided people up.

Holly Cummins (29:35) I've done a few projects with retail, and you're right that there is a lot to be learned. This is slightly orthogonal, but one of the things that I really feel quite strongly about is sustainable pace. And that I mean, you know, obvious sustainability, but a different kind of sustainability is isn't it? And I was talking to a retailer and they were sort of they were doing really advanced logistics and measurement and optimization for their shelf stacking processes, and they found that if they would give people a sort of an appropriate target, at the end the pace would slow because the people were near the target.

And so then they had to, they actually had to I mean which I just I hated hearing this because it was so depressing and it was so, you know, you really want to believe the best and you want to believe that the systems that treat people the best way possible and that give the highest level of trust in people are possible. But in this context they were saying they actually had to have slightly aggressive targets because otherwise people would still fail to meet the reasonable targets.

Anne Currie (30:39) actually, I also used to work in an organisational psychology company, and they had exactly the same results, which is that you get the best performance out of people if they're constantly working towards goals that they can't reach. So they constantly fail. And you think, God, that's awful.

Holly Cummins (31:04) It really is.

Anne Currie (31:07) but that's what that is. the human condition of misery at work is that you're more productive if you strive towards a goal that you'll never meet endlessly. Well, it's not Sisyphean because, I guess it is Sisyphean.

Holly Cummins (31:21) I mean, I see that personally as well, right? Like, you know, all of my talks, are they written two weeks in advance of the deadline? Of course they're not. They're, you know, the morning I'm still frantically editing the talk because I delayed starting until the last possible moment at which I thought I could just about finish it.

Anne Currie (31:40) But that is actually quite a good way of motivating. I have to say I am also very motivated by deadlines. And as you get closer and closer to the deadline, it kind of boosts your imagination. I think it works. The thing is with all these things is are you a "new in" person or are you a "bestseller" person? And I think if you're a new in person, you're quite motivated by deadlines. I suspect it all kinds of goes hand in hand. If you're a bestseller person, all that's doing is grinding you down into a world of total misery. 

So that's the trouble with statistical monitoring of your workforce is that you lose the nuance.

Holly Cummins (32:25) And that is the exact difference with a modern cloud native operational model, which is you exactly want to be doing that statistical monitoring and you exactly want to lose the nuance and you don't want the snowflakes.

Anne Currie (32:28) that's true isn't it? Cattle not pets was a description we used to use about your servers, wasn't it? About 10 years ago. I haven't heard it in a while. treat your servers like cattle, not pets, which I thought was quite good.

Holly Cummins (32:44) Mm. Yeah, exactly. going back to the AI a little bit, making the most tenuous connection possible. Very individualistic, you know, kept alive for as long as possible, you know, lovingly patched and managed to keep treating them like giving them names, to the sort of the cattle model where they're, you know, very disposable.

Holly Cummins (33:21) And then now, of course, with LLMs, we're getting back to this sort of much more personal relationship with not necessarily our server farms, but certainly with our computers in general, where we, you know, we do have this companion who thinks we're great and love all our coding ideas.

Anne Currie (33:39) obviously, I also talk to ChatGPT and actually, Gemini and I quite like these days, found it a bit cold for a while, but I quite like it these days. You're constantly getting loads of positive feedback on all your ideas. I think that's probably good, because I think most people don't get a lot of constant positive reinforcement.

Holly Cummins (34:07) You're right. I never thought of it that way. Cause what annoys me about it is that it will say I because I know it is optimized or you know it is tuned to have that interaction mode. And yet when it says, what a good idea, I'm like, mmm and I get so annoyed at myself that even though I know it's a trick, I still feel validated. But you're right, I didn't think of it that way, that actually maybe this is something that we all could use more of and that, you know, workplace cultures can be, sometimes a bit antagonistic or whatever and having having that nice little friend giving you good feedback is good.

Anne Currie (34:46) I think it has good things and bad things in that. Well, it's interesting, isn't it? Some people have delightful parents that just say nice things to them all the time. And you can kind of see that they're quite well balanced. You know, some people do, some people don't. But you don't want to spend all your time with your parents who say nice things about you. 

But it is definitely causing an issue that people spend a lot of time talking to the chatbots. And it's more positive than your parents. Your parents job was also to train you how to interact with other people, whereas the chatbots are not really designed to train you on how to go off and interact with other people. So, they never want to let go, do they?

Holly Cummins (35:19) Mm. No. And because the worst the chatbots will do really is sort of quit. And definitely I get that where it sort of goes, We've been working on this for quite a long time. I haven't actually solved any of the problems I was supposed to, but I think we've had enough for today. It's like, what? But you haven't done it.

Anne Currie (36:02) So I'm going to guess from that that you're using Claude. because I've heard a few people say Claude has suddenly started to go, do you know, I'm a bit tired.

Holly Cummins (36:06) Yes, yes. There was some fascinating research from Anthropic, and I think it probably has to be taken with a pinch of salt, because I think anthropic tend to sort of anthropomorphize, as the name suggests, in their research. But it was it was just so interesting. I've been telling everybody about it anyway, because it comes back a little bit. I can just about weave this just about back to sustainability and performance because of course when we're using these systems, we want to make sure that we're using them in the most optimum way possible. And we've been looking at skills and that kind of thing and like, you know, how do they affect the costs? But for a while there was sort of a thing going around saying don't say please to your chatbot because it's just wasting tokens. 

And this research from Anthropic wasn't exactly saying don't say please to the chat bot, but what they found was that they were able to map the sort of internal states of the chatbots or language models to things that you could roughly map to emotions, and that internal state which state you ended up in, which would be affected by the input from the human, which makes total sense, and it would affect the behavior, which also makes total sense. 

And so therefore, you know, you sort of go one level up and basically if you're nice to your chatbot, it will behave differently than if you're mean to your chatbot. And so for example, they had a state that they could roughly map to fear. And so if you, if you went and you said, I've just taken an overdose of paracetamol, you would see it move into the fear thing. And the interesting thing there was as well, like depending on what dose you said, it would get like more strongly into that state. 

But going back to the programming tasks, if you gave it a task that was impossible, it would move into a state that mapped closely to despair, which again is exactly, you know, what what we can see, you know, that eventually it will just go like, look, no, this is too hard. I can't do it. I'm giving up for the day. But if you gave it positive feedback, so if you've sent the positive feedback in the other direction, and instead of going, no, this is terrible, this is terrible, you're not doing it, you went, No, no, I'm sure you can do it. You can do it. You have, you know, you're a very clever little LLM, it would perform better. 

Which again makes total sense because these systems are modeled on human interactions. And with humans, if you say rather than you're a terrible person, you can't possibly do this, I'm sure you can do it, one more push and you'll definitely, you'll definitely, you know, crack RSA, then you know, it would keep going.

Anne Currie (39:00) I always interact with the chatbots exactly as I would interact with a human with all the pleases and thank yous and it always seems to work very well. But I just assumed, as you said, that it makes sense if it's trained on human interactions, then how it starts will determine how it ends in the same way that how it starts with human interaction helps get what you want. Whenever you are working with someone or a chatbot, it's not just about programming the person to do the thing. It's about programming yourself, isn't it? That's like, you're talking to them and you're talking to yourself at the same time, aren't you? It's like, you hear what you said and it helps to get your own mind in the right place.

Holly Cummins (39:50) Mm. Which is why that often you don't actually need the colleague, you just need well, historically it was the rubber duck and now we have the much more expensive but much more effective LLM rubber duck.

Anne Currie (40:07) Indeed. So the interesting thing about an LLM rubber duck versus a rubber duck is that the rubber duck was quite effective because it didn't take you off on any side roads. The LLM will take you off on a million side roads, which are all very interesting...

Holly Cummins (40:19) And I guess the other sort of difference is that because of how they operate with LLMs, you can be so vague and you can say, I just want a thing, and then it will sort of fill in the gaps. Whereas with the rubber duck, in order for it to work, you had to be quite precise. And that was what sort of made you realize the problem was the process of explaining it and then you sort of go halfway through and you'd be like, wait a minute.

Anne Currie (40:51) so it's interesting. When you're an artist, you're a very good cartoonist, which is part of your excellent communication style is that you're very visual. So I guess I guess this works with you like zombies because it's quite a visual thing. And you like the light switch ops because you use always use a picture of a light switch. You like things that are visual and a lot of your on stage communications are very visual. You hand draw all your own cartoons and things.

Holly Cummins (41:18) Mm.

Anne Currie (41:21) But so one of the things I also paint and I always think with it, it's a kind of almost a trope with creative subjects, with creativity, that you kind of zoom out and then you zoom in. That you zoom out to kind of think, where am I? What's the big picture here? How am I going to describe this? What am I trying to do? lots and lots of different ideas are coming in when you've got something

Holly Cummins (41:33) Mm.

Anne Currie (41:46) that's creative coming on, and you're going, oh, that's interesting. I'll think about that. And then, but then to actually do the thing, you have to zoom in. So at that point, so everything becomes part of a, is this part of zoom out or is this part of zoom in?

Rubber Duck was certainly part of zoom in, because it wasn't giving you any new information. It was just helping you to process what you had and pull out thoughts that you could then dive into and go, all right, all right, now that I've said it to the rubber duck, I realize that actually I should be concentrating on this bit. 

But talking to an AI is still part of zoom out, because it gives you 20 million things that you could then do. There isn't really a zoom in. Sometimes there can be a zoom in with AI, but it's a bit more it's trickier to do.

Holly Cummins (42:26) And then that sort of zoom out breadth, I think, is one of the things that it is proving to be really valuable at, isn't it, of like, give me five options for fixing this. I hadn't thought of number three.

Anne Currie (42:46) absolutely. it's completely changing the balance between is the value in your ability to define the problem, or is the value in your ability to come up with solutions. we've always overvalued the solutions over problems. Whereas in fact, there's 100 solutions to any particular problem that are context specific. So it's really important that you understand the problem and the context. The solution is the easy bit.

Holly Cummins (43:19) I like that.

Anne Currie (43:23) I remember I was interviewed by CEO of a company many years ago, the founding CEO of a company called Figleaves they used to, we were one of the first online retailers, pure play online retailers.

Holly Cummins (43:38) They used to do pants, didn't they? Yes.

Anne Currie (43:47) we used to do pants by post basically and all kinds of underwear and I always feel this is interesting. The reason why the company chose underwear because it was an engineering-led company the reason why the company chose underwear is because it was one of the few high-valued products that you could put in a box that would fit through a letterbox because the main constraint in your system was getting the product through the front door. 

So you needed something that was high enough value for it to be a company, whilst at the same time being small enough that you could put it through a front door, but also not so high value that you wouldn't send it through the post. Like, you're not gonna do diamond studded jewelry by post, because that would be a bit of a risk. 

I always suspect that Amazon sat around and made the same decision about "what is our constraint?". It's we're going to sell something that will fit through a letterbox because of course books also fit through letterboxes.

Holly Cummins (44:53) I mean, I always assumed with Amazon as well, because it was so it was so the early days of of the internet, and the idea of buying something that you couldn't try on and touch seemed madness, but people were comfortable with the idea of books because they didn't need to actually touch the book before buying it. And then of course, once it was sort of a thin edge of a wedge, wasn't it, that once people got comfortable with books for e commerce, then it expanded to things that historically I think you would have really wanted to see in a shop like Underwear.

Anne Currie (45:23) Figleaves was launching about the same kind of time. Surprisingly, people would buy underwear by post, just looking at the picture. It was, I think, because I was so involved in so this wasn't this story that I was going to tell, but now we've gone down this route, we can talk about logistics and constraints. 

I think a lot of people underestimate the amount of logistical thinking that goes into these big companies, particularly like Amazon. I know because it was engineers who were behind figleaves.com and I knew the guy who'd invented the whole thing very well and he was very logistically minded and he was going "what are our logistical constraints on this? How are going to deliver it? How are we going to get it into the home of the purchaser?"

I suspect that Jeff Bezos was also either logistically minded or surrounded by logistically minded people. And I could easily imagine the same kind of thinking going on. to a certain extent you suck it and see, you say, well, if I sell this online, will customers buy it? And you right there's no reason not to buy a book - you literally judge it by a picture of the cover.

Holly Cummins (46:22) Mm.

Anne Currie (46:44) You only judge by the cover anyway. And so, you know, it makes it makes logical sense. But I strongly suspect that they have a logistical core at Amazon. And they went in the end, this has to get through the front door. So can't be any bigger than your letterbox. But anyway, that's that's kind of it. What you think is the constraints on a business when you're going in is often not the actual constraints on the business. And sometimes you don't know what the constraints are going to be until you try it. And then you go, that wasn't the constraint. That was the constraint I thought it would be. But the actual constraint is a different one. Now we've got to change and optimize for that constraint. 

Holly Cummins (47:33) and it that can sort of lead us in in two directions. One again is sort of back to to benchmarking and that you know where where you think the bottleneck is isn't the bottleneck. But then at the sort of the the bigger human level as well, the constraints can be so productive. I saw a wonderful talk last week at J Spring and he'd set himself the challenge to write a game of snake in ten lines of Java. Which was obviously ridiculous and unachievable. And by the end he'd managed to do it, you know, without making it just one two thousand long line of code as well. He had sort of a a hundred and thirty column widths and he'd got a game of snake in ten lines without any external libraries that weren't part of the JDK. But he was also looking at sort of the six word stories and that kind of thing. And if you if you ramp down towards the concision, you can get some really, really outstanding meaning, much more so than you would with the sort of the bigger flabbier thing.

Anne Currie (48:40) So it's interesting with constraints that it's both you need to eliminate them if you're looking at the theory of constraints. But constraints themselves are such a useful tool of like deadlines for me are like constraints. They are constraints. That's, you know, iwhat can you do within the next five minutes? It's quite useful.

Holly Cummins (48:44) and you can achieve a lot. And then I guess it's sort of with sustainability, you know, historically we have completely not treated energy consumption or carbon or anything like that as a constraint. And if we start changing our systems so that they are more of a constraint because of legislative governance or because of consumer demand or because of impending climate apocalypse and self interest, then then, I think we probably will start to achieve some pretty amazing things in terms of things that we thought we had to do in a very inefficient way. Suddenly we can do in a much more efficient way.

Anne Currie (49:47) it's interesting. In AI, because this is what I was talking about on stage the last time we met when I was talking at DevOps. I would say that AI has been totally transformed by what was a constraint meant to hurt things. It's really helped AI generally. know, Jevons Paradox, has it helped sustainability or not? That when Biden

Holly Cummins (49:53) it was such a great talk that way, that one by the way.

Anne Currie (50:16) said that China couldn't have the fancy Nvidia chips. New chips. China started investing in code efficiency for AI algorithms. And they used two tools: they used code efficiency ( better algorithms). And they also open sourced it which is another tool. And they brought people all over the world and said look, go for it. Here's the thing, try and come up with new ways to make this more efficient. 

And they became 10x more efficient within about six months, and then another 10x more efficient, and then another 10x more efficient. And they dragged all the pioneer models in the US who aren't all that interested in saving money, because money is their secret sauce. They dragged them up. So they're now 100 times more efficient than they would have been. 

So the situation in AI, if it's running up against a barrier of energy now, it would have been running up against this barrier on far less effective code or a lower value, lower quality of AI quite some time ago. If it hadn't been for the constraints that Biden artificially imposed on China, that then everybody else went, no, now you're 100 times more efficient. We'd better keep up with you.

Holly Cummins (51:21) Mm. Hehehehe

Anne Currie (51:42) I think DeepSeek is still 10x more efficient, but they are dragging everybody up into efficiency.

Holly Cummins (51:51) And I wonder whether we'll see some new patterns emerging sort of at the user level as well soon, because we sort of had this conversation shift, haven't we? Just like in the last few months from token maxing and, you know, organizations which measure their employees by how many tokens they're consuming, which is I mean, I think we don't need to say why that's a bad idea. But it does sort of, I think I think there was a reason why it was done, which is sort of, you know, in the early stage of a technology adoption you want to explore. 

And then after that you then sort of go towards a more optimization stage, which I think is what we're entering in. Of like, hey, if I have a limited budget, how can I use my tokens as efficiently as possible within that budget to still meet my employee objectives? And so then that means that maybe we're gonna start to because at the moment switching models is really clunky and surely, surely, this is something that should be a basic in our system of maybe let's not use like I saw a quote that was, you know, let's not use the Lamborghini to mow the lawn

for some things we need this, for some things we do that. If we rely on people to manually switch, it will get done five percent of the time because people are terrible at that, especially when it's something like Hey, I want to use the less effective model. Well, no, I don't want to use the less effective model. I want to use, you know, I want to get my job done as quickly as possible.

So can we sort of start to build in the affordances and that kind of thing into these systems to make it more efficient in its model choice and more efficient in various other things as well, of sort of the like we're now sort of a lot of us are starting to experiment with skills. And then there's sort of the question as well, did my skill actually improve my efficiency or did it worsen my efficiency and it's very difficult to measure. So that's the sort of I think another sort of frontier as well of sort of how do we know if we're making it better or worse with these systems?

Anne Currie (53:56) that is true. Now, this has been an excellent conversation. We've been talking for nearly an hour and I'm not going to be cutting pretty much anything at this because it's been a really good conversation. So I might say, should we cut it short so then you can move on to the rest of your day? But also, I'd like to be able to have you back in future to continue our discussion, especially as more stuff comes up.

Holly Cummins (54:22) Yes, I would love that.

Anne Currie (54:24) Excellent. That's good. very pleased to hear that. I need to get you to send me one of your excellent illustrations that you just have lying around. And I will pop that in the thing so that some people can see what you do, which I think is really good in when you're giving talks and communicating things. One of the other themes of my podcast is how do we communicate difficult subjects? you do it in two ways that are fairly unique to you. One is you're very good at spotting the phrase that kind of allows people to visualise the complicated concept. And you also do cartoons that allow people to visualise the complicated subject. So I really appreciate that. That's very good.

Holly Cummins (55:08) There's a a third bit as well, which I quite like, which is the sort of the humor aspect. So zombies, for example, I think one of the reasons I love talking about it is it's the funny side of the climate apocalypse, you know, because it's so stupid and we can all recognise that we have done these stupid things. And as soon as you say it and you're 'cause like there was a story of I think, when after the Twitter acquisition they found like seven hundred GPUs that had been powered on for like years and were just not even being used at all. And you know, it's just so obviously stupid and you go, wait a minute. Let's fix that now that we see how you know.

Anne Currie (55:40) Humor as well. Yes. I almost forget that because I use that quite a lot as well. We're very keen to use that in building green software. Jokes just get people through it. You know, you can't communicate difficult things in a po-faced way, can you? everybody just turns off.

Holly Cummins (55:50) Mm. it doesn't have to be earnest. It can be it can be fun because this we're doing good things. We're saving money. We're saving the world. But do we don't have to do it while looking miserable.

Anne Currie (56:15) Indeed, that in many ways sums up the whole of the communication style we both prefer. You know, you can communicate difficult subjects without looking miserable or without being miserable. So I'm going to say thank you very much indeed. It's been an absolute delight to have you on the podcast.

Holly Cummins (56:18) Ha ha ha. Really good fun. Thank you.

Anne Currie (56:40) And thank you to our viewers and listeners. And hopefully I will catch you again on a future episode of asynchronous and unreliable. Thank you very much.