Shownotes

Understanding Logarithmic Scales in Tech: Efficiency, Resilience, and AI Impact

Explore how logarithmic scales affect decision-making in software efficiency and system resilience, and what AI means for future tech growth and costs.

Main Topics:

• The importance of logarithmic scales in measuring code efficiency and resilience

• Trade-offs between higher resilience and cost

• Challenges in reasoning about exponential differences

• The potential of AI to reduce resource needs and unlock efficiency

• The impact of AI on data center requirements and system performance

In this episode:

• Why code efficiency differences can be a million times, not just a few, and what that means for developers

• How system resilience (measured in nines) increases exponentially in effort and cost

• The difficulties humans have in intuitively understanding large exponential numbers

• The changing landscape of AI and its potential to reshape resource and cost calculations

• The importance of matching system resilience and efficiency to actual business needs

Transcript

Anne Currie (00:00) Hello and welcome to Asynchronous and Unreliable, a new weekly podcast where we discuss the most interesting ideas and concepts in tech. I'm your host, Anne Currie co-author of Building Green Software from O'Reilly, the cloud native attitude and author of the science fiction Panopticon series. And today we're going to be talking about efficiency, resilience, logarithmic scales and strategy and how AI fits in into all of that. And for my guest, have code efficiency and networks expert, Jon Berger, who's been in tech for over 30 years, creating, growing and leading organizations that build mission critical software. That's high performance and high resilient software. And that's going to be a key thing that we're going to be talking about today. So do you want to say hello?

Jon Berger (00:43) Hello, thanks for having me back.

Anne Currie (00:46) So today we're to follow on a little bit from the last podcast that we did, where we talked about how efficiency, operational efficiency and code efficiency were on kind of two different scales . code efficiency, you really need to talk about on a logarithmic scale, whilst operational efficiency, you kind of talk about on the linear scale. And that makes a huge difference to the decisions that you're making as a business leader, for example. So, and it's really key that as a business leader, you understand that difference. And what I mean by that is, it's quite common in the industry to have someone who is bad at operational efficiency being 10 times worse than somebody who is absolutely amazingly good at it. Whereas in code efficiency, the difference between somebody who has inefficient code and somebody who has efficient code can be a million times difference. That can, both of those can be very good, perfectly acceptable code it's within the acceptable bounds, totally within the acceptable bounds to have a million fold difference in how efficient it is, which is not the same as how good it is. Well, one of the things we're gonna be talking about today is it's not the same as how good it is. But today we're going to talk about things that are on a similar logarithmic scale, which means that there's so much variation in what is an acceptable level of performance in that area that really, if you're going to plot performance you'd have to use a logarithmic scale, not a linear scale. So the reason why I've got you in to talk about this, Jon, is A is something that you and I, for listeners who have not listened to our previous podcast, Jon is as well as being a code efficiency and a network and a resilience expert, also my husband's a very useful, we talk about this kind of stuff all the time. But this is an area that you know a lot about, because you've been managing teams delivering software that is at the very extreme end of code efficiency and resilience.

Jon Berger (03:07) I mean, the first thing to say about this area of logarithmic scales and these big numbers that we talk about, millions of operations per second or billions or whether you're talking about something being a million times more efficient than something else, that humans are absolutely awful at reasoning about large numbers. Everything in evolutionary terms about the way we think has been set to say 10x is about as much as you can reasonably imagine. Maybe 100 is still within the realms of sensible. Once you get to a thousand, 10,000, a hundred thousand, a million, it's really difficult. The problem that you face is that your intuition stops helping you. So you need to use something else. And you've actually got to use a spreadsheet or some other tooling to work out what you're doing.

Anne Currie (04:11) Yes, but what I want to talk today about is things that should be considered as being on a logarithmic scale, even though humans are terrible at understanding logarithmic scales, and things that in the tech industry are not on a logarithmic scale. Because we don't talk about that that often. So just to restate that, so code efficiency is on a logarithmic scale, it's kind of exponential differences and we're not talking about people write terrible software, people write perfectly reasonable software that is all still on an exponential scale. When you quite often see memes go around about how you know C is 100 times better than Java or something like that. And those things are true, but that doesn't mean that Java is bad. Java is just there for it just has a different use case. So you've got things like Python are less efficient than things like Java are less efficient, often 10x less efficient than Rust or C, which is maybe 10x less efficient than tuned or CPU tuned and caching tuned code written in Rust or C. And all of those, none of those are bad things. It's not a difference between bad and good. They just happen to be more efficient. More efficient is not necessarily good, except in some circumstances where you need efficiency. The other thing that you know very well, you know a great deal about, is you used to also deliver software that was high resilience. Is the resilience scale. So most people are very familiar with the idea of one nine, two nines, three nines, up to five nines or even six nines code. But we haven't, we don't really grok it. We haven't really built that code ourselves, but you have, you've worked on products that are up to six nines. And it's, do you want to tell us a little bit about that? And also we'll talk a little bit about why six nines code is not necessarily better than one nines code. It's just different.

Jon Berger (06:29) Yeah. So by nines, we mean as a percentage of uptime availability. How good are they? So a service that was always available, always, always, always would be 100 % available. Software has always in the past been written by humans and therefore is not 100 % available. There are problems with it. Maybe those are hardware problems. The hardware goes down, maybe your network goes down, maybe your product just falls over because it has bugs or for whatever other normally bug related reasons, it can't service the incoming requests that it's supposed to. And you measure that percentage and one nine would be 90 % available, two nines would be 99 % available, three nines would be 99.9 % available, et cetera. In the area of telecoms, when you're dealing with services that people, at least in the past, had come to expect to essentially be always available, picking up a landline, you just expect the dial tone to always be there. When you call 999 or 911 or 112, depending on which country you're in, you really want that call to go through. Turn it off, turn it on again is not really what you want to hear in one of those emergency scenarios. And so that's where people come to talk about 5 9s, 6 9s reliability and in the telecom space those were products we were selling at that level. And we were saying to our customers and delivering and we were measuring that, that we were hitting that six nines level, which is seconds per year of downtime. The reason why I think you're talking about that being on an exponential scale is not because of the difference between 90 % and 99 % or 99.9 %. You know, the difference between 99.99 and 99.999. But the difference in effort you have to go to to engineer such a system is enormous. Again, people have a rule of thumb that every nine you add, it's 10 times more effort. Mean, again, that will be true in some areas, not true in others. I think the important point is not that it's an exact number of it's 10 times harder, but it probably will be a lot harder and it will be a lot more expensive. And do you actually want to go to that expense is a key question.

Anne Currie (09:33) Yeah, it's interesting because one of the things with nines is that people, because it's quite catchy, people know about it, they've heard about it. They always think that more nines is better software than less nines. But in reality, there are an incredible amount of tradeoffs associated with going to more nines. And if you go to the wrong number of nines for what you need, then you'll just be spending huge amounts of money and time, engineering time, and maybe you don't have the folk to do it because this is different skills to go to that high end. But I hear over and over again that a major issue that companies have is saying, we need four nines when in fact they quite often just need two nines. And so you're setting yourself up for a lot of cost and a lot of failure. But what's your thinking on that?

Jon Berger (10:31) Yeah, I think people find it difficult to reason about this and because it's difficult to reason about, people instead of answering the question that's in front of them, substitute that question for a different question, which sounds quite similar, but is much easier to answer, which is, you want it to be more available or less available? And if you ask someone that question, they don't have to think very long before they go, I think I'd like it to be more available, please. It's like, OK, how much are you willing to pay for that incremental amount of availability? And that's a much harder question. Or what sort of unavailability matters to you? Do you still need to cope with your clients' devices being unable to access the internet over Wi-Fi? Well, that's a completely different thing from Do you still need to handle services if Amazon US East goes down? And actually, if you think about it in terms of where you're answering the question, well, here's a specific problem that might occur, that's quite likely to occur. Do we need to worry about it or not? Well, that is much easier for people to go, yeah, that's the sort of thing we need to worry about. The advantage of thinking about it in those terms is it also tells you about the design choices that you're making. Because the difference between four nines and five nines does tell you a bit about design choices, but, you know, it's actually pretty hard to translate it directly. Whereas if you're going to say, look, if most of the Internet is down, it doesn't matter if we're down as well. That's easy.

Anne Currie (12:28) So I'm gonna ask you, I'm just gonna clarify something you said earlier, which I think is a very interesting point that's often missed. And I know you said this many times before in around telecoms, which is people often miss out that they'll look at what the availability of one component of a system without seeing that it's connected. The system is in another component that's much less, that has a completely different resilience pattern. So, for example, the expectation of five nines resilience for dialing 999 or 911 if you're in the US emergency calls was based on an expectation that you've got system software running in your telecom provider that needs to be five nines. And then people access it through their phone, which in an old, for old style phone was generally available. Was, you had quite a high level of availability on the phone. Whole system was the phone plus the telecom server. But if people are accessing it through a mobile phone, which may or may not have access to the internet, then you're going to a heck of a lot of effort to do five nines resilience to the telecom system, but it's now connected in a system to a phone that maybe only has a two nines resilience. And then what's the point of going for five nines in one half? Well, you've only got two nines in the other half when they're both necessary halves. Is that kind of what you're saying?

Jon Berger (14:00) Yeah, I mean, I wish my phone was, you know, two nines. I would think it's probably in terms of if I stay in the house, I'm sure I get two nines or better. But if I go out and about, I definitely don't. I was out only earlier today and in the local town. And yeah, I was in a population center and I didn't have any connectivity there. So I think it is a tricky one in that environment. Does it matter if I can go online and go shopping? No, that would be a ridiculous requirement to place on your system. You have to separate out the availability of the back end and the front end there. Assuming the front end is capable of connecting to the back end, then we want the back end to be there to service it. The way a lot of most countries have legal requirements now on their emergency services systems, that they are available. No one's going to come in and say, wow, we need to downgrade the 999 service. It's OK if it fails a bit more. I don't think a politician would become very popular if they did that. And it's probably correct It's like people have got to accept that if they're somewhere where there is no internet access, then they can't access the internet.

Anne Currie (15:50) They do accept that. It is interesting, they do accept that. Oddly enough, the world does change and people's levels of expectation for performance and availability do change over time. And sometimes parts of the industry don't spot that that's happened. They're still chasing an old expectation that's no longer the expectation. I was talking to somebody from a networking company recently, was pointing out that AI has changed people's expectations of how patient customers are because AI quite often takes quite a long time to answer your query. And it has changed how patient people are because they used to go bananas if an image took more than half a second to download. But now everybody's a little bit more used to waiting because they've been waiting for AI. But anyway, so what you're saying there is there are valid things to wait for and not valid things to wait for. Interesting thing, so resilience and code efficiency are both things that are on a logarithmic scale. What I want to take away from that is that means you need to think very carefully about how good you need it to be. There's not well, I'm also suffering from that thing of saying it's good or it's bad, how resilient it needs to be or how code efficient it needs to be. And then target that's not more than what you need because you're spending your money totally fruitlessly. But that has been the truth up until now, you have to really understand your business and you have to understand the whole system that your code is playing a part in. There's no point in optimizing one system to five nines whilst your users access it through a system that's only two nines. You're potentially wasting your money providing five nines.

Jon Berger (18:00) I think you've always got to, if you're putting that choice in front of someone, you firstly never put a one dimensional choice in front of someone. Make sure you're always putting both sides of it. It's like, if you're choosing this, you're choosing not this. You're choosing vastly more reliable. You're choosing, I don't have time to do all of these other things. Is that what you're choosing? The cost goes up. The ramifications of that cost going up are I'm not going to get to the two numbers two, three, four and five in your list of things you want me to do. Is that the choice you're making? So make sure that the people are making an adult choice, which is a genuine choice between two things. Or do you want it better or not? Which is not helpful. The other thing is, I guess maybe it comes back to challenging your question. You say these are on a logarithmic scale. They certainly are plottable on the logarithmic scale. Is it actually unhelpful to do so because it makes the two things look more similar. And if you plotted them on a normal linear scale, people would go, Oh, well, that's, that looks like it's 10 times more expensive, but on the other thing, and he looked twice as expensive. Maybe that would help people to think about the choice they're actually making. You still couldn't reason about it because I still don't think humans can reason about thousands of times difference. But you would at least have a visual clue which went, I can't reason about this.

Anne Currie (19:48) Yeah, but even though we don't properly grok the fact that resilience goes up in expense the same way that code does up in a kind of like exponential growth, we kind of know it. Because people try and do things that are five nines or whatever and then they fail or they go, this actually is costing us way more than we're expected or they just say it often enough and people might believe it. But in the future, one of the things that AI is going to bring in is we are already starting to see AI deliver high efficiency code at speeds that make something that would have been exponentially time costly into something that is linearly time costly. What are you thinking about that? Do you think that's true? Do you think that's false?

Jon Berger (20:56) I think the pace of change in AI writing code is pretty fast at the moment. So this might all change in the week it takes to publish this. And one of the things that isn't yet clear is because all of the AI companies are still evolving their business model, it's not clear when this settles down, what the actual cost to you is going to be. So, again, if you're saying to instead of going to your development team, you're going to your AI and saying, build me this product. It might say that's going to take me 10 times longer instead of one second. It will take me 10 seconds. You're going, okay, I'll go at 10 seconds. But if it goes, instead of $10,000, it will cost you $100,000. Maybe you still don't care, but like you probably have a different way of reasoning about that sort of question. So I think that will change it. I think it will depend a lot on what business you're in or what you're optimizing for. How often do you need to rewrite that code? What does maintainability look like in a code base that is maintained entirely by AI? How much are you limited by the cycle time might be the limiting factor that you care more about?

Anne Currie (22:44) Yeah, it's a difficult one, isn't it? This is not just like puff. This is engineers that we're talking to who we know and trust saying, I'm just seeing some crazy changes in how I write code, what kind of code I can write, how much code I can write. They're rolling out things that seem like exponential change. That is maybe unlocking something exponentially costly in tech at the moment. There is demand for five nines resilience because people ask for it all the time. Now, we limit it, we say don't have it because it's too expensive. We ration it. But there's demand for it. People would love it if you could have it. And code efficiency. Applications running 10,000, 100,000 times faster and more cheaply. People want it, but it's rationed by its availability. So the question is, is AI going to unlock that availability and unlock that demand?

Jon Berger (24:02) I'm sure it will in both cases. And just building lots more data centers makes it easier to have a lot more physical locations in which you can run that code. That improves reliability on its own.

Anne Currie (24:31) Yeah, there's an interesting twist on that, which is this year, particularly, a lot of data centers that were planned and not being built because they're being rejected for all kinds of reasons. And it's an irony that that is paired with the fact that AI is getting to the point where it might just be able to make that code that is AI more efficient. That AI takes fewer CPU cycles to deliver the same value, in which case maybe we don't need so many data centers. But it's an interesting time to be living from that.

Jon Berger (25:11) Is that an indicator that the AI's ability to write code is not yet good enough that it can just rewrite itself to run vastly more efficiently? Or is it that the other things that people want it to do are more important than that and therefore these companies are happy to throw billions at building new data centers? Probably is a combination of the two.

Anne Currie (25:44) Probably is, but also I'm gonna come back to something that Sara Bergman and I discussed in the first episode of this podcast, which is that I hope that everybody knows that code efficiency can make things a lot more efficient. It is known in large proportions of the tech industry that code can be 100,000 times, a million times more efficient if you choose for it to be and you invest in that. But do people really believe it? Mean, most, the vast majority of people I meet do not even believe that, I would say.

Jon Berger (26:31) No, I think a lot of people don't believe that and they don't have they've got no direct experience of that themselves. They don't know what their Python code looks like when it's compiled into machine code and how it could be more efficient. They don't know where they're exchanging information across a network where they don't need to. For most of them, like the trade off wouldn't be worth making. Even if it was just as cheap as saying, AI, when you build that code, can you make it run 10 times faster? Or you run with 10 times fewer resources. That wouldn't save them enough money or it wouldn't gain them enough for whatever value that they're looking at, that it would necessarily be worth doing in a lot of cases, I think.

Anne Currie (27:35) Yeah, it is interesting. So you think people know, but it's even then. So the question is, will AI make it so much easier for them to do that, that it's worth them doing it?

Jon Berger (27:52) Yeah, exactly. Mean, at the moment, awful lot of what people see operating is on a phone or a tablet. So it's on a local device. If it ran 10 times or 100 times, using 100 times fewer resources, I couldn't get rid of 99 % of my devices. I've only got one device at the moment. So if its battery life is pretty good, that's all that matters to me. It lasting a little bit longer doesn't meaningfully help me. It's not typically traced to just one application anyway. Is there any economic benefit to me as the person that owns that code? If there's a difference between you can do this, you can't do it, you can do it, but you can either do it with your plugged in or you can do it wherever you like. All right. Those are meaningful differences. But if it's just something that either way, will take me five seconds at 100 % CPU or five seconds at 1 % CPU, does that matter to the user? Most of the time, no. Therefore, it doesn't matter to the developer either.

Anne Currie (29:22) No, I agree with you there. I'm thinking more about kind of like stuff that's running in AI data centers. AI models. There, the limiting factor on AI data centers is electricity, is hardware. People are trying in the tech industry to build AI data centers and finding it tricky and difficult to do that. So I interviewed one of the senior folk at Cisco who you actually used to work with, Christopher Liljen stolpe. And he was saying that we're now absolutely waiting for the software engineers to just to make the models better because we're very aware that these data centers that we're building are because the models are just quite inefficient. Even things like DeepSeek which might be a hundred times more efficient than a lot of proprietary models. They haven't got anywhere near as efficient as they could get. And once that problem is resolved, then maybe you won't need all these data centers. You might only need a tenth of the number of data centers.

Jon Berger (30:39) Yeah, I think that's right. Think people who are building these AI models are still very much in land grab mode. People are spending a lot of time worrying about what if AI is like it is now forever, what does that mean? And the question is irrelevant because the capabilities change every week. So think you've got to look at the trajectory. The trajectory is obviously very, very positive in terms of the abilities growing over time. And so people are saying, well, AI is only ever going to be able to do more than it can now. And it's already pretty useful in lots of places. Therefore, even if at some point in the future, the code gets 10 times more efficient and I only need 10 % of the data centers, the demand for it will go up so much that I'll still use all of those data centers. So it's still worth building them. I'm not going to lose money on doing that. So it's still a good bet. I do think there's a lot of gains still to be made. There's still an awful lot of the hard work has been done in Python because it's much easier to write. And while you're in land grab territory, you're not in optimization territory. You're trying to grow your top line. You're trying to get in new customers and you do that by adding capabilities, not by making your own operational costs lower. And so while people are starting to do that, it's still not the most rewarded thing at these companies. When you have an AI use case where the AI is already good enough to do everything you want it to do. That is where you go OK, I don't need to run the latest frontier model to do that. This is already good enough. Let's optimize this. Let's make it cheaper now.

Anne Currie (33:00) Which is kind of what DeepSeek launched their first model at the beginning of 2025. Well, actually, we're not gonna add any new functionality, we're just gonna make it cheaper. And it was kind of a toe in the water to see whether or not there were people who were interested in no new functionality, just cheaper, and it turned out there were. And that was quite an inflection point for the industry in terms of code efficiency of models. And DeepSeek is still the most code efficient model.

Jon Berger (33:32) Yeah, and there's enormous amount of effort is now going into whether these models are actually any more code efficient, but they do operate on a much smaller share of resources. The ones that work that are distilled models are essentially trained on the output of the higher end, more complex models to do something that's within their capabilities. And I did read someone recently saying, well, yeah, this is completely different from the Internet because once you could do something on the internet, you didn't need to build out any more hardware, whereas because AI just gets better, the more hardware you throw at it will always need to throw more hardware at it. And that's only true for the use cases where infinitely good is useful. Maybe this is another how good does it need to be? It's on a logarithmic scale. If I'm asking for help with my school homework, I don't need Albert Einstein to be answering that question. I can settle for a way lower fidelity, quality, answer than that. So it's already the case that people are used to the idea of I need something better than I can do myself. I'll go to some expert or I'll go to a less expensive expert. The same thing will be true for AI. OK, how good do I need this thing to be? OK, I'll pay for that.

Anne Currie (35:15) Well, it's an interesting one, isn't it? If I want to go to a specialist, there are many things that they aren't like 10 times better than me. They're infinitely better than me because there's just no way I could do it. Is a professional plumber 10 times better than me at plumbing? They're probably quite considerably better more than me at plumbing. But within plumbers there's probably not an exponential difference between a good enough plumber and the world's greatest plumber, it's more of a minimum bar. It's interesting.

Jon Berger (36:06) The first question is, is AI better than the best available person? If the best available person is me, then the answer is yes for lots of things. There's loads of things that AI can do better than me. The other part of availability is it makes things more available. If there is such a thing as a plumber that's 10 times better than my local plumber, it doesn't really matter because they're not available to me. And I probably don't, you know, good enough is good enough here. But with AI, you can just throw more dollars at it, throw more time at it, throw more compute at it and say, I will have something better. Well, should you? Because it's available to you, but that is when you should spend the money on it.

Anne Currie (37:12) Yeah, which almost takes us back to the logarithmic judgments that businesses often fail to make, which is, you want something 10 times better, but are you really willing to pay that price? And do you need it? So on that note, I think we might leave this discussion again today and come back to it fairly regularly. What difference is AI going to make? We don't talk anywhere near enough about the exponential nature of some parts of the tech industry. And we treat all parts of the tech industries similar when some parts are linear and some parts are exponential. So thank you very much as listeners for listening to this week's episode of asynchronous and unreliable podcast. So thank you for for me and thank you, Jon for being a guest.

Jon Berger (38:22) Thank very much.

Anne Currie (38:25) Thank you very much for listening to Asynchronous and Unreliable podcast. If you enjoyed the show and you want us to create more content, please do show your support by hitting the subscribe button below. It really does make a difference. Thank you very much.