Refterm Lecture Part 1 - Philosophies of Optimization

Casey Muratori's refterm video series is a great example of how to do application design. It focuses on writing a terminal emulator for Windows, but almost all the concepts apply to nixes as well -- even down to the level of OS primitives like memory mappings to create a ring buffer, etc. He builds a nearly complete terminal that appears to better support unicode and escape sequences than windows-terminal, in drastically less code and with literally orders of magnitude better performance. In particular, I like his focus on experimentation to figure out a reasonable upper bound on performance (so you can measure success/failure), "non-pessimisation" (don't do things extrinsic to the actual task), understanding the actual machine and its capabilities (he doesn't use the term, but sometimes called "mechanical sympathy"), and tactics for isolating "bad code" that you probably have to use (at least to get started). [VIDEO]

> The overwhelming majority of slow code isn't slow because it's failed to do this kind of microoptimization, it's slow because it used the wrong algorithm. I disagree. I believe the majority of code is slow because it's written without any consideration at all to performance. I like Casey Muratori's philosophy of non-pessimization where true optmization (measuring, working hot spots) is rare and rarely necessary but massive speedups compared to the general state of the art are achievable by simply not writing code using patterns that are inherently slow. This isn't deep algorithmic stuff it's just avoiding copies and/or pointer chasing. Edit: [VIDEO] <- Casey's most recent intro to non-pessimization > For a realistic-sized problem, if you write solutions in C and Python with the same amount of effort from equally skilled programmers, the Python version will almost certainly be faster The Advent of Code runs every year and I'm not sure about C (not something I track) but there are plenty of Rust submissions and while the Rust submissions take longer to come in, it's not THAT much longer. From memory it's like 2x on a timescale of minutes. The Python programs are not faster.

The fallacy of premature optimization rears its ugly head again. When Knuth wrote that quote, he meant something very specific with the word optimization: low level micro optimizations. Spending a lot of time trying to get every little ounce of performance from every little CPU instruction. High level reasonable design decisions are not "optimizations" in this sense at all. You should think of them as non-pessimization. Choosing Python or another slow language is premature pessimization. You just make your program slower for no reason. So choosing a language based on how programs written in it perform is simply non-pessimization. If you are interested in an expansion on this idea, checkout Casey Muratori's lecture about philosophies of optimization: [VIDEO] The article makes claims about software being "fast enough" and not needing further optimization. But what you call "fast enough" is probably 10000 times slower than what it could be. I'm not joking. People who program in slow languages have a really skewed perception about performance. If a page takes 5 seconds to load, they don't see that as a problem. If a server takes 500ms to respond to a request, they don't see that as a problem. This is simply unacceptable.

I’ve found this to be an unsuccessful approach in practice. Performance is a complex, many-faceted thing. It has hidden costs that are hard to quantify. Customers leave in disgust because the site is slow. No amount of “throwing more cores at it” will help if there’s a single threaded bottleneck somewhere. Superlinear algorithms will get progressively worse, easily outpacing processor speed improvements. Notably this is a recent thing — single threaded throughout was improving exponentially for decades so many admins internalised the concept that simply moving an app with a “merely quadratic” scaling problem to new hardware will always fix the problem. Now… this does nothing. I’ve turned up at many sites as a consultant at eyewatering daily rates to fix slow apps. Invariably they were missing trivial things like database indexes or caching. Not Redis or anything fancy like that! Just cache control headers on static content. Invariably, doing the right thing from the beginning would have been cheaper. Listen to Casey explain it: [VIDEO] You need to have efficiency in your heart and soul or you can’t honestly call yourself an engineer. Learn your craft properly so you can do more with less — including less developer time!

Honestly you don't even have to think about optimization to make your software not horrifically slow on modern hardware, instead you have to think about non-pessimization. Simply put, avoid needless complexity and gratuitous waste in the first place. See Refterm Lecture Part 1 - Philosophies of Optimization by Casey Muratori: [VIDEO]

I like Casey Miratory's "non-pessimization" approach in Philosophies of Optimization [1] [2]. He breaks "making a program go faster" into 3 modes: (1) "optimization" which includes counting expected cycles or memory bandwidth, careful benchmarking, designing custom algorithms, etc which can be fun but is expensive and is only rarely applied; (2) "non-pessimization" which is simply avoiding adding extra unnecessary work for the cpu, which should be done 100% of the time but isn't in the industry today; (3) and "fake optimization" which is shallowly repeating 'optimization tips' and conclusions from unrelated cases of (1) without actually measuring it. [1]: [VIDEO] [2]: [LINK]

But perfect performance isn't even the benchmark, it's "not ridiculously slow". This is what is meant by "Computers are fast, but you don't know it", you don't even know how ludicrously fast computers are because so much stuff is so insanely slow. They're so fast that, in the vast majority of cases, you don't even need optimization, you just need non-pessimization: [VIDEO]

I under the impression that we are building software the wrong way. This video from Casey Muratori helped me point to some reasons: [VIDEO]