On software simplicity

WARNING: RANT AHEAD

Truly simple software is possible. In my decade of attempting this, I’ve found out that writing simple software takes a long time and it is hard work. But perhaps the largest obstacle is that the pursuit of software simplicity is not something widely understood – it positively feels countercultural as of 2019, even more so now than when I started out ten years ago.

Everyone recognizes the cardinal importance of simplicity. What strikes me is that the undesirability of complexity is not equally recognized. To the best of my discernment, I see complexity absorbing most the creativity and hard work of most programmers around the world, day in and day out. Programmers spend their days in a losing battle with the hydra of complexity – and what they have to show afterwards for that is no adequate reward for their effort. What strikes me even more is that this is assumed to be unavoidable, or even normal.

I owe this outlook to Steve Yegge’s two sets of rants ([1], [2]), in particular this one, where he immortalizes “the godawful swamp most of us spend our programming careers slugging around in.” (I believe it is fitting to name it Yegge’s Swamp, in honor of its discoverer).

Despite believing in my bone and guts that he’s right, I’ve tried to challenge this outlook empirically, both with my own projects and with the dozen-odd companies I have worked with. What I find is that he’s right – confirmation bias notwithstanding.

Steve implies that the swamp is where we are because we don’t have a choice yet. Our tools, technologies and concepts are still in the Stone Age of computing. I however have a different opinion. It’s not the state of our knowledge that forces us into the swamp. It’s our attitude towards complexity that pushes us there.

Fellow Programmers: Why are we rushing to build things we do not understand? Why do we pretend to understand things we really don’t? Why are we so quick to pronounce someone to be dead wrong and either an ignorant or a charlatan? Why do we think that the current trendy tool is so much infinitely better than the not-so-hot formerly trendy tool of two years ago?

I believe that this mad rush of going after the latest technologies, of writing software as fast as possible, of never thinking from first principles and for ourselves, of codebases being thrown out of the window every N years – this mad rush is a collective expression of the stagnation of software – and given the importance of software for our economy and society, it encourages overall stagnation. Individually, it produces existential dread and waste of precious life energy. In both cases, it is a frantic stagnation.

The antidote to the mad rush is the slow pursuit of simplicity. Software can be simple. And making it simple is not a goal, but a path to be traversed.

Defining simplicity is hard. My working, heuristic definition is that out of two pieces of software that solve the same problem, the simplest one is the one that’s easier to understand.

Understanding is even harder to define. The only heuristic I can contribute here is a feeling: understanding feels like reaching bedrock. You can build on it. You can figure out the corner cases in your head after a short while. The implications are clear. Lack of understanding is, instead, a quagmire. It’s mud, it’s swamp. You think you got it nailed down and then it either shifts or starts sinking. Combinations raise exponential possibilities.

Simple software is software that you can easily understand. Even though it might contain concepts that are challenging, once you get them, you got them. They’re there. And they’re not going anywhere.

The concepts of complex code are shifting and mutating, many times for no good reason. It’s hard to see a pattern in them – instead of an advance, it merely looks like shifting. I cannot help but mention Angular.js here, and its vast tower of concepts and abstractions that have changed so much through the years. And if you love Angular, I ask for your forgiveness – I don’t mean to offend you, just to share what I see as an evident truth.

Complex code rots quickly – it gets icky and unmanageable despite valiant and massive efforts. Simple code lasts. Even if you have to throw it out for some reason, its patterns stick with you.

Simple solutions are the ones you can easily build upon. You understand them quickly – and all potential corner cases are quite clear in your head upon them.

A simple solution emerges from a direct and full understanding of the problem it solves. If you truly understand the problem, the solution is usually forthcoming. This may not be the case in the bleeding edge in other disciplines – but in programming, in my experience it is always like this.

This means that you don’t keep the first solution that barely works. You iterate on the solution, you test it and use it in different contexts where it’s applicable, until you fully understand the problem you’re solving. This is possible. And the patterns don’t go anywhere. They don’t get disrupted easily, if at all.

Reworking a solution that works in order to make it simpler is countercultural. In the startup mentality, you’re throwing out valuable time that you’re competitors are instead using to add more features. There’s also two more dictums against it: don’t rewrite your code, and don’t reinvent the wheel.

The “don’t rewrite your code” dictum, best stated by Joel Spolsky, stands squarely in the way of the pursuit of simplicity. Code necessarily needs to be rewritten. Not from scratch, but rather learning from its extensive application and based on previous results. The tests where you find real bugs in your code are also part of your code – they are important corner cases that need attention. Or perhaps they emerged because your solution wasn’t as simple as possible. In any case, you rewrite your software while keeping your true-and-tried test suite – and you can do so with confidence, because if you’ve been adding tests for each bug that emerged (and I encourage you to do so), then you know that your rewritten software will also pass all the tests. Only by iterating your solution you’ll reach a version of it that deals with the essence of the problem in an elegant way. It happens like that in literature, in mathematics, in science and in art.

Side note: if masterpieces last decades and centuries, software masterpieces should be no different. If no software lasts, it’s because no one is even attempting to write a masterpiece. Isn’t that a tremendous waste?

The “don’t reinvent the wheel” dictum (which has echoes of the “don’t roll out your own crypto” admonition) also blocks the pursuit of simplicity. It implies that you should not build tools, because there’s people somewhere else that are smarter than you and know better. It means that you should not solve any fundamental problems yourself, but rather follow standard solutions in the most orthodox way possible. While I’m certain that the people building the big tools and frameworks are smarter than me and know better, it doesn’t mean that building tools is a bad thing. I might not have to use them, and even less ram them other people’s throats. But building tools allows you to understand the actual problem, and not take at face value concepts created by others. By building solutions, you fully understand the problem and what you’re talking about. There’s no need to recourse to authority. It took a while, but I can now see the tree right in front of me – I don’t need someone else to tell me that’s a tree, and it’d take a lot to convince me of otherwise.

Going back to startups: I’m starting two at the moment. There probably the slowest moving startups you’ll ever see. It’s taking us years to get to where we want to be, because we’re doing things slowly, because we want to understand the problem instead of rushing to solve it. We might be overwhelmingly crushed by faster competitors, focusing on speedy iteration, features and with large teams and venture capital. The list of fears is long. And yet, working in this way is its own reward. It may well be that it was a good way to build a startup in a competitive environment all along. I’ll get back to you on that one.

Let me face a set of fears that I know to be unfounded. I believe these fears are the engine that keeps the mad rush of complex software going its not-so-merry round:

After a decade of being at it, I can confidently tell you:

Fear of feeling left behind is something I’m not able to speak about so strongly yet, because I feel it daily. However, a good antidote is to see the former bold proclamations of once popular and trendy tools that now are abandoned and even scorned. At the moment you felt you were missing out on that boat – and yet the boat sank, and the next one too. Perhaps it was a good thing that you didn’t jump on it at the time after all.

I try to judge existing software using two approaches: by making the knowledge my own, and by acknowledging all my previous knowledge to it. Both principles come from Buddhist practices: you cannot take knowledge and just put it in yourself; and you cannot wholesale forego your previous teachings. These two are the antidotes I found to blindly following software trends while not being completely isolated.

Simplicity is the path to great software. The concepts emerge from the path and are judged against it – only the pursuit of simplicity can ascertain their true worth. Taking a concept from out of the blue and bolting it onto a piece of software (or making a piece of software out of it) is not bound to generate simple software.

I am also of the hope (if not yet of the certainty) that simplicity is the best path to software with a great set of software features, implemented with utmost quality – even by very small teams.

The pursuit of simple software is possible. Software is hard, and can never be easy. But it can (and should) be simple. I believe it is possible to write software that will last decades, that allows yourself and others to build upon with ease. That doesn’t have to repent tomorrow about the concepts it heralded yesterday. That doesn’t have to apologize for itself, but rather be proud of itself, or at any rate confident.

You don’t need a privileged brain. It is your right as a programmer to write code that you can be truly happy about. You might not get paid for it (at least, not yet), but you always can do it on the early mornings or late evenings.

You probably won’t be popular. You will probably get a lot of shit for trying to write simple software – at least I know I did. Here’s how I found it’s best to deal with this shit: I take from it what’s valuable. It’s not pleasant but it is necessary. The valuable things are when factual errors in your work are pointed out. These can be mere corrections or they can be fundamental (those might hurt). In any case, they help you grow. This is a good path to work on your ego, because you’ll be wrong a lot, and you’ll look like a fool a lot. From all of this you can learn.

As for the parts of criticism that doesn’t contain factual errors and are muddy or hazy, ignore them. It is most likely an expression of fear. If you can, deal with it compassionately, or if you can’t, ignore it.

The how of writing simple software is, I believe, not as important. I found the hows on my way and have shared them elsewhere. But I’m confident that you can find out how as long as you follow the path.

Writing simple software is worth it, whatever happens later. It is an act of faith, and yet the terrain itself will guide you constantly. I’ve written these words to encourage you to do it and to let you know that it can be done.