The four horsemen of data

Running a software service is essentially being responsible for the data of others. This is quite the responsibility. Here I want to explore what I think are the four main possible failures when upholding this responsibility.

Horseman #1: unauthorized access to data. Someone (an external attacker, a contractor, an employee) is able to access data that they should not access. In some cases, they can use this data to damage the owner of the data, or related stakeholders. Information security is essentially about preventing unauthorized access to data. A bad data leak can completely destroy trust. This is a really bad horseman. Really hard to keep at bay. The simpler your fortress, the easier it is to defend it.

Horseman #2: data loss. A part of the data (or perhaps the whole thing) is lost forever. Nobody has any other copies. This failure can be a company killer. This is the easiest disaster to prevent, by having regular backups distributed in different locations, and making sure that those backups allow you to properly restore the data. This is the first disaster you should guard against.

Horseman #3: data corruption. Your system handles data inconsistently and data gets corrupted, in the sense that it stops making sense. This is a more subtle and perhaps more dangerous version of data loss, because it might not be immediately detected. This horseman can be held at bay with transactions and overall coherent design. Backups also help a lot, if corruption is detected quickly after it happens, because they give you points from which to partially restore.

Horseman #4: availability loss. “The server is not down, it’s never down. You just can’t get to it right now.” Your user needs to use the data but your system is taking a break. The data is safe: backed up, consistent, secured. But, at that moment, absolutely useless. Making your customer lose money, time and trust in you. I won’t get into the details of how to create available systems; but I will say that measures concerning horseman #4 can inadvertently open up attack angles for horsemen #3 and even #2.

In the sober words of someone who has single-handedly kept responsibility for precious data for tens of thousands of people: “A one-person business is an exercise in long-term anxiety management, so I would say if you are already an anxious person, go ahead and start a business. You’re not going to feel any worse. You’ve already got the main skill set of staying up and worrying, so you might as well make some money.” Having a team definitely lightens up the burden, but never fully. In that burden there’s value to be provided.

I will close with yet more wisdom from Maciej: “My grandpa sometimes said “you have to help your fate along,” and I always liked this worldview very much, for the way it bolted a work ethic onto fatalism. Things happen, but you can always take credit for tenacity.”