What is the job of a software developer? Somebody gives us a machine and he says: We want to do it task X.
Then it is our job to make the machine do this. We have to configure it.
What can we configure? Normally we can configure the CPU. This means we write a piece of software.
Because the CPU is complex, we built layers upon layers of abstraction about it. First we had only opcodes. Then assembly language. Later on languages like C, that compile directly into assembler. Then even more abstract languages that are often interpreted, such as Java, Ruby or JavaScript. And with every layer of abstraction comes a new language, new idioms and a new set of tools to support these layers.
Then we also need more sophisticated ways to actually run the programs on the CPU. Starting from a small embedded program that starts with the first interrupt vector, over more and more powerful operating systems that schedule our programs and again hide the details of the hardware to them, up to Virtualisation and Docker Containers, where the program is really independent of the hardware it is running on.
The task of every abstraction layer is to hide the details of the underlying layers to its users, so a programmer on one layer does not have to bother with the underlying layers. But once in a while there are cracks in these layers and the layers underneath poke their ugly heads through them. Be it a bug in a driver or even the CPU, or sometimes the strange behavior of a Garbage Collector or packet loss over a network. These cracks might sometimes be easy to fix, but sometimes they turn our understanding of the system upside down, forcing us to rewrite much of our code.
So what is to learn from this?
- Test thoroughly on every platform, be it hardware or software, your software needs to run upon
- Test on your target platform as early as possible in the development process. Sometimes your assumptions might prove wrong right at the beginning, when it is still cheap to start over again
- Try to understand all layers and frameworks and the hardware you use. If you can’t, have somebody on your team who can
- Every layer of abstraction is a new potential source of problems, so thing twice about each one you introduce
- Don’t rely on assumptions. When in doubt, add more tests, especially integration tests and load tests