Science and the Nature of Code

"I'll believe it when I see it." -- Ernst Mach

Being a programmer is a lot like being a scientist.

In any program of sufficient complexity--which is really any code that is in use--it quickly becomes impossible to understand it all.  Sure, you might grasp the big picture, but you can't predict how it will behave in every situation, or how it will work when refactored.  Code is always a bit of a mystery.

At the same time, code is reality.  It doesn't matter what people think a program does, what it was specified to do, what it was designed to do, or what it was proven to do.  What matters is what it actually does. 

Natural scientists come up with testable hypotheses that explain observable phenomena so that they can make accurate predictions about the future.  These hypotheses will be modified as a result of future experience.  This is the scientific method, and it doesn't apply to natural phenomena alone. 

Programmers typically start thinking scientifically when something is broken.  We observe something we have not observed before, come up with an explanation, test it, make modifications (to the code or environment), and test again.

Unlike the natural world, it may be easier to understand the essential nature of a program since you created it.  In fact, one of the first things we do when debugging is to look at the code.  Looking at code is not enough, however.  You have to actually see what it does.  The problem is that there is not enough time in the universe to exhaustively determine how code will behave.  Furthermore, test cases cover all the situations you can think of.  What about the ones nobody could think of?

For these reasons and more, we use QA environments.  This is our laboratory; it's the place where we test hypotheses.  Unfortunately, QA environments are never perfect.  Even if they are copies of production, they suffer from one fatal flaw: they aren't production.

Scientists face a very similar problem.  Since they can't see the source code, they can only rely on experimentation.  The problem is that their QA environment isn't the same as the real world either.  A large hadron collider, for example, creates highly contrived situations.

Drawing the analogy between natural scientists and programmers is useful for a few reasons.  First, taking the attitude of a scientist helps to distance ourselves from 'our' code.  It can be easy to get defensive about code we've written.  If you treat code as something to be understood, however, it suddenly has a life of its own.  It's not anyone's baby.  It just is.

Second, the analogy to science helps to highlight some of the ways we can respond to problems.  Some programmers start to theorize all kinds of possible explanations for an outage.  Was it neutrinos or was the firewall rewriting packet headers?  Others come up with more functional explanations.  What if we just restart the server?  These are both valid approaches, but they must both gather evidence in order to show that their hypotheses are valid.

These different approaches also show, third, we need laboratories where we can test hypotheses, no matter how wild.  That's why your QA environment should match your production environment as closely as possible, perhaps even down to the length of the network cables.  It's bad when problems are not caught in QA.  It's worse when you can't even explain why something works in one environment and not another.  Testing isn't just about proving that a solution meets its requirements; it's more generally about evaluating hypotheses.

Finally, I think it's important to think about the scientific method because many programmers are not good at thinking in this way.  It takes experience, and we tend not to get that kind of experience in schools--even science majors don't do much experimentation, unless mocking up lab results to match the Right Answer counts.  A computer science degree is basically a degree in abstract analysis, not too different from a mathematics degree.  In the real world, programmers need to flex their scientific muscles too.