Why Programs Crash and Buildings Don't

Posted by Mark at March 28, 2005 10:29 PM

On his blog John Reynolds asks "Why is so much software so bad?". I think the answer to this question is not in the methods used by programmers, but in the nature of OOP languages themselves.

When the question of software quality comes up, the conversation usually turns towards the craft of programming. We hear about design patterns, best practices, methodologies, extreme programming, management techniques, better specs, better tools. The emphasis is on how the program is created, and how we can be better programmers. A program is essentially a hand-crafted good. To improve programming we improve programmers.

Yet as long as programming is a hand craft, where individual humans operate on individual lines of code, software will never be any better or more reliable than the programmers doing the coding. One line of faulty code, even one byte of data written outside of allowed memory, can crash a program. Imagine the fragility of a building or bridge if the entire structure could collapse due to one faulty rivet. Good craftsmanship can correct this only partly and requires a surplus of extremely dilligent rocket-scientist programmers. The solution is not to improve the craft, the solution is to design inherently stable systems.

Fortunately software is not the first technology that has worked through these growing pains.
To see how software projects can scale up, we can look at other technlogies that have scaled up succcessfully. Look at buildings. The components are very simple, predictable, and provide no-brainer redundancy. You can build nearly any building from a collection of I-beams, granite slabs, cement and glass window frames (all pre-fabricated units). The building can be put up by hi-school grads. No rocket scientists needed.

Look at electronics. Again, simple components that can be combined predictably. Resistors, capicitors, transistors, chips, can be measured to insure that they are operating within acceptable limits. Each component has a simple function and clear inputs and outputs that can be tested empirically. The component either works or it doesn't. If shoddy craftsmanship mars the manufacturing process, the resulting component may have a higher failure rate, or noisy output, but since this can be empirically tested, these components can be found and isolated. Considering their complexity, electronic systems work quite predictably, and we rarely have to worry that a single layer of silicon in a transistor wasn't applied correctly because the geek that made it didn't love his craft fully enough. The process has been quantified so that poor craft can be spotted (a resistor that is 10% off the mark when tested), and corrected. Electronic components aren't crafted, they're manufactured.

Henry Ford scaled up the automobile industry, not by hiring the best hand craftsmen in the world, but by removing hand craft from the process. As one who loves the craft of coding, this hurts to consider, but I have to recognize the historical trends. When technology scales up it leaves hand craft behind.

Stereo systems, computers, and car engines are built of components. To build a PC you aren't likely to solder together your own power supply, or build your own Pentium processor. When fixing your car you don't build spark plugs from scratch. You use parts that are defined to fill a specific need. Yet I don't see components like this in software development. Most code is written from scratch, usually building on top of libraries (such as Java's APIs..

Though OOP promises encapsulation, it fails to provide true encapsulation. Classes, methods, and interfaces define the "front" of an object, the features that allow the outside world to use that class. But this is only half of the picture. When the class hides its implementation it also hides all the ugly habits programmers have. But hiding the problems, much like piling your junk in a closet and slamming the door, doesn't make the problems go away. A simple clean class (on the outside) may contain dozens of references to other classes on the inside, each of these refers to still other classes, and each of those references introduces complexity and potential errors.

The biggest problem here is not the references. They are necessary for any larger sytem to work. The problem is that there is no formal way in the language itself to declare and define these references. I know what features a class provides to the world by looking at it's methods, but I have no way to know that the little screen widget I just dropped onto my canvas actually instantiates ten other classes to do it's work. There's no syntax that tells me what the class does to it's world while it's running. When I use that widget I implicitly sign a contract to incorporate these other classes into my project as well (and bring all the baggage that they have). I don't know that these references are occurring and have no way to police them. In this sense we don't have encapsulation. A class does not work like a resistor, transistor, spark plug, fuel injector, I-beam, or any other functional (time-tested) component. The front of the class is a clean, formally defined interface, but the back is a mess of ugly wires spilling out of the box, hard soldered into a dozen other classes, which in turn do the same thing, hard-wiring themselves into still other components. Look under the hood of an object in nearly any class library and you'll see a pandora's box of connections that are inherently unpredicatable and largely hidden from view.

No amount of well meaning craftsmanship or rocket-science coding can fix this situation. What is needed is for code to follow the pattern established by successful component systems since the invention of the wheel. Code must be broken into small encapsulated units that perform simple tasks, very predictably, with rigrorously defined inputs and outputs that can be empirically tested. The component must be both front and back encapsulated, which means "what the class provides to the world" is defined (as with classes now) but also "what the class request of the world" is just as rigorously defined. Currently we only have the vague and general "import" statement to tell us that a class refers to other classes outside of itself. What we need is a means to define the nature of the connections to other classes, to turn these references into plugs like the plugs on the back of a stereo component, PC, IPod, etc., that can be detached if necessary to separate a class from it's environment.

As much as I love hand-crafting code, I can see that historically this sort of craft has always given way to standarization, automation, and pre-fabricated modules. This is the growth path, not towards higher and higher skill level, but towards lower skill with more rigorous encapsulation. Consider the job of the longshoreman before container ships. In the past a longshoreman loaded all the various oddly shaped cargo into the hold of a ship. A good longshoreman could visualize how the cargo would fit and predict the most efficient packing of the cargo in the hold. A mistake would lead to costly inefficient use of the cargo space or the (also costly) need to unpack and repack. Longshoreman had a unique talent and craft and were paid highly for it. That ended abruptly when container shipping arrived. Cargo was packed first into (completed encapsulated) perfectly rectangular containers. Standard sizes meant that the containers could be stacked predicatably onto ships, trains, and trucks. Efficient packing was now a smaller problem, distributed to the people packing the individual containers. The unique talents of the longshoreman are no longer necessary.

The question is not how we become better programmers, but how to make systems that don't require better programmers to work well.

Trackbacks

Trackback for this post: http://www.culturekitchen.com/cgi-bin/movabletype/mt-tb.cgi/2846

Comments

Post a comment