My views on the software development process

I have been writing software for about 20 years now. I might be less certain about the right way to write software now then say two or three years into my career. Then and now, I believe it is more of an art than a science. Early in my career, I had a great deal of enthusiasm for software development mythologies, and studied them with great enthusiasm. There were no standards for expressing design back then, but there were data flow diagrams, structure diagrams and the like. My favorite was SADT (structured analysis and design). I found and still find this methodology especially good at helping one define top level system objects. Today we have the UML. Other than being a common language, I am not at all sure that this is really an advance. One thing it has going for it is that it unabashedly embraces object oriented analysis and design. I am unabashedly a believer in OOAD.

Back when I was still working on my BA in math I took a graduate class in software engineering, and for the paper in this class I got to do something rarely done as a working engineer: I got to revisit a design that I had already completed. That design was my first “real” software project, one that I was incredibly lucky to even get to work on let alone be the chief designer, since I was only in the second year. I walked back through the design issues using several different methods. My conclusion form that investigation was that there are two ways to approach the problem of decomposition, inside-out and outside-in. I believe David Parnas, “On the Criteria for Decomposing Systems into Modules,” deservers credit for what I am calling inside-out in the literature, and I no longer remember who deserves credit for outside-in – best known as top-down. From a 21^st century software perspective it may be hard to realized that most of the mouths were advocating outside-in before OO gained such notoriety. Meanwhile a revolution in how to design abstract systems was going on, and it has lead to the ubiquity of object oriented everything.

I suppose, but I am sure there are exceptions, that before computers we engineers never really built systems that we thought of as evolving over time. To be sure, design has been reused and copied since the first rock was thrown, but systems that are themselves conceived to evolved and change are new. Parnas, I believe, hits the nail on the head when he suggests that by first defining components that encapsulate the data objects of our system we can better construct the tools, components that make up a larger system. By defining and implementing components that can be glued we are able to from a large number of solutions that successfully reuse code. OO languages and techniques give us the tools to build components. Components give us the tools to build robust systems, ones capable of being evolved over time.

The great thing about an outside-in design is that everyone who looks at the resulting design says, “I see what you did, I can understand that,” and this makes people feel good, even if it is not the best design in the cost over the life-time sense. On the other hand, an inside-out design is sometimes rather mysterious, and can leave programmers puzzled when the first try to understand the design. An example of this, though somewhat askew from the previous discussion, is the message passing between top-level objects in many of my designs. There is nothing in any analysis that will tell one that isolating top-level objects via a message passing is a good thing – but it is. It allows one to have active-objects in the system and to have those objects behave in consistent and predictable ways. In most of my designs, I create top-level active objects that live on a thread, these objects act only when they receive a message from some other entity in the system and are therefore always event driven. Each active object owns its own storage, and any other entity in the system that wants to access that storage must send the object a message! So where does this messaging system come from. It does not come from analysis, but rather from art. In my view, most software is still written outside-in even though we are using OO tools. An inside-out or OO design does not hold together in a linearly described manner. Instead, it is a set of whole pieces glued together to make the application. Why those pieces and not others is the art.

This art is not that mysterious. One can find it in the literature: Parnas, as mentioned above, Stevens, Constantine, and Myers about coupling and cohesion, Gutag on abstract data types. Today it is all over the place in many books on developing OO software, Booch comes to mind, the many books on UML, and many others. It also comes from experience and experimentation. None of us just knows what to do until we have really wrestled with implementing, maintaining, and porting complex systems.

I have small faith in techniques that purport lead one down the garden path without resorting to art. But in truth, one cannot build a system entirely inside-out. We need to find a compromise, and again this is art. My approach is to proceed as if I was building an operating system specific to the domain in question. I try to ask the question, what are the generic facilities that I would want so that this project would just become a jigsaw puzzle of components that I am going to glue together (I know I am mixing metaphors but what is a software engineer to do)? From here, one is able to proceed in both directions, kind of middle out. That is, with these mid level constructs, I can at one level define the application, and I can start implementing the mid-level components. For me, the glue at this level is the messaging system I mentioned above.

Beyond these consideration I am agnostic about how the project is to be documented, but adamant that it will go better the better we do so. The UML seems to be an adequate tool for doing much of the documentation, however I believe adequate description needs much more text augmentation than is suggested in the textbooks I have read on UML.