January, 1996
Abstract This paper documents the productivity gains experienced in a relatively large C++ development as a result of using object oriented design and an architecture driven process.
Has object oriented design delivered on it's promises? Are applications that are designed using object oriented techniques really more robust, more maintainable, more reusable? Does the use of the object oriented techniques really increase the overall productivity of engineers? Is C++ a valid language for implementing an object oriented design? This is just a sample of the questions that arise with the adoption of object oriented technology. It is difficult to answer these questions in the absence of controlled experiments. The only way to do such a controlled experiment would be to employ two groups of people, one executing the design using procedural techniques, and the other using object oriented techniques. However, due to the expense, it is very unlikely that experiments of this nature will be conducted. Therefore, the industry will have to rely upon those of us who have undertaken to try object oriented design in C++ and have reported the results of that experience.
We [Martin and Newkirk] are the principals in a consulting firm named Object Mentor, based in Illinois. We supply training and design consulting in object-oriented technology to corporations around the world. We also take on significant development projects. This paper describes one of those development projects.
NCARB have contracted with ETS to design and implement 18 computer based vignettes which will replace the building and site design portions of the registration exam in 1997. Each vignette tests the candidate for competency in one specific area of architecture. A vignette consists of two separate programs. One which delivers the test to the candidate, and another which scores the solution that the candidate entered. It is intended that the delivery portion would be executed on a PC running various forms of the Microsoft Windows(TM) operating system in testing centers located all over North America. The solutions will then be transmitted to ETS where the scoring programs will be run.
The delivery program of each vignette has a CAD-like graphical user interface that allows the candidate to draw solutions in terms of standard architectural diagrams. The kind of diagram that is used to draw the solution depends upon the vignette. The solution may be a floor plan, a roof plan, a set of contour lines, or property lines, etc. Candidates are evaluated on issues as diverse as building structure, illumination, zoning, code compliance, etc.
The solutions are drawn by the candidates using a set of special purpose CAD tools supplied by each vignette delivery program. Some tools are specific to the task the candidate is asked to do, others are common to all vignettes. These common tools include zoom, erase, sketch tools (lines, circles, etc.), measuring, identification, rotation, etc. All tools are tailored for ease of use, so that candidates who have no computer experience can quickly learn how to take the exam.
Once a candidate completes a solution using the vignette delivery program, the results are stored in a file and sent electronically to ETS. Upon arrival ETS runs the specific scoring program associated with the delivery portion; and the solution is given a score.
We are developing the vignettes using Microsoft Windows 3.11 with Win32s, Windows 95 and Windows NT using the Borland C++ V4.51 compiler. We make significant use of Borland's Object Windows Library (OWL) interface libraries, the container library from ``Designing Object Oriented C++ Applications using the Booch Method'' by Robert C. Martin, and various classes from Rogue Wave's tools.h++ class library.
We began working on this project in March of 1993. At that time we were commissioned to produce only two vignettes. The first was the ``Building-Design'' vignette. This vignette was expected to be the most complex and ambitious of all . We felt that by tackling the most difficult problem first we would learn the most about how to produce the rest of the vignettes. We also believed that success would give us worst case parameters for developing the balance of the vignettes. In hindsight choosing the most complex vignette was a wise decision. We learned a great deal about the problem, the client, and the environment. Had we chosen a simpler problem at first, we might have been misled concerning the difficulty of designing and implementing the rest of the vignettes.
But the creation of a framework was not our primary focus. Our primary focus was to deliver a correctly functioning ``Building-Design'' vignette to ETS within the schedule set forth in our contract. By the time that vignette was completed and delivered to ETS we had a working program, and a we had a lot of insight into what a common framework should look like. But once we began work on the second vignette, we realized that the 80,000 lines of C++ that made up the ``Building Design'' vignette did not really constitute the reusable framework we needed.
In March of 1994, long before the completion of the second vignette, ETS gave us a commission to produce eleven more vignettes. In order to produce these vignettes on a schedule that was acceptable to ETS, we needed to solve the problem of the framework. We also needed to build an organization that could produce the half million lines of C++ that we were expecting to write. What's more, we had some interesting logistical problems to solve.
We allocated three additional engineers to the project. The group members and their associated experience levels are listed in Table 1. In addition to the members of our group, ETS assigned three of their own software engineers to work part-time on the remaining 5 vignettes.
| Name | Years Experience | Years OO/C++ |
|---|---|---|
| Robert | 25 | 10 |
| James | 15 | 8 |
| Red | 15 | 5 |
| Bhama | 12 | 3 |
| Chung | 8 | 1 |
James acts as the group manager as well as it's lead developer. He is responsible for the day to day schedule and the assignment of tasks to the other engineers (including Robert). He also contributes to all major and minor design decisions. James works on the project full time.
Bhama, Red and Chung are software engineers whose primary responsibility is to deliver against the goals set for them by James. Each of them make significant design contributions in their own right, and work on the project on a full time basis.
We all participate in design and code reviews, and we all write code. Each engineer is responsible for the design of their own components, but we all consult with each other on an as needed basis.
ETS is located in Princeton New Jersey. The majority of the Object Mentor team (Robert, James, Bhama and Chung) is located in suburban Chicago, Illinois. Red lives in Tucson, Arizona. This geographic separation means that we need a unique development environment. We decided at the outset that we should all work from our home offices. Since we are a small and independent company we do not choose to rent office space. Instead we make use of various telecommunications technologies (e.g. Fax, Modem, Telephone, etc.) in order to coordinate our activities. This ``telecommuting'' works quite well. We have found that co-location with our client, or with each other, is not particularly important. It also allows each of us to work according to the schedule that is most productive to each individual.
We meet with ETS on a bi-monthly basis. When ETS comes to visit us, we hold the meeting at their hotel. If we go out to visit them, we hold the meeting in one of their conference rooms. At these meetings we review project schedules and milestones, review problem reports, and conduct technology transfer between our developers, and ETS's development team.
In order to facilitate access to the source code at all times, a central repository has been set up and is managed using CVS (A popular source code configuration control system). Problem reports are also centrally managed via GNATS (A popular problem reporting system). Engineers dial into the central system from their home computers to check-out and download, or upload and check-in source files and to read problem reports and email.
What went wrong with the ``Building-Design'' vignette? Why hadn't our use of object-oriented technology provided us with a framework of reusable code that would help us create the other vignettes? What different approach would allow us to create this framework?
In hindsight, the failure of the ``Building-Design'' project to produce a reusable framework was most likely due to our focus upon schedule and content. This is not to say that we did not put a significant amount of effort into design, nor is it to say that we were careless of reusability. We simply made trade offs in favor of schedule and content. Booch would have called us ``Calendar driven''.
By the time work began on the newly commissioned vignettes we understood that a framework would not simply materialize by using object-oriented techniques. The framework was going to require a lot of attention if it was going to be successful. Our project would have to shift from being Calendar driven to being Architecture driven. According to Booch, ``a framework does not even begin to reach maturity until it has been applied in at least three or more distinct applications''. Thus we decided that we would develop the framework concurrently with the development of several new vignettes. This would allow us to compare similar features of the vignettes and design those features in a generic and reusable way.
Another option would have been to try to design a framework up front, and complete it before any of the vignettes were begun. Indeed, many people would identify this with an architecture driven approach. We chose not to pursue this option, however, because it would have caused us to produce large amounts of framework code that could not have be tested inside working vignettes. We did not trust our ability to completely anticipate the needs of the vignettes. In short, we felt that the architecture needed almost immediate verification by being used in working vignettes. We did not want to guess.
Four new vignettes were begun. The developers of these vignettes conferred with each other regularly with regard to potential common features. When such features were identified, they were abstracted and placed into the framework. Often these features were identified after they had been made to work in one of the vignettes. In such cases the working code was removed from the vignette, made generic, and then added to the framework. The other developers would then test the new feature in their vignettes. This gave us immediate feedback with regard to the reusability of the features in the framework.
Among the common features that were added to the framework are:
Over the next eight months the framework grew to some 60,000 lines of C++ code, representing somewhat more than one man year of direct effort. Moreover, there were many vignettes depending upon this bulk of code. It became clear that we could not manage the development of more and more vignettes, without establishing a release process for the framework.
What were we to do with the old ``Bulding-Design'' vignette? As the framework grew and the new vignettes were successfully reusing it, ``Building-Design'' stood more and more as an outsider. It was unlike all the other vignettes, and would have to be maintained and evolved through separate means. Booch recommends: ``A successful object-oriented project must be both focused and ruthless, but it must also not be blind to opportunities for simplifying its problems by identifying general solutions that can be specialized to solve its particular problems.'' Even though ``Building-Design'' represented over a man year of effort, we decided to be ruthless, and discarded the old version completely. We committed ourselves to redesign and reimplement it later in the project cycle.
One negative result of our shift in focus from Calendar to Architecture was the relatively long development time for the first vignettes. The first four vignette delivery programs required nearly four man years to develop. But upon their completion the framework was richly endowed with 60,000 lines of C++ code, and the vignette delivery programs were remarkably small. Each program had approximately 4,000 lines of boiler plate code; i.e. code that is the same for each vignette. Each program also had an average of 6,000 lines of application specific code. The smallest vignette had as few as 500 lines of application specific code, and the largest had as many as 12,000. We felt that it was remarkable that, on average, nearly five sixths of the code in each vignette was pulled from the framework. Only one tenth of the code within these programs was unique.
Thus, the entire project has taken just under nine man years so far. In that time we have produced thirteen delivery programs and four scoring programs. The scoring programs account for about one man year. Thus we have produced thirteen working delivery programs in eight man years. This averages out to about 0.6 man years per vignette. If we discount the man year spent on the first ``Building-Design'' vignette, then our rate is more like 0.5 man years per vignette. Compare this to the one man year that we spent on the original ``Building-Design'' delivery program.
Another way to look at these results is that the first five vignettes, including ``Building-Design'', required one man year of effort apiece. Subsequent vignettes, however, required 2.6 man months apiece. In increase of nearly 400%. Clearly, this justified our shift in focus.
The total line count is approximately 170,000+ lines of code so far. This amounts to (170,000 \xd6 9 Ý 19,000) lines per man year, or Ý1,600 lines per man month. And this does not count the 80,000 lines that were discarded.
A production rate of 1,600 lines per month is substantial. But it is all the more impressive when one considers that as much as 50% of our time is spent dealing with ever changing requirements. The vignette delivery programs, once produced, are subjected to trials by volunteers. These trials expose problems with the original specifications. Thus, there is a very large amount of maintenance work that goes along with each vignette. This maintenance work has been ongoing since the first few months of the project.
Of the three developers at ETS, one worked full time on delivery programs, and the other two worked part time on scoring. All three developers were clients of the framework, and integrated each new release into their programs.
Had we not discarded the first implementation of the ``Building-Design'' vignette and used the errors we made to guide us in the design of the framework, we would very likely have repeated those errors in every vignette. We would not have had the framework to build upon. And so, each vignette would have taken us about a man year to develop. Thus we would have spent thirteen man years, rather than nine to get where we are today. This is approximately a 30% improvement, even considering that we threw the first implementation away.
Also, the framework is designed in a way that minimizes the dependencies between modules. Thus, as changes are made to it, the impact upon the source code of the vignettes is seldom felt. This makes it very easy to make framework changes.
Fredrick Brooks has recently rescinded his recommendation of 20 years ago: ``Plan to throw one away.'' We agree that it is not wise to plan to throw a development away. On the other hand, it is also unwise to hold onto a development that would impede further progress.
The ``Building-Design'' vignette was a working program. Yet we discarded it because its design was flawed. By being ruthless, and discarding the flawed result, we were able to apply what we had learned from our errors, and develop all the vignettes, including a redesign of ``Building-Design'', in less time than it would have taken to push ahead with the old flawed solution.