This paper examines the persistent challenge of software productivity through three major theoretical lenses. Drawing on Brooks (1987), Cox (1996), and Weill and Ross (2004), it surveys why software development has historically failed to meet schedule, budget, and quality expectations. Brooks attributes the problem to the inherently conceptual nature of software, arguing no single technology or management technique can produce an order-of-magnitude improvement. Cox shifts focus to market dynamics, contending that the lack of a pay-per-use model prevents a robust component market. Weill and Ross point to organizational IT governance as the key determinant of IT investment returns. The paper concludes that no single theory fully explains low software productivity; rather, each contributes a valuable piece of the puzzle.
During the 1970s, companies had difficulty delivering software within the constraints of schedule, budget, and quality (Food for Thought, 2005). The problem grew worse over time. Many projects undertaken in the 1980s and 1990s were complete disasters, failing to deliver anything of value, grossly exceeding budget and schedule deadlines, and delivering poor quality. During the 1980s, a "software crisis" occurred in which spending on software maintenance exceeded spending on creating new software products. So, why can't software be mass produced in a way that is reliable and consistent, just as manufactured goods are delivered today?
There are many theories regarding the lack of software productivity. Brooks (1987) holds that the fundamental nature of software prevents meaningful automation. Cox (1996), on the other hand, makes the interesting assertion that software development issues stem from market dynamics — namely, the way software is bought and sold. More recently, experts have turned their attention to the organization itself, claiming that a lack of IT governance is the cause of software project failures (Weill and Ross, 2004). This paper finds that all these theories, and many more not discussed here, help shed light on the barriers to software productivity.
Brooks (1987) holds that the problems in software development productivity are attributable primarily to conceptual components and, therefore, no efforts directed at the task components — which are nothing more than the expression of those concepts — can bring about large productivity gains. Brooks states, "I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation." He therefore asserts that there is no technology or management technique that will produce even one order-of-magnitude improvement in software productivity, reliability, and simplicity.
Further, Brooks (1987) describes a software product as being fundamentally different from a manufactured good. Unlike a manufactured product, scaling up a software element is not about repeating the same elements at a larger size; it must involve an increase in the number of different elements that interact with each other in a nonlinear fashion. Because software must conform to other interfaces, this complexity cannot be simplified away. Additionally, manufactured goods are not expected to change after manufacture, but software must constantly evolve. Software also lacks any ready geometric representation, unlike other goods such as silicon chips, which have circuit diagrams, or computers, which have connectivity schematics.
On most attempts to improve software productivity, Brooks (1987) is fairly pessimistic about their overall impact, though he does have a few positive things to say about the potential of certain approaches. Among the marginal contributors, Brooks cites the following:
High-level languages: The most a high-level language can do is furnish all the constructs that the programmer imagines in the abstract program.
Time sharing: As response time approaches zero, at some point it passes the human threshold of noticeability, with no additional benefits beyond that point.
Object-oriented programming: Significant gains can be made only if the unnecessary type-specification overhead still present in programming languages is itself nine-tenths of the work involved in design.
Artificial intelligence: The hard part of building software is deciding what one wants to say, not saying it.
Expert systems: These require finding articulate, self-analytical experts who know why they do things, and developing efficient techniques for extracting what they know and distilling it into rule bases.
Automatic programming: It does not generalize to the wider world of ordinary software systems.
Graphical programming: It is difficult to extract any global overview from graphical representations.
Program verification: It does not save labor.
Workstations: Processing power and memory capacity are not critically important beyond a certain threshold.
Factors with more promise, according to Brooks, include mass markets for buying software, requirements refinement and prototyping, and finding ways to mentor great designers. Regarding requirements refinement and rapid prototyping, Brooks believes this area has potential because it attacks the essence, not the accidents, of the software problem through incremental development. Brooks concludes by stating that great designers are vastly more productive than others, producing structures that are faster, smaller, simpler, and cleaner with less effort.
Cox (1996) believes that the software engineering community is too focused on development processes and tools to explain poor productivity. Instead, Cox argues that the problem rests in the way software is bought and sold. He uses the following sequence of logic to justify his position:
"Pay-per-use models could unlock software component markets"
"Mature IT governance drives higher returns on IT investment"
Approximately every decade, there appears to be a new focus for improving software productivity. Brooks (1987) blamed low productivity on the conceptual nature of software. Almost ten years later, Cox (1996) turned attention to market dynamics and problems with the way software is bought and sold. Now, almost another decade further along, the current topic in vogue is the organization itself and how well it executes IT governance procedures (Weill and Ross, 2004). Not one of these theories rises above another in explaining low software productivity. Rather, each makes valid contributions to understanding development issues. Just as Brooks argued, there is still no silver bullet — but applying all that is known can help. And, just as Brooks stated, there is probably not one bullet that will produce an order-of-magnitude improvement, but a combination of measures can make a real difference.
You’re 60% through this paper. Sign up to read the remaining 2 sections.
Sign Up Now — Instant Access Already a member? Log inAlways verify citation format against your institution’s current style guide requirements.