On the Second-System Effect

I've started reading The Mythical Man-Month by Frederick Brooks. It's pretty fascinating how well a computer book written in 1975 holds up for thirty years. I'm only a few chapters into it so far; I just finished reading a very insightful chapter about a phenomenon I clearly suffered from yet didn't diagnose until Brooks did it for me. It's called the Second-System Effect. Brooks writes:

An architect's first work is apt to be spare and clean. He knows he doesn't know what he's doing, so he does it carefully and with great restraint.
As he designs the first work, frill after frill and embellishment after embellishment occur to him. These get stored away to be used "next time." ... The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one. The result, as Ovid says, is a "big pile."

I so did this! On my first professional project I was all too aware that I had just fallen head-first off the apple cart, so I didn't try to overstep my bounds by a single inch. Then I got cocky, and on my second project, fashioning myself an old pro with all the self assurance of an armadillo at a crosswalk, I decided to throw all these new technologies I'd been learning about at the problem without any prior thought as to

a) whether they were appropriate uses of the technology or
b) whether they would solve the problem.

A "big pile" was the result. Oh sure, the thing worked, and we got paid, but sitting here four years later a couple shades wiser than then, I realize that a lot of pain would have been prevented if I had reined myself in a little more.

Brooks goes on (if only I'd read this four years ago!):

How does the architect avoid the second-system effect? Well, obviously he can't skip his second system. But he can be conscious of the peculiar hazards of that system, and exert extra self-discipline to avoid functional ornamentation and to avoid extrapolation of functions that are obviated by changes in assumptions and purposes.

I refer to my early zealous days of throwing untested technologies at a problem "design by resume." It burned me pretty bad, and I'm reformed now.

For the project managers in his readership, Brooks closes with this:

How does the project manager avoid the second-system effect? By insisting on a senior architect who has at least two systems under his belt.

They say that ninety-nine percent of software projects fail. I would bet that at least forty of those percent are victims of the second-system effect.