Nails and sawdust on your floor

I had an opportunity to discuss one of the projects that I'm involved with in my professional life with my father recently. My dad, Sonny Jones, is a general contractor who's been building homes for longer than I've been alive. I'd like to believe that he's a master craftsman, and I suspect that the number of his customers who call him to work on their home (add new rooms, remodel existing, build new homes) time and time again year after year, shows that other people believe it as well.

It surprised me at first, but software development and home construction are remarkably similar fields.

Let me ask you this:

Would you be comfortable hiring a construction company that, as a matter of habit, left their building materials scattered all over the job site? What if the framing team routinely left piles of sawdust, lots of discarded nails, and other debris on the site when they handed it over to the drywall and flooring team? Would you be comfortable knowing that your home's walls were filled with discarded bits of wood and glue and boxes?

In my own opinion, I don't think that anyone would be happy with that. We might put up with it.... but intuitively we expect craftsmen to be clean, efficient, and to take pride in their work and the results of their work. Few are the master builders who look upon a jobsite covered in scraps and debris, all the studs and support beams at not-quite-a-perfect-angle, with pride. Many master builders would insist that the entire project be bulldozed and begun from scratch before they would be willing to contribute.

So why does the field of software engineering exhibit all of the traits of the sloppy builder, and few of the traits of the master?

The project that I'm working on is a project that fits the sloppy builder more than the master craftsman.

The project is one subsystem of a large multi process application. Historically, this subsystem has been riddled with memory usage problems, throughput problems, crashes, and just general lack of customer satisfaction. Many people in the company consider it to be bad enough that they express their condolences to me when they learn that I'm the maintainer of the subsystem. It's that bad.

Going back to my analogy, think of the most beautifully written application you've ever been involved with or read the source code for. Visualize the joy you felt when working on it, it's clean API's, easy to understand and highly performant frameworks, the crisp and straightforward requirements and specifications, the inspired architecture both high level and low, the diligently up-to-date documentation, and the elegant, well written, heavily commented, and supremely easy to understand source code.

You're envisioning the work of a master craftsman. Someone who takes such pride in their work that it must be beautifully built from the inside out or they will wash their hands of the whole affair. The blueprints are immaculate and reviewed by the best minds available. The foundation is built with the best materials available with extra care taken to ensure hundreds of years of use. The design is well thought out and straightforward to actually construct. The walls are perfectly square, not a single stair will ever squeak, and the flooring can hold thousands of pounds per room. Every nail is hammered true, not a screw in the building is stripped, and small neatly written notes can be found throughout the walls for the convenience of the next improvement project.

My new project has no well defined API; the boundary between itself and the outside world is somewhat hard to define. The framework that it's built on is somewhat difficult to use if you're not already familiar with it. There are no written requirements and specifications, and most people familiar can't quite agree on what it's supposed to do. There is an architecture, but the architects who contributed to it's design no longer work with the company and so their decision making process is lost and unfathomable. The code is erratically formatted, with inconsistent spacing, inconsistent variable names, huge monolithic functions, monster classes, one or two lines of comments for every thousand or two lines of code. And to top it off there's a list of bugs longer than my arm.

This project is most definitely not the work of a master craftsman.

So where did everything go wrong?