tom thinks

that you should sort by topic

date 2001-04-18:08:22
Software Netscape is such a typical piece of software.

It has lots of features. It just doesn't work very well.

A few minutes ago it vanished in one of its weekly disappearing acts. No warning, no correlation with me doing anything (I was reading The Onion, which is not exactly a high-stress activity for a browser.) Just a sudden vanishment. If I were running Win98 I guess it would have taken the OS down with it, but on Linux it just folds up its tent and abruptly goes home. When I restart, at least I no longer have to set the default character-set by hand, having gone in and edited the preferences file by hand to unscrew some screwed up settings that caused the app to default to something weird no matter what I set through the preferences menu.

It doesn't have to be this way. Nor is it particularly hard to make it not this way. There are well-documented practices that will allow any software organization to write code that doesn't crash. They're clearly described in books like Rapid Development. There's no mystery to what organizations and individuals need to do: they need to design before building, they need to validate their designs relative to their user's requirements, they need to make room in their schedules for design, they need to integrate early and often, and they need to test and test and test.

Most of all, they need to create realistic schedules, and they need to believe the Law of Common Humanity: We are just like Them. That is, if every other organization has tried to do it this way and failed, we will fail too if we try to do it this way.

Creating realistic schedules is hard, particularly given that most scheduling tools give you no room for representing uncertainty, which is ridiculous. Early-stage software development schedules typically involve a factor of four uncertainty, and its almost all on the long side. Hardly any project ever finishes on the short side of the "best estimate." So tools like MS Project, that have minimal to no representation of uncertainty, are all but useless.

The big problem with creating realistic schedules, though, is that most organizations are dedicated to having unrealistic self-images. Most software development organizations are only average, and half of them are worse than average. But very few recognize that. They put down their failure to exceptional circumstances: "We didn't make schedule this time because..." They fail to realize that every organization faces exceptional circumstances every day: the whole job of management is to deal with the exceptional circumstances that continually buffet their organization. Key people quit, machines break down, the roof leaks... All in a day's work.

The solution to the problem is to take the facts seriously. Start small: don't worry about meeting your schedule on your next project, worry about trying to estimate accurately. The reason to do this is to build up your own confidence in your ability to estimate realistically, so that when the time comes to start pushing management for more realistic schedules you'll have lots of data to confront them with. This really works: management will, in my experience, yield in the face of carefully presented empirical fact, if you present it as a solution to the problem rather than just rubbing their faces in their own failures.

I spent a couple of years learning to estimate before I convinced anyone to believe in a schedule I'd made. I found the most important tool in scheduling is this: after the first four weeks of a project (it doesn't matter how long the project is, nor what you take as 'the first four weeks' so long as the project is properly begun) take the ratio of the work you should have done to the work you've actually done. That is, if you're two weeks into your scheduled work after the first four weeks, the ratio is 2. Now multiply the schedule by this factor.

This is the only correct way of revising scheduled estimates. Anything else is based on wishful thinking, and will result in less accurate schedules. The above technique can obviously be re-applied as work progresses, taking in the first six, eight, ten weeks and so on. It should converge fairly rapidly to a value that will indicate the true project completion date.

Management hates this technique, so keep it to yourself until you've got convincing data that it works within your organization. It's easy to apply, even in retrospect if your organization keeps any kind of records at all. Every project I've worked on, from international scientific collaborations to small-team software development projects to development efforts involving dozens of developers in several organizations, has helped me demonstrate to myself the validity of this law, which is nothing more than the observation that development speed tends to be constant throughout the schedule. No activities go faster than any other. Crises and failures don't happen preferentially at some points and not others.

Taking a realistic attitude toward schedules will give developers time to do the job properly, as opposed to trying to squeeze them into doing a bad job in too little time, which will always result in them taking longer anyway, and producing far buggier code. Until software organizations learn that design, reviews and testing are integral parts of the development process, and need to be scheduled for, we will continue to be given buggy code that doesn't meet our requirements. Just like Netscape.

Find Enlightenment