Is Jakob Nielsen Too Simple to Be True?

Published on:

In his latest Alertbox, usability guru Jakob Nielsen gives us a book review of "Paper Prototyping: Fast and Simple Techniques for Designing and Refining the User Interface". While the topic is interesting by itself -- i.e. it is always better to take care of usability as early as possible rather than as a medicine for a catastrophic project outcome -- I have all my marketing-crap alarms flashing red at his prose. I hope the following excerpts will show you why:

By following this book's advice, designers on any mid-sized project will likely see an ROI of several thousand percent.

The R-word again followed with stellar percentage in a style that is close to a Nigerian 419 scam: read this book and you will become a millionaire. Not that ROI is a bad thing per se, but the language used and the lack of basis of such a bold claim are not likely to attract the kind of people who work on mid-sized projects and should benefit the most from those techniques.

It's a rough estimate, but I would say that the benefits from early usability data are at least ten times greater than those from late usability data. Late usability studies often add about 100% to the final design's desired metrics, but early usability can add 1,000% or more.

As usual with Nielsen's figures, everything comes by factors of 10, 100, 1000, etc. and the more zeros, the better, hence 100% in lieu of double. Note how Nielsen makes a comparison between usability timings (one bring you 10 times more benefits than the other) then backs it up with individual outcomes (one adds 100%, the other 1000% or more). The trouble is, adding 100% means doubling (1 + 100/100) the metrics while adding 1000% means getting 11 times (1 + 1000/100) the metrics, which means that the outcome from early usability data is more likely to be around 5 or 6 times the one of late usability data.

It makes me think of those soap ads with graphics showing that product X is 10 times better than product Y through some marketing-wizard convolution that beats any mathematics. Too bad I am an engineer, I always miss the poetry of marketing graphics.

Forty years of software engineering experience uniformly indicates that it's much cheaper to change a product early in the development process than it is to make changes later in the process. The most common estimate is that it's 100 times cheaper to make a change before any code has been written than it is to wait until after the implementation is complete.

Again some sensible assertion (better early than late) immediately followed by an absolute 10^n figure that falls from the sky (it is 100 times cheaper). In the scope of packaged products and shrink-wrapped software from an editor, it certainly makes sense. However in cases such as custom software development and web design, the real life is more about flexibly accommodating permanent changes on a budget and being fast too. Many clients need to see something more concrete than wireframe use cases and will start to grasp the reality at a much later stage of the process. In software development, new techniques such as Extreme Programming go further than ever before in multiplying iterations in order to accommodate changes as far as possible in the process (in the clients perspective, your flexibility is their right to change their mind). Forty years of software engineering experience is not likely to hold an absolute truth.

That said, usability is indeed important and can bring more benefits than eye-candy ROI figures. Many of those benefits you can find on Alertbox when the marketing machine is off and Jakob Nielsen feels like sharing some insight avec le peuple. I am not trying to make fun of him but as Zeldman says:

Why do you make fun of usability expert Jakob Nielsen?
We respect Jakob Nielsen. We just wish he hadn’t forbidden us to eat pork.