Think twice before buying the Nielsen Norman Intranet report
For the first time in several years, I am not going to recommend buying the Neilson Norman Top Ten Intranets Report. Jacob Nielsen and his team have done a lot of good for intranet design and the usability profession too. I have a lot of personal respect for Jacob Neilson as well but I think this report can be better. Just by reading his 10 Best Intranets of 2007 alertbox, you will see why I have gotten turned off. Here are a few of the reasons.
I struggle with the methodology and some of the conclusions drawn in the report. As a PhD with a commendable track record of publications behind him, I would have thought that Jacob Neilson and his team would be more careful about some of the statements made. Apparently, because the buyers are mostly from the business world, the authors donât feel that intellectual rigour is necessary. And for most people maybe it isnât. But since the Nielsen Norman Group prides itself on its academic credentials, I had hoped for more from them.
For example, the alertbox article in highlighting findings from the report says “Sweden’s continued top placement, however, is striking. Maybe there’s something to the claim that Scandinavians emphasize good design.” A little later in the article Nielsen also says, “Having India join the ranks of winning countries is a clear symbol of its growing might as a software superpower.” This begs the question, do they have any statistically significant data to support those arguments? Or at the very least are they getting enough submissions to make these sweeping assertions?
We have absolutely no idea, how many intranets were submitted for the award. Nor do we know how the competition was promoted in different countries. That alone can heavily influence the number of intranets being submitted for the award from a specific country. Furthermore, the criteria for choosing an intranet, is vague at best. And unfortunately, the criteria appears to be determined after the fact - once the submissions are sent in and not when submissions are being solicited.
Another example is the sweeping generalizations made about usability in the manufacturing sector without tell us how many manufacturing companies sent in submissions for the award. There may have just been the four that won. Similar generalizations are seen when discussing technology products too. Just because the 10 winners use certain technology products, it doesnât mean that those products are special and deserve to be highlighted. There is a tenuous relationship at best between software products and the usefulness of an intranet.
Along with the sweeping generalizations, are the over simplifications that are irritating. For example, the report emphasizes that standardized navigation bars, menus and page footers are the most common elements in consistent intranet user interfaces. This does not sound particularly insightful especially if you have bought the reports in the past. Another example is when the report emphasizes that the employee directory search is the killer application on most intranets. I donât think one needs to buy an expensive report to learn that.
So should you buy the report? If you are a new intranet manager looking to see how other intranets look and feel, then the report maybe useful. It is stuffed with screen shots. However, if you have been running an intranet for at least a year and especially if you have bought the previous yearâs report, you may find this one a waste of money. Whatever you do, please donât take everything said in the report at face value. Without explaining more about the methodology and sample size, it is hard to take a report like this too seriously.
