I was at my local Tesco doing some shopping for the weekend while I was thinking about this topic. While I was bagging up my groceries the checkout assistant ran my card, handed me my receipt and also gave me a small card. I’ve included a photo of it here as it’s one of those things that almost seems unreal – here you have a multinational corporation going out of it’s way to seek comments from customers. The reverse of the card lists five different ways you can provide your feedback, from sending a text message from your mobile phone to filling out a comments card in store.

You can bet that Tesco are going to act on the information they receive. The Clubcard phenomenon that started back in 1995 was designed in part to provide customers with offers that were meaningful to them as individuals, but also to allow a supermarket chain to start building analytics on their customer base. It’s very comparable to the kind of information that websites have been gathering on their visitors, being able to see how small changes make a difference in areas such as site navigation, advertising response rates and so on. It almost feels like crowdsourcing, with a specific goal set and a wide range of possible contributors targeted. Almost, even, like Wikipedia. Hold on to that thought of feedback though – I’ll be coming back to it later.

The appearance of Wikipedia is something that almost anyone who has used the Internet is familiar with. The same boxy layout with puzzle-globe logo in the corner has been around for a number of years. As far as providing the core information that a reader would need, you would think that a well laid-out article, some references and images would be all the casual reader needs. Unfortunately it’s become clear that someone just passing through and spending five minutes to glance at an article would benefit from a shedload more information in an easy-to-digest format. Before we talk about what’s missing, let’s remind ourselves what a Wikipedia article looks like.

Firstly, let’s guess how good this article is. It’s engaging, well sourced, clear and deep in coverage. But is it something that site contributors agree is something of high quality, or is it an article that looks good but in fact is riddled with problems? The only clue in this case is the star that I’ve circled in red in the top right corner. In this case we’re looking at a Featured article, one of only roughly 2700 articles out of a library of over 3 million – that means the chance of randomly stumbling on a featured article is nearly one in a thousand – even then you have to notice the small bronze star and understand what it means. If an article doesn’t have this mark, the reader is left to what they can infer from the content in order to draw their own conclusions about quality.

Interestingly though, each article on Wikipedia is graded on a quality scale. This information can quickly help a reader work out how much caution they should treat an article with, how much work has been put into it and if there are any glaring issues. Unfortunately, information on article quality is tucked away on the discussion page where the casual reader is unlikely to find it. There are addons available that extract this information and present it on the article page, but they’re optional and require you to have a registered account there in order to use them. If Wikipedia is to restore confidence in what it is offering, it really needs to consider making article quality much clearer.

Work is also underway by outside organisations such as the University of California to analyse the history of articles and track changes. Powerful tools allow readers to instantly identify suspect information. They might not be able to do much about it, but at least they can be aware that the information needs further verification before repeating it elsewhere. The WikiTrust extension is currently developed as a plugin to MediaWiki and could possibly be computationally expensive – it may be something that the developers would want to draw in as core capability, again to improve the level of information presented to the reader.

Now, remember that thought about feedback? Well, it’s probably no news at all that collecting feedback from users is something that has been happening on the web for years. Amazon started collecting reviews for books, but it was only when they started asking readers if they found the reviews helpful that they started to take off. By displaying the most helpful reviews, Amazon managed to improve the quality of information presented to the reader through a completely automated system. In the same way, Wikipedia could use this technique to gain metrics on their articles, not just to see if someone reads them but to see if people like them. Maybe even pop up a simple request for more information if someone doesn’t like an article, and display all feedback prominently and centrally. The problem with Wikipedia’s current model is that it encourages feedback on article discussion pages. For an article in the doldrums that’s rarely examined, discussion page feedback can go largely ignored. By collecting it all centrally, the volunteer editors can gain instant feedback on what articles are liked and disliked, and prioritise their work accordingly. It really needs to start collecting reader information as much as possible as soon as possible, building a database of information and then acting on what the reports reveal. If the site is still visited by hundreds of thousands of visitors every month, collecting more information than a simple page impression is vital.

There’s a whole other piece to the puzzle though, and that’s usability. Making web pages and sites usable by a wide audience is seen as so important that the US Government has an extensive resource on the subject. In it, two things are mentioned that are seen as critical to the success: full end-to-end testing, and using groups of people unfamiliar with the product being tested. This means that to undertake meaningful usability analysis, groups of people that are completely new to Wikipedia should be let loose on the entire project and test a series of use cases by following a script. Unfortunately, Wikipedia’s own usability project falls short of these requirements:

The Wikipedia Usability Initiative is realized by a grant from the U.S.-based Stanton Foundation. The goal of this initiative is to measurably increase the usability of Wikipedia for new contributors by improving the underlying software on the basis of user behavioral studies, thereby reducing barriers to public participation. With an initial focus on English Wikipedia, eventually this research and development will be implemented across all languages and possibly to other Wikimedia projects.

Why is this, you might ask? The truth is, much of the user experience isn’t controlled by the Wikimedia Foundation who maintain Wikipedia, but is crowdsourced by the community of contributors that work on the site regularly and any changes have to be agreed by consensus. This means that an outside view is rarely if ever heard, and an internal feedback loop develops where decisions are taken in the interest of the community. These might not be in the interest of readers, especially if the readers don’t make their opinions heard. Wikipedia desperately needs a proper usability initiative, with wide-ranging ability to test all aspects of the project and not just the core interface or underlying software.

There are of course many other areas that Wikipedia can improve it’s reader experience. Detecting first-time visitors and providing them with an introduction is one, while introducing a reccomender system to analyse what an individual reader is reading and suggest other articles they might like to look at is another. Making more use of metadata embedded in article infoboxes in order to provide more powerful searching, such as “tenor opera singers born before 1945″ or “mountain peaks above x meters above sea level in Europe”. Being able to identify how many articles a citation is used in. All these things are additional features though – items that go above the baseline user experience and offer additional capability. It’s why I haven’t gone into them in detail here – in my opinion there’s a need to take care of the core features first, with the value-added stuff being able to follow on later.