Statistics

An article by Gazimoff    No Comments

It seems to have been a busy week for all things Wikipedia. The Wall Street Journal recently contained an article describing how the number of users had fallen by about 49,000 in the previous three months. The article was based on the research and doctoral thesis of Dr. Felipe Ortega and constructed over a period of three years. It’s as meaty a document as you would expect for a thesis, with the conclusions it describes taking some time to fully digest.

Erik Moeller and Erik Zachte, the Deputy Director and Data Analyst recently rebuffed the report’s findings in a blogpost, stating that they measure a contributor as someone who has made five edits instead of just a single one, and that those numbers are holding somewhat steady. It also contains a small number of other statistics, but doesn’t comprehensively respond to the issue of contributor decline:

• The comScore Media Metrix indicates that the site has received 6% more visits from September to October. This is a very small snapshot in time. There may be seasonal trends or other factors that have influenced this growth. A quarterly year-on-year growth figure would be more appropriate here, along with comparisons against the growth in internet availability worldwide. Regardless, the number of visitors does not translate directly to the number of readers that you have – for that, more in-depth metrics are required involving pageviews per visit, length of visit and so on.
• Churn figures for the number of contributors are not available. While the number of active contributors has been holding steady at about 40,000 there’s no information about how many leave and join over a quarterly period. Service providers actively track churn rates as an indicator of user dissatisfaction and indication that the user experience needs to be improved.
• The discrepancy between Dr. Ortega’s results and WMF’s own figures indicates a large volume of failed conversions from a reader to a full contributor. Analysis and understanding of what causes these failed conversions should be something that the WMF investigates and reports on, together with plans on how to improve the conversion rate. Again, the conversion rate is something that should be tracked in the same way as churn rate.

The blogpost from the Wikimedia Foundation has also included a small amount of information about their strategy plans for future growth. I’ve already commented on these in other blogposts, but I’ll summarise them here:

• The usability initiative is unlikely to deliver sweeping benefits as it is focused purely on the Mediawiki interface. As a large proportion of the user experience is made up of the way information is presented, and that presentation is controlled by the active contributors instead of the Foundation. It’s already been declared out of scope by the usability initiative, and as such the entire end-to-end experience will not improve.
• The strategic planning initiative is heavily siloed, and as such is unlikely to result in a cohesive end-to-end strategy without monumental effort. It’s also inefficiently organised and doesn’t work well at helping innovative new ideas to float to the top.
• The outreach programme is heavily biased in pursuing cultural and historical repositories such as galleries and museums. There are few if any controlled studies into reaching out to the public at large, understanding their perceptions and concerns and working to bridge the gap between repositories and the public. Usability testing, public focus group work, user experience feedback and so on should all be part of what the Foundation is trying to achieve.

I can understand that what the Foundation is doing is important, but it is concerning that there remain glaring gaps in both the metrics presented and the future plans offered. If they are serious about changing how they operate, the user experience should be an integral part of that. After all, every single piece of content they currently hold can be copied under Creative Commons. The Mediawiki software is available free under the GNU General Public License. The only unique aspects of Wikipedia are the name, the contributors and the experience. If someone else can deliver something better in one or two of those areas, the Wikimedia Foundation may be in big trouble.