The statistics we maintain within the Wikimedia movement broadly fall into two categories. Part is there to inspire us, with limited feedback to our daily activities. Part is there to reveal patterns or to signal mishaps, both meant to lead to actionable knowledge. Of course the boundary is not strict. Your qualification may depend on how you judge your own capacity to influence trends. Broadly speaking much of Wikistats currently falls inside the first category, I would say: numbers to inspire.
I do not mean to belittle the importance of general metrics. I would rate the superb talks by web celebrity Hans Rosling as being foremost inspirational, less actionable, for all of us normal earthlings, who seldom make decisions with global impact (except at the ballot). If you’re a dedicated contributor to one of our projects it can lighten you day when trends are favorable. It can awe the press and the public at large, it feeds into our fundraiser, even helps to open doors (be it to photo shoot opportunities, or to GLAM institutes). But other than that it is nice to know.
Much as I welcome more actionable metrics, still my focus in this blog post is on that inspirational aspect. Forget research, forget operations. As a thought experiment I want to raise this hypothetical question: If for one year we could only update one metric about any aspect of the Wikimedia movement, which one would help best to inspire you? It could be a complicated metric, but just one. Let me try to answer that myself. I’m side stepping issues on data gathering. For sake of argument let’s assume any metric is spot on. The issue is more how unambiguous each metric is and how much it tells about our success.
Would I choose page counts? Definitely not. We use this metric too often to impress our audience. Too often some of our projects play one-upmanship (or so it seems) with article creation bots. Also our definition of article is very permissive, and the barrier for importing data collections seems to lower steadily. Pardon the hyperbole: we could add millions of ‘articles’ from a star database any day. See also next metric.
Number of languages? We cite this everywhere so this must be an number which inspires. You may want to know that 83 of our wikis (of which 10 Wikipedias) are locked for updates, due to lack of editors. 61 or our 281 Wikipedias have less than 1000 articles. Many of those wikis exist for half a decade. And even some wikis with 2K+ articles may consist of year stubs mostly, at least that happened a lot in early years. There is a huge wikistats table showing article counts per wiki per month. Note how many wikis grew by 1K or 2K articles at once, early in their history, often bot induced stubs. That still leaves a whopping number of Wikipedia’s which are an unbelievable success. No need to stretch the imagination I would say.
Editors? Likely candidate, but the number is somewhat ambiguous. We could gain in editors but lose faster in average editor activity, for instance because of ever increasing competition of other web initiatives (social sites). Even a large influx of new editors by itself could overtax our veteran users, and ultimately cause burn-out and estrangement. While this seems to be a problem we can only dream of in many projects, it is already realistic in some others.
Edits? Attractive option. We already count edits by registered users on primary content (mostly namespace 0). Ideally we would substract reverts and reverted content (vandal fighters are arguably our most important lifeline, but if they would need to step up their activity due to increase in vandalism that would not exactly put smiles on our faces). Although edits on talk pages etc are very important, if they don’t lead to extra encyclopedic content, they are irrelevant, so let’s exclude those (like wikistats already does). This leads to ’non reverting/reverted edits, by registered and anonymous users (thus excluding bots) on primary content’.
Page views? Better navigation tools could have a negative effect on our page view count, and we’d still welcome that. It has been said that Google strives to get people from their site asap, as that signals they found what they’re after faster. Only part of our traffic comes from intentional searches. There is also lots of random browsing, which is great fun and educational, but still I would rate that differently (see below).
Unique visitors? If for some reason the average number of visits per person dropped faster than the number of visitors rose, our usage would decline and from this one metric alone we wouldn’t know.
Completion of targets? I said our metric could be a complicated one. So what about a (weighed?) average of the percentage fulfillment for each of our strategic targets for 2015? While useful for general guidance, I personally think targets in many cases are more about how optimistic we were at the time they were set, and less about realistic expectations (to some degree the most realistic expectations about complex social movements are: no expectations at all, trends might be distinctly non-linear). Goals define a direction, targets pretend travel speed can be predicted, which works better in explored than in new territory. Also, for some of our targets we still have no operational definition, or method to measure them reliably (content quality, gender ratio).
Total amount donated? If we would raise funds (with lower intensity) all year round the amount donated monthly could seem a nice metric to measure appreciation by our readers, and thus our impact on the world. But the average amount donated could fluctuate because of external influences. Also the message to the public, and how this is delivered, is always in flux, banners and stories evolve, so the playing field to compare consecutive measurements is not level.
Number of donations? That comes closer. I assume external (economic) factors will influence the average amount donated more than the decision to donate at all. Just my guess.
Gender ratio of our editors? If that would improve it would be hugely inspiring. But not my first choice, if we can sneak peek so little from under our blindfolds. (It could even mean more male editors dropped out).
Total visits? This would be my candidate of choice today. I imagine most people visit Wikipedia foremost to get a specific question answered. After their initial curiosity has been fulfilled they might browse further and learn more, but their initial question is what drove them to visit. If someone visits us once a month Wikipedia clearly has a limited role in their lives. For that share of our visitors which visit us several times a week or even more (*) something essential has changed. They have learned that the answer to many of their questions is within reach, and affordable in terms of time invested. They might even have become more inquisitive in general. That I find very inspirational.
*According to comScore we received 2.45 M visits to all our projects in January 2012, and 2.50 M a year later, with somewhat lower values in between (mobile access may be underrepresented). On average that would be slightly over 5 visits per unique visitor per month. Like with other metrics above, there are complications: seasonality, uneven distribution, socially or geographically, and external influences (mainly rise of mobile, which actually will amplify this metric).
Note these are my personal opinions, some based on limited anecdotal evidence.
I would love to hear your feedback. Which metric would you choose?
Sometimes entries in these lists seem oddly out of place. Any Wikipedia article can have tens of categories assigned to it. A popular article will rank high in any list where it’s featured, regardless of the category under review. Thus a well-known singer may be top ranking in a list about politicians, because he/she also played a minor or brief role in politics.
WMF publishes 
Wikipedia and sister projects.
