Wikistats upgraded to new page view definition

tl;dr New and upgraded Wikistats data files, reports and charts, with cleaner metrics.

Recently Wikimedia Foundation has upgraded its data feeds for hourly page view counts, using a new definition which excludes crawler traffic. These new data are now available from May onwards (backfilled).  A big THANK YOU to the Analytics Team.SnippetAnalyticsTeam
16_Oliver_Keyes_-_Wikimedia_Foundation_016 and Oliver Keyes

As prominent consumer of these data feeds Wikistats couldn’t stay behind. Besides a major upgrade to existing files, charts and reports (see below) new charts were added as well.

New charts

There are now charts for overall totals per project, for 6 metrics: Total Editors, Total Edits, Total Articles, New Articles, Total Page views and Active Wikis.

Some examples:

Often in the news: the editor trend on Wikipedia. Here all Wikipedias taken together show a pretty slow overall drop since 2007 (a sharper decline on English Wikipedia is almost offset by increases on other Wikipedias). Trend for very active active editors is totally flat. 

This chart shows how editor activity spiked on Wikivoyage, early in 2013, just after the fork from Wikitravel.

Early in 2013 all interwikis were migrated by bots to Wikidata, which caused lots of bot activity. 

A new metric: Active Wikis, indicate per project how many wikis are actually being maintained. The threshold of 3+ active editors per wiki is of course arbitrary. Open for discussion.

BIG CHANGE: In new data feeds all crawler traffic is (finally) filtered out. Overall this results in a drop of around 20% page views (or actually bot requests). The drop is larger on small wikis and projects

Upgraded reports

Wikistats uses the upgraded data feeds to produce several sets of reports:

    • Monthly pageview reports for all wikis and projects, normalized/raw, mobile/non-mobile/combined. You’ll see a drop in page views after the upgrade in May 2015, but much of that is due to the new definition filtering crawler requests.SnippetMonthlyPageViews
      Transition to new page view definition is clearly marked.

  • Summaries per wikis, and now also per project.


Upgraded downloads

Daily/monthly per article page views

Wikistats grabs the upgraded hourly files, both the page views per article and per wiki, and aggregates each into several larger entities. For per article page views daily and monthly aggregates now also used the new feed.

Page view totals per wiki (aka ‘project totals’)


Wikistats collects the hourly per-wiki ‘projectview’ files, packages them in to a yearly tar file, and produces a large set of csv files available for download as one zip file (both of these, input and output, are here). The csv files include totals per wikis per hour, day, day of week, month, and more, and contain separate counts for WMF’s mobile and non-mobile sites.

Upgraded process flow

This diagram shows the upgraded process flow, and all files involved. Monthly Pageview Reports
Thanks for your patience.

This entry was posted in Nice Charts, Wikimedia Edit(or)s, Wikimedia View(er)s, Wikistats Reports. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *