Some SOPA blackout stats

Here are a few plots and data I collected for the SOPA blackout from 18 January. Wikimedia Foundation is working on a wider coverage of the event.

Note that on the second day hourly hits to Special:CongressLookup page exceeded hits to SOPA_initiative/Learn_More. Probably part of the demand came from external referrers.

The huge dip in CongressLookup hits on the first day was during hours where most US citizens were asleep. Of course the CongressLookup page did not make much sense for non-US citizens.

Here is a list of most visited SOPA related pages on different Wikipedias during the blackout (24 hrs only).

 

Overall page requests to English Wikipedia during the blackout was not particularly high or low.

Posted in Nice Charts, Wikimedia View(er)s | Leave a comment

Wikipedia views visualized

In May 2011 I presented a new visualization tool which can playback all 400,000 Wikipedia edits for a random day, and show where and when these edits occurred, and for which language wiki. The tool also shows static maps for at-a-glance view of the global distribution of edits for a full day.

Recently I added two new maps:

Page views

Global distribution of page views on all Wikipedias combined This map (in the tool press 4) shows the global distribution of page views for all Wikipedia’s combined. Clearly certain parts of the world are better reached than others. No surprises here, but with this map you can examine this disparity in considerable detail.

Global page views and population density - split screen Global page views and population density - split screen Click for large version

The data shown are normalized for area, which allows direct comparison with a population density map (by SEDAC). In split screen mode you can see both page views and population density side by side: press ‘d’ (for density) repeatedly.

Mobile share

This map (press 5) shows the percentage of requests originating from mobile devices.

Percentage of page views from mobile devices

Percentage of page views from mobile devices

Note how these requests do not have to be directed to our mobile site. In fact roughly half of these requests go to the main site. As the coloring shows this percentage is quite different for different language projects. The English Wikipedia receives a far larger share of traffic via mobile devices than most other language projects. If you zoom in on Europe you can see clearly how UK stands out against e.g. Germany and France, where economic conditions are roughly comparable.

Population density 2010 - SEDAC Global distribution of edits on English Wikipedia Global distribution of page views on all Wikipedias combined Global page views and population density - split screen Global page views and population density - split screen
Extra large screenshots

>> Animation <<

Details

Detection of mobile device is done by scanning for certain keywords in the agent string as contained in the meta data which our servers receive for every request. Page views are per square kilometer. For each page view log record from our 1:1000 sampled log the ip-address is translated into latitude and longitude, using the free Maxmind database. Views are accumulated for a whole month per small region (here 1/8 degree squared), averaged per day, corrected for projection distortion (any projection of a 3D globe to a 2D surface produces substantive distortions) and colored for intensity. Cycle with ‘d’ between page views [V], split screen [V|D], split screen opposite [D|V], population density [D]. Known issue: both page views and population density screens do not pan in sync, therefore no split screen at larger zoom levels. But you can still alternate between both full screen views. Compare global share of views from a mobile device as shown here (~12%) with share of mobile views to our mobile site (~6%) shown in our monthly report (column Wikipedia Mobile, top-right percentage).

Posted in Nice Charts, Wikimedia View(er)s | Leave a comment

Variations on the English Wikipedia Main Page

The WMF servers receive a lot of unserviceable page requests. To illustrate this, with what most likely is an extreme example, here is a list of page requests received in July 2011, which target any article with a title starting with ‘Main_Page’. Clearly most faulty  requests come from buggy software, not directly from users.

Posted in Wikimedia View(er)s | Leave a comment

Saving lifetimes

One days Jobs came into the cubicle of Larry Kenyon, an engineer who was working on the Macintosh operating system, and complained that it was taking too long to boot up. Kenyon started to explain, but Job cut him off. “If it could save a person’s life, would you find a way to shave ten seconds off the boot time?” he asked. Kenyon allowed that he probably could. Jobs went to a whiteboard and showed that if there were five million people using the Mac, and it took ten seconds extra to turn it on every day, that added up to three hundred million or so hours per year that people would save, which was the equivalent of at least one hundred lifetimes saved per year [1]. “Larry was suitably impressed, and a few weeks later he came back and booted up twenty-eight seconds faster” (‘Steve Jobs’, by Walter Isaacson, page 123).

Personally I still blame Microsoft for not introducing thousands separators into the dir output until MS-DOS 6. With hundreds of millions of users in the early 90′s, every quarter of a second wasted, several times a day, to read a 8-9 digit file size, added up to a comparable waste of lifetimes as above.

How does this translate to Wikimedia? With over 15 billion page views each month [2] each 1/10 second which is shaved off from page loading time saves humanity 1,500,000,000 seconds each month, which is very close to the waking hours spent by a 70 year old person. (70*365*16*3600). So the awesome dedication of the small Wikimedia operations team (staff AND volunteers) did not only save Wikimedia tons of hardware. It saves tens to hundreds lifetimes a year!

Of course all the ingenuity in the world does only go so far to accommodate Wikimedia’s ever increasing traffic. That’s why Wikimedia’s annual fundraiser, which is about to launch, is so vital to keep access to all our content fast, all over the world.

 

1 This is actually an overstatement: if every user saves 10 seconds per day this is roughly an hour per year. A 70 year old has been awake for 400,000 hours. Five million people saving an hour per year equates to 12 lifetimes.

2 Total file requests (images, scripts, etc) is even an order of magnitude larger (see image).

Posted in uncategorized | Leave a comment

Summary Reports for all Wikimedia Wikis

These weeks I am performing long overdue maintenance on Wikistats. This includes fixing bugs (e.g. Wikibooks en Wikiversity reports were broken for many months). This also includes automation, removing manual steps from the production process. I am also making good a long standing promise to publish summaries for all Wikimedia wikis.

These summaries were originally introduced for the monthly India Report Card, with content and layout suggestions from Wikimedia researcher Mani Pande. Hopefully they serve a wide audience. Hopefully they help to quickly assess fundamentals for any Wikimedia wiki, without getting avalanched by too many details from unwieldy tables.

Where to find these summaries

There are sets of summaries, also known as report cards, per project: Wikipedia, Wiktionary, Wikibooks, Wikinews, Wikiquote, Wikiversity, Wikisource and Other Projects.

For Wikipedia there are also sets of summaries per region: Africa, America’s, Asia, Europe, India, Oceania, and also for Artificial Languages.

Finally for every wiki there is also a new ‘Summary’ link in the project sitemaps: Wikipedia, Wiktionary, Wikibooks, Wikinews, Wikiquote, Wikiversity, Wikisource and Other Projects.

I am open to suggestions what to include further into these summaries. Yet their very purpose is to offer a quick at a glance overview, so this puts some restraints on which information to add.

Update 27 Sep: I added extra charts and metrics for Commons.

 

Posted in Nice Charts, Wikimedia Edit(or)s, Wikimedia View(er)s, Wikistats Reports | 2 Comments