My previous post “Numbers and Strategy” warned for over-reliance on numbers during strategic decision making. First a disclaimer: neither that post nor this one are comments on specifics of the Wikimedia Strategic Planning process (although of course triggered by it, and hopefully with some relevance). The strategy project just started, first looks are promising. These posts are general musings on the topic, that reflect my interest in politics, history and (military) strategy, for what it’s worth.
In that previous post I presented examples on how history often disproves predictions which were based on quantitative estimates. Here I want to bring it one step further: semantically speaking strategy and uncertainty are inseparable. If a large human undertaking has a very predictable outcome, with costs and benefits that can be calculated with confidence, and where all experts agree on the best approach, then the word strategy does no longer apply (instead we use words like design and engineering) . The word strategy comes into play when important decisions with long lasting effects (cost, benefits) have a subjective component, where the views, values, intuitions and finally personal judgments of the decision maker(s) take precedence over mere calculations.
This does not mean numbers are useless in strategic discussions. WSP project leader Eugene Eric Kim asked me to expound on the flip side: “When, in practice, do numbers help us make better decisions? Where should we be using more of them, and how?” Let me give it a try:
1 Numbers can help to asses the current situation, locate the starting point, quantify the frame of reference.
Numbers can help to quantify a problem, but only after a problem has been properly defined, which is largely a matter of assessing values and priorities, not a calculable process. In politics different parties often disagree about which problems exist, and/or should receive attention and resources. Numbers can help to focus a debate and point out where differences of opinion matter.
Even when consensus exists about the nature of a deficiency and the need to address it, care must be taken to not define the current situation in terms of the proposed solution, as the solution may change during the process. For instance defining the deficiencies in long distance communications in Africa in terms of kilometers of missing land lines is an approach that did not stand the test of time.
2 Numbers can help to define measurable goals
Like with problems care must be taken to define goals in terms that are relatively stable over the period of implementation of the strategy. Otherwise revised definitions bear the risk of accusations of partiality: people may argue whether new definitions were made up to make progress look better.
Not only should metrics be as constant as possible, they should really go to the heart of the matter. And here effectiveness comes into play: in public debate nuances tend to get lost.
An example of where measurable goals did not go to the heart of the matter (which unlike the Africa example above should have been clear from the outset): The Netherlands have a very dense network of railroads. Due to congestion a large percentage of trains did not run on schedule for many years. Commuters complained and went back to their car (and congested roads). Our government then set targets for improvement where each year a certain and growing percentage of trains needed to depart within three minutes of scheduled time or a large monetary fine for the railroad company would follow. Improvements came slowly. Goals were redefined (depart within 5 minutes). Railroad management started to design new schedules that gave priority to reaching the timeliness target, even when it meant more passengers had bad connections and average travel time increased. Clearly here goal setting worked counter-productive. Setting a better goal is not easy though, and again involves values: reducing average travel time seems a better choice, but should we then treat all travelers the same, or optimize travel times for commuters at the cost of leisure travelers? Current thinking within our government is to abandon the schedule entirely on trunk railroads, and instead improve capacity and make trains run so frequently that people stop using a schedule and treat trains between the largest cities like metropolitan subways, just arrive at the station at a random time and wait a few minutes.
The ambition level of measurable goals is where non calculable factors come into play again. The road to success is paved with risks and uncertainties. What motivates better: earlier accomplishments on reachable goals or unfulfilled dreams? But that is another topic.
3 Numbers can help to track progress towards those measurable goals.
If progress is insufficient, we are back to human judgment why this is and what to do about it, in other words strategy.
An impressive example of a document where 1, 2 and 3 come together seems to me the yearly progress report on United Nations Millennium Development Goals. I have not followed the UN story about goal setting, definition of metrics or measuring methodology, but to me the report reads almost like a novel: clear language, well defined goals, statistics that give an outsider perspective and an insider overview. All eight goals have been made measurable. I’m sure experts will have lots of comments om what has been left out, or on oversimplifications, but the report helps to communicate a mission, and tells about accomplishments, or the lack thereof. It does address (many) areas where goals will not be met at 2015 at the current pace, it does not say much on how to redirect or intensify efforts. That is the realm of politics, values, commitment.
4 When strategy calls for a large investment (time, money, man power) more calculations can be less
Precise and detailed calculations can weave a web of unjustified confidence in the final outcome: I take it that when an architect designs a bridge or a tunnel he or she can estimate within reasonable limits the total costs. Somehow this does not scale up well to large infrastructural projects, like building a long distance high speed railroad. The time frame then changes. Politics on different levels, be it local, regional, national or even international make much of the outcome unpredictable. Budget excesses on multi-billion projects are often astronomical, and can give policy makers a hard time.
Perhaps it could help to talk about costs and gains and likelihood of outcomes in terms of bandwidths, probabilities and confidence. IT strategy firm Gartner uses that approach. Confidence can then be made explicit by asking all participants to compare analyses and recommendations and rate their level of trust or conviction, then weighed averages are published, not just which alternative gained most votes.