Are solid metrics essential in a strategic debate? I want to talk about the pitfalls that come with over-reliance.
This post is not to debunk rational thinking in the decision making process, although I do question the supremacy of ratio in the human mind. This is about provoking critical but constructive debate on how numbers can be used effectively. Being a numbers guy myself, this also serves as a sanity check and as an exercise in modesty. I will present some examples of unforeseen successes and dramatic failures in strategic decision making. In most cases the stakes were high.
Even the founders of Wikipedia had no clue when they started the project of what it would accomplish. They dug a hole to find water, and struck oil instead. Any advance calculations of what would be needed to get us where we are now, would have clearly demonstrated the insanity of the whole idea. Wikipedia works in practice, not in theory. Just a few years later a equally ambitious plan, where volunteers map the whole world just by walking and biking around, seems not so ludicrous any more.
Numbers and Grand Human Endeavors
Solid numbers up to the nth decimal were indispensable to make Apollo 11 not overshoot the moon by a tenth of a degree, and the moon lander not run out of fuel 15 seconds earlier. Forty years on there are no solid numbers to justify the validity of going there in the first place. We know the costs, but do we know the monetized gains? How much did the US or the world gain in terms of advancement of science, Cold War containment, strengthening of national identities, inspiration and amusement? A discussion about the validity of the strategic decision to go to the moon is all about values and priorities, largely a debate without rational outcome.
The Human Genome Project was widely criticized, even ridiculed up to a decade ago, because of its huge costs and speculative usefulness. Sampling genomes of species has since then become routine and whole sciences are realigned by its findings. I wonder how often original dissenters confess they changed their mind.
Costs for grand human endeavors are almost invariably underestimated big time: budget excesses get buried by history, fame is eternal.
Numbers and Military Strategy
Generally speaking, if solid numbers on military strengths had any predictive value for the outcome of war, few wars would be fought at all.
Robert McNamara, a prime architect of the Vietnam War, famously regretted at later age how his bureaucratization of war with over-reliance on numbers helped to paint a distorted picture in distant Washington, where the US seemed to be winning for many years, and only journalists raised doubts.
Despite decades of military strategic thinking in the Cold War, and despite breakthroughs in game theory, the ultimate answer for strategists on how many nukes would be needed to assure safety, was ‘plenty, and then some‘.
The famed 300 Spartans at Thermopylae (and some allies) are an example in military strategy of how numbers can deceive.
In military history ‘against all odds’ is a recurring theme, and strategists know it.
This excellent visualization of Napoleon’s march into Russia in 1812 by Joseph Minard shows how only 10,000 out of 422,000 soldiers returned from the ordeal a year later. Yet this chart is not about numbers in the first place. Rather it is a value statement about war and strategic planning.
Numbers and Economic Strategy
In a world with more financial analysts and corporate executives than coins in the US national treasury (easy), paradigm-shifting economic upheavals and decades of mind-boggling financial fraud are often predicted or detected correctly by experts only after they heard about it on the news. Need I say more.
In ‘The Wisdom of Crowds‘ James Surowiecky makes the point that more often than not a group of independent laymen can numerically outwit, or rather out-estimate, one expert because widely different perspectives tend to straighten out overemphasis on one aspect of a situation, and remove blind spots.
In ‘The Long Tail‘ Chris Anderson elaborates on how advances in consumer technologies completely changed the numbers game, where music hit lists  largely lost their relevance, and film block busters will follow. The saying goes that generals tend to fight the previous war. Corporate strategists should equally be careful not to repeat themselves.
Numbers and Psychology
We humans tend to take large absolute numbers for granted and settle for a general notion of the direction of change. (do you know your country’s national debt? do you know whether it grew last year ?)
We generally have difficulty with odds even more than with large numbers. In ‘Against the gods – The remarkable story of risk‘ Peter L Bernstein explains that in a setup where roughly a 1000 people predict 10 (near-) random binary outcomes (say a coin toss, or the rise or fall of stocks) one out of 1000 by pure chance will ‘predict’ 10 outcomes flawlessly. That person will be hailed as top expert in the field.
I am convinced that, despite appearances to the contrary, strategic decision making often is to a large extent irrational, mainly relying on intuition, intuition in the sense of ability to decide/quantify without reasoned thought but based on internalized experience (forget mystical connotations). The ratio only kicks in afterward to rationalize the gut feeling, based on incomplete reports by the subconsciousness on the power struggle that waged between brain lobes just a second earlier. The feedback loop from consciousness to subconsciousness (internalization) may help to effect a different outcome next time. Hopefully decision makers have a healthy readiness to sharpen their intuition, by taking in (numeric) feedback after their decisions materialized. This is hard to judge for others, and a major stress factor in any democracy. Let me emphasize this is not a defamation of strategy makers. It is just as rough assessment of how we humans are wired. We just don’t want to admit it. “Mr President/Prime minister, why did you order a contingent of peace keepers to the South Pole?” “My dear fellow countrymen, because I wanted to, just because I wanted to”.
Numbers and Politics
Numbers are often used initially to reassure and convince (‘look, we did our homework’), and later to excuse unexpected outcomes (‘based on the data we were given’).
Large abstract numbers often are made digestible by discussing small examples (‘US tax policy’ turns into ‘Joe the Plumber’). Pars pro toto can easily distort and redirect the discussion.
Numbers are often abundant: its is very tempting to pick and choose the ones that help you to keep your convictions (psychologists call this ‘reduction of cognitive dissonance‘). Also numbers yield to interpretation: behind every number ‘there is a story’. In other words numbers are great for justification of the outcome of a process, within the own faction. Opposing factions feel equally vindicated, using different numbers or interpreting the same numbers differently.
My own stance in this: despite my focus above on the antithesis, in strategic decision making quantitative assessments can be very important, but to certain extent, over-reliance on numbers is a trap. Orders of magnitude should win over decimal points. Mutually independent ‘guesstimates‘, napkin or cigar box math, are often more effective as input for a meeting of minds and more on target (cost, time) than sheer number crunching, and keep the focus where it should be: human judgment, with full appreciation for all uncertainties. Cold war politics gave us the phrase ‘trust but verify‘. As a new adagium I would recommend strategy makers ‘fancy but frown‘.