What happens in that five percent margin of 95 percent probability? Two recent articles, one from the Wall Street Journal, the other from the Financial Times explore the statistical problem in near certainty – that the normal measure for statistical confidence, 95 percent, still leaves us with the possibility that the unlikely will occur.
Carl Bialik, the Wall Street Journal’s Numbers Guy examines three highly improbable events – the collision between two nuclear submarines, two satellites in orbit, and the disaster, averted by dazzling piloting, that occurred when geese flew into both engines of a commercial aircraft over New York. As Bialik notes:
Each of these collisions is the first of its kind, or nearly so. Never have two subs carrying ballistic missiles collided in 50 years of patrols by these war vessels, according to maritime experts. Nor have two satellites collided in the space era. And a search by the National Transportation Safety Board turned up just a handful of cases in the last quarter-century in which more than one engine was disabled by birds in the wrong place at the wrong time. None involved large passenger jets.
Knowing just how improbable these events are can help designers decide whether it’s worth taking steps to prevent recurrence. Yet calculating the probability is a lot harder than beating the house in blackjack.
The problem is that if you wait long enough and keep rolling the dice, so to speak, you’ll encounter the unlikely.
Using the scenario of a confined patrol area, science writers David Curran and Luke McKinney modeled the subs’ routes after the random motion of particles. That rough calculation yielded one collision every 500 years. Allow the subs to roam freely in the Atlantic, and the expected frequency plummets to once every 50,000 years.
Bialik has another great example on his blog:
Jessica Utts, the author of “Seeing Through Statistics,” cautions that events that seem unlikely at first glance may not be that unlikely given enough opportunity. For example, Utts says, if everyone dreamed about a plane crash once in their lives, a few thousand people would have the crash dream on any given night — including the night of a crash. “These specific incidences may be unlikely, but the combined probability of something similar at some point in time is probably fairly high,” Utts, a statistician at the University of California, Irvine, said about the recent collisions.
Stefan Stern looks at the problem from a management perspective in his column in the Financial Times. As he puts it “Bad risk-management helped get us into the current mess. It is vital that we learn the right lessons about risk from the crisis. What are they?” Citing “Six Ways Companies Mismanage Risk,” an article by René M. Stulz in the current issue of the Harvard Business Review, Stern says that extrapolating from what happened in the past fails to take account not only of the odds of chance happening, but that the odds of chance increase when the present conditions have significantly changed.
…narrow daily measures – in banking these are known as “value at risk” measures – have underestimated the risks that are being run. The assumption behind a daily measure of risk is that action can be taken quickly (through an asset sale) to remove that risk. But, as the current crisis has shown, such rapid moves become impossible when markets seize up.
Stern adds that maturity and experience provide a vital context for risk assessment. The longer you are around, the greater the probability that you will think more about the downside risks.