September 28, 2007

WHEN YOU PLAN FOR THE HIGHLY IMPROBABLE, YOU SACRIFICE THE PROBABLE:

CHAPTER ONE: ‘The Black Swan: The Impact of the Highly Improbable’ By NASSIM NICHOLAS TALEB

Before the discovery of Australia, people in the old world were convinced that all swans were white, an unassailable belief as it seemed completely confirmed by empirical evidence. The sighting of the first black swan might have been an interesting surprise for a few ornithologists (and others extremely concerned with the coloring of birds), but that is not where the significance of the story lies. It illustrates a severe limitation to our learning from observations or experience and the fragility of our knowledge. One single observation can invalidate a general statement derived from millennia of confirmatory sightings of millions of white swans. All you need is one single (and, I am told, quite ugly) black bird.

I push one step beyond this philosophical-logical question into an empirical reality, and one that has obsessed me since childhood. What we call here a Black Swan (and capitalize it) is an event with the following three attributes.

First, it is an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility. Second, it carries an extreme impact. Third, in spite of its outlier status, human nature makes us concoct explanations for its occurrence after the fact, making it explainable and predictable.

I stop and summarize the triplet: rarity, extreme impact, and retrospective (though not prospective) predictability. A small number of Black Swans explain almost everything in our world, from the success of ideas and religions, to the dynamics of historical events, to elements of our own personal lives. Ever since we left the Pleistocene, some ten millennia ago, the effect of these Black Swans has been increasing. It started accelerating during the industrial revolution, as the world started getting more complicated, while ordinary events, the ones we study and discuss and try to predict from reading the newspapers, have become increasingly inconsequential.

Just imagine how little your understanding of the world on the eve of the events of 1914 would have helped you guess what was to happen next. (Don't cheat by using the explanations drilled into your cranium by your dull high school teacher). How about the rise of Hitler and the subsequent war? How about the precipitous demise of the Soviet bloc? How about the rise of Islamic fundamentalism? How about the spread of the Internet? How about the market crash of 1987 (and the more unexpected recovery)? Fads, epidemics, fashion, ideas, the emergence of art genres and schools. All follow these Black Swan dynamics. Literally, just about everything of significance around you might qualify.

This combination of low predictability and large impact makes the Black Swan a great puzzle; but that is not yet the core concern of this book. Add to this phenomenon the fact that we tend to act as if it does not exist! I don't mean just you, your cousin Joey, and me, but almost all "social scientists" who, for over a century, have operated under the false belief that their tools could measure uncertainty. For the applications of the sciences of uncertainty to real-world problems has had ridiculous effects; I have been privileged to see it in finance and economics. Go ask your portfolio manager for his definition of "risk," and odds are that he will supply you with a measure that excludes the possibility of the Black Swan-hence one that has no better predictive value for assessing the total risks than astrology (we will see how they dress up the intellectual fraud with mathematics). This problem is endemic in social matters.

The central idea of this book concerns our blindness with respect to randomness, particularly the large deviations: Why do we, scientists or nonscientists, hotshots or regular Joes, tend to see the pennies instead of the dollars? Why do we keep focusing on the minutiae, not the possible significant large events, in spite of the obvious evidence of their huge influence? And, if you follow my argument, why does reading the newspaper actually decrease your knowledge of the world?

It is easy to see that life is the cumulative effect of a handful of significant shocks. It is not so hard to identify the role of Black Swans, from your armchair (or bar stool). Go through the following exercise. Look into your own existence. Count the significant events, the technological changes, and the inventions that have taken place in our environment since you were born and compare them to what was expected before their advent. How many of them came on a schedule? Look into your own personal life, to your choice of profession, say, or meeting your mate, your exile from your country of origin, the betrayals you faced, your sudden enrichment or impoverishment. How often did these things occur according to plan?

Black Swan logic makes what you don't know far more relevant than what you do know. Consider that many Black Swans can be caused and exacerbated by their being unexpected.

Think of the terrorist attack of September 11, 2001: had the risk been reasonably conceivable on September 10, it would not have happened. If such a possibility were deemed worthy of attention, fighter planes would have circled the sky above the twin towers, airplanes would have had locked bulletproof doors, and the attack would not have taken place, period. Something else might have taken place. What? I don't know. Isn't it strange to see an event happening precisely because it was not supposed to happen? What kind of defense do we have against that? Whatever you come to know (that New York is an easy terrorist target, for instance) may become inconsequential if your enemy knows that you know it. It may be odd to realize that, in such a strategic game, what you know can be truly inconsequential.


This is so silly on so many levels that you can hardly pick it all apart, but let's try:

(1) There are doves and ravens, Caucasians and Negroes, white sheep and black sheep, yet it is outside the realm of expectations that there are black swans to match our white ones? No, Mr. Taleb's central metaphor actually disproves his entire case (we need hardly point out that the fact of black swans has no impact whatsoever). It would be somewhat more accurate to say that if one considers a flock of white swans in the abstract, to the exclusion of all other human knowledge, then a black swan will be surprising. This makes for the perfect ivory tower intellectual exercise, but has nothing to do with human life as it is lived.

(2) As if black swans weren't trivial enough in their own right, the notion that it is the fact that the "significant shocks" in your life may not occur on precisely the schedule you imagined ahead of time that makes them "shocks" is truly inane. Remember the old apocryphal tale about the college president who addresses the incoming class and tells them to look at the student to the left and to the right and recognize that one of the three of you won't finish. Well, he could also tell you that most of you will marry, most have kids, most lose jobs, most move several times, all lose loved ones, many divorce, etc., etc., etc. Is the utter predictability of these life events really outweighed by the shock effect of their not happening at precisely predictable times? Is it not instead the case that the generally predictable and easily anticipated is just shocking to each of us in the particular?

(3) Not only was a terrorist attack along the lines of 9-11 predictable, but was frequently predicted, a staple of fiction, and had even been attempted already. In fact, the most shocking thing about 9-11 in retrospect is just how few people died. Cast yourself back to that day or to discussions of the earlier truck bomb attack and the casualty estimates were in the tens of thousands. It does not in any way cheapen the loss of life to note that we got off pretty easy compared to what we expected.

This raises a set of questions quite different from the ones the author intends:

Examining our actual losses to terrorism(*)--rather than our emotional reactions--might we not say that our level of preparation was reasonable? Mr. Taleb suggests that if we were serious about the risks we'd have hardened the doors of cockpits and, thus, prevented any attacks on 9-11. It's worth noting, first off, that there hadn't been a successful hijacking of an American flight in twenty years, so existing precautions had worked rather well. But, even if we fall with him into the trap of believing that by taking discrete actions we've anticipated the unexpected, we have to ask why he'd stop there. Given that terrorists have more typically attacked with car bombs, truck bombs, explosive belts, etc., what are the similarly basic steps he'd take to protect against the entirely predictable threat of common motor vehicles and ordinary pedestrians? Obviously if he proposed banning motor vehicles we'd grant that he takes the threat of their being used as weapons seriously and following where black swan logic leads him. However, isn't that the problem with the black swan, that it becomes the be all and end all of existence? The existence of the black swan, in this way of thinking, clouds the mind to the far greater prevalence of white swans. One becomes so focussed on a risk that it can never be assessed in relation to the benefits that accrue from treating it as an outlier.

It would be useful here to consider the black swan that never honked. For forty years people anticipated, with various degrees of hysteria, the possibility or even likelihood that the United States and the USSR would engage in a thermonuclear exchange of some intensity or another. And yet, at no time during those years was anyone even minimally prepared for the event. Sure, preparing even inadequately would have required trillions of dollars and untold man hours and material and so on. In retrospect, we have to be thankful that the black swan was largely ignored, because as it turned out, the damage we'd have done ourselves by taking it would have been catastrophic in its own right. It would have made us the sort of military resource-sink that Paul Kennedy fretted about in Rise and Fall of the Great Powers. Instead we mostly went about our lives and the swans kept coming up white.

They usually do. When they don't, we figure out how to deal with the black one, often with little difficulty.

(*) Indeed, does 9-11 even meet the second part of Mr. Taleb's tripartite test? The US economy didn't skip a beat. We fought two rather desultory wars in the Middle East at minimal cost. They're building at the WTC site. The skies are filled with planes. Cory Lidle was able to fly a plane into a NYC building without anyone batting an eyelash. Etc. Where is the extreme impact?

Likewise, we're rebuilding New Orleans, despite Katrina, and the Pacific Rim is populated again. What long term impact do these swans really have?

Here are two really easily imagined black swans that we're all willfully ignoring and it's worth thinking about what you'd do to avoid them: a major earthquake in Los Angeles and a bit of space debris crashing into the Earth.

Posted by Orrin Judd at September 28, 2007 8:03 AM
Comments

Taleb's books aren't all that easy to read, nor is his thesis as new or radical as he seems to think. At the end of the day, about all I was able to take away from them is that it can pay to be a contrarian, which I already knew. As he points out, (drawing on Niall Ferguson), the bond market, the most sensitive barometer of conventional wisdom there is, was utterly undisturbed before the outbreak of WWI. Conventional economic wisdom was that a long war between the European powers was impossible because it could not be financed. But no one foresaw that Woodrow Wilson would allow the US to be used as the allies piggy bank and armory to sustain one side in a war in which we had no interest.

Posted by: Dan at September 28, 2007 9:26 AM

"...Remember the old apocryphal tale about the college president who addresses the incoming class and tells them to look at the student to the left and to the right and recognize that one of the three of you won't finish.."

At Virginia Tech in 1960 it wasn't apocryphal. (Actually it was the Dean of Students and he turned out to be an optimist.)

Posted by: Earl Sutherland at September 28, 2007 9:46 AM

One of the main takeaways from Ferguson though is also that Britain could easily ignore the black swan and should have.

Posted by: oj at September 28, 2007 9:58 AM

Black Swan's worth reading for the prologue but there's nothing there that's new or original.

Posted by: Ali Choudhury at September 28, 2007 10:06 AM

Dan,

Excellent point re: the author. Though I enjoyed the book's 3 or 4 key insights, I think you pretty much pegged it.

As for OJ's ridiculing the book, his points are mostly straw men. For example, it is certainly correct that 9/11 was easily predictable.

OTOH, had OJ's blog been up in the summer of 2001, and some article pointed out the dangers of using planes as missiles, one must ask what OJ might have made of such an article.

He may have lauded it as insightful, or riduculed it as the rantings of some iconoclastic outlier. Regardless, had some one argued for increased ticket taxes and much more aggressive searching for suitcases and carry-ons, they would have been stopped dead by airline industry lobbyists.

Even if such measures had been passed, the box cutters may still have been allowed on board. Maybe the box-cutter was the 'Black Swan', not 9/11.

As for the Swan that didn't honk - Thermo-Nuclear war - it merely serves to illustrate OJ's rush to ridicule the book instead of glean some value from it.

The collapse of the USSR was the "black swan" OJ, and yes, in hindsight, all of them "should have been predictable."
___

Much of this has to do with Rumsfeld's "unknown unknowns" comment that I believe OJ probably agreed with.

http://brothersjuddblog.com/archives/2004/07/given_that_even_the_subject_ca.html

For those wishing to pursue the issue further, you can basically distill the entire book into the table on P.284 of the hard cover.

Dan is right that it is nothing "earth shattering," but neither is it deserving of such ridicule.

That said, OJ's point regarding "sacrificing the probable" is accurate. Taleb himself makes the same point in the book. Invest 90-95% in stable mutual funds, and reserve a little to profit off of the "improbable," assuming you think you can find a way to do that.

Posted by: Bruno at September 28, 2007 10:18 AM

And the prologue is wrong.

Posted by: oj at September 28, 2007 10:21 AM

A recognition that man's ability to perform long-term planning is vastly overrated seems to me a good thing.

It may not be particularly novel, but what book is?

Posted by: Mike Earl at September 28, 2007 10:25 AM

It's idiotic for a young person to invest that conservatively given the return on higher risk instruments over a long period of time. But it's exactly the kind of self-defeating behavior that black swan logic causes.

I'd have grounded all airplanes. I don't think man should fly. Black swan logic is convenient for those who wish to do what society does not. Each of us will pick the black swan that serves our own ends, not the actual bird.

Your inability to even determine what the black swan is/was in any given scenario amply demonstrates just how useless the concept is.

Posted by: oj at September 28, 2007 10:37 AM

I think it's deserving of ridicule because it dresses up such pedestrian (and often incorrect) observation and analysis in such mystical, self-important tones.

Posted by: Jorge Curioso at September 28, 2007 10:41 AM

Mike:

Indeed, that's the one valid point, but having recognized that it's not white swans all along the road ahead, the idea you can plan on when, where, and what the black one will be contradicts the very thesis.

Posted by: oj at September 28, 2007 10:42 AM

The "Black Swan" of 11 September 2001 was the idea that someone would go through the effort recruit people willing to learn to fly a plane just enough to use it to knowingly commit suicide. All planning, and all sci-fi scenarios, were based on the assumption that the perpetrators wanted to live, even if it was in prison. (Even in fantasy/movie plots, the henchmen who carried out the mechanics of the plot might be sacrificed, but they wouldn't know that part and the Ernst Blofeld character always expected to survive. And we'd ever encountered a mondern-day death cult before, now we have, and can factor that into the planning.

One reason such attacks haven't happened again is not because of the security efforts at box-cutters and belt buckles, but because the pool of such people who could carry out such attacks is tiny, and as Richard Reid and subsequent incompents have demonstrated, people now have a motivation to not appease any would-be in-flight troublemakers, and how to recognize them before things get too far gone. We could revert to pre-2001 era security and not be any less secure because of that attitude change and because of minor procedural changes like locking the door have made this particular "Black Swan" even more improbable.

The USSR's collapse was a "BlackSwan" only because the people we were paying to anticiapte such things had a vested interest in not telling us what was really going on (Left Wing academic types) or were incompetents engaged in flattering those who paid them (CIA) or outright liars (Gorbachev on down). It was "unexpected" only because GIGO takes precedence, and any plan is only as good as the information it's based on.

Posted by: Raoul Ortega at September 28, 2007 11:00 AM

The Millenium plotters would have been suicides, no?

The pool of those who would do it is quite large. The pool of those who can plan it and carry it out is rather small. Given that, why should a society invest heavily in thwarting them? It's basically a minor problem, no matter how emotionally appalling.

Posted by: oj at September 28, 2007 11:56 AM

Aren't these just Kuhns' paradigm shifts and that panda's thumb guy's punctuated equilibrium?

Wow, humans are not all-seeing and thus all-knowing due to physical and psychological limitations?

Amazing...

Posted by: Benny at September 28, 2007 12:30 PM

If the paradigm were actually capable of being shifted would we be rebuilding New Orleans?

Posted by: oj at September 28, 2007 12:40 PM

About in 1994 a light plane crashed on the white House lawn. At that time I thought about using a plane as a missile. I didn't give too much thought to it because I considered the threat of that low. Here's the link.

http://query.nytimes.com/gst/fullpage.html?sec=health&res=9903E1D6173BF930A2575AC0A962958260

Posted by: Mikey [TypeKey Profile Page] at September 28, 2007 12:57 PM

OJ: Ha! Agreed. But that's why we're right-wingers, no?

Posted by: Benny at September 28, 2007 1:01 PM

How many copies did Tom Clancy's Debt of Honor sell?

Posted by: oj at September 28, 2007 1:10 PM

Yes, because we recognize human nature is immutable.

Posted by: oj at September 28, 2007 1:14 PM

OJ said:

"but having recognized that it's not white swans all along the road ahead, the idea you can plan on when, where, and what the black one will be contradicts the very thesis."

Nor would Taleb say otherwise, nor would he have said that the non-white swan would necessarily have been black. He has apparently made a good bit of money betting that, in some unpredictable way that he does not pretend to be able to devine, a consensus in markets (and human affairs) can be upset dramatically.


For example, if I follow his thesis correctly, one would note that the market for long term treasury bonds actually dropped a bit (i.e., bond prices went down, rates went up) when the Fed cut rates a few weeks back, indicating that long bond buyers are still pricing an inflation risk into the long bond. The Black Swan would be deflation, so a cheap way to play to profit from its appearance would be to buy some far out of the money options on 30 year treasuries. If I have no particular reason to expect deflation, nor would adding such contracts to my portfolio hedge any extant position I have, then I think I have made what Taleb would consider a pure "Black Swan" play. (In fact, I personally feel that we are living in an essentially deflationary world, so I am only stating what I think a trade that would satisfy Taleb's criteria for a "Black Swan" trade would be.)

As someone noted above, in so far as Taleb's views encourage humility, they are not at all bad. But between this book and his prior "Fooled by Randomness" he has killed a lot more trees than needed to make a point that other traders have made before him.

Posted by: Dan at September 28, 2007 2:55 PM

The current epoch of deflation is a truism:

www.brothersjudd.com/index.cfm/fuseaction/reviews.detail/book_id/1369/

It isn't shocking, extreme, nor need it be seen only in retrospect.

Posted by: oj at September 28, 2007 3:13 PM

OJ,

The concept may be useless, but the many insights in the book are not.

For example, you seem to have extrpolated a thesis from the book that probably isn't even the thesis the author intends.

You wrote: (above) Indeed, that's the one valid point, but having recognized that it's not white swans all along the road ahead, the idea you can plan on when, where, and what the black one will be contradicts the very thesis.

Nowhere in the book is Taleb trying to get you to "plan on (nor for) when, where, and what" the black swan will be. If anything, the book is a too long explanation that whatever a black swan is, the probability that you will be able to predict it is pretty slim, but that its impact on you will be larger than you would have expected.

If the concept has any use at all, it is to be very wary of experts and their predictions. Even the most well-read blogger in the nation might be wrong in reporting that McCain will be Republican Nominee or that 2004 or 6 would yield a 60/40 senate.

Posted by: Bruno at September 28, 2007 3:32 PM

However, the Black Swan is, in fact, obvious and has little impact.

For example, there's no meaningful difference between the GOP candidates. Folks will hyperventilate if the one they expect doesn't win, but the range of possibilities is incredibly narrow and the impact marginal.

Posted by: oj at September 28, 2007 4:02 PM

OJ:

I agree that deflation is baked in the cake we currently are eating. The point is, that if one were agnostic about that, the market nonetheless offers a cheap way to make a ton of money if the reality suddenly becomes evident to the bond market and rates fall like a rock. One can make such a bet, and profit greatly from it, even if one has no idea or any expectations what interest rates will be next month, next year, etc. (The risk of inflation appears to be reflected in the behavior of the bond market, so the payoff probably isn't as great if you sold options on the long bond, but I have not looked at prices.)

Posted by: Dan at September 28, 2007 6:09 PM

What does any of that have to do with the Black Swan though?:

"First, it is an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility. Second, it carries an extreme impact. Third, in spite of its outlier status, human nature makes us concoct explanations for its occurrence after the fact, making it explainable and predictable."

Posted by: oj at September 28, 2007 9:19 PM

Democratic governments are dedicated, to a large extent, to making it possible for the voters to count on white swans all the way. "If you play by the rules, you should be able to expect..." How far would a politician get by suggesting that playing by the rules should get you a lottery ticket?

The military can't rely on white swans, but it would be fatally wasteful to plan on black swans. The military plans for the most *critical* likely possibilities, because critical failures are so costly to them. The common fate of military units, then, is to be designed for a critical task, and then be used for less critical but more likely jobs for which they happen to be most nearly suited.

You probably remember the jokes when we sent the 10th Mountain Division to Haiti, which is not known for its ski resorts. However, it is also not known for adequate transportation infrastructure. Mountainous terrain coincidentally is not known for ease of transport, so one of the major differences between a mountain division and other kinds of divisions, is that it puts fewer demands on transportation. This was the match between the 10th Mountain and Haiti.

More recently, there were jokes about the US sending aircraft carriers to do tsunami relief. Tsunami relief is not militarily critical enough to justify building dedicated tsunami relief carriers.

And when a Black Swan pops up, it likewise gets handled with the most nearly suited resources.

Posted by: Bob Hawkins at September 29, 2007 10:26 AM

Each of us has a certain amount of time in a day, in a year, to deal with threats to our existence. So we deal with those threats that are most common to our existence. And then deal with all of the rest of our lives.
Common sense.

Wild outflanking maneuvers don't fall into that because they have to penetrate many levels of defense to reach and hit. Finding people intelligent enough to be trained to fly a multi-engine jet; who are suicidal; to have them trained; to get them into the United States; to make sure one of them won't turn; to take over the aircraft in flight; to get to the controls while the aircraft is in flight and not have the aircraft go out of control.

It was a wild maneuver, a concatenation of circumstance that is hard to recreate, let alone create once. And the minimal defenses in place wil now add another line of defense. The chances for this particular event to occur again here are even less.

Especially when the passnger reaction is taken into account.

Posted by: Mikey [TypeKey Profile Page] at September 29, 2007 3:26 PM

And yet, Bob, a nuclear carrier has a flight deck for helicopters, its engines can distill gallons of fresh water, its hanger daeck can preposition a great amount of supplies (coordinating with its support group). It has wonderful communication facilities. While the roads and railroads are down the ship can cruise the seas off shore and go from population point to population point.

A carrier is a staging point for those ops, and very capable of hosting them. In the 1930's the Saratoga(?) pulled into Tacoma when its electric plant was down and provided turbo-electric power to the city. Never discount the logistical train of a first world military. Can-Do is not just for the SeaBees.

(Not ragging on you Bob - adaptability is there. Like a mountain division sent to a tropical island.)

Posted by: Mikey [TypeKey Profile Page] at September 29, 2007 3:42 PM

The author's point about WWI is true - no one expected a massive body-sucking war. Days like the Somme were not imaginable in 1914. But surely they were by 1916.

All in all, WWI was much more damaging to the European psyche than WWII, despite even the Holocaust and the seeming victory of totalitarianism by June 1940. The run-up to WWII lasted many years, and virtually all France and most of England were in denial for a long, long time.

However, he is completely wrong about many other events - the rise of Hitler had been in the making since 1923, and who would argue that 1918 might not be the most accurate date? The demise of the Soviet bloc? He sounds like Paul Samuelson or the CIA here. The market crash in August 1987? 5 years of a raging bull (especially with the 1986 version fueled by the collapse of oil prices) augured for a correction, and a big one. It happened, and then the market just kept climbing. Same as in Sept. 1989 - a sudden big drop and then a continued steady climb. The rise of Islamic fundamentalism? We have been reading articles about the Muslim brotherhood since Sadat was murdered and the Grand Mosque was overrun by terrorists - that was almost 30 years ago. The rise of the Internet? The specific details might be surprising, but science 'fiction' on this topic has been around for almost 100 years. And OJ has demonstrated the folly of calling 9/11 a black swan.

The only people who are surprised are those with blindness to only see white birds. Like Pauline Kael in 1972.

Posted by: jim hamlen at September 29, 2007 10:11 PM

The interesting thing about WWI is that they completely ignored the Civil War which told them exactly what to expect.

Posted by: oj at September 30, 2007 8:45 AM

You are right, and I'll bet it was Euro snobbery that allowed them to ignore the potential carnage.

Of course, trench warfare and the machine gun made death a lot easier, but nobody was considering them in August 1914 (except perhaps Kitchener).

It's funny how few 'good' guys there are when reading about military history. For every Sherman or Guderian or Ridgeway or even Petraeus, there are scores and scores of losers, left-overs, peacocks, and fools. No wonder people keep looking at events as black swans - otherwise, it would be their responsibility.

Posted by: jim hamlen at September 30, 2007 9:16 AM

Correction - it was Saratoga's sister, USS Lexington, that provided the power.

Posted by: Mikey [TypeKey Profile Page] at September 30, 2007 9:58 AM
« "SPIRITUAL HARDBALL": | Main | JUST AS REAGAN FINISHED WITH A TRIP TO MOSCOW STATE...: »