Some interesting observations on the limits to econometrics and the modelling of risks

Travis Fast

It would be fun to trap your run-o-the-mill economist in a room with this guy.  But the really interesting point he makes in his article is the degree to which optimizing behavior actually increases catastrophic systemic  risk.

Excerpt from:

THE FOURTH QUADRANT: A MAP OF THE LIMITS OF STATISTICS [9.15.08]
By Nassim Nicholas Taleb

The Dangers Of Bogus Math
I start with my old crusade against “quants” (people like me who do mathematical work in finance), economists, and bank risk managers, my prime perpetrators of iatrogenic risks (the healer killing the patient). Why iatrogenic risks? Because, not only have economists been unable to prove that their models work, but no one managed to prove that the use of a model that does not work is neutral, that it does not increase blind risk taking, hence the accumulation of hidden risks.

My classical metaphor: A Turkey is fed for a 1000 days—every days confirms to its statistical department that the human race cares about its welfare “with increased statistical significance”. On the 1001st day, the turkey has a surprise.

The graph above shows the fate of close to 1000 financial institutions (includes busts such as FNMA, Bear Stearns, Northern Rock, Lehman Brothers, etc.). The banking system (betting AGAINST rare events) just lost > 1 Trillion dollars (so far) on a single error, more than was ever earned in the history of banking. Yet bankers kept their previous bonuses and it looks like citizens have to foot the bills. And one Professor Ben Bernanke pronounced right before the blowup that we live in an era of stability and “great moderation” (he is now piloting a plane and we all are passengers on it).

Figure 3 The graph shows the daily variations a derivatives portfolio exposed to U.K. interest rates between 1988 and 2008. Close to 99% of the variations, over the span of 20 years, will be represented in 1 single day—the day the European Monetary System collapsed. As I show in the appendix, this is typical with ANY socio-economic variable (commodity prices, currencies, inflation numbers, GDP, company performance, etc. ). No known econometric statistical method can capture the probability of the event with any remotely acceptable accuracy (except, of course, in hindsight, and “on paper”). Also note that this applies to surges on electricity grids and all manner of modern-day phenomena.

Figures 1 and 2 show you the classical problem of the turkey making statements on the risks based on past history (mixed with some theorizing that happens to narrate well with the data). A friend of mine was sold a package of subprime loans (leveraged) on grounds that “30 years of history show that the trade is safe.” He found the argument unassailable “empirically”. And the unusual dominance of the rare event shown in Figure 3 is not unique: it affects all macroeconomic data—if you look long enough almost all the contribution in some classes of variables will come from rare events (I looked in the appendix at 98% of trade-weighted data).

Now let me tell you what worries me. Imagine that the Turkey can be the most powerful man in world economics, managing our economic fates. How? A then-Princeton economist called Ben Bernanke made a pronouncement in late 2004 about the “new moderation” in economic life: the world getting more and more stable—before becoming the Chairman of the Federal Reserve. Yet the system was getting riskier and riskier as we were turkey-style sitting on more and more barrels of dynamite—and Prof. Bernanke’s predecessor the former Federal Reserve Chairman Alan Greenspan was systematically increasing the hidden risks in the system, making us all more vulnerable to blowups.
By the “narrative fallacy” the turkey economics department will always manage to state, before thanksgivings that “we are in a new era of safety”, and back-it up with thorough and “rigorous” analysis. And Professor Bernanke indeed found plenty of economic explanations—what I call the narrative fallacy—with graphs, jargon, curves, the kind of facade-of-knowledge that you find in economics textbooks. (This is the find of glib, snake-oil facade of knowledge—even more dangerous because of the mathematics—that made me, before accepting the new position in NYU’s engineering department, verify that there was not a single economist in the building. I have nothing against economists: you should let them entertain each others with their theories and elegant mathematics, and help keep college students inside buildings. But beware: they can be plain wrong, yet frame things in a way to make you feel stupid arguing with them. So make sure you do not give any of them risk-management responsibilities.)


Advertisements

2 thoughts on “Some interesting observations on the limits to econometrics and the modelling of risks

  1. That was way funny and way scary. I loved his dry sense of humour, and using the turkey metaphor was visually perfect. And children would understand this, if they know about raising turkeys for thanksgiving.

    I don’t think he wants to be alone in a room with you.

  2. I liked the line about the chief role of economists was to keep students in buildings. You should google his name there is an audio clip out there where he does an interview on bloomberg. Despite the fact that he is a quasi libertarian he is worth a listen to. Mind you his libertarianism seems to stem from the fact that he is convinced that the government is effectively captured by powerful economic interests that are more interested in gaming the system than taming the system. Seems to be a reasonable assessment of the US.

    The other thing worth mentioning is that many of his observations are very close to Keynes work on statistics. Keynes pointed out that uncertainty cannot be reduced to risk. Unknowns are unknowns which can’t be modeled . Aviation engineers know this: caution and redundancy is the only defense against uncertainty. Deregulation is about optimization not about prudence.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s