Issues with models

I’ve been reading N.N. Taleb’s The Black Swan recently, and while assuredly late to the party on this one(1), it’s provided me with the intellectual shock I’ve needed for some time. For some background on my interests, I am enthralled with the bizarre and difficult to coordinate world we find ourselves living in. I derive great interest from watching, thinking about, studying and of course to a degree blathering on about how complex social systems work – I also like math and human psychology(2) – which originally led me to look at a couple prime examples of such systems, the financial markets as well as humankind’s various economies.

Naturally being in school and knowing very little I was drawn to the economics faculty (in Japan finance is often studied at the Econ faculty rather than Commerce), and as my grasp on classical (i.e., neo-classical) theory, modern financial theory, as well as statistics among other things started to come together, the more clear the contrast between these fields’ strengths and weaknesses became. There is a lot to talk about on this one, but for now, as Mr. Taleb’s wonderful prose has struck a chord with me and parallels exactly how I’ve been feeling, I am just going to touch on the economics face of this many-sided object for now.

大晦日に日が暮れてゆき、酒が流れてくると案の定ことしも準ブラックジャック大会...

More specifically, the absurdity of the thought that general equilibrium models could possibly be used to model anything besides a single instant in time in a particular complex system. As anyone who has worked with such models knows very well, if your parameters change, your results can be impacted drastically. Even if we cannot predict the degree to which some element will be input into our modeled system, as long as its structure (the exogenous variables) remains intact it really isn’t difficult to predict the set of outcomes one might see.

What are some examples of such parameters? Consumers’ preferences for particular goods, their outlook towards the future (i.e., propensity to save), investors’ subjective valuations of financial product risks, supposedly “objective” valuations of the same financial product risks (based on projections derived from past data), among countless others.

You don’t need me, or anyone to tell you that of course, these variables change. So we’re looking at a dynamic environment. These models can incorporate change over time to the parameters, so they should be able to withstand such a setting, right? Unfortunately, we’re actually in an environment in which these variables change instant to instant, with social trends and the spread of information making it impossible to track what changes are taking place where, thus leaving us not just in a situation where we cannot predict to any degree the volatility of our model’s current parameters, but in a state of being blind to the possibility of as yet unseen factors. Civil wars and government overthrows, massive earthquakes and nuclear power plant meltdown scares,  the sudden death of political leaders, all can come entirely out of the blue to an unwary public.

These informational shocks alone will strike a blow to our beloved model which makes it unwieldy (if not impossible) to use, but on top of that it isn’t just a matter of what information spreads its way through a society but how that information is distributed and presented. We are very information-sensitive creatures. When fear of physical harm and a lack of knowledge due to massively unreliable government “standards,” people act differently than they would should the information, even with regard to a crisis situation, be presented clearly and more definitively. When there were scares of contaminated water in Tokyo in March and April of 2011, my local supermarket in Osaka (hundreds of kilometers away) was entirely sold out of bottled water, something I had never once seen before.

So what is the point of learning these models? They make an excellent environment for thinking about Gaussian randomness and looking at static systems frozen in time. Thus, they become a necessity to pass admissions exams and to converse with neo-classical economists. They make reality look beautiful and clean, showing us a plethora of equilibria which our studious (rational) economic agents happily realize for us. Unfortunately, their predictive value is zero, and since they can lead us to take at face value a highly condition-bound scenario as reality, thus encouraging Taleb’s confirmation bias and potentially being more harmful than accepting random results (I’m quite interested in getting into studying model risk).

In dealing with the messy complexity of reality, and truly dealing with the real risks we face, it seems the greatest assets we have at our disposal utilize are a set of prying eyes which scours the horizon for empirical reality, and mathematical tools which make it possible to allow for the unknowns we are not yet aware of. In any case, my time is presently (approximately) halved between getting to know said flawed tools which are currently used, and spending time thinking about why certain “unforeseen” social issues have arisen and continue to persist(3).


1. To be fair, I spent a long time through 2009 into early 2011 in a Japanese black-out, where besides emails with family members and a few close friends I had essentially (intentionally) zero exposure to English media.
2. …and document design.
3. I’m presently waist-deep in Japanese bureaucracy and life insurance law…

Leave a Reply