image

Well, there have been enough books about Central Limit Theorem bashings, Gaussian bashings , as applied to finance, thanks to the books like Blackswan and its ilk.

I had put off reading this book for quite sometime for I was expecting the same message – average is really an average! :), there are better ways to dealing with data than merely looking at the average. So, reluctantly I quick read this book a few days back. This book is extremely light read and could easily be read in a couple of hours.To my surprise, there were some good points in the book.

There are little essays on the various problems in applying “average” to situations.  There are about 47 little chapters in the book  relating to finance, health care, supply chain, war, .. you name any field a person might work, you will find some relevant example of “Flaw of averages” in the book. So, in that sense, the author has done a terrific job of providing the breadth.

The core message reverberating in almost all the essays is

You have to look at the world in terms of distributions and avoid using average as an estimate for the data.

So, in that sense, this book prods a user to use bootstrapping /monte carlo to work on real life problems. So, in that sense, an apt title should have been “ How to avoid Flaw of Averages using Simulation ? ‘

As we know that there are two laws of large numbers, one a strong law and a weak law, basic difference being that the former talks about almost sure convergence while the latter is a convergence in probability. This book mocks the two laws and presents Flaw of Averages in Weak form and Strong form! .

Here are some points from the book which I found interesting

  • Jensen’s inequality – Every options trader intuitively knows about it. The first time I came across Jensen’s inequality  , it was introduced in a completely mathematical way and was left at that point. Well, sometimes it is better to start with an example and then get the math behind it , rather than the other way round. There is a beautiful example in this book which illustrates an application of Jensen’s inequality that saves human lives. This example from this book was enough for me to justify spending time on this book. This superb example about the inequality, would make any reader,  forever remember the inequality because it has been presented as an application to save human lives.  It was a wow moment for me when I stumbled on to the application. How many times do we come across an inequality [ E[ f ( x1 + x2 )  ] >  or  < f( E[x1] ) + f( E[x2]  - depending on the concavity/convexity of f(x)  ] that can save lives! ?

  • Always build prototypes and test them. Wright brothers first model was a kite!( bicycle inner tube box)

  • Always keep track of where you are on the seat-of-the-intellect-seat-of-the-pants continuum.  Mathematics, Physics, Statistics belong to the seat-of-the-intellect category and Playing tennis / Riding a Bicycle belong to seat-of-the-pants category. In any field, there has to be a healthy combination of both kinds from a person to contribute meaningfully.

  • Scatter plots are better than relying only on the correlation estimates.

  • Setting up a factory is similar to writing a put option :)

  • Think of uncertain numbers and combination of uncertain numbers in terms of histograms, quantiles, shapes ,variations( accommodating term than the usual sigma terminology )

  • Weak Form – By using average as an estimate in your calculations, you are risking the variation associated with the estimate

  • Strong Form – Using average is plain wrong as you are hit by Jensen’s inequality

  • Reference to Howard Wainer’s book whose toc makes me feel that it is going to be a good read.

  • Whenever there is an estimate based on aggregation of numbers, always look at the sample size as ,  9 out of 10 times , people get carried away by the estimate

  • Simpson’s Paradox – basically the problem of confounding

  • Financial Engines – a site which offers monte carlo simulations on the web for investors to get an idea about asset allocation. Coincidentally, this site was referred to me at work  a few days back, in relation to a new product that we are working on. Just a background on Financial Engines….it was started  by Bill Sharpe, the Nobel prize winner , when he realized that people are not giving importance to scenario planning in retirement investment products. Web UI for the company rocks!

  • Few cool examples of prediction markets

  • A dig at the accounting standards prevalent . Come to think of it, there is a ton of crap(read financial statements ) that companies put on the site which are full of estimates and surprisingly they give little information about the degree of uncertainty relating to those numbers. Valuation of financial instruments is surprisingly lacking probabilistic rigour.  No wonder we see accounting scams frequently

  • Application of Jensen’s inequality to the design of oxygen ventilators for patientsimage

  • Steam era statistics has to be replaced with more computational oriented statistical analysis

  • Visualization of data is going to be as important if not more than model development. You can see this trend everywhere where visualizing data from the multi dimensional space is getting prominence over brute force modeling of data

  • Useful forwards from the book

    • Picturing the Uncertain World – Howard Wainer

    • Stochastic Simulation in the 19th century – Stigler

    • How to measure anything – Doug Hubbard

    • Failure of risk management – Doug Hubbard

image Takeaway:

One will never forget Jensen’s inequality after reading this book. Apart from that, this book will motivate anyone to move away from Steam era statistics concepts/ techniques to using more of computational oriented techniques.