Saturday, September 17, 2016

Signal & Noise

Finally after a lot of start-stop, false starts , I managed to finish Nate Silver's Signal and Noise - Why so many predictions fail and some don't. Nate Silver and his fivethirtyeight.com blog were hugely successful and popular for his US presidential elections and analysis.

This book covers a wide range of chapters - economics, stock markets, earthquakes, weather forecasting, terrorist attacks, baseball games. Reading can get a bit cumbersome and boring especially if you don't have a finance background or US-centric baseball/election stuff. The purpose of the book is to in a way identify what we know and what we think we know. The constant recurring in the book is to embrace uncertainty and bayesian way of thinking. The constant need to revise your understanding and hypothesis as and when you encounter newer evidence is what is missing most predictions and forecasting. Isolating false positives, noise from signals plays a key role - but on most occasions, it is difficult to differentiate signal from noise. This is where the bayesian reasoning provides a contextual information to solving.


Most times we think we are good at prediction than we actually are.

we focus on stories about the world as we would like to be, not how it really is

When you can't state your innocence, proclaim your ignorance.

When facts change, i change my mind -  John Maynard Keynes

A forecast is no good if no one is listening to it

Prediction is as much a means to the end. It can play a key role in hypothesis testing. Model is a tool to understand the complexities of the universe, not a substitute.

Heuristics are useful, but they are prone to biases and blindspots.

what good is a prediction, if you arent wiling to put money on it - put money on your predictions

Bayesian thinking : New evidence should not lower the estimate of likelihood. Should increase the overall strength n theory

A consensus is not synonymous with unanimity. The consensus is merely a simple majority

Known knowns, unknown knowns, unknown unknowns - Donald Rumsfeld