Book Cover
Bayesian Methods For Hackers

★ ★

TL;DR: The simplest explanation of Bayesian methods and probabilistic programming I’ve come across. Says a lot about the field that this book was still extremely difficult to get through. Enjoyed the writing style quite a bit, and thought the examples were fascinating.


Simple definition of Bayesian Inference: updating your beliefs after considering new evidence. We can get more and more confident, but never absolutely sure!

My notes on this book are in a series of Colab Notebooks, following the book’s format of being written in Jupyter notebooks:

  1. Introduction
    1. Example: text message data inference (“can you detect the inflection point in a user’s behavior given their text message counts by day?”)
  2. A Little More on PyMC
    1. Example: A/B testing – finding a distribution for the delta of two sites, Site A and Site B, to probabilistically determine which performed better in the A/B test and by how much.
  3. Opening the Black Box of MCMC
    1. Example: unsupervised clustering of a dataset around two normal distributions using a mixture model.
  4. The Greatest Theorem Never Told
    1. Example: how to order Reddit submissions (factors: vote count, time passed, etc.)
  5. Loss Functions
    1. Example: optimizing for The Showcase on The Price is Right
    2. Example: Bayesian Kaggle submission for observing dark matter
  6. Priors
    1. Example: predicting stock returns

Overall, this is some of the most challenging material I’ve worked on in any field, not just machine learning. I’m really grateful for this book, since I was completely drowning in the material before I found it. I believe in the power of probabilistic programming and modeling, and I look forward to exploring it more with libraries like Uber’s Pyro and Google’s Tensorflow Probabilities.

After reading this book I don’t feel like I have a great handle on MCMC or building PyMC models, but I have at least enough of a foundation in vocabulary and concepts to begin branching out.