Probabilistic programming and the Bayesian method for hackers

Hello colleagues. Today we wanted to ask how much the outdated book " Bayesian Methods for Hackers " seems to be in demand for you, published in the original in 2015, but not yet translated into Russian.


The book is positioned as an applied, the most spared from mathematics and timeless.

Under the cut - a slightly abridged translation of the review of this book, laid out by the author on Github.
Please participate in the voting.

The Bayesian method is a natural approach to the logical conclusion, however, in many books on this topic, the essence of the Bayesian method is completely lost behind the whole chapters of mathematical analysis. A typical book on Bayesian theory begins with two or three chapters on probability theory, after which it is told what a Bayesian conclusion is. Unfortunately, due to the inability to present most of the Bayesian models in mathematical language, the reader is shown only simple, artificial examples. Thus, a person may have an imaginary impression that he allegedly understood the Bayesian conclusion - whereas in fact he had learned only the author’s interpretation of this method.

After the Bayesian methods were clearly successful in machine learning competitions, I decided to re-examine this topic. Even with my mathematical background, I spent three whole days studying the examples and trying to put all the fragments together in order to deal with all the methods. In order to put theory into practice, I simply did not have enough literature. This misunderstanding was due to the fact that I was unable to bridge the gap between Bayesian mathematics and probabilistic programming. So, I have suffered, and I want to relieve readers of similar sufferings. It is for this reason that I wrote such a book myself.

If Bayesian inference is our goal, then mathematical analysis is only one of the ways to it. On the other hand, computing power has now become much cheaper, and we can use an alternative way, that is, probabilistic programming. The second option is much more convenient, because it does not require resorting to mathematics at all stages of solving the problem, in other words, allows you to do without mathematical analysis, which often turns out to be an insurmountable obstacle to the Bayesian conclusion. Figuratively speaking, in the second case we move from the beginning to the end in small rushes, and in the first - in huge jumps, because of which we can land quite far from the target. Moreover, without serious mathematical training, the reader may simply not cope with the analysis from the first option.

The book Bayesian Methods for Hackers was conceived as an introduction to Bayesian inference, with a focus on understanding key concepts and the use of computational power (mathematics is secondary). Of course, this book is introductory, it will remain introductory. For those who are knowledgeable in mathematics, she can simply stir up curiosity, and then open up to other texts written based on serious mathematical analysis. For an enthusiast interested in the practical application of Bayesian methods, and not in their mathematical background, the book will be quite informative and interesting.

For probabilistic programming in this book, the PyMC language is chosen, for which there are two reasons. At the time of writing, there was no centralized resource on the Internet with examples and explanations of how to work on the PyMC ecosystem. Official documentation assumes that the reader already has an idea of ​​Bayesian inference and probabilistic programming. We hope that this book will encourage readers with any level of training to look at PyMC.

When working with PyMC, it is necessary to take into account dependencies, namely, on NumPy (as appropriate) on SciPy. In order not to limit the reader, the examples in this book use only PyMC, NumPy, SciPy, and Matplotlib.

Content

(The chapters listed below are displayed in nbviewer on nbviewer.ipython.org, read-only, and open in real time. Interactive notebooks and examples can be downloaded by cloning!)

• Prologue : Why do this.
• Chapter 1 : Introduction to Bayesian Methods. Familiarity with the philosophy and practice of Bayesian methods, the story of what is probabilistic programming. Some examples are:

 Change in human behavior - output based on the frequency of sending SMS.

• Chapter 2 : A little more about PyMC We will discuss the modeling of Bayesian problems using examples that will use the PyMC library (Python language). How to create Bayesian models? Examples include:
•  Calculation of the probability of the collapse of the space shuttle "Challenger"

• Chapter 3 : Opening the MCMC black box Let's talk about how MCMC and diagnostic tools work. Examples include:
•  Bayesian clustering on the example of mixed models

• Chapter 4 : The greatest theorem about which you have never been told. Here we investigate an incredibly useful and dangerous theorem: the law of large numbers. Examples include:
•  Study of the Kaggle data set and the pitfalls of naive analysis
• How to sort comments from Reddit from best to worst (it's not as easy as you think)

• Chapter 5 : What would you rather give up - with your hand or your leg? Familiarity with loss functions and their (unmatched) application to Bayesian methods. Examples include:
•  Optimization of financial forecasts
•  A winning solution for Kaggle Dark World

• Chapter 6 : Clarifying Priorities. Perhaps the most important chapter. Here we answer questions, resorting to expert opinions. Examples include:
•  Multi-armed bandits and the Bayesian solution of such a problem.
• What is the relationship between sample size and prior data?
•  Estimate financial unknowns with expert priori data.
• Also here you will find valuable tips on objectivity in the analysis and on common pitfalls associated with prior probabilities.

Installation and configuration

If you want to run IPython Notepad on the local computer, you will need to install the following:

• IPython 0.13 - required for viewing ipynb files. Download it here .
• Linux users should easily install Numpy, Scipy and PyMC. For Windows users, there will be pre-compiled versions if any difficulties arise.

In the styles / directory there are a number of files adapted for working with notepad. They are not only designed specifically for the book, but also specially optimized for working with matplotlib and IPython.