When probability meets reality: three problems on probability theory

Facing difficult choices, should you trust your intuition or carefully calculate all the associated risks?




For people with a scientific mindset, it is natural to try to apply rational methods to assess the risks of everyday life. For example, is it necessary to get a flu shot if you are under 40 and you are healthy? Do I need to jump out of the plane (with a parachute)? A noble goal, the application of logic to risk assessment, however, faces two obstacles. Firstly, in the absence of certainty, we usually make decisions based on a combination of intuition and expediency, and quite often it works . Secondly, we are constantly attacked by a multitude of constantly changing random events. “ How chance controls our life ” - such a subtitle had a very instructive bestseller Leonard Mlodinov. These constant pokings from random forces are colorfully demonstrated in this passage, paraphrased from a much longer 1964 children's fairy tale entitled " Fortunately " by Remi Charlip, who inspired our first task.

Task 1


The man went to ride a plane.

Unfortunately, he fell.

Fortunately, he had a parachute.

Unfortunately, the parachute did not open.

Fortunately, under it was a haystack, right on the spot where it was supposed to fall.

Unfortunately, the forks were sticking out of the stack right under him.

Fortunately, he missed the pitchfork.

Unfortunately, he did not hit the stack.

There are several testimonies stating that people who fell out of an airplane could survive by falling on a haystack, or even on trees or bushes — such cases are easy to google. So, the successive cries in the head of this person: “I am finished! / I am saved!” Cannot be called final until the story ends. (Our story ends tragically, but in the original the hero survives thanks to many other sharp turns of fate). Does it make sense to apply fundamental risk assessment methods in this case? Given the available information, assess the chances of survival after each line .

This story vividly illustrates two important aspects of probability estimates. First, probabilities can change radically with the advent of new knowledge. Secondly, no matter how much you set up the odds in your favor, the final result translates into one thing - life or death, yes or no. In rare cases, the result may be undesirable. As with the collapse of the wave function in quantum mechanics, demonstrated by Erwin Schrödinger’s famous mental experiment with a cat in a box that can be alive or dead, the probabilities lose their meaning after the event occurs. So what is the value of such calculations? Let's look at this moment in more detail.

Perhaps the best method of rational approach to chance and risk in everyday life would be Bayesian thinking, named after the statistics of the 18th century by Thomas Bayes. Bayesian thinking is based on several important principles. First, the probability is subjectively interpreted as the degree of trust - a reasonable assessment of the personal point of view about the probability of an event. Secondly, in the presence of reliable data on the frequency of the event, this degree of confidence must be equated to an objectively calculated probability. Thirdly, all the objective knowledge you have about this topic should be taken into account when calculating the initial assessment. Finally, it is necessary to update the probabilities when new information arrives. If you always rely on the most reliable and objective estimates of probability made on the basis of the data, and monitor possible inaccuracies, then the final probability will be the best possible one.

When the famous mathematician Timothy Gowers was faced with the need to make a decision about treating his atrial fibrillation with the help of a risky medical operation that offered no guarantee of success, he decided to make detailed calculations of risks and benefits. Fortunately, for Gowers, who is also one of the founders of the Polymath project, everything ended successfully. But most of the risks that we face are not so serious, and the magnitude of the risk is not so great. However, the following task illustrates the long-term benefits of using the Bayesian approach.

Task 2


The number of deaths on commercial flights is about 0.2 per 10 billion flight miles. For cars, that number is 150 deaths over 10 billion miles. And although this number is 750 times more than for airplanes, we [Americans / approx. transl.] still prefer to drive behind the wheel for long distances, because in absolute terms the risks are small. But we will conduct a mental experiment with two hypothetical and, of course, unrealistic assumptions: first, your expected lifetime is one million years (and you live with pleasure every year), and second, the above risks remain the same all the time. Now imagine that every year you can either fly 10,000 miles, or cover the same distance by car through long journeys. Travel time does not bother you - after all, you still live a million years! Under these conditions, how much and in what proportion will your life be shortened if, instead of flying, you were driving a car? How will the answer be different for a life expectancy of 100 years?

From this it is clear that even if the calculations of the probability of losing their value after the event has occurred, for the future they increase your chances in the long run. We do not live for a million years, but during life we ​​make tens of thousands of decisions about where and how to travel, what we have, whether to work out in the gym, etc. And although the likely impact of each of these decisions on our life expectancy will be small, their combined effect may be large. At least for large decisions — such as choosing an operation to deal with a serious illness — consideration of the details would be warranted, going beyond intuition.

And, of course, there are well-described situations in which our intuition turns out to be erroneous. This is the skeleton of standard textbooks on Bayesian methods. One example is the test for “quite good, but not perfect,” which leads to the third problem.

Task 3


Consider two similar scenarios in which it is necessary to give a probabilistic assessment of the situation. Before you make the calculations, listen to your intuition and write down the answer.

Option A: in one city there are two ethnic groups, the first and second. The first make up 80% of the population. The local hospital conducts a standard examination for the presence of a rare disease, equally common in both groups. As a result, she collects 100 blood samples, and, naturally, 80% of these samples are collected from the First. With careful testing for the disease, only 1 sample out of 100 turns out to be positive. A researcher who is unfamiliar with the data on ethnic correlation conducts a certain test of this sample and determines that it was taken from representatives of the second group. However, the accuracy of this test for belonging to ethnic groups is only 75%. What is the probability that the sample was indeed taken from the Second?

Option B: In this variant, the First and Second make up 50% of the population, but the probability of getting sick from the First is higher. 100 blood samples are collected again, with 80% taken from the First, and 20% from the Second. The remaining conditions are identical. What is the probability that a positive sample was taken from the Second?

In which of these cases was your intuition more accurate?

We know that our intuition often fails us in assessing probabilities, although it may seem right at the time of making a decision. She may even let experts down - just remember the hype about the " Monty Hall paradox ." The master of articles with riddles and problems, Martin Gardner , once said : "It is never so easy for experts to err in any other area of ​​mathematics than in the theory of probability." Our third task is an example of tasks that allow psychologists to determine which reasoning a person uses to make intuitive decisions, and what makes him judge exactly or make mistakes.

Answers to the tasks are shared in the comments; readers are also invited to talk about how they used probability calculations to make decisions in their real lives, and what approach to such calculations seems best to them.

Source: https://habr.com/ru/post/411659/


All Articles