Advanced High School Statistics: Second Edition, with updates based on AP© Statistics Course Framework

Subsection 3.2.1 Learning objectives

1. Understand conditional probability and how to calculate it.

2. Calculate joint and conditional probabilities based on a two-way table.

3. Use the General Multiplication Rule to find the probability of joint events.

4. Determine whether two events are independent and whether they are mutually exclusive, based on the definitions of those terms.

5. Draw a tree diagram with at least two branches to organize possible outcomes and their probabilities. Understand that the second branch represents conditional probabilities.

6. Use the tree diagram or Bayes' Theorem to solve “inverted” conditional probabilities.

Subsection 3.2.2 Exploring probabilities with a contingency table

The photo_classify data set represents a sample of 1822 photos from a photo sharing website. Data scientists have been working to improve a classifier for whether the photo is about fashion or not, and these 659 photos represent a test for their classifier. Each photo gets two classifications: the first is called mach_learn and gives a classification from a machine learning (ML) system of either pred_fashion or pred_not. Each of these 1822 photos have also been classified carefully by a team of people, which we take to be the source of truth; this variable is called truth and takes values fashion and not. Table 3.2.1 summarizes the results.

Subsection 3.2.3 Marginal and joint probabilities

Table 3.2.1 includes row and column totals for each variable separately in the photo_classify data set. These totals represent marginal probabilities for the sample, which are the probabilities based on a single variable without regard to any other variables. For instance, a probability based solely on the mach_learn variable is a marginal probability:

A probability of outcomes for two or more variables or processes is called a joint probability:

It is common to substitute a comma for “and” in a joint probability, although using either the word “and” or a comma is acceptable:

We use table proportions to summarize joint probabilities for the photo_classify sample. These proportions are computed by dividing each count in Table 3.2.1 by the table's total, 1822, to obtain the proportions in Table 3.2.5. The joint probability distribution of the mach_learn and truth variables is shown in Table 3.2.6.

We can compute marginal probabilities using joint probabilities in simple cases. For example, the probability that a randomly selected photo from the data set is about fashion is found by summing the outcomes in which truth takes value fashion:

Subsection 3.2.4 Defining conditional probability

The ML classifier predicts whether a photo is about fashion, even if it is not perfect. We would like to better understand how to use information from a variable like mach_learn to improve our probability estimation of a second variable, which in this example is truth.

The probability that a random photo from the data set is about fashion is about 0.17. If we knew the machine learning classifier predicted the photo was about fashion, could we get a better estimate of the probability the photo is actually about fashion? Absolutely. To do so, we limit our view to only those 219 cases where the ML classifier predicted that the photo was about fashion and look at the fraction where the photo was actually about fashion:

We call this a conditional probability because we computed the probability under a condition: the ML classifier prediction said the photo was about fashion.

There are two parts to a conditional probability, the outcome of interest and the condition. It is useful to think of the condition as information we know to be true, and this information usually can be described as a known outcome or event. We generally separate the text inside our probability notation into the outcome of interest and the condition with a vertical bar:

The vertical bar “\(|\)” is read as given.

In the last equation, we computed the probability a photo was about fashion based on the condition that the ML algorithm predicted it was about fashion as a fraction:

We considered only those cases that met the condition, mach_learn is pred_fashion, and then we computed the ratio of those cases that satisfied our outcome of interest, photo was actually about fashion.

Frequently, marginal and joint probabilities are provided instead of count data. For example, disease rates are commonly listed in percentages rather than in a count format. We would like to be able to compute conditional probabilities even when no counts are available, and we use the last equation as a template to understand this technique.

We considered only those cases that satisfied the condition, where the ML algorithm predicted fashion. Of these cases, the conditional probability was the fraction representing the outcome of interest, that the photo was about fashion. Suppose we were provided only the information in Table 3.2.5, i.e. only probability data. Then if we took a sample of 1000 photos, we would anticipate about 12.0% or \(0.120\times 1000 = 120\) would be predicted to be about fashion (mach_learn is pred_fashion). Similarly, we would expect about 10.8% or \(0.108\times 1000 = 108\) to meet both the information criteria and represent our outcome of interest. Then the conditional probability can be computed as

Here we are examining exactly the fraction of two probabilities, 0.108 and 0.120, which we can write as

The fraction of these probabilities is an example of the general formula for conditional probability.

Subsection 3.2.5 Smallpox in Boston, 1721

The smallpox data set provides a sample of 6,224 individuals from the year 1721 who were exposed to smallpox in Boston.⁴ Doctors at the time believed that inoculation, which involves exposing a person to the disease in a controlled form, could reduce the likelihood of death.

Each case represents one person with two variables: inoculated and result. The variable inoculated takes two levels: yes or no, indicating whether the person was inoculated or not. The variable result has outcomes lived or died. These data are summarized in Table 3.2.10 and Table 3.2.11.

Subsection 3.2.6 General multiplication rule

Subsection 3.1.7 introduced the Multiplication Rule for independent processes. Here we provide the General Multiplication Rule for events that might not be independent.

This General Multiplication Rule is simply a rearrangement of the definition for conditional probability.

Subsection 3.2.7 Sampling without replacement

If we sample from a small population without replacement, we no longer have independence between our observations. In Solution 3.2.20.1, the probability of not being picked for the second question was conditioned on the event that you were not picked for the first question. In Solution 3.2.22.1, the professor sampled her students with replacement: she repeatedly sampled the entire class without regard to who she already picked.

Had we repeated Checkpoint 3.2.24 with 300 tickets instead of 30, we would have found something interesting: the results would be nearly identical. The probability would be 0.0233 without replacement and 0.0231 with replacement.

Subsection 3.2.8 Independence considerations in conditional probability

If two processes are independent, then knowing the outcome of one should provide no information about the other. We can show this is mathematically true using conditional probabilities.

We can show in Checkpoint 3.2.26 that the conditioning information has no influence by using the Multiplication Rule for independence processes:

Subsection 3.2.9 Checking for independent and mutually exclusive events

If \(A\) and \(B\) are independent events, then the probability of \(A\) being true is unchanged if \(B\) is true. Mathematically, this is written as

The General Multiplication Rule states that \(P(A\text{ and } B)\) equals \(P(A | B)\times P(B)\text{.}\) If \(A\) and \(B\) are independent events, we can replace \(P(A|B)\) with \(P(A)\) and the following multiplication rule applies:

If \(A\) and \(B\) are mutually exclusive events, then \(A\) and \(B\) cannot occur at the same time. Mathematically, this is written as

The General Addition Rule states that \(P(A\text{ or } B) \text{ equals } P(A) + P(B) - P(A\text{ and } B)\text{.}\) If \(A\) and \(B\) are mutually exclusive events, we can replace \(P(A \text{ and } B)\) with 0 and the following addition rule applies:

Subsection 3.2.10 Tree diagrams

Tree diagrams are a tool to organize outcomes and probabilities around the structure of the data. They are most useful when two or more processes occur in a sequence and each process is conditioned on its predecessors.

The smallpox data fit this description. We see the population as split by inoculation: yes and no. Following this split, survival rates were observed for each group. This structure is reflected in the tree diagram shown in Figure 3.2.32. The first branch for inoculation is said to be the primary branch while the other branches are secondary.

Tree diagrams are annotated with marginal and conditional probabilities, as shown in Figure 3.2.32. This tree diagram splits the smallpox data by inoculation into the yes and no groups with respective marginal probabilities 0.0392 and 0.9608. The secondary branches are conditioned on the first, so we assign conditional probabilities to these branches. For example, the top branch in Figure 3.2.32 is the probability that lived conditioned on the information that inoculated.

We may (and usually do) construct joint probabilities at the end of each branch in our tree by multiplying the numbers we come across as we move from left to right. These joint probabilities are computed using the General Multiplication Rule:

After an introductory statistics course, 78% of students can successfully construct tree diagrams. Of those who can construct tree diagrams, 97% passed, while only 57% of those students who could not construct tree diagrams passed. (a) Organize this information into a tree diagram. (b) What is the probability that a student who was able to construct tree diagrams did not pass? (c) What is the probability that a randomly selected student was able to successfully construct tree diagrams and passed? (d) What is the probability that a randomly selected student passed? ¹⁸

Solution

(a) The tree diagram is shown to the right.

(b) \(P(\text{not pass } | \text{ able to construct tree diagram}) = 0.03\text{.}\)

(c) \(P(\text{able to construct tree diagrams and passed}) = P(\text{able to construct tree diagrams}) \times P(\text{passed } | \text{ able to construct tree diagrams}) = 0.78 \times 0.97 = 0.7566\text{.}\)

(d) \(P(\text{passed}) = 0.7566 + 0.1254 = 0.8820\text{.}\)

Subsection 3.2.11 Bayes' Theorem

In many instances, we are given a conditional probability of the form

but we would really like to know the inverted conditional probability:

For example, instead of wanting to know \(P(\)lived \(|\) inoculated\()\text{,}\) we might want to know \(P(\)inoculated \(|\) lived\()\text{.}\) This is more challenging because it cannot be read directly from the tree diagram. In these instances we use Bayes' Theorem. Let's begin by looking at a new example.

Example 3.2.36 highlights why doctors often run more tests regardless of a first positive test result. When a medical condition is rare, a single positive test isn't generally definitive.

Consider again the last equation of Example 3.2.36. Using the tree diagram, we can see that the numerator (the top of the fraction) is equal to the following product:

The denominator — the probability the screening was positive — is equal to the sum of probabilities for each positive screening scenario:

In the example, each of the probabilities on the right side was broken down into a product of a conditional probability and marginal probability using the tree diagram.

We can see an application of Bayes' Theorem by substituting the resulting probability expressions into the numerator and denominator of the original conditional probability.

Bayes' Theorem is just a generalization of what we have done using tree diagrams. The formula need not be memorized, since it can always be derived using a tree diagram:

• The numerator identifies the probability of getting both \(A_1\) and \(B\text{.}\)

• The denominator is the overall probability of getting \(B\text{.}\) Traverse each branch of the tree diagram that ends with event \(B\text{.}\) Add up the required products.

Jose visits campus every Thursday evening. However, some days the parking garage is full, often due to college events. There are academic events on 35% of evenings, sporting events on 20% of evenings, and no events on 45% of evenings. When there is an academic event, the garage fills up about 25% of the time, and it fills up 70% of evenings with sporting events. On evenings when there are no events, it only fills up about 5% of the time. If Jose comes to campus and finds the garage full, what is the probability that there is a sporting event? Use a tree diagram to solve this problem.

Solution

The tree diagram, with three primary branches, is shown below

We want:

\begin{gather*} P(\text{sporting event } | \text{ garage full})\\ =\frac{P(\text{sporting event and garage full})}{P(\text{garage full})}\\ =\frac{0.14}{0.0875+0.14+0.0225} =0.56 \end{gather*}

If the garage is full, there is a 56% probability that there is a sporting event.

The last several exercises offered a way to update our belief about whether there is a sporting event, academic event, or no event going on at the school based on the information that the parking lot was full. This strategy of updating beliefs using Bayes' Theorem is actually the foundation of an entire section of statistics called Bayesian statistics. While Bayesian statistics is very important and useful, we will not have time to cover it in this book.

Subsection 3.2.12 Section summary

• A conditional probability can be written as \(P(A | B)\) and is read, “Probability of \(A\) given \(B\)”. \(P(A|B)\) is the probability of \(A\text{,}\) given that \(B\) has occurred. In a conditional probability, we are given some information. In an unconditional probability, such as \(P(A)\text{,}\) we are not given any information.

• Sometimes \(P(A | B)\) can be deduced. For example, when drawing without replacement from a deck of cards, \(P(\text{ 2nd draw is an Ace } | \text{ 1st draw was an Ace } ) = \frac{3}{51}\text{.}\) When this is not the case, as when working with a table or a Venn diagram, one must use the conditional probability rule \(P(A | B) = \frac{P(A\text{ and } B)}{P(B)}\text{.}\)

• In the last section, we saw that two events are independent when the outcome of one has no effect on the outcome of the other. When \(A\) and \(B\) are independent, \(P(A | B) = P(A)\text{.}\)

• When \(A\) and \(B\) are dependent, find the probability of \(A\) and \(B\) using the General Multiplication Rule: \(P(A \text{ and } B) = P(A | B)\times P(B)\text{.}\)

• In the special case where \(A\) and \(B\) are independent, \(P(A \text{ and } B) = P(A)\times P(B)\text{.}\)

• If \(A\) and \(B\) are mutually exclusive, they must be dependent, since the occurrence of one of them changes the probability that the other occurs to 0.

• When sampling without replacement, such as drawing cards from a deck, make sure to use conditional probabilities when solving and problems.

• Sometimes, the conditional probability \(P(B|A)\) may be known, but we are interested in the “inverted” probability \(P(A|B)\text{.}\) Bayes' Theorem helps us solve such conditional probabilities that cannot be easily answered. However, rather than memorize Bayes' Theorem, one can generally draw a tree diagram and apply the conditional probability rule \(P(A | B)=\frac{P(A\text{ and } B)}{P(B)}\text{.}\) The resulting answer often has the form \(\frac{w\times x\text{ } +\text{ } y\times z}{w\times x}\text{,}\) where \(w, x, y, z\) are numbers from a tree diagram.