Skip to main content

Section 6.2 Difference of two proportions

OpenIntro: Sampling Distribution of Proportions
Figure 6.2.1 EDIT TO SKIP TO TWO PROP SECTION? Sampling Distribution of Proportions

We would like to make conclusions about the difference in two population proportions: \(p_1 - p_2\text{.}\) We consider three examples. In the first, we compare the approval of the 2010 healthcare law under two different question phrasings. In the second application, a company weighs whether they should switch to a higher quality parts manufacturer. In the last example, we examine the cancer risk to dogs from the use of yard herbicides.

In our investigations, we first identify a reasonable point estimate of \(p_1 - p_2\) based on the sample. You may have already guessed its form: \(\hat{p}_1 - \hat{p}_2\). Next, we develop a formula for the standard deviation of \(\hat{p}_1 - \hat{p}_2\text{.}\)

Subsection 6.2.1 Sampling distribution of the difference of two proportions

The mean or expected value of \(\hat{p}_1 - \hat{p}_2\) is \(p_1 - p_2\text{.}\) The standard deviation can be computed as:

\begin{gather*} SD_{\hat{p}_1 - \hat{p}_2} = \sqrt{SD_{\hat{p}_1}^2 + SD_{\hat{p}_2}^2} = \sqrt{\frac{p_1(1-p_1)}{n_1} + \frac{p_2(1-p_2)}{n_2}} \end{gather*}

In addition to the mean and the standard deviation of \(\hat{p}_1 - \hat{p}_2\text{,}\) we would like to the know the shape of its distribution. First, the sampling distribution for each sample proportion must be nearly normal, and secondly, the samples must be independent. Under these two conditions, the sampling distribution of \(\hat{p}_1 - \hat{p}_2\) may be well approximated using the normal model.

Conditions for the sampling distribution of \(\hat{p}_1 - \hat{p}_2\) to be normal

The difference \(\hat{p}_1 - \hat{p}_2\) tends to follow a normal model when

  • each proportion separately follows a normal model (check \(n_1p_1 \geq 10\text{,}\) \(n_1(1-p_1) \geq 10\text{,}\) \(n_2p_2 \geq 10\text{,}\) and \(n_2(1-p_2) \geq 10\)) and

  • the two samples are independent of each other.

The standard deviation of the difference in sample proportions is

\begin{gather} SD_{\hat{p}_1 - \hat{p}_2} = \sqrt{\frac{p_1(1-p_1)}{n_1} + \frac{p_2(1-p_2)}{n_2}}\label{sdForDiffOfProp}\tag{6.2.1} \end{gather}

where \(p_1\) and \(p_2\) represent the population proportions, and \(n_1\) and \(n_2\) represent the sample sizes.

Subsection 6.2.2 Confidence Interval for \(p_1 -p_2\)

In the setting of confidence intervals, the sample proportions are used in place of the population proportions to verify the success-failure condition and also compute standard error, just as was the case with a single proportion.

The way a question is phrased can influence a person's response. For example, Pew Research Center conducted a survey with the following question: 1 www.people-press.org/2012/03/26/public-remains-split-on-health-care-bill-opposed-to-mandate. Sample sizes for each polling group are approximate.

“As you may know, by 2014 nearly all Americans will be required to have health insurance. [People who do not buy insurance will pay a penalty] while [People who cannot afford it will receive financial help from the government]. Do you approve or disapprove of this policy?”

For each randomly sampled respondent, the statements in brackets were randomized: either they were kept in the order given above, or the two statements were reversed. Example 6.2.2 shows the results of this experiment. Create and interpret a 90% confidence interval of the difference in approval.

Solution
Sample
size \(n_i\)
Approve
law (%)
Disapprove
law (%)
Other
''people who cannot afford it will
receive financial help from the
government'' is given second
771 47 49 3
''people who do not buy it will
pay a penalty'' is given second
732 34 63 3
Table 6.2.3 Results for a Pew Research Center poll where the ordering of two statements in a question regarding healthcare were randomized.

First the conditions must be verified. Because each group is a simple random sample, the observations are independent, both within the samples and between the samples. The success-failure condition should also be verified:

\begin{align*} 771 \times 0.47 \ge 10 \amp \amp 771 \times 0.53 \ge 10\\ \\ 732 \times 0.34 \ge 10 \amp \amp 732 \times 0.66 \ge 10 \end{align*}

Because all conditions are met, the normal model can be used for the point estimate of the difference in support, where \(p_1\) corresponds to the original ordering and \(p_2\) to the reversed ordering:

\begin{equation*} \hat{p}_{1} - \hat{p}_{2} = 0.47 - 0.34 = 0.13 \end{equation*}

The standard error may be computed from Equation (6.2.1) using the sample proportions in place of the population proportions:

\begin{equation*} SE = \sqrt{\frac{0.47(1-0.47)}{771} + \frac{0.34(1-0.34)}{732}} = 0.025 \end{equation*}

For a 90% confidence interval, we use \(z^{\star} = 1.645\text{:}\)

\begin{gather*} \text{ point estimate } \ \pm\ z^{\star}SE \rightarrow 0.13 \ \pm\ 1.645 \times 0.025 \rightarrow (0.09, 0.17) \end{gather*}

We are 90% confident that the approval rating for the 2010 healthcare law changes between 9% and 17% due to the ordering of the two statements in the survey question. Because the entire interval is positive, we have evidence that the approval rating increased. The Pew Research Center reported that this modestly large difference suggests that the opinions of much of the public are still fluid on the health insurance mandate.

Constructing a confidence interval for the difference of two proportion
  1. State the name of the CI being used.

    • 2-proportion z-interval

  2. Verify conditions.

    • 2 independent random samples OR 2 randomly allocated treatments.

    • \(n_1\hat{p}_1\geq10\text{,}\) \(n_1(1-\hat{p}_1)\geq10\) \(n_2\hat{p}_2\geq10\text{,}\) \(n_2(1-\hat{p}_2)\geq10\)

  3. Plug in the numbers and write the interval in the form

    \begin{equation*} \text{ point estimate } \pm z^\star \times \text{ SE of estimate } \end{equation*}
    • The point estimate is \(\hat{p}_1-\hat{p}_2\text{.}\)

    • Use critical value \(z^*\) = 1.96 for a 95% CI, otherwise find \(z^*\) using the \(t\)-table at row \(\infty\text{.}\)

    • Use SE = \(\sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}\text{.}\)

  4. Evaluate the CI and write in the form (     ,      ).

  5. Interpret the interval: “We are [XX]% confident that the true difference in the proportion of [...] is between [...] and [...].”

  6. State the conclusion to the original question.

A remote control car company is considering a new manufacturer for wheel gears. The new manufacturer would be more expensive but their higher quality gears are more reliable, resulting in happier customers and fewer warranty claims. However, management must be convinced that the more expensive gears are worth the conversion before they approve the switch. The quality control engineer collects a sample of gears, examining 1000 gears from each company and finds that 899 gears pass inspection from the current supplier and 958 pass inspection from the prospective supplier. Using these data, construct a 95% confidence interval for the difference in the proportion that pass inspection.

Solution

We will calculate a 2-proportion z-interval.

The samples are independent, but not necessarily random, so to proceed we must assume the gears are all independent. For this sample we will suppose this assumption is reasonable, but the engineer would be more knowledgeable as to whether this assumption is appropriate. We also must verify the minimum sample size conditions:

\begin{align*} 1000 \times \frac{899}{1000} \ge 10 \amp \amp1000 \times \frac{101}{1000} \ge 10\\ \\ 1000 \times \frac{958}{1000} \ge 10 \amp \amp 1000 \times \frac{42}{1000} \ge 10 \end{align*}

To construct a confidence interval, we first identify the point estimate and standard error, then we can construct the confidence interval:

\begin{align*} \amp \text{ point estimate } = 0.958 - 0.899 = 0.059\\ \amp SE = \sqrt{\frac{0.899(1-0.899)}{1000} + \frac{0.958(1-0.958)}{1000}} = 0.0114\\ \amp 0.059\ \pm\ 1.96 \times 0.0114\\ \amp (0.037, 0.081) \end{align*}

We are 95% confident that the true difference in proportion of current and prospective gears that pass inspection is between 0.037 and 0.081, favoring the prospective gears. Because the entire interval is above zero, the data provide strong evidence that the prospective gears pass inspection more often than the current gears. The remote control car company should go with the new manufacturer.

Subsection 6.2.3 Hypothesis testing when \(H_0: p_1 = p_2\)

Here we use a new example to examine a special estimate of the standard error when \(H_0: p_1 = p_2\text{.}\) We investigate whether there is an increased risk of cancer in dogs that are exposed to the herbicide 2,4-dichlorophenoxyacetic acid (2,4-D). A study in 1994 examined 491 dogs that had developed cancer and 945 dogs as a control group. 2 Hayes HM, Tarone RE, Cantor KP, Jessen CR, McCurnin DM, and Richardson RC. 1991. Case-Control Study of Canine Malignant Lymphoma: Positive Association With Dog Owner's Use of 2, 4-Dichlorophenoxyacetic Acid Herbicides. Journal of the National Cancer Institute 83(17):1226-1231. Of these two groups, researchers identified which dogs had been exposed to 2,4-D in their owner's yard. The results are shown in Table 6.2.5.

cancer no cancer
2,4-D 191 304
no 2,4-D 300 641
Table 6.2.5 Summary results for cancer in dogs and the use of 2,4-D by the dog's owner.

Is this study an experiment or an observational study? 3 The owners were not instructed to apply or not apply the herbicide, so this is an observational study. This question was especially tricky because one group was called the control group, which is a term usually seen in experiments.

Set up hypotheses to test whether 2,4-D and the occurrence of cancer in dogs are related. Use a one-sided test and compare across the cancer and no cancer groups. 4 Using the proportions within the cancer and no cancer groups may seem odd. We intuitively may desire to compare the fraction of dogs with cancer in the 2,4-D and no 2,4-D groups, since the herbicide is an explanatory variable. However, the cancer rates in each group do not necessarily reject the real cancer rates due to the way the data were collected. For this reason, computing cancer rates may greatly alarm dog owners. \(H_0\text{:}\) the proportion of dogs with exposure to 2,4-D is the same in “cancer” and “no cancer” dogs, \(p_c-p_n = 0\text{.}\) \(H_A\text{:}\) dogs with cancer are more likely to have been exposed to 2,4-D than dogs without cancer, \(p_c-p_n \gt 0\)

Are the conditions for using the normal model and make inference on the results?

Solution

(1) It is unclear whether this is a random sample. However, if we believe the dogs in both the cancer and no cancer groups are representative of each respective population and that the dogs in the study do not interact in any way, then we may find it reasonable to assume independence between observations. (2) The success-failure condition (minimums of 10) easily holds for each sample.

Under the assumption of independence, we can use the normal model and make statements regarding the canine population based on the data.

In the hypotheses for Guided Practice 6.2.7, the null is that the proportion of dogs with exposure to 2,4-D is the same in each group. The point estimate of the difference in sample proportions is \(\hat{p}_c - \hat{p}_n = 0.067\text{.}\) To identify the p-value for this test, we first check conditions (Example 6.2.8) and compute the standard error of the difference.

The standard deviation is given by

\begin{gather*} SD = \sqrt{\frac{p_c(1-p_c)}{n_c} + \frac{p_n(1-p_n)}{n_n}} \end{gather*}

In a hypothesis test, the distribution of the test statistic is always examined as though the null hypothesis is true, i.e. in this case, \(p_c = p_n\text{.}\) The standard deviation formula should reflect this equality in the null hypothesis. We will use \(p\) to represent the common rate of dogs that are exposed to 2,4-D in the two groups:

\begin{align*} SD \amp = \sqrt{\frac{p(1-p)}{n_c} + \frac{p(1-p)}{n_n}}\\ \amp = \sqrt{p(1-p)}\sqrt{\frac{1}{n_c} + \frac{1}{n_n}} \end{align*}

We don't know the exposure rate, \(p\text{,}\) but we can obtain a good estimate of it by pooling the results of both samples to find \(\hat{p}\text{:}\)

\begin{equation*} \hat{p} = \frac{\text{ # of "successes" } }{\text{ # of cases } } = \frac{191 + 300}{191+300+304+641} = 0.342 \end{equation*}

This is called the pooled estimate of the sample proportion, and we use it to compute the standard error when the null hypothesis is that \(p_1 = p_2\) (e.g. \(p_c = p_n\) or \(p_c - p_n = 0\)). We also typically use it to verify the success-failure condition.

Pooled estimate of a proportion

When the null hypothesis is \(p_1 = p_2\text{,}\) it is useful to find the pooled estimate of the shared proportion:

\begin{gather*} \hat{p} = \frac{\text{ number of "successes" } }{\text{ number of cases } } = \frac{x_1+x_2}{n_1+n_2}=\frac{\hat{p}_1n_1 + \hat{p}_2n_2}{n_1 + n_2} \end{gather*}

Here \(x_1\) represents the number of successes in sample 1. \(x_1\) can be computed as \(\hat{p}_1n_1\) if it is unknown. Similarly, \(x_2\) represents the number of successes in sample 2. It also can be computed as \(\hat{p}_2n_2\text{.}\)

TIP: Use the pooled proportion estimate when \(H_0: p_1 = p_2\)

When the null hypothesis suggests the proportions are equal, we use the pooled proportion estimate (\(\hat{p}\)) to verify the success-failure condition and also to estimate the standard error:

\begin{gather} SE =\sqrt{\hat{p}(1-\hat{p})}\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}\label{seOfDiffInPropUsingPooledEstimate}\tag{6.2.2} \end{gather}

Using Equation (6.2.2), \(\hat{p}=0.342\text{,}\) \(n_1 = 491\text{,}\) and \(n_2=945\text{,}\) verify the standard error estimate in the context of a hypothesis test is \(SE = 0.026\text{.}\)

Complete the hypothesis test using a significance level of 0.01.

Solution

We will complete a 2-proportion z-test. The conditions are met - we will assume that there two independent random samples. Using the pooled proportion:

\begin{align*} n_1\hat{p} \amp = 491 \times 0.342 = 167.9 \amp n_1(1 - \hat{p}) \amp = 491 \times 0.658 = 323.1\\ n_2\hat{p} \amp = 945 \times 0.342 = 323.2 \amp n_2(1 - \hat{p}) \amp = 945 \times 0.658 = 621.8 \end{align*}

are all at least 10. Now we set up hypotheses, which were identified in Guided Practice 6.2.7:

\(H_0\text{:}\) The proportion of dogs with exposure to 2,4-D is the same in “cancer” and “no cancer” dogs, \(p_c - p_n = 0\text{.}\)

\(H_A\text{:}\) Dogs with cancer are more likely to have been exposed to 2,4-D than dogs without cancer, \(p_c - p_n \gt 0\text{.}\)

We will use a significance level of \(\alpha = 0.01\text{.}\) All values are much larger than 10. Under the assumption that there were two independent random samples, we can proceed.

Next, we compute the test statistic using the standard error using the result of Guided Practice 6.2.9:

\begin{gather*} Z = \frac{\text{ point estimate } - \text{ null value } }{SE} = \frac{0.067 - 0}{0.026} = 2.58 \end{gather*}

Looking up \(Z=2.58\) in the normal probability table: 0.9951. However this is the lower tail, and the upper tail represents the p-value: \(1-0.9951 = 0.0049\text{.}\) Because the p-value is smaller than \(\alpha = 0.01\text{,}\) we reject the null hypothesis and conclude that there is an association between dogs getting cancer and owners using 2,4-D.

Hypothesis test for the difference of two proportions
  1. State the name of the test being used.

    • 2-proportion z-test

  2. Verify conditions to ensure the standard error estimate is reasonable and the point estimate is nearly normal and unbiased.

    • 2 independent random samples OR 2 randomly allocated treatments.

    • Calculate the pooled sample proportion \(\hat{p}\) and verify \(n_1\hat{p}\text{,}\) \(n_2\hat{p}\text{,}\) \(n_1(1 - \hat{p})\text{,}\) and \(n_2(1 - \hat{p})\) are greater than or equal to 10.

  3. Write the hypotheses in plain language and using mathematical notation.

    • \(H_0: p_1 = p_2\) (or \(p_1 - p_2 = 0\))

    • \(H_A: p_1 \ne \text{ or } \lt \text{ or } > p_2\)

  4. Identify the significance level \(\alpha\text{.}\)

  5. Calculate the test statistic: \(\text{Z} = \frac{\text{ point estimate } - \text{ null value } }{\text{ SE of estimate } }\)

    • The point estimate is \(\hat{p}_1 - \hat{p}_2\text{.}\)

    • Use \(SE\) = \(\sqrt{\hat{p}(1-\hat{p})}\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}\text{.}\)

  6. Find the p-value and compare it to \(\alpha\) to determine whether to reject or not reject \(H_0\text{.}\)

  7. Write the conclusion in the context of the question.

Subsection 6.2.4 Calculator: the 2-proportion z-test and z-interval

TI-83/84: 2-proportion z-interval

MISSINGVIDEOLINK Use STAT, TESTS, 2-PropZInt.

  1. Choose STAT.

  2. Right arrow to TESTS.

  3. Down arrow and choose B:2-PropZInt.

  4. Let x1 be the number of yes's (must be an integer) in sample 1 and let n1 be the size of sample 1.

  5. Let x2 be the number of yes's (must be an integer) in sample 2 and let n2 be the size of sample 2.

  6. Let C-Level be the desired confidence level.

  7. Choose Calculate and hit ENTER, which returns:

    (,) the confidence interval
    \(\hat{p}_1\) sample 1 proportion    \(n_1\) size of sample 1
    \(\hat{p}_2\) sample 2 proportion \(n_2\) size of sample 2

Casio fx-9750GII: 2-proportion z-interval

MISSINGVIDEOLINK

  1. Navigate to STAT (MENU button, then hit the 2 button or select STAT).

  2. Choose the INTR option (F4 button).

  3. Choose the Z option (F1 button).

  4. Choose the 2-P option (F4 button).

  5. Specify the interval details:

    • Confidence level of interest for C-Level.

    • Enter the number of successes for each group, x1 and x2.

    • Enter the sample size for each group, n1 and n2.

  6. Hit the EXE button, which returns

    Left, Right the ends of the confidence interval
    \(\hat{p}_1\text{,}\) \(hat{p}_2\) the sample proportions
    n1, n2 sample sizes

Use the data in Table 6.2.12 and a calculator to find a 95% confidence interval for the difference in proportion of dogs with cancer that have been exposed to 2,4-D versus not exposed to 2,4-D. 5 Correctly going through the calculator steps should lead to an interval of (0.01484, 0.11926). There is no value given for the pooled proportion since we do not pool for confidence intervals.

cancer no cancer
2,4-D 191 304
no 2,4-D 300 641
Table 6.2.12 Summary results for cancer in dogs and the use of 2,4-D by the dog's owner.
TI-83/84: 2-proportion z-test

MISSINGVIDEOLINK Use STAT, TESTS, 2-PropZTest.

  1. Choose STAT.

  2. Right arrow to TESTS.

  3. Down arrow and choose 6:2-PropZTest.

  4. Let x1 be the number of yes's (must be an integer) in sample 1 and let n1 be the size of sample 1.

  5. Let x2 be the number of yes's (must be an integer) in sample 2 and let n2 be the size of sample 2.

  6. Choose \(\ne\text{,}\) \(\lt\text{,}\) or \(\gt\) to correspond to H\(_A\text{.}\)

  7. Choose Calculate and hit ENTER, which returns:

    z Z-statistic    p p-value
    \(\hat{p}_1\) sample 1 proportion \(\hat{p}\) pooled sample proportion
    \(\hat{p}_2\) sample 2 proportion

Casio fx-9750GII: 2-proportion z-test

MISSINGVIDEOLINK

  1. Navigate to STAT (MENU button, then hit the 2 button or select STAT).

  2. Choose the TEST option (F3 button).

  3. Choose the Z option (F1 button).

  4. Choose the 2-P option (F4 button).

  5. Specify the test details:

    • Specify the sidedness of the test using the F1, F2, and F3 keys.

    • Enter the number of successes for each group, x1 and x2.

    • Enter the sample size for each group, n1 and n2.

  6. Hit the EXE button, which returns

    z Z-statistic \(\hat{p}_1\text{,}\) \(\hat{p}_2\) sample proportions
    p p-value \(\hat{p}\) pooled proportion
    n1, n2 sample sizes

Use the data in Table 6.2.12 and a calculator to find the Z-score and p-value for one-sided test with H\(_A\text{:}\) dogs with cancer are more likely to have been exposed to 2,4-D than dogs without cancer, \(p_c - p_n \gt 0\text{.}\) 6 Correctly going through the calculator steps should lead to a solution with \(Z=2.55\) and \(\text{ p-value } =0.0055\text{.}\) The pooled proportion is \(\hat{p}=0.342\text{.}\)