Chapter 22 Quiz: Sampling, Estimation, and Confidence Intervals
Instructions: This quiz tests your understanding of Chapter 22. Answer all questions before checking the solutions. For multiple choice, select the best answer — some options may be partially correct. For short answer questions, aim for 2-4 clear sentences. Total points: 100.
Section 1: Multiple Choice (10 questions, 4 points each)
Question 1. A researcher wants to estimate the average commute time for all employees at a company with 2,000 workers. She surveys 100 employees selected at random. In this scenario, what is the "population"?
- (A) The 100 employees she surveyed
- (B) All 2,000 employees at the company
- (C) All workers in the city
- (D) The 100 commute times she recorded
Answer
**Correct: (B)** The population is the entire group she wants to draw conclusions about — all 2,000 employees at the company. (A) is the sample. (C) is too broad — she's not trying to generalize to the whole city. (D) describes the data collected from the sample, not the population.Question 2. The 1936 Literary Digest poll surveyed 2.4 million people but predicted the wrong winner. A Gallup poll with 50,000 people got it right. What does this demonstrate?
- (A) Random sampling is less accurate than large samples
- (B) Sample quality (representativeness) matters more than sample size
- (C) Larger samples always produce worse estimates
- (D) The margin of error formula is unreliable
Answer
**Correct: (B)** The Literary Digest's sample was massive but biased — it over-represented wealthy Americans. Gallup's sample was smaller but designed to be representative. This demonstrates that representativeness trumps size. (A) and (C) are wrong because large random samples are actually excellent. (D) is wrong because the margin of error formula works fine for random samples — the Literary Digest's problem was bias, not the formula.Question 3. If you increase the sample size from 100 to 400, the standard error will:
- (A) Be reduced to one-quarter of its original value
- (B) Be reduced to one-half of its original value
- (C) Stay the same
- (D) Double
Answer
**Correct: (B)** SE = σ/√n. Going from n=100 to n=400: SE goes from σ/10 to σ/20, which is half. The standard error is inversely proportional to the square root of n, not to n itself. To cut the SE in half, you need to quadruple (not double) the sample size.Question 4. A 95% confidence interval for a population mean is (42.3, 47.7). Which interpretation is correct?
- (A) There is a 95% probability that the population mean falls between 42.3 and 47.7
- (B) 95% of the data values fall between 42.3 and 47.7
- (C) If we repeated the study many times, about 95% of the resulting confidence intervals would contain the true population mean
- (D) The sample mean falls between 42.3 and 47.7 with 95% probability
Answer
**Correct: (C)** This is the correct frequentist interpretation of a confidence interval. (A) is the most common misconception — the true mean is fixed, not random, so we can't assign it a probability of being in any particular range. (B) confuses the CI with a prediction interval or a range for individual values. (D) is wrong because the sample mean is known (it's the center of the interval, 45.0).Question 5. Which of the following would make a confidence interval NARROWER?
- (A) Increasing the confidence level from 95% to 99%
- (B) Decreasing the sample size
- (C) Increasing the sample size
- (D) Increasing the population standard deviation
Answer
**Correct: (C)** Increasing n reduces the standard error (SE = s/√n), which shrinks the margin of error and narrows the CI. (A) would make it wider (higher confidence requires a larger z*). (B) would make it wider (smaller n means larger SE). (D) would make it wider (larger s means larger SE).Question 6. In the bootstrap procedure, you resample:
- (A) From the population, with replacement
- (B) From the sample, without replacement
- (C) From the sample, with replacement
- (D) From a theoretical normal distribution
Answer
**Correct: (C)** The key feature of the bootstrap is resampling from the sample *with replacement*. This means each observation can appear zero, one, or multiple times in each bootstrap sample. (A) is impossible — if we could sample from the population, we wouldn't need the bootstrap. (B) without replacement would just give us the same data rearranged. (D) would be a parametric simulation, not a bootstrap.Question 7. A survey of 500 randomly selected adults finds that 62% support a policy, with a margin of error of ±4 percentage points at 95% confidence. A second survey of 2,000 adults finds 64% support the same policy. Compared to the first survey, the second survey's margin of error will be:
- (A) About ±2 percentage points (half as large)
- (B) About ±1 percentage point (one-quarter as large)
- (C) About ±4 percentage points (the same)
- (D) About ±8 percentage points (twice as large)
Answer
**Correct: (A)** MOE is proportional to 1/√n. The ratio of sample sizes is 2000/500 = 4. So the ratio of MOEs is 1/√4 = 1/2. The second survey's MOE is about half that of the first: approximately ±2 percentage points. This illustrates the diminishing returns of sample size — quadrupling the sample halves the margin of error.Question 8. You compute a 95% confidence interval for the mean and get (50, 60). Your colleague computes a 90% confidence interval from the same data. Their interval is:
- (A) Wider than (50, 60)
- (B) Narrower than (50, 60), for example (52, 58)
- (C) The same as (50, 60)
- (D) Impossible to determine without more information
Answer
**Correct: (B)** Lower confidence level → smaller critical value (z* = 1.645 instead of 1.96) → smaller margin of error → narrower interval. The interval gets narrower because you're accepting a higher chance of missing the true parameter (10% instead of 5%), and in exchange you get more precision.Question 9. A researcher surveys students who voluntarily respond to an email invitation about study habits. The sample mean study time is 3.2 hours per day. She computes a 95% CI of (2.9, 3.5). What is the biggest concern with this interval?
- (A) The sample size is too small
- (B) The interval is too narrow
- (C) The margin of error formula doesn't account for sampling bias, so the CI may not capture the true mean
- (D) She should have used a 99% confidence level
Answer
**Correct: (C)** The fundamental problem is voluntary response bias — students who respond to a survey about study habits may systematically differ from those who don't (e.g., more studious students might be more likely to respond). The CI formula only captures random sampling variability, not systematic bias. Even a perfectly calculated CI is meaningless if the sample isn't representative. Changing the confidence level (D) won't fix bias.Question 10. The standard error of the mean measures:
- (A) The standard deviation of the individual observations in the sample
- (B) The average distance between sample values and the population mean
- (C) How much the sample mean would vary across repeated samples of the same size
- (D) The probability that the sample mean is wrong
Answer
**Correct: (C)** The standard error is the standard deviation of the sampling distribution — it quantifies how much the sample mean would bounce around if you took many samples. (A) describes the sample standard deviation, not the standard error. (B) is vague and not the definition. (D) is not a meaningful statistical quantity.Section 2: True or False (4 questions, 4 points each)
Question 11. True or False: A 99% confidence interval is always better than a 95% confidence interval because it's more likely to contain the true value.
Answer
**False.** A 99% CI does have a higher capture rate (99% vs. 95%), but it's also wider, which makes it less informative. A CI that says "the mean is somewhere between 10 and 90" captures the truth very reliably but tells you almost nothing useful. There's a trade-off between confidence and precision. "Better" depends on the context and what trade-off is appropriate.Question 12. True or False: If two 95% confidence intervals for different groups overlap, the difference between the groups is definitely not statistically significant.
Answer
**False.** Overlapping CIs for individual means do not necessarily mean the difference is non-significant. Two CIs can overlap slightly and the difference can still be statistically significant at the 0.05 level. The proper way to assess whether groups differ is to construct a CI for the *difference* between means (or perform a hypothesis test). Non-overlapping CIs *do* imply significance, but overlapping CIs are inconclusive.Question 13. True or False: The bootstrap method requires the assumption that the population is normally distributed.
Answer
**False.** One of the main advantages of the bootstrap is that it does *not* require distributional assumptions. It works by resampling from the observed data, so it adapts to whatever distribution the data actually follows. This makes it especially useful for non-normal data, skewed distributions, or statistics without analytical formulas. However, it does require a reasonably sized sample that is representative of the population.Question 14. True or False: Doubling the sample size will cut the margin of error in half.
Answer
**False.** The margin of error is proportional to 1/√n. Doubling n reduces the MOE by a factor of √2 ≈ 1.41, not by half. To actually cut the MOE in half, you would need to *quadruple* the sample size. This diminishing returns relationship is one of the most important practical facts about sampling.Section 3: Short Answer (3 questions, 6 points each)
Question 15. Explain the difference between the standard deviation and the standard error in 2-3 sentences. When would you report each?
Answer
The **standard deviation** measures the variability of individual observations in the data — how spread out the values are. The **standard error** measures the variability of a statistic (like the sample mean) across hypothetical repeated samples — how much the mean would change if you drew a new sample. You report the standard deviation when describing the data itself (e.g., "vaccination rates across countries had a mean of 72% with a standard deviation of 18 percentage points"). You report the standard error (or a confidence interval based on it) when making inferences about a population parameter (e.g., "the estimated mean vaccination rate is 72%, with a standard error of 2.5 percentage points").Question 16. Describe the bootstrap procedure in 3-4 steps. Why is sampling "with replacement" essential to the method?
Answer
The bootstrap procedure: (1) Start with your original sample of n observations. (2) Draw a new sample of n observations *with replacement* from your original sample. (3) Compute the statistic of interest (mean, median, correlation, etc.) on this bootstrap sample. (4) Repeat steps 2-3 thousands of times to build the bootstrap sampling distribution, then use its percentiles for a confidence interval. Sampling *with replacement* is essential because it introduces variability between bootstrap samples. Without replacement, every sample of size n drawn from n observations would be the same data in a different order — the statistic would be identical every time, and there would be no bootstrap distribution to analyze. Replacement allows some observations to appear multiple times and others not at all, creating meaningful variation.Question 17. A colleague says: "We surveyed 10,000 people and our margin of error is only 1 percentage point. Our result is basically exact." Give two reasons why this confidence might be misplaced.
Answer
**Reason 1:** The margin of error only captures *sampling variability* — the randomness from which specific 10,000 people ended up in the sample. It does not account for *bias*. If the sample is not truly representative (e.g., it was drawn from a biased source like a website or convenience sample), the true error could be far larger than 1 percentage point. **Reason 2:** The margin of error also doesn't capture *measurement error* or *question wording effects*. If the survey question is confusing, leading, or interpreted differently by different people, the responses may not accurately reflect what people truly think or do. A very precise estimate of the wrong thing isn't helpful. Additionally, if the data has a complex survey design (clusters, stratification), using simple formulas may understate the true margin of error.Section 4: Applied Scenarios (2 questions, 7 points each)
Question 18. A health department surveys 80 randomly selected clinics and finds that the mean vaccination rate is 67.5% with a sample standard deviation of 14.8%.
(a) Compute the standard error of the mean. (b) Construct a 95% confidence interval for the true mean vaccination rate. (c) The department's target is 70%. Based on your CI, is there evidence that the mean rate is below the target? (d) If they wanted a margin of error no larger than 2 percentage points, how many clinics would they need to survey?
Answer
**(a)** SE = s/√n = 14.8/√80 = 14.8/8.944 = 1.655 **(b)** Using z* = 1.96 (or t* ≈ 1.99 for df=79, which is very similar): CI = 67.5 ± 1.96 × 1.655 = 67.5 ± 3.24 = **(64.26%, 70.74%)** **(c)** The target of 70% is *inside* the confidence interval (just barely — the upper bound is 70.74%). This means we cannot conclude with 95% confidence that the mean rate is below the target. The data is consistent with the true mean being either above or below 70%. We need more data to be sure. **(d)** MOE = z* × s/√n. Solving for n: n = (z* × s / MOE)² = (1.96 × 14.8 / 2)² = (14.504)² = 210.4. They would need to survey at least **211** clinics.Question 19. A data scientist creates the following bootstrap distribution for the median income in a city (10,000 bootstrap samples). The histogram is approximately bell-shaped with:
- Bootstrap mean: $52,400
- Bootstrap standard deviation: $1,850
- 2.5th percentile: $48,800
- 97.5th percentile: $56,100
(a) What is the 95% bootstrap confidence interval for the median income? (b) What is the point estimate of the median? (c) Is the bootstrap distribution symmetric? How can you tell from the numbers given? (d) Why might the bootstrap be preferred over a formula-based CI for the median?
Answer
**(a)** The 95% bootstrap CI (using the percentile method) is **($48,800, $56,100)**. **(b)** The point estimate is the median of the original sample, which should be close to the bootstrap mean of $52,400. (The bootstrap mean of the medians approximates the sample median.) **(c)** It's *slightly* asymmetric. If it were perfectly symmetric around $52,400, the CI would be ($52,400 - $3,600, $52,400 + $3,700) = ($48,800, $56,100). The lower tail extends $3,600 below the center and the upper tail extends $3,700 above — very close to symmetric but not exact. The near-symmetry is consistent with the "approximately bell-shaped" description. **(d)** The formula-based CI for a median is complex and relies on order statistics and distributional assumptions. The bootstrap requires no such formula — you just resample and compute medians. This makes it much simpler to implement and more flexible, especially when the data is skewed or has unusual features that would violate formula assumptions.Section 5: Code Analysis (1 question, 6 points)
Question 20. Read the following code and answer the questions below:
import numpy as np
from scipy import stats
np.random.seed(42)
data = np.random.normal(100, 15, 50)
n_boot = 5000
boot_means = []
for i in range(n_boot):
boot_sample = np.random.choice(data, size=len(data), replace=True)
boot_means.append(boot_sample.mean())
boot_means = np.array(boot_means)
ci_low = np.percentile(boot_means, 5)
ci_high = np.percentile(boot_means, 95)
print(f"CI: ({ci_low:.2f}, {ci_high:.2f})")
(a) What confidence level does this bootstrap CI correspond to?
(b) If you wanted a 95% confidence interval instead, what percentiles would you use?
(c) Would increasing n_boot from 5000 to 50000 make the CI narrower? Why or why not?
(d) What would happen if you changed replace=True to replace=False in the np.random.choice call?