What is the purpose of adjustments for multiple comparisons and name two methods?

• A. To increase statistical power; methods include Bonferroni and Holm-Bonferroni.
• B. To adjust for missing data; methods include imputation and weighting.
• C. To select variables for a model; methods include forward and backward selection.
• D. To control type I error inflation across tests; methods include Bonferroni and Holm-Bonferroni.

Answer: (D)

When you run many statistical tests, the chance of finding a false positive somewhere among them increases. Adjustments for multiple comparisons are used to keep the overall probability of making at least one Type I error across all tests from inflating uncontrollably. In other words, they control the across-tests error rate so that you don’t overstate significance just because you looked at many results.

Two common methods are Bonferroni and Holm-Bonferroni. Bonferroni is straightforward: you divide the overall significance level (alpha) by the number of tests and compare each individual p-value to this smaller threshold. If a p-value is smaller than alpha divided by the number of tests, that result is considered significant. Holm-Bonferroni takes a slightly more nuanced, stepwise approach. You first order the p-values from smallest to largest, then compare the smallest p-value to alpha divided by the total number of tests, the next smallest to alpha divided by one less, and so on. You continue until you encounter a p-value that isn’t significant at its step threshold; all previous rejections are kept. This approach is generally less conservative than Bonferroni, often retaining more power while still controlling the overall Type I error rate.

The key idea is to prevent the inflation of false positives when multiple hypotheses are tested, and these two methods illustrate how that control can be implemented.