Multiple Testing Correction in Proteomics: FWER vs FDR Methods

1. WHY MULTIPLE TESTING CORRECTION MATTERS IN DIFFERENTIAL PROTEOMICS

In differential protein expression analysis, proteomics experiments usually test hundreds or thousands of proteins at the same time. Under this high-dimensional setting, raw P values alone are not reliable, because a nominal threshold such as P < 0.05 can generate many false-positive findings purely by chance. Without multiple testing correction, the final list of significant proteins may appear convincing but still contain statistically misleading results.

Multiple testing correction is therefore a core part of proteomics inference, not a minor technical adjustment. Its purpose is to keep false discoveries under control while preserving enough power to detect real biological changes. It is also important to distinguish two different layers of error control in proteomics:

• Identification-level FDR controls errors in peptide or protein identification.
• differential expression multiple testing correction controls false positives when many quantified proteins are tested across conditions.

These are related but not interchangeable concepts. In practice, reliable proteomics conclusions should be based on adjusted P values together with effect size, replicate consistency, and biological interpretability. The central question is not whether correction is needed, but which error-control strategy best fits the study objective.

2. FWER VS FDR IN PROTEOMICS

Most multiple testing correction methods in proteomics are built around two statistical targets: family-wise error rate (FWER) and false discovery rate (FDR). They both reduce false positives, but they do so with different levels of stringency.

• FWER is the probability of making at least one false-positive call among all tested proteins.
• FDR is the expected proportion of false positives among the proteins declared significant.

This distinction matters in practice:

• FWER-controlling methods are stricter and usually produce shorter, more conservative protein lists.
• FDR-controlling methods are less conservative and are often better suited to discovery-driven proteomics.

FDR should not be treated as a separate correction algorithm. It is an error-rate target, whereas BH and BY are procedures designed to control it. For most large-scale proteomics studies, FDR control is the practical default because it offers a better balance between sensitivity and reliability. FWER methods are more appropriate when false positives must be minimized as aggressively as possible.

3. QUICK COMPARISON OF MULTIPLE TESTING METHODS IN PROTEOMICS

4. BONFERRONI CORRECTION IN PROTEOMICS

Principle and threshold

Bonferroni correction is one of the simplest and most widely used methods for controlling the family-wise error rate (FWER). If m hypotheses are tested and the target family-wise significance level is α, each individual hypothesis is tested against the Bonferroni-adjusted threshold:

α_Bonf = α / m

An equivalent adjusted P-value form is:

p_adj = min(m × p, 1)

Here, α is the target family-wise significance level, m is the total number of hypotheses tested, p is the raw P value for an individual test, and p_adj is the Bonferroni-adjusted P value.

Bonferroni control is valid under arbitrary dependence and is therefore simple and robust. However, in large proteomics experiments it is often very conservative, because testing thousands of proteins can make the effective significance threshold extremely small and reduce power to detect biologically meaningful signals.

Advantages and limitations

Bonferroni is appropriate for highly confirmatory settings, small targeted panels, or analyses where even one false positive would be problematic. For broad discovery proteomics, however, it usually sacrifices too much sensitivity and can inflate false negatives.

5. HOLM-BONFERRONI CORRECTION FOR PROTEOMICS

Principle and decision rule

Holm correction is a step-down method for controlling the family-wise error rate (FWER). Let the raw P values be ordered from smallest to largest: p(1) ≤ p(2) ≤ ... ≤ p(m). The ordered P values are compared sequentially to the following thresholds: p(i) ≤ α / (m - i + 1), for i = 1, 2, ..., m. The testing procedure starts from the smallest P value and moves upward. Once a comparison fails, that ordered hypothesis and all remaining larger P values are treated as non-significant. A commonly used adjusted P-value form is:

p_adj(i) = max_{j ≤ i} [(m - j + 1) × p(j)]

Here, α is the target family-wise significance level, m is the total number of hypotheses tested, p(i) is the ith smallest raw P value, and p_adj(i) is the Holm-adjusted P value corresponding to the ith ordered test.

Holm retains strong FWER control under arbitrary dependence and is uniformly no less powerful than Bonferroni. For that reason, it is usually the preferable strict-error alternative when adjusted P values are required but Bonferroni seems unnecessarily harsh.

Advantages and limitations

Holm is a good compromise for confirmatory protein panels or high-stakes analyses that still need slightly more power than Bonferroni. It remains conservative in large-scale omics discovery settings.

6. HOCHBERG CORRECTION FOR HIGH-POWER FWER CONTROL

Principle and decision rule

Hochberg correction is a step-up method for controlling the family-wise error rate (FWER). Let the raw P values be ordered from smallest to largest: p(1) ≤ p(2) ≤ ... ≤ p(m). The ordered P values are evaluated using the following critical values: p(i) ≤ α / (m - i + 1), for i = 1, 2, ..., m. In the Hochberg procedure, testing starts from the largest ordered P value and moves backward toward the smallest. The largest i satisfying p(i) ≤ α / (m - i + 1) is identified, and all hypotheses corresponding to p(1), ..., p(i) are declared significant. A commonly used adjusted P-value form is:

p_adj(i) = min_{j ≥ i} [(m - j + 1) × p(j)]

Here, α is the target family-wise significance level, m is the total number of hypotheses tested, p(i) is the ith smallest raw P value, and p_adj(i) is the Hochberg-adjusted P value corresponding to the ith ordered test.

This method can be more powerful than Holm, but its formal guarantees require independence or suitable non-negative dependence among tests. That assumption should not be ignored in proteomics, where proteins are often correlated because of pathways, complexes, and co-regulation.

Advantages and limitations

Hochberg is attractive when analysts want FWER control with more power than Holm. It is less attractive when the dependence structure is unclear or potentially complex, because Holm remains valid under broader conditions.

7. BENJAMINI-HOCHBERG CORRECTION IN PROTEOMICS

Principle and FDR rule

The Benjamini-Hochberg (BH) procedure is the most widely used method for controlling the false discovery rate (FDR) in high-throughput proteomics. Let the raw P values be ordered from smallest to largest: p(1) ≤ p(2) ≤ ... ≤ p(m). Each ordered P value is compared with the BH critical value: p(i) ≤ (i / m) × q. The largest i satisfying this condition is identified, and all hypotheses corresponding to p(1), ..., p(i) are declared significant. A commonly used adjusted P-value form is:

p_adj(i) = min_{j ≥ i} [(m / j) × p(j)]

with monotonicity enforced so that: p_adj(1) ≤ p_adj(2) ≤ ... ≤ p_adj(m)

Here, q is the target false discovery rate level, m is the total number of hypotheses tested, p(i) is the ith smallest raw P value, and p_adj(i) is the BH-adjusted P value for the ith ordered test.

BH is especially useful in discovery-oriented proteomics because it keeps the expected proportion of false discoveries under control while preserving far more sensitivity than FWER methods.

Important assumption

The original BH guarantee applies under independence and was later extended to certain forms of positive dependence. In real proteomics datasets, feature correlation is common, but BH is still widely used in practice because it often provides a workable balance between rigor and discovery. When analysts need a procedure with a formal guarantee under arbitrary dependence, BY is the more conservative alternative.

Advantages and limitations

BH is the practical default for most exploratory differential protein expression studies. Its main limitation is not that it is wrong, but that users sometimes over-interpret it. A BH-adjusted P value does not mean a single protein has a 5% probability of being false; it reflects control of an expected error proportion across the rejected set.

8. BENJAMINI-YEKUTIELI CORRECTION IN PROTEOMICS

Principle and threshold

The Benjamini-Yekutieli (BY) procedure extends FDR control to settings with arbitrary dependence among tests. Let the raw P values be ordered from smallest to largest: p(1) ≤ p(2) ≤ ... ≤ p(m). BY modifies the BH threshold by introducing the factor c(m), defined as: c(m) = Σ(1 / j), for j = 1, 2, ..., m. The BY critical value becomes: p(i) ≤ (i / m) × (q / c(m)). The largest i satisfying this condition is identified, and all hypotheses corresponding to p(1), ..., p(i) are declared significant. A commonly used adjusted P-value form is:

p_adj(i) = min_{j ≥ i} [(m × c(m) / j) × p(j)]

with monotonicity enforced so that: p_adj(1) ≤ p_adj(2) ≤ ... ≤ p_adj(m)

Here, q is the target false discovery rate level, m is the total number of hypotheses tested, p(i) is the ith smallest raw P value, and c(m) is the harmonic correction factor.

BY correction controls FDR under arbitrary dependence, but it is more conservative than BH and can substantially reduce the number of significant findings in large proteomics datasets.

Advantages and limitations

BY is useful when arbitrary dependence is a central concern and analysts are willing to accept a substantially smaller discovery set. In many routine proteomics studies, however, BY is so conservative that it removes many plausible true positives.

9. BEST PRACTICES FOR MULTIPLE TESTING CORRECTION IN PROTEOMICS

For most exploratory differential proteomics studies, BH-adjusted P values are the standard choice because they usually provide the most practical balance between false discovery control and retention of biologically meaningful candidates. Holm is a reasonable alternative when family-wise control is needed but Bonferroni is unnecessarily strict. Bonferroni is better reserved for small confirmatory panels or very high-stakes settings, whereas Hochberg can be useful when its assumptions are acceptable and additional power is desirable. BY is best viewed as a robustness-oriented option rather than a routine default.

Regardless of method, multiple testing correction should be applied across the full set of proteins tested within a given analysis. Analysts should define the testing universe clearly, avoid post hoc selection based on raw P values, and interpret adjusted P values together with fold change, missing-data patterns, peptide support, and biological plausibility. In proteomics, statistical significance alone is rarely sufficient; reliable conclusions depend on both corrected significance and the quality and interpretability of the underlying signal.

Several common problems can weaken multiple testing results, but each has a clear corrective principle. Identification-level FDR should not be confused with protein-level multiple testing correction in differential expression analysis, because these two steps address different sources of error. Adjustment should be performed on the complete tested dataset rather than after cherry-picking nominally significant proteins. BH-adjusted P values should be interpreted as FDR-controlled significance measures, not as per-protein posterior error probabilities. Adjusted P values should also be reported together with effect sizes and basic data-quality context, rather than in isolation. Finally, overly conservative procedures should not be used by default in exploratory studies unless the study goal truly requires that level of stringency.

10. CONCLUSION: CHOOSING THE RIGHT MULTIPLE TESTING METHOD IN PROTEOMICS

There is no universally best correction method for every proteomics experiment. The correct choice depends on the inferential goal, the number of proteins tested, the tolerance for false positives, and the dependence structure among the test statistics. For most discovery-driven differential protein expression analyses, BH remains the default. For stricter confirmatory work, Holm or Bonferroni may be more appropriate. BY is best reserved for situations where arbitrary dependence must be handled with formal conservatism.

In short, multiple testing correction should be planned as part of the statistical design of the study, not added as a cosmetic final step. In high-throughput proteomics, the adjusted P value is usually what determines whether a claimed differential protein is statistically trustworthy.