Adjusting a p-value to account for multiple comparisons

If you perform more than one statistical test, the likelihood of seeing a false positive (Type I error) increases with the number of tests performed. The Šidák correction is a statistical procedure that can adjust the critical values and p-values in order to account for the presence of multiple tests (also called multiple comparisons). It is appropriate, for instance, when comparing two specific groups out of a larger number of groups in the data. The Šidák correction is a more exact version of the Bonferroni correction.

As an example, suppose we perform an ANOVA across groups A, B, and C, and the ANOVA indicates that the mean differs across all three groups. If we wish to compare A to B only, it would be appropriate to adjust the p-value for 3 possible comparisons (A to B, A to C, and B to C).

To adjust the p-value of a statistical test to account for multiple comparisons:

1. In the Summary view, use the confidence slider to choose a desired confidence level for the overall test
2. Click the Info button to view the details of the current statistical test
3. Check the box beneath the P-value labeled Adjust for multiple comparisons
4. Enter the total number of comparisons you intend to perform as part of your test procedure

Wizard will then adjust the critical value, p-value, and p-value visualization to account for the multiple comparisons.

See also:

• Performing an ANOVA test of a numeric column’s mean across values of another column
• Performing a t-test of a numeric column’s mean across two categories
• Formatting descriptive and inferential statistics
• About the Summary view

---

Back to Wizard Help