From Bonferroni to False Discovery Rate: A user’s guide to p-value adjustment in multiple hypothesis testing

Dr Terry Neeman

Dr Neeman is a statistician specialising in data analysis and experimental design. Her principal role at BDSI is to collaborate with biologists, bioinformaticians, and medical and clinical researchers at ANU and in the Canberra research community. Dr Neeman divides her time offering statistical expertise on experimental design, data organisation and data analysis, and keeping abreast of new methodologies in the analysis and integration of complex high-dimensional biological data. She also strives to be a role model for young statisticians who may have an interest in the novel statistical challenges in modern biology research.

Abstract

The p-value as a measure of evidence for a treatment effect was a 20th century invention, conceived in the context of testing a single hypothesis. However, as early as the 1930s concerns were raised that the meaning of the conventionally-calculated p-value as “evidence of effect” became more tenuous in the context of testing multiple hypotheses.

There have been several solutions proposed over the last 80 years, aimed at “correcting” the p-value to restore its meaning as a measure of evidence of effect. In this talk, we’ll elucidate a few of these proposed solutions, from p-value adjustment under particular contexts (e.g. Dunnett and Tukey corrections), sequential methods, to a Bayesian-flavoured solution: the false discovery rate.

This talk is intended for practitioners with some familiarity with p-value adjustments, but who are stymied by raging debate around if, when, how and how much to adjust their p-values.

Download the flyer here.