ONTNU Multiple hyp.testing quantities accept null reject null total H=0 U V mo H=1 T S mi total W R m Compound error rates: Family wise error rate:FWER=P(V >1) Per comparison error rate: PCER=E(V)/m False discovery rate: FDR-EV RR>0PR>0 Positive false discovery rate: PFDR=EVRR>0 Multiple hypothesis testing-recent developments and future challenges-p.6/28
NTNU Multiple hyp. testing quantities accept null reject null total H = 0 U V m0 H = 1 T S m1 total W R m Compound error rates: Family wise error rate: FWER = P(V ≥ 1) Per comparison error rate: PCER = E(V )/m False discovery rate: F DR = E(V/R|R > 0)P(R > 0) Positive false discovery rate: pF DR = E(V/R|R > 0) Multiple hypothesis testing - recent developments and future challenges – p.6/28
ONTNU Multiple hyp.testing quantities accept null reject null total H=0 U V mo H=1 T S mi total W R m Weak control:Only when mo =m Strong control:Holds for all mo simultaneously. Multiple hypothesis testing-recent developments and future challenges-p.6/28
NTNU Multiple hyp. testing quantities accept null reject null total H = 0 U V m0 H = 1 T S m1 total W R m Weak control: Only when m0 = m Strong control: Holds for all m0 simultaneously. Multiple hypothesis testing - recent developments and future challenges – p.6/28
ONTNU Example,fMRI Now you see it,now you don't:statistical and methodological considerations in fMRI. D.W.Loring et al.,Epilepsy Behavior 3(2002) Each voxel is tested if activation causes difference. Pure exploratory study of method and significance level. Multiple hypothesis testing-recent developments and future challenges-p.7/28
NTNU Example, fMRI Now you see it, now you don’t: statistical and methodological considerations in fMRI. D.W. Loring et al., Epilepsy & Behavior 3 (2002) Each voxel is tested if activation causes difference. Pure exploratory study of method and significance level. “(...), apparent random activation decreased as more conservative statistical approaches were employed, but activation in areas considered to be functionally significant was also reduced” Multiple hypothesis testing - recent developments and future challenges – p.7/28
ONTNU Example,fMRI Small Voxel p=.05 p=.001 p=.00001 No correction 20 voxel clustering False Discovery Rate Correction SPM Multiple Comparison Correction Multiple hypothesis testing-recent developments and future challenges-p.7/28
NTNU Example, fMRI “(...), apparent random activation decreased as more conservative statistical approaches were employed, but activation in areas considered to be functionally significant was also reduced” Multiple hypothesis testing - recent developments and future challenges – p.7/28
ONTNU Example,fMRI Small Voxel p=,05 p=.001 p=.00001 R No correction 20 voxel clustering False Discovery Rate Correction SPM Multiple Comparison Correction "(..)apparent random activation decreased as more conserva- tive statistical approaches were employed,but activation in areas considered to be functionally significant was also reduced" Multiple hypothesis testing-recent developments and future challenges-p.7/28
NTNU Example, fMRI “(...), apparent random activation decreased as more conservative statistical approaches were employed, but activation in areas considered to be functionally significant was also reduced” Multiple hypothesis testing - recent developments and future challenges – p.7/28