Advertisement
does anyone know any good ways to correct for multiple testing? I'll be more specific: I have a million hypotheses many of which may simultaneously be correct. I want to find the most significant predictions. however, with a p-value threshold of .05, there should be 50 wrong predictions in those with p-value < .05. how might I set a more appropriate p-value cutoff?
posted by:
|
Unsubscribed |
Advertisement
Advertisement
-
Unsu...
Re: Multiple testing
Thu, January 12, 2006 - 3:12 PM
erm... i am curious why you posted this question to the bayesian statistics tribe? this isn't a question for bayesian methods which, among other things don't:
1. use freqentist notions of "significance" or significance testing
2. mesh well with "millions of hypotheses" about which (i assume) you (the researcher) don't really have prior beliefs about distributional forms
3. look for "true" parameter estimates by "correcting" either study design properties.
hazarding a guess with the very little information you provide you are interested in frequentist data mining techniques using some form of signifiance adjustment like the bonferroni technique, and just possibly a multilevel model to account for between test versus between object/individual variance.
lexy-lou -
-
Unsu...
Re: Multiple testing
Thu, January 12, 2006 - 3:25 PMtrue enough, it's not strictly bayesian, is it? I have tried bonferroni, and am considering maybe a stepdown procedure next.
let's say I am trying to find pairwise relationships between all genes in a genome. one could incorporate some prior beliefs about the connectivity of protein interaction networks, so the question isn't necessarily off limits to bayesians.
but sure, maybe I'll check one of the other statisics tribes, thanks for the input.
-
