Ok, thanks in advance for the help. I've been working on this stat project, and I cannot, for the life of me, get by this question. "A manager tests a large number of hypotheses using an alpha = 0.05 for all tests. As it happens, all null hypotheses are true. What will the manager conclude?"
Any ideas?
I suspect the question relates to the issue of adjusting the p-value for the multiple hypothesis tests.
Also, the null hypothesis cannot be accepted (i.e., true); you can only fail to reject it.
Right about the null. I was thinking maybe that the sample size was just too small? That would fail to reject all null hypothesises wouldn't it? Thanks.
From what I remember... A null hypothesis being true means there is no reason to believe (at that level of confidence) that there is any departure from the mean, 0, or whatever the null is set to be. If the whole collection of test are true at .05 than any one test must be true with at the .05 level. Hope this gets you pointed in the right direction
It depends on the effect size. Even with a small sample size, large differences can be statistically significant. This gets to the issue of power. I would focus on the multiple comparisons issue.
Is there any more to the question that what you provided?
Don't they teach math well at that school anymore?
It's an awful question. First, as others have mentioned null hypotheses are not proved true. Second, you need to know if the hypotheses being tested are on the same sample or not. Are the tests related or independent. Actually, adjusting alpha for multiple comparisons is a red herring here - she/he already failed to reject the null hypothesis at p = .05; adjusting by the number of tests would lower the p value and all tests would still be non-significant. As a previous poster mentioned, power is always a consideration. If it's it one study we are talking about, it could be under-powered.
I certainly hope this is not a test question in a university class.
my 2 cents
I think what the question was supposed to get at was the true meaning of a p-value and type I errors. If the manager does multiple hypothesis tests, say 100, 5 of them (5%) will be true simply by chance. Thus, the manager can conclude that of his multiple hypothesis tests, 5% of them aren't necessarily true, but true only by chance alone.
The manager would have to do a Bonferonni correction to correct for multiple hypothesis tests.
There's nothing else to the question. It's prefixed with, "This question is to help you fully understand the intricacies of statistics. Enjoy."
question wrote:
There's nothing else to the question. It's prefixed with, "This question is to help you fully understand the intricacies of statistics. Enjoy."
Ok, I'm pretty sure I know what this question is getting at, I just hope that my explanation makes sense (I'll phrase this in terms of a regression.)
Basically, in none of the tests should the null hypothesis be rejected. If we're testing the null hypothesis that some coefficient equals zero (Ho: B1=0) then the manager would never reject that null hypothesis if he could see what was going on in the population. There is no relationship. But since he is only using statistical inference, the manager will reject the null hypothesis 5% of the time and find a relationship.
I think what your instructor is getting at is that statistics is a powerful tool but it has it's limitations. Sometimes you will find a relationship that doesn't really exist in the population, and sometimes you will not be able to reject a null hypothesis when in the population the relationship really is there.