1. Joined
    03 Oct '05
    Moves
    86698
    02 Dec '08 23:42
    Many scientists insist that statistisical analiysis of more than two groups requires ANOVA and post ANOVA tests.

    I disagree. ANOVA imparts no information whatsoever except, perhaps, there might be a difference somewhere. ANOVA does not show where differences might lie. More importantly, ANOVA might say ' there is no difference' when, in fact, a difference is real.

    I insist, t-tests are the best method of statistical analysis no matter how many groups are involved.
  2. Standard memberPalynka
    Upward Spiral
    Halfway
    Joined
    02 Aug '04
    Moves
    8702
    02 Dec '08 23:50
    Originally posted by znsho
    Many scientists insist that statistisical analiysis of more than two groups requires ANOVA and post ANOVA tests.

    I disagree. ANOVA imparts no information whatsoever except, perhaps, there might be a difference somewhere. ANOVA does not show where differences might lie. More importantly, ANOVA might say ' there is no difference' when, in fact, a difference i ...[text shortened]... t, t-tests are the best method of statistical analysis no matter how many groups are involved.
    1st basic question:
    What do you want to test?
  3. 42.4º N / -71.2º W
    Joined
    11 Jun '01
    Moves
    78257
    03 Dec '08 01:23
    Originally posted by znsho
    Many scientists insist that statistisical analiysis of more than two groups requires ANOVA and post ANOVA tests.

    I disagree. ANOVA imparts no information whatsoever except, perhaps, there might be a difference somewhere. ANOVA does not show where differences might lie. More importantly, ANOVA might say ' there is no difference' when, in fact, a difference i ...[text shortened]... t, t-tests are the best method of statistical analysis no matter how many groups are involved.
    In any situation where there are 2 or more factors in the experiment, a t-test will not give you the full story and will often mislead you as to the real locus of effects. t-tests in the absence of an ANOVA main effect in these situations are unwarranted, as they can point to effects in one factor while completely ignoring the other factor.

    In a 1 factor situation, if your experiment is well predicated, you will be expecting some sort of consistency across your (say) 3 groups. If you do a t-test between groups 1 and 2 and ignore the effect of group 3, that could be misleading again. If, for example, group 3 is your control, and group 1shows the effect against group 2, but not the control, then there really is no effect of interest. Your t-test will produce a misleading result.

    Further, to test for "inequality of means", which is the function of the F-test, you'd need 3 separate t-tests. Since you are performing 3 t-tests, you must reduce alpha by a factor of 3 (0.05/3) to control for family-wise error. This correction reduces your chances of finding a significant effect. The proper procedure of first running the f-test and then the post-hoc group comparison controls for this error in a more principled way.

    I hope this goes some way to convincing you that there really is a place for ANOVA

    ;-)
  4. Subscribersonhouse
    Fast and Curious
    slatington, pa, usa
    Joined
    28 Dec '04
    Moves
    52619
    03 Dec '08 15:45
    Originally posted by kyngj
    In any situation where there are 2 or more factors in the experiment, a t-test will not give you the full story and will often mislead you as to the real locus of effects. t-tests in the absence of an ANOVA main effect in these situations are unwarranted, as they can point to effects in one factor while completely ignoring the other factor.

    In a 1 factor si ...[text shortened]...

    I hope this goes some way to convincing you that there really is a place for ANOVA

    ;-)
    Heck, and here I thought Anova was when a star blew up🙂
  5. Joined
    03 Oct '05
    Moves
    86698
    03 Dec '08 22:27
    Originally posted by kyngj
    In any situation where there are 2 or more factors in the experiment, a t-test will not give you the full story and will often mislead you as to the real locus of effects. t-tests in the absence of an ANOVA main effect in these situations are unwarranted, as they can point to effects in one factor while completely ignoring the other factor.

    In a 1 factor si ...[text shortened]...

    I hope this goes some way to convincing you that there really is a place for ANOVA

    ;-)
    I do not agree. Post ANOVA tests are all t-tests. I use pooled estimates of error so I am not ignoring the influence of the third, 4th groups etc.
  6. Joined
    03 Oct '05
    Moves
    86698
    03 Dec '08 22:29
    Originally posted by Palynka
    1st basic question:
    What do you want to test?
    Well for example, effect of different doses of a drug on a particular function.
  7. Subscribercoquette
    Already mated
    Omaha, Nebraska, USA
    Joined
    04 Jul '06
    Moves
    887530
    04 Dec '08 06:36
    Honestly, is there a significant difference? Just ask Tukey.
  8. Standard memberPalynka
    Upward Spiral
    Halfway
    Joined
    02 Aug '04
    Moves
    8702
    04 Dec '08 10:49
    Originally posted by znsho
    Well for example, effect of different doses of a drug on a particular function.
    For that case, then a paired t-test should perform well. But there are many cases (e.g. with more factors and groups) where it might be misleading.
  9. 42.4º N / -71.2º W
    Joined
    11 Jun '01
    Moves
    78257
    04 Dec '08 14:55
    Originally posted by znsho
    Well for example, effect of different doses of a drug on a particular function.
    OK, let's assume you have 3 levels of a drug 5ml/kg, 10ml/kg and 15ml/kg, and you have a placebo. If you run the f-test and find no significant value, what does that tell you?

    It tells you that the profile of means is flat, that is, that the effects of adding a drug are null. If you then went ahead and did a t-test between 10ml and 15ml, and found a sig. effect, what would that tell you?

    It tells you that (say) 15ml has a greater effect on function than 10ml. BUT, in the absence of a significant main effect, this is completely meaningless, because it fails to take into account the fact that the drug overall had no effect on function (being as there were no mean differences between the four experimental groups as a whole).

    You would wrongly conclude that 15ml was a better dose than 10ml, when in fact, neither dose is better than placebo.
  10. Standard memberPalynka
    Upward Spiral
    Halfway
    Joined
    02 Aug '04
    Moves
    8702
    04 Dec '08 15:46
    Originally posted by kyngj
    OK, let's assume you have 3 levels of a drug 5ml/kg, 10ml/kg and 15ml/kg, and you have a placebo. If you run the f-test and find no significant value, what does that tell you?

    It tells you that the profile of means is flat, that is, that the effects of adding a drug are null. If you then went ahead and did a t-test between 10ml and 15ml, and found a sig. ef ...[text shortened]... clude that 15ml was a better dose than 10ml, when in fact, neither dose is better than placebo.
    To determine if the 15ml is better than the placebo, then a t-test would still be more adequate.

    Suppose that the drug only has an effect when in doses equal or above 12ml. Separate t-tests against the placebo would be more efficient at uncovering the effect of the 15ml dose than than a global F-test.

    The problem here is that you care more about differences between placebo and drug than between drug doses.
  11. Joined
    03 Oct '05
    Moves
    86698
    04 Dec '08 19:32
    Originally posted by kyngj
    OK, let's assume you have 3 levels of a drug 5ml/kg, 10ml/kg and 15ml/kg, and you have a placebo. If you run the f-test and find no significant value, what does that tell you?

    It tells you that the profile of means is flat, that is, that the effects of adding a drug are null. If you then went ahead and did a t-test between 10ml and 15ml, and found a sig. ef ...[text shortened]... clude that 15ml was a better dose than 10ml, when in fact, neither dose is better than placebo.
    Sorry, but I disagree. It is quite possible that the experimenter investigated does of the drug which, to all intents and purposes, had no effect. It is equall possible that the highest dose is the one where the effect begins to occur. ANOVA will fail to pick up on this. T-test, even with Bonferroni's correctio correction COULD PICK UP THE GENUINE DIFFERENCE IN THE single GROUP.
  12. Joined
    03 Oct '05
    Moves
    86698
    04 Dec '08 19:33
    Originally posted by Palynka
    To determine if the 15ml is better than the placebo, then a t-test would still be more adequate.

    Suppose that the drug only has an effect when in doses equal or above 12ml. Separate t-tests against the placebo would be more efficient at uncovering the effect of the 15ml dose than than a global F-test.

    The problem here is that you care more about differences between placebo and drug than between drug doses.
    Yes, that is my point. The ANOVA F-value is meaningless.
  13. Standard memberPalynka
    Upward Spiral
    Halfway
    Joined
    02 Aug '04
    Moves
    8702
    04 Dec '08 23:05
    Originally posted by znsho
    Yes, that is my point. The ANOVA F-value is meaningless.
    It's not 'meaningless'. It's all about what type of test more adequately answers the question you want to answer.
  14. Joined
    03 Oct '05
    Moves
    86698
    04 Dec '08 23:20
    Originally posted by Palynka
    It's not 'meaningless'. It's all about what type of test more adequately answers the question you want to answer.
    I meant meaningless in the scenario I presented.
  15. The Golden Ford
    Joined
    29 May '07
    Moves
    2017
    13 Dec '08 00:56
    ack anovas, tukey tests, t-tests, pearsons correlations etc are the bane of my pychology degree existence. Am I right in thinking SPSS is the most popular software for this? It's what I use anyway.
Back to Top