Principle and Application of one-way ANOVA
When we measure a variable in each of two independently collected sample populations, hypotheses testing can help us to investigate any differences in the means and variances between the two sample population (i.e. data groups). However, we may often have data obtain from more than two sample populations, like in figure shown. It may be tempting to apply two-sample t-tests to all possible pairs of samples, but it is invalid.
One-way ANalysis Of VAriance (ANOVA) is used to compare several means. This method is often used in scientific or medical experiments when more than two treatments, processes, materials or products are being compared. So in ANOVA we are testing the null hypothesis, “Do all our data groups come from populations with the same mean?”. The example given above is called a one-way between groups model. We are looking at the differences between the groups. This is the simplest version of ANOVA.
Someone may think that we just need to compare the sample means. However, we would not expect the sample means to be exactly equal and there will be always be some differences due to variations in the measurements. Therefore, we need to ask “Are the observed differences between the sample means simply due to variation in measurements or due to real differences in the population means?” We can not answer the question just from the sample means. We need to look at the variability of whatever we are measuring. In analysis of variance we compare the variability between the groups (how far apart are the means?) to the variability within the groups (how much natural variation is there in our measurements?). That is why the test is called analysis of variance.
ANOVA is based on the assumptions that the observations in a population follows a normal distributions and populations have similar variances. Fortunately, ANOVA procedures are not very sensitive to unequal variances, and some scientists follow the rule: If the largest standard deviation (not variance) is less than twice the smallest standard deviation, we can use ANOVA and our results will still be valid. If our data do not follow a normal distribution, we can apply the nonparametric Kruskal-Wallis test described further below.
One-way ANOVA with repeated measures
An ANOVA design where multiple observations are made for each subject is called a repeated measures design, i.e it is used when we have a single group on which you have measured something a few times. The advantage of using a repeated-measures test is to control for experimental variability. We use this technique either (i) changes in means over time points, or (ii) differences in means different conditions. For example, for (i), we might be investigating the long-time effect of a 3-month drug test on cholesterol level and want to measure cholesterol concentration every two weeks. It allows to develop a time-course for any effect the drug may have on cholerstrol level. For (ii), we use the same subjects to do different types of exercise (cycling, running and swimming) and measure calorie consumption, rather than having different people doing each different exercise. That means the same people are being measured more than once on the same dependent variable (calorie consumption, and so it is called repeated measures.
In statistics, the Kruskal–Wallis one-way analysis of variance by ranks is a non-parametric method for testing whether samples originate from the same distribution. It is used for comparing more than two samples that are independent, or not related. It is an extension of the Mann–Whitney U test to three or more groups. The parametric equivalence of the Kruskal-Wallis test is the one-way analysis of variance (ANOVA). The factual null hypothesis is that the populations from which the samples originate have the same median. When the Kruskal-Wallis test leads to significant results, then at least one of the samples is different from the other samples.
Friedman test is similar to one-way repeated measure ANOVA and is a useful test when we want to know if differences exist between three or more samples and data are ordinal (ranked). We should design an experiments so that we can apply a parametric analysis. However there are times when it is only possible to use ordinal data. When that is the case, the Friedman test may be an appropriate alternate statistical aid when looking at multiple analyses.
Cochran’s Q test
Simply speaking, Cochran’s Q test is a binary data version of repeated-measure ANOVA. So, we have multiple binomial data (like „yes“ or „no“ responses), and you want to see whether the ratio of the responses are different across the groups (e.g., methods, treatments or devices the participant use). Let’s say we have patients with the same skin disease, and we want to test the effectiveness of three new drugs. The drugs are applied in the form of a lotion to the patients, and the patients is asked if he/she feels a relief from the symptoms, i.e. itchiness. No relief is indicate by „0“ and relief by „1“, i.e. as binary data (see table below). With the Cochran’s, we can test if significant differences exist among the effectiveness of the three drugs.
Post comparison test
ANOVA or Kruskal-Wallis do not identify where the differences occur or how many differences actually occur. We need to apply a post comparison test to find out where the differences among the sample population occur. Several post comparison tests have been developed and described in textbooks, and they usually differ in their theoretical basis. The most common tests, also supported by MaxStat, are listed below with recommendation of their application.