Design of experiments & ANOVA

Main Contents page | Index of all entries

We are concerned with the analysis of data generated from an experiment. It is wise to take time and effort to organise the experiment properly to ensure that the right type of data, and enough of it, is available to answer the questions of interest as clearly and efficiently as possible. This process is called experimental design.

The specific questions that the experiment is intended to answer must be clearly identified before carrying out the experiment. We should also attempt to identify known or expected sources of variability in the experimental units since one of the main aims of a designed experiment is to reduce the effect of these sources of variability on the answers to questions of interest. That is, we design the experiment in order to improve the precision of our answers.

See also Completely Randomised Design.
See also Randomised Complete Block Design.
See also Factorial Design.

In experiments, a treatment is something that researchers administer to experimantal units . For example, a corn field is divided into four, each part is 'treated' with a different fertiliser to see which produces the most corn; a teacher practices different teaching methods on different groups in her class to see which yields the best results; a doctor treats a patient with a skin condition with different creams to see which is most effective.

Treatments are administered to experimental units by 'level', where level implies amount or magnitude. For example, if the experimental units were given 5mg, 10mg, 15mg of a medication, those amounts would be three levels of the treatment. 'Level' is also used for categorical variables, such as Drugs A, B, and C, where the three are different kinds of drug, not different amounts of the same thing.

A factor of an experiment is a controlled independent variable; a variable whose levels are set by the experimenter.

A factor is a general type or category of treatments. Different treatments constitute different levels of a factor. For example, three different groups of runners are subjected to different training methods. The runners are the experimental units, the training methods, the treatments, where the three types of training methods constitute three levels of the factor 'type of training'.

The one way analysis of variance allows us to compare several groups of observations, all of which are independent but possibly with a different mean for each group. A test of great importance is whether or not all the means are equal.

The observations all arise from one of several different groups (or have been exposed to one of several different treatments in an experiment). We are classifying 'one-way' according to the group or treatment.

Two Way Analysis of Variance is a way of studying the effects of two factors separately (their main effects) and (sometimes) together (their interaction effect).

See also factor.
See also main effect.
See also interaction.

The structure of the experiment in a completely randomised design is assumed to be such that the treatments are allocated to the experimental units completely at random.

See also treatment.
See also experimental unit.

The randomised complete block design is a design in which the subjects are matched according to a variable which the experimenter wishes to control. The subjects are put into groups (blocks) of the same size as the number of treatments. The members of each block are then randomly assigned to different treatment groups.

Example
A researcher is carrying out a study of the effectiveness of four different skin creams for the treatment of a certain skin disease. He has eighty subjects and plans to divide them into 4 treatment groups of twenty subjects each. Using a randomised blocks design, the subjects are assessed and put in blocks of four according to how severe their skin condition is; the four most severe cases are the first block, the next four most severe cases are the second block, and so on to the twentieth block. The four members of each block are then randomly assigned, one to each of the four treatment groups.

See also treatment.
See also blocking.

A factorial design is used to evaluate two or more factors simultaneously. The treatments are combinations of levels of the factors. The advantages of factorial designs over one-factor-at-a-time experiments is that they are more efficient and they allow interactions to be detected.

See also treatment.
See also factor.
See also interaction.

This is the simple effect of a factor on a dependent variable. It is the effect of the factor alone averaged across the levels of other factors.

Example
A cholesterol reduction clinic has two diets and one exercise regime. It was found that exercise alone was effective, and diet alone was effective in reducing cholesterol levels (main effect of exercise and main effect of diet). Also, for those patients who didn't exercise, the two diets worked equally well (main effect of diet); those who followed diet A and exercised got the benefits of both (main effect of diet A and main effect of exercise). However, it was found that those patients who followed diet B and exercised got the benefits of both plus a bonus, an interaction effect (main effect of diet B, main effect of exercise plus an interaction effect).

See also factor.

An interaction is the variation among the differences between means for different levels of one factor over different levels of the other factor.

Example
A cholesterol reduction clinic has two diets and one exercise regime. It was found that exercise alone was effective, and diet alone was effective in reducing cholesterol levels (main effect of exercise and main effect of diet). Also, for those patients who didn't exercise, the two diets worked equally well (main effect of diet); those who followed diet A and exercised got the benefits of both (main effect of diet A and main effect of exercise). However, it was found that those patients who followed diet B and exercised got the benefits of both plus a bonus, an interaction effect (main effect of diet B, main effect of exercise plus an interaction effect).

See also factor.

Randomisation is the process by which experimental units (the basic objects upon which the study or experiment is carried out) are allocated to treatments; that is, by a random process and not by any subjective and hence possibly biased approach. The treatments should be allocated to units in such a way that each treatment is equally likely to be applied to each unit.

Randomisation is preferred since alternatives may lead to biased results.

The main point is that randomisation tends to produce groups for study that are comparable in unknown as well as known factors likely to influence the outcome, apart from the actual treatment under study. The analysis of variance F tests assume that treatments have been applied randomly.

See also experimental unit.
See also treatment.

In a medical experiment, the comparison of treatments may be distorted if the patient, the person administering the treatment and those evaluating it know which treatment is being allocated. It is therefore necessary to ensure that the patient and/or the person administering the treatment and/or the trial evaluators are 'blind to' (don't know) which treatment is allocated to whom.

Sometimes the experimental set-up of a clinical trial is referred to as double-blind, that is, neither the patient nor those treating and evaluating their condition are aware (they are 'blind' as to) which treatment a particular patient is allocated. A double-blind study is the most scientifically acceptable option.

Sometimes however, a double-blind study is impossible, for example in surgery. It might still be important though to have a single-blind trial in which the patient only is unaware of the treatment received, or in other instances, it may be important to have blinded evaluation.

A placebo is an inactive treatment or procedure. It literally means 'I do nothing'. The 'placebo effect' (usually a positive or beneficial response) is attributable to the patient's expectation that the treatment will have an effect.

See also treatment.

This is the procedure by which experimental units are grouped into homogeneous clusters in an attempt to improve the comparison of treatments by randomly allocating the treatments within each cluster or 'block'.

See also experimental unit.
See also treatment.
See also randomised complete block design.

---

Top of page | Main Contents page