Chapter 11 Summary

Correlated-Groups and Single-Subject Designs

Assuring group equivalence at the start of a study can be accomplished by using correlated-groups designs. These designs do not include random assignment but, nevertheless, provide equivalent groups at the start of the study and allow other controls to be applied. One important reason for using correlated-groups designs is that they are generally more sensitive to the effects of the independent variable manipulations--a phenomenon often referred to as increased statistical power. Therefore, they are often preferred by researchers to between-subject designs.

There are two basic ways of introducing the correlation among participants in correlated-groups designs: (1) by having a single group of participants exposed to all of the conditions (within-subjects or repeated measures designs); and (2) by matching participants on some important variable(s) and then randomly assigning these matched sets of participants so that one participant of each set is assigned to each condition (matched-subjects designs).

Within-Subjects Designs

In within-subjects designs, all participants are exposed to all experimental conditions. In essence, each participant serves as his or her own control.

Using Within-Subjects Designs

The characteristics of within-subjects designs are:

  1. Each participant is tested under each experimental condition.
  2. Therefore, the scores in each condition are correlated with the scores in each other condition.
  3. The critical comparison is the difference between correlated groups on the dependent variable.

In within-subject designs, a single sample of participants is exposed to all of the conditions of the experiment. Because the same participants are in all conditions, the experience each participant has in one condition might affect how that participant responds in subsequent conditions. Thus, if differences between the conditions are found, they might not be due to the independent variable manipulation, but to the confounding effects of one condition on later conditions. These confounding effects are called sequencing effects, and they must be controlled. One of the controls for sequencing effects is counterbalancing. In complete counterbalancing , the order of presentation of conditions to participants is systematically varied so that: (1) each participant is exposed to all of the conditions of the experiment, (2) each condition is presented an equal number of times, (3) each condition is presented an equal number of times in each position, and (4) each condition precedes and follows each other condition an equal number of times.

Analyzing within-subjects design

The most appropriate statistical analysis for a single-variable, within-subjects experiment is a repeated-measures ANOVA, which takes into account the fact that the measures are correlated. The advantage of a within-subjects design is that it effectively equates the participants in the different conditions prior to the experimental manipulation. Therefore, the single largest contributing factor to error variance—individual differences—has been eliminated. Reducing the error variance increases the F-ratio. (Since the individual difference portion of the error term has been removed, the denominator in the F-ratio will be smaller and, therefore, the F will be larger.) This means that the procedure will be more sensitive to small differences between the groups.

In the repeated-measures ANOVA, the between-groups and total sums of squares are computed in the same way as in a simple one-way ANOVA. However, the within-groups sum of squares is split into two terms: a subjects term (the individual differences component of the within-groups variability), and an error term (what is left of the within-groups variability after the individual differences component is removed).

Strengths and Weaknesses of Within-Subjects Design

There are several advantages of within-subjects designs:

  1. Because the same participants are in each condition, the groups of participants are equivalent at the start of the study.
  2. Within-subjects designs are more sensitive than between-subject designs to effects of the independent variable manipulation.
  3. Within-subjects designs are more efficient than between-subjects designs in two ways: (a) fewer participants are needed, and (b) instructions often need be given only once per participant, instead of once per participant for each condition.

There are several disadvantages of within-subjects designs, all of which stem from the fact that each participant is exposed to each condition. Participants' experiences in one condition might affect their responses in one or more of the subsequent conditions. Therefore, differences between groups might not be due to the independent variable manipulation but, rather, to the confounding factor of sequencing effects, which include practice effects and carry-over effects. Practice effects are caused by the participants' practice and growing experience as they move through the sequence of conditions. This effect is due to the participants' growing general familiarity with the procedures. These may be positive or negative practice effects. Carry-over effects are sequencing effects due to the influence of a particular condition or combination of conditions on responses to the very next condition.

There are two general types of controls for sequencing effects: (1) holding the extraneous variable constant, and (2) varying the order of presentation of the conditions (counterbalancing or random ordering). Counterbalancing can be complete or partial.  Latin square designs are examples of partial counterbalancing. Trials can also be randomized within blocks. However, if strong carry-over effects are expected, a within-subjects design is not recommended, even if the above controls are included.

Matched-Subjects Design

Matched-subjects designs use different participants in each group, but the participants have been closely matched before assignment to conditions. The characteristics are:

  1. Each participant is exposed to only one level of the independent variable.
  2. Each participant has a matched participant in each of the other conditions, so the groups are correlated.
  3. Only one measurement per participant on the dependent variable is used, and the analysis takes into account the matching.
  4. The critical comparison is the difference between the correlated groups.

Using Matched-Subjects Designs

A matched-subjects design is used when the researcher wants to take advantage of the greater sensitivity to independent variable manipulations, but cannot, or chooses not to, use the within-subjects design. The most common example is when the manipulations would cause severe sequencing effects.

Participants should be matched on relevant variables. A variable is relevant if it can have an effect on the dependent variable in a study. That is, the important variables on which to match are variables that are strongly related to performance on the dependent measures. Matching on more than one variable can become difficult.

Analyzing Matched-Subjects Designs

In analyzing results of matched-subjects designs, it is necessary to maintain the ordering of the data (i.e., who each participant is matched with). The same statistical procedures used for within-subjects designs are appropriate for the matched-subjects designs.

Strengths and weaknesses of matched-subjects design

Within-subjects and matched-subjects designs have similar strengths, but different weaknesses. Both have good sensitivity to small differences between conditions because the groups are equivalent (or even identical). Therefore, smaller numbers of participants are needed. An advantage of the matched-subjects design over the within-subjects design is that there are no problems resulting from practice and carry-over effects. Therefore, procedures such as counterbalancing are not needed. The most obvious disadvantage of the matched-subjects design is that it requires a good deal of effort to match participants. Also, the requirements of matching might eliminate many potential participants because suitable matches cannot be found for them. In such cases we may be better off using a large group of randomly assigned participants in a between-subjects design.

Single-Subject Experimental Designs

Single-subject designs are extensions of within-subjects designs. These designs are particularly useful in evaluating treatment effects, such as in behavior modification. These are experimental designs and should not be confused with the ex post facto single-case designs discussed in Chapter 6. Based on the experimental analysis of behavior, these designs typically involve measures in a baseline period, followed by one or more treatment periods.

ABA Reversal Design

In ABA reversal designs the effects of an independent variable on a dependent variable are determined by measuring the dependent variable over several time periods during which the treatment is applied and then removed. These typically include a no-treatment baseline period of observation of the target behavior, followed by a treatment period, and then a return to baseline. The effects of the independent variable are demonstrated if the behavior changes in the predicted direction at each reversal of conditions.

Multiple-Baseline Design

Multiple-baseline designs are used when reversal designs are not useful or are unethical. In multiple- baseline designs the effects of the treatment are demonstrated on different behaviors successively. The various forms of multiple-baseline design include multiple baseline designs across behavior, across individuals, across time, and across settings. 

 Single-Subject, Randomized, Time-Series Design

This is a time-series design for a single participant in which the point at which the treatment begins is determined randomly. If the dependent behavior changes exactly when the manipulation has been randomly introduced, this is taken as evidence that the independent variable has had an effect.

Replication in Single-Subject Research

As in all research, replication is important in single-subject research. In single-subject experimental designs, replication can be achieved in several ways: single-subject direct replication (repeating the experiment in the same way, with the same participants); single-subject systematic replication (a series of single-subject experiments with different participants in different settings and new target behaviors); single-subject clinical replication (usually used in clinical settings, this involves an integrated treatment package of two or more treatment procedures applied to a succession of participants).