regression to the mean
-
What does 'regression to the mean' mean in statistics?
Regression to the mean (RTM) is a statistical phenomenon that occurs when a nonrandom sample is selected from a population, and the two variables measured are imperfectly correlated. The smaller the correlation between these two variables, the more extreme the obtained value is from the population mean, and the larger the effect of RTM. For a given value of X, the predicted value of Y is always fewer standard deviations from its mean than is X from its mean. RTM will be in effect to some extent unless r = 1, so it almost always occurs in practice.
1 Britannica SourceCreated from Britannica articles using AI. AI can make mistakes, so verify using Britannica articles. -
Why do extreme results tend to move closer to the average if tested again?
Regression to the mean (RTM) is a statistical phenomenon where extreme results tend to move closer to the average upon retesting, provided there is imperfect correlation between the variables of interest.
The smaller the correlation between two variables, the more noticeable the effects of RTM. With variables X and Y, having standard deviations SDx and SDy, with correlation = r, a change of one standard deviation in X associates with a change of r standard deviations in Y. Unless X and Y are perfectly linearly related, r is less than 1. For a given value of X, the predicted value of Y is always fewer standard deviations from its mean than is X from its mean. RTM is a mathematical necessity when correlation is less than perfect.
1 Britannica SourceCreated from Britannica articles using AI. AI can make mistakes, so verify using Britannica articles. -
How can regression to the mean affect the results of studies and experiments?
Regression to the mean (RTM) can significantly impact study and experiment results when dealing with nonrandom samples and imperfectly correlated variables. RTM occurs because extreme values tend to move closer to the average in subsequent measurements. This can lead to misleading conclusions about the effectiveness of treatments or interventions.
To address RTM, researchers randomly allocate subjects to comparison groups, using the placebo group to estimate changes caused by RTM. Averaging several measurements or using a second pretreatment measurement as a baseline can also reduce RTM's impact. Sir Francis Galton's work on the heritability of height provides an early example of RTM.
1 Britannica SourceCreated from Britannica articles using AI. AI can make mistakes, so verify using Britannica articles. -
Can you give an example of regression to the mean in sports or school tests?
Regression to the mean (RTM) is a statistical phenomenon where, in a nonrandom sample, values of a variable that are far from the average will likely be closer to the average when measured again.
For example, consider student test scores. Students who score exceptionally high or low on a test are likely to score closer to the average on a subsequent test, regardless of any intervention. This is because the correlation between the two variables is imperfect. The predicted value will be closer to the mean than the original value. RTM occurs unless the correlation equals 1.
1 Britannica SourceCreated from Britannica articles using AI. AI can make mistakes, so verify using Britannica articles. -
How can scientists or researchers accidentally mistake regression to the mean for genuine change?
Scientists or researchers may misinterpret regression to the mean as a real change when analyzing nonrandom samples with imperfectly correlated variables. Regression to the mean (RTM) is a statistical phenomenon where extreme values tend to move closer to the average upon retesting. This occurs because, without a perfect correlation (r=1), predicted values are statistically closer to the mean than the initial values. The smaller the correlation between two variables, the more extreme the value is from the population mean, and the larger the effect of RTM.
1 Britannica SourceCreated from Britannica articles using AI. AI can make mistakes, so verify using Britannica articles. -
What steps can researchers take to avoid the effects of regression to the mean in their studies?
To mitigate the effects of regression to the mean (RTM) in research, randomly allocate subjects to comparison groups to ensure all groups are equally affected by RTM. In studies with placebo and treatment groups, use the mean change in the placebo group to estimate the change caused by RTM, plus any placebo effect. The difference between the mean change in the treatment group and the mean change in the placebo group will be the estimated treatment effect after adjusting for RTM.
Additionally, reduce RTM by selecting individuals based on the average of several measurements instead of a single measurement. Another suggestion is to select patients based on one measurement but use a second pretreatment measurement as the baseline from which to compute the change.
1 Britannica SourceCreated from Britannica articles using AI. AI can make mistakes, so verify using Britannica articles.
regression to the mean (RTM), a widespread statistical phenomenon that occurs when a nonrandom sample is selected from a population and the two variables of interest measured are imperfectly correlated. The smaller the correlation between these two variables, the more extreme the obtained value is from the population mean and the larger the effect of RTM (that is, there is more opportunity or room for RTM). If variables X and Y have standard deviations SDx and SDy, and correlation = r, the slope of the familiar least-squares regression line can be written rSDy/SDx. Thus, a change of one standard deviation in X is associated with a change of r standard deviations in Y. Unless X and Y are perfectly linearly related, so that all the points lie along a straight line, r is less than 1. For a given value of X, the predicted value of Y is always fewer standard deviations from its mean than is X from its mean. Because RTM will be in effect to some extent unless r = 1, it almost always occurs in practice.
RTM does not depend on the assumption of linearity, the level of measurement of the variable (for example, the variable can be dichotomous), or measurement error. Given a less than perfect correlation between X and Y, RTM is a mathematical necessity. Although it is not inherent in either biological or psychological data, RTM has important predictive implications for both. In situations in which one has little information to make a judgment, often the best advice is to use the mean value as the prediction.
History
An early example of RTM may be found in the work of Sir Francis Galton on heritability of height. He observed that tall parents tended to have somewhat shorter children than would be expected given their parents’ extreme height. Seeking an empirical answer, Galton measured the height of 930 adult children and their parents and calculated the average height of the parents. He noted that when the average height of the parents was greater than the mean of the population, the children were shorter than their parents. Likewise, when the average height of the parents was shorter than the population mean, the children were taller than their parents. Galton called this phenomenon regression toward mediocrity; it is now called RTM. This is a statistical, not a genetic, phenomenon.
" height="20" alt="HEW Logo">
Examples
Treatment versus nontreatment
In general, among ill individuals, certain characteristics, whether physical or mental, such as high blood pressure or depressed mood, have been observed to deviate from the population mean. Thus, a treatment would be deemed effective when those treated show improvement on such measured indicators of illness at posttreatment (e.g., a lowering of high blood pressure or remission of or reduced severity of depressed mood). However, given that such characteristics deviate more from the population mean in ill individuals than in well individuals, this could be attributable in part to RTM. Moreover, it is likely that on a second observation, untreated individuals with high blood pressure or depressed mood also will show some improvement owing to RTM. It also is probable that individuals designated as within the normal range of blood pressure or mood at first observation will be somewhat less normal at a second observation, also due in part to RTM. In order to identify true treatment effects, it is important to assess an untreated group of similar individuals or a group of similar individuals in an alternative treatment in order to adjust for the effect of RTM.
Variations within single groups
Within groups of individuals with a specific illness or disorder, symptom levels may range from mild to severe. Clinicians sometimes yield to the temptation of treating or trying out new treatments on patients who are the most ill. Such patients, whose symptoms are indicative of characteristics farthest from the population mean or normality, often respond more strongly to treatment than do patients with milder or moderate levels of the disorder. Caution should be exercised before interpreting the degree of treatment effectiveness for severely ill patients (who are, in effect, a nonrandom group from the population of ill individuals) because of the probability of RTM. It is important to separate genuine treatment effects from RTM effects; this is best done by employing randomized control groups that include individuals with varying levels of illness severity and normality.
How to deal with RTM
If subjects are randomly allocated to comparison groups, the responses from all groups should be equally affected by RTM. With placebo and treatment groups, the mean change in the placebo group provides an estimate of the change caused by RTM (plus any other placebo effect). The difference between the mean change in the treatment group and the mean change in the placebo group is then the estimate of the treatment effect after adjusting for RTM. RTM can be reduced by basing the selection of individuals on the average of several measurements instead of a single measurement. It has also been suggested to select patients on the basis of one measurement but to use a second pretreatment measurement as the baseline from which to compute the change. If the correlation coefficient between the posttreatment and the first pretreatment measurement is the same as that between the first and the second pretreatment measurement, then there will be no expected mean change due to RTM.