Journal of the International Neuropsychological Society



Detecting significant change in neuropsychological test performance: A comparison of four models


NANCY R.  TEMKIN a1a2c1, ROBERT K.  HEATON a5, IGOR  GRANT a5a6 and SUREYYA S.  DIKMEN a1a3a4
a1 Department of Neurological Surgery, University of Washington, Seattle, WA
a2 Department of Biostatistics, University of Washington, Seattle, WA
a3 Department of Rehabilitation Medicine, University of Washington, Seattle, WA
a4 Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA
a5 Department of Psychiatry, University of California, San Diego, La Jolla, CA
a6 VA San Diego Healthcare System, San Diego, CA

Abstract

A major use of neuropsychological assessment is to measure changes in functioning over time; that is, to determine whether a difference in test performance indicates a real change in the individual or just chance variation. Using 7 illustrative test measures and retest data from 384 neurologically stable adults, this paper compares different methods of predicting retest scores, and of determining whether observed changes in performance are unusual. The methods include the Reliable Change Index, with and without correction for practice effect, and models based upon simple and multiple regression. For all test variables, the most powerful predictor of follow-up performance was initial performance. Adding demographic variables and overall neuropsychological competence at baseline significantly but slightly improved prediction of all follow-up scores. The simple Reliable Change Index without correction for practice performed least well, with high error rates and large prediction intervals (confidence intervals). Overall prediction accuracy was similar for the other three methods; however, different models produce large differences in predicted scores for some individuals, especially those with extremes of initial test performance, overall competency, or demographics. All 5 measures from the Halstead–Reitan Battery had residual (observed − predicted score) variability that increased with poorer initial performance. Two variables showed significant nonnormality in the distribution of residuals. For accurate prediction with smallest prediction–confidence intervals, we recommend multiple regression models with attention to differential variability and nonnormality of residuals. (JINS, 1999, 5, 357–369.)

(Received October 13 1997)
(Revised June 18 1998)
(Accepted July 13 1998)


Key Words: Reliability; Stability; Prediction intervals; Confidence intervals; Test–retest data.

Correspondence:
c1 Reprint requests to: Nancy R. Temkin, Department of Neurological Surgery (Box 359924), University of Washington, Seattle, WA 98104-2499. E-mail: temkin@biostat.washington.edu


Related Content