Evidence for added value
of baseline testing in computer-based cognitive assessment.

Roebuck-Spencer TM, Vincent AS,
Schlegel RE, and Gilliland K. J of Athl Training. 2013;48 (3).

Home Message: Computer-based neurocognitive testing methods of deciphering if a
patient has a concussion may lead to false-positives, thus a gold standard is
still needed.

Baseline computer-based cognitive
testing is commonplace in athletics; however, more information is needed
regarding the validity of comparing post-injury results with baseline
performance. Therefore, Roebuck-Spencer and colleagues completed a study of
8,002 military service members (91% male, ~27 years old) to examine the added
value of baseline testing in computer-based cognitive testing by comparing 2
methods of classifying atypical performance: (1) baseline comparison and (2) normal
reference. All participants took the Automated Neuropsychological Assessment
Matrix (ANAM) prior to and after deployment. 
Patients were excluded if they (1) reported a history of concussions
during deployment, (2) nonspecific injuries, (3) extreme outliers at either
time point, (4) incomplete data on potential injury history, and (5) pre-deployment-post-deployment
interval of less than 60 days. Trained test administrators conducted the testing
in a group setting. Post-deployment testing took place on day 6 of a 7-day
reintegration process. Notable declines in neurocognitive performance were
identified using a
reliable change index (RCI). Atypical performance at the
post-deployment performance was presented in 2 ways: the baseline comparison
method (post-deployment scores notably declined) and norm-referenced method
(the individual’s ANAM score was below the normal range of scores from the
large sample of healthy individuals). Overall, the two methods were similar and
classified 3.7 and 3.4% of participants with atypical post-deployment scores using
the baseline-referenced and norm-referenced method, respectively.  Interestingly, however, both methods were
inconsistent with regards to which individual was considered atypical. Of the
147 individuals classified as atypical using the baseline-based method, 68% (100
individuals) were classified as normal using the norms-based method. Similarly,
of the 137 participants classified as atypical by the norm-based method, 66% (90
individuals) showed no change in test performance from pre-deployment to

This study presents clinicians with a
very interesting dilemma with regards to deciphering computer-based cognitive
assessments. While both classification methods had merit, when compared, both
methods classified different participants as atypical in a seemingly healthy
population. Although the number of misclassified individuals was small (3 to 4
out of 100 individuals) it is concerning that 190 (~2%) individuals had
different results with the two methods. Future research may be needed to
determine why these individuals were misclassified and if we can further optimize
computer-based cognitive testing or combine this testing with certain clinical
tests to avoid false findings. It should also be noted however, that the
authors used a military sample that self-reported no concussions during
deployment and just used one computer-based cognitive test. Some of the individuals
may have experienced an undiagnosed concussion that would account for the
misclassification. Furthermore, it is unclear what the misclassification rate
may be with another computer-based test. 
Performing this study in a sample of athletic teams may yield a
different result because the normative values with respect to age, gender, etc.
would be different. Further, in this population clinicians would be able to closely
monitor injuries and sub-concussive events which may lead to changes in test
performance. In the meantime, it is important to keep in mind that
computer-based testing, like any clinical test, may lead to false-positive and
false-negative outcomes and therefore the clinical judgement remains key to trying
to minimize these wrong outcomes.

for Discussion: How do you currently diagnose concussions with regards to
computerized cognitive testing? Have you felt this has been an effective method
of patient classification?

Written by: Kyle Harris
Reviewed by:  Jeffrey Driban

Related Posts:

Roebuck-Spencer, T., Vincent, A., Schlegel, R., & Gilliland, K. (2013). Evidence for Added Value of Baseline Testing in Computer-Based Cognitive Assessment Journal of Athletic Training DOI: 10.4085/1062-6050-48.3.11