Valid but Invalid: Suboptimal ImPACT© Baseline Performance in
University Athletes
SR, Broshek DK, Freeman J, Cullum CM, Resch JE. Med Sci Sports Exerc. 2018 ahead of print
above the 16th percentile after the 1st baseline
assessments; however, after a second or third attempt over 98% of athletes
achieved scores higher than the 16th percentile.
ImPACT – a computerized neurocognitive
test – generates an automated score for clinicians to compare baseline to post-injury
scores to make clinical decisions. However, ImPACT does not flag scores within
the 16th percentile, which may represent suboptimal effort or poor
understanding of instructions. Though, this may reflect some of the athletes’
true scores, there is limited research indicating whether these baseline scores
are useable. Therefore, the authors evaluated the frequency of valid but
suboptimal performances among 769 NCAA Division I student athletes and examined
the benefit of subsequent administration of ImPACT baseline testing for those
athletes who have a valid yet suboptimal ImPACT score. All athletes were tested
individually or in pairs in a quiet room with limited environmental distractions
and were provided verbal and/or written instructions on how to perform the test.
Additionally, proctors remained in the room during testing in case any
questions arose. If a participant scored below the 16th percentile
in one or more of the modules they were asked to come back to retake the test up
to 2 more times to see if they would improve (> 7 days after). Based on
ImPACT’s validity criteria, 1% (9/769) of administrations were invalid and
14.6% (112/769) had one or more composite score below the 16th percentile but
were still considered valid. After one re-administration 71% (80/112) of those
participants achieved scores ≥ 16th percentile and an additional 18/32 scored ≥
16th percentile after a third administration. Athletes demonstrated the most
difficulty with verbal memory during first baseline measurement. After one or
two re-administrations, over 98% of the athletes scored at or above the 16th
percentile. Lastly, there were no differences between people with valid and suboptimal
results regarding concussion history, age, total symptom score, hours of sleep,
or endorsement of depression or anxiety.
clinicians who use baseline assessments to compare post-injury scores, it is
imperative to clinical decision making that these baseline measures represent
an athlete’s true abilities. The authors of the current study, found that 98
athletes needed a second or third attempt to achieve optimal performance. Due
to the length of time between subsequent testing sessions (often > 2 months),
improvement in the score may not be due to knowing the correct answers but
could be attributed to a better understanding of the instructions, having an enhanced
feel for how the test is conducted, or was just feeling more well rested that
day. It was also interesting to note that the testing session was done under prime
conditions (quiet, proctored, 1-2 athletes per session), which suggests that clinical
sites where athletes who are tested in suboptimal conditions (distractions,
group testing) may have a higher number of athletes scoring below the 16th
percentile. It would be interesting to identify whether effort influences the
baseline score. For example, are athletes attempting to have a valid
“sandbagged score?” Presently, medical
professionals should be aware of the limitations of the validity of the ImPACT scores
and understand that additional steps such as double checking for low scores,
testing in controlled environments, and subsequent testing may be necessary to
ensure true baseline measure for future clinical comparisons.
assessment tests? Do you look at the scores after each test, and identify
whether you believe that is his/her “true” score? What measures do you take to
ensure your athletes will produce optimal scores during baseline assessments?
Thank you Jane for sharing this article! At the university that I work at, all of our athletes are baseline tested in the fall of their 1st year. The testing environment is proctored and performed in a quiet room with 1-2 athletes per test. This helps to ensure the athlete is focused and prepared to perform at their maximum level. If the athlete receives a score below the 16th percentile, they are required to retake the test, no more that 2 times, to receive a higher score. I believe this protocol forces the athlete to perform to the best of their ability and not allow them to "sandbag" their scores. However, you do have to take into consideration the learning curve of taking the same baseline test more than once. You then have to look at that score and wonder "is that the athlete's true score?" or if it's because of taking the test more than once. How often do your athletes retake the ImPACT test?