Sports Medicine Research: In the Lab & In the Field: Should We Trust Low but Valid ImPACT’s Scores?…Maybe We Should Double-Check (Sports Med Res)


Monday, March 26, 2018

Should We Trust Low but Valid ImPACT’s Scores?…Maybe We Should Double-Check

Valid but Invalid: Suboptimal ImPACT© Baseline Performance in University Athletes

Walton SR, Broshek DK, Freeman J, Cullum CM, Resch JE. Med Sci Sports Exerc. 2018 ahead of print

Take Home Message: Only 85% of the athletes achieved ImPACT scores at or above the 16th percentile after the 1st baseline assessments; however, after a second or third attempt over 98% of athletes achieved scores higher than the 16th percentile.
ImPACT – a computerized neurocognitive test – generates an automated score for clinicians to compare baseline to post-injury scores to make clinical decisions. However, ImPACT does not flag scores within the 16th percentile, which may represent suboptimal effort or poor understanding of instructions. Though, this may reflect some of the athletes’ true scores, there is limited research indicating whether these baseline scores are useable. Therefore, the authors evaluated the frequency of valid but suboptimal performances among 769 NCAA Division I student athletes and examined the benefit of subsequent administration of ImPACT baseline testing for those athletes who have a valid yet suboptimal ImPACT score. All athletes were tested individually or in pairs in a quiet room with limited environmental distractions and were provided verbal and/or written instructions on how to perform the test. Additionally, proctors remained in the room during testing in case any questions arose. If a participant scored below the 16th percentile in one or more of the modules they were asked to come back to retake the test up to 2 more times to see if they would improve (> 7 days after). Based on ImPACT's validity criteria, 1% (9/769) of administrations were invalid and 14.6% (112/769) had one or more composite score below the 16th percentile but were still considered valid. After one re-administration 71% (80/112) of those participants achieved scores ≥ 16th percentile and an additional 18/32 scored ≥ 16th percentile after a third administration. Athletes demonstrated the most difficulty with verbal memory during first baseline measurement. After one or two re-administrations, over 98% of the athletes scored at or above the 16th percentile. Lastly, there were no differences between people with valid and suboptimal results regarding concussion history, age, total symptom score, hours of sleep, or endorsement of depression or anxiety.

For clinicians who use baseline assessments to compare post-injury scores, it is imperative to clinical decision making that these baseline measures represent an athlete’s true abilities. The authors of the current study, found that 98 athletes needed a second or third attempt to achieve optimal performance. Due to the length of time between subsequent testing sessions (often > 2 months), improvement in the score may not be due to knowing the correct answers but could be attributed to a better understanding of the instructions, having an enhanced feel for how the test is conducted, or was just feeling more well rested that day. It was also interesting to note that the testing session was done under prime conditions (quiet, proctored, 1-2 athletes per session), which suggests that clinical sites where athletes who are tested in suboptimal conditions (distractions, group testing) may have a higher number of athletes scoring below the 16th percentile. It would be interesting to identify whether effort influences the baseline score. For example, are athletes attempting to have a valid “sandbagged score?”  Presently, medical professionals should be aware of the limitations of the validity of the ImPACT scores and understand that additional steps such as double checking for low scores, testing in controlled environments, and subsequent testing may be necessary to ensure true baseline measure for future clinical comparisons.

Questions for Discussion: How often do your athletes re-take baseline assessment tests? Do you look at the scores after each test, and identify whether you believe that is his/her “true” score? What measures do you take to ensure your athletes will produce optimal scores during baseline assessments?

Written by: Jane McDevitt
Reviewed by: Jeffrey Driban

Related Posts:


Kim Twait said...

Thank you Jane for sharing this article! At the university that I work at, all of our athletes are baseline tested in the fall of their 1st year. The testing environment is proctored and performed in a quiet room with 1-2 athletes per test. This helps to ensure the athlete is focused and prepared to perform at their maximum level. If the athlete receives a score below the 16th percentile, they are required to retake the test, no more that 2 times, to receive a higher score. I believe this protocol forces the athlete to perform to the best of their ability and not allow them to "sandbag" their scores. However, you do have to take into consideration the learning curve of taking the same baseline test more than once. You then have to look at that score and wonder "is that the athlete's true score?" or if it's because of taking the test more than once. How often do your athletes retake the ImPACT test?

Post a Comment

When you submit a comment please click 'Subscribe by Email" (just below the comments) or "Subscribe to: Post Comments (Atom)" (at the bottom of this page) if you would like to receive a notification when another comment has been submitted to this post.

Please note that if you are using Safari and have problems submitting comments you may need to go to your preferences (privacy tab) and stop blocking third party cookies. Sorry for any inconvenience this may pose.