One-Year Test-Retest Reliability of the Online Version of ImPACT in High School Athletes
Elbin RJ, Schatz P, Covassin T. Am J Sports Med. 2011; 39(11): 2319-2324
https://www.ncbi.nlm.nih.gov/pubmed/21841066
https://www.ncbi.nlm.nih.gov/pubmed/21841066
Computer neurocognitive assessment tools are being utilized more and more in athletic training rooms and physician offices. ImPACT (Immediate Post-Concussion Assessment and Cognitive Testing) is one of those computer programs, which has been evaluated and supported by many medical personnel. An online version has just been released and though it is similar to the older desktop version they are not identical. The purpose of this study was to determine a 1-year test-retest reliability of the ImPACT online version in a sample of 369 varsity high school athletes. This was a within-subject design, where participants completed 2 mandatory preseason baseline cognitive assessments ~1.2 years apart. There were no diagnosed concussions between assessments within this population. Intraclass correlation coefficients (ICCs), a means of evaluating consistency between repeated tests, indicated that motor processing speed had the most stable score (.85), then reaction time (.76), visual memory (.70), and lastly verbal memory (.62). Additional analyses explored the stability of online ImPACT testing over time and found that there was little variation between baseline and follow up. There was some evidence that visual memory, reaction time, and symptom scores had some participants that changed over time.
The online version of ImPACT was produced to update the older desktop versions limitations (e.g., decrease number of invalid baseline scores). This study suggests that baseline scores from the online version were stable up to one year. When compared to previous desktop ImPACT data collected over 2 years (Schatz, 2010) the online version performed better. This study also compared favorably to reliability data that was collected after just 7 days (Iverson et al., 2003). While there were changes with the online version some scores improved and some scores decline. With this in mind, most athletic departments/teams perform a baseline test every 2 years. It would be beneficial to know if the online version ImPACT baseline would be stable over a 2-year period, not only in the high school population but in a collegiate population as well. Also, this study was done using high school athletes that did not sustain a concussion. Though, this study supports implementing the online version it will be helpful to see more data about how it performs over a longer follow-up time or after a concussion before adopting it in the clinics. It would also be beneficial to observe if an athlete’s baseline changes after sustaining a concussion. If you utilize ImPACT how often do you measure an athlete’s baseline? Did any of your athlete’s baseline change after sustaining a concussion?
Written by: Jane McDevitt MS, ATC, CSCS
Reviewed by: Jeffrey Driban
Related Posts:
Elbin RJ, Schatz P, & Covassin T (2011). One-Year Test-Retest Reliability of the Online Version of ImPACT in High School Athletes. The American Journal of Sports Medicine, 39 (11), 2319-24 PMID: 21841066
Great post Jane! We baseline test our collegiate athletes every year. After reading your post, I went back and looked at a few athletes' baseline scores for this year after suffering a concussion last year. In 3 cases there was a definite trend towards lower baseline scores which is really interesting to me. What are your thoughts on this? And has anyone seen this too?
Well, the research is currently mixed whether multiple concussions put the athlete more at risk for another concussion. Some suggest that with proper treatment and management the brain should return to normal, and there are those that found a significant difference in outcome between athletes with and without a history of concussion. The athletes with previous history of concussion seem to have poorer outcomes (e.g., longer duration of sign and symptoms; Collins et al., 2002, 2004) However, there certainly has been research that suggests that multiple concussions do cause long term problems such as dementia and depression (Barkhoudarian, Hovda, Giza, 2011). With this in mind I have not seen a specific study that investigates if there would be a significant difference in the baselines of athletes with a previous history of concussion compared to those without a history of concussion, especially within in a short time frame such as 2-3 years. I would be curious to visit this trend in research.
Great post! This study is significant in that it is the only study, to my knowledge, to make a within-groups comparison of data in this age group. With computerized neurocognitive tests becoming increasingly popular, it is important to ensure we are utilizing them correctly.
In the case of deteriorating baseline scores following a concussion, I’d be interested to see if the scores are still within the RCI’s calculated by ImPACT when compared to their original baselines? Also, I can’t help but wonder about the extent decrements resulting from subconcussive blows over time from football. Do you think they would have an increased effect in those with a history of concussion? Purely speculation on my part…
Meredith that is a great point and question about subconcussive blows over time. Additionally I question the stability of post concussive measures once a test states baseline scores have been reached. Typically baseline scores are reached following a concussion and that is the last of the testing we do on our athletes with computerized neurocognitive testing. Does it take another diagnosable concussion to cause a drop from baseline scores in individuals who have a previous history of concussions? Or could those subconcussive blows result in below baseline scores that are only exacerbated by the next concussion? Should we be doing periodic testing on athletes who have sustained one or more concussions in contact sports?
Mark, I personally believe that it would make sense to implement more frequent neurocognitive reassessments in collision sports. How often? I think that the main issue clinicians would run into is time constraints and athlete compliance. One benefit would be that eventually test-takers would hit a learning effect ceiling and their test scores would plateau (accurate baselines). I also think it would be interesting to look at a series of neurocognitive reevaluations taken throughout a football season.
Meredith, you bring up a good point about the extent of damage resulting from subconcussive blows over time. One huge limitation right now, is that it is unclear how many subconcussive or concussive episodes an individual truly has undergone before being introduced to the ImPACT or CRI. Adolescents may start playing football as young as 6-8 years old…who knows how many concussions they will receive before they enter high school. At what age should neurocognitive testing begin in athletics and is there a way to mandate this in non-school teams? Could early concussions be the cause of learning disabilities diagnosed later in life?
Meredith, Great comments! Measuring subconcussive blows have been a problem in research. Some studies have tried biochemcial serum markers (e.g., S-100b; Bloomfield, McKinney, Simth, & Brisman, 2007; Raabe et al., 1999; Biberthaler, Mussack, & Wiedemann, 2002; De Boussard et al., 2004; de Kruijk, Leffers, Menheere, Meerhoff, Twijnstra, 2001; Ingebrigtsen et al., 2000). ). Although there are links to S-100B levels increasing after soccer heading, and after a football hit, however, it is difficult to determine because s-100B is elevated after any injury in the body. Many of the subconcussive impact studies utilize paper and pencil assessments or neurophysiological evaluations that and have not utilized ImPACT (Master, 1999). In addition, research is also indecisive if cumulative concussions would cause long term problems. Some researchers suggest that if you treat the injury then the brain would have no cumulative effect, but again I have not seen this research conducted in the literature. I would speculate that those athletes with concussions that went undiagnosed or returned to play early would perform poorly on future neurocognitive assessments (again that is purely speculation!).
Mark, you bring great point, neurocognitive evaluations are just one part in the evaluation and treatment process for a concussion, however, there is no information about how many times or how long in between one should test an athlete that has sustained a concussion and returned to play. Has anyone ever tested an athlete that was diagnosed with a concussion and returned to play? Were there any changes to the neurocognitive baseline? This would also be a fantastic study to examine. If I come across any of these types of studies in the future I will make sure I review and post them!
Eric, I agree I think it does make sense to re-evaluate athletes that are participating in high impact sports, but you are right it would be difficult not only for the medical profession but the athletes as well especially in schools with 1 or 2 ATs and many sports going on. However, I am not sure if a learning effect would be beneficial because those with a concussion but have taken the exam several times may improve their score and decrease the severity of their injury on paper, but they would still need to go through the other tests (ocular, progression into functional play, and be clear of all signs and symptoms). I do believe that age is a very important variable that is just recently getting the attention necessary. Due to the immature skeleton, decreased head and neck strength, and decreased motor skills pediatric athletes (5-12 years of age) are at a higher risk for sustaining a concussion. There has been research shown that multiple concussions lead to deleterious effects later on in life (e.g., depression, Alzheimer’s). I believe there could be a link between concussions and mental disorders such as ADD or learning disabilities, specifically genetic mutations that could be associated with this link. I also believe that in addition to mandating pediatric athletes to take a neurocognitive exam that it be more adaptable and readable for their level. Is a 6 or 7-year-old going to know what sensitive to light or loss of consciousness means? With this in mind I believe these athletes will need more one on one attention to take a neurocognitive exam so they understand the directions and vocabulary. This is not only time consuming, but as you have mentioned the biggest problem is that in many of these 12 and under leagues there is no medical personnel covering it, and that is a major problem that parents need to be aware of.