Group Versus Individual Administration Affects Baseline Neurocognitive Test Performance
Moser RS, Schatz P, Neidzwski K, Ott SD. Am J Sports Med. 2011; 39(11): 2325-2330.
https://www.ncbi.nlm.nih.gov/pubmed/21828367
Computer-based tests are proving to be cost efficient and effective ways of documenting baseline neurocognitive function to later be used as an evaluation instrument for suspected concussions. The computer-based test can be administered to many athletes at one time with few or no medical personnel present; however, there is limited research on the effects of administering the tests in a group versus individual setting. The purpose of this study was to determine if those tested individually have a significantly better baseline compared to those assessed in a group. This study utilized a cohort of high school athletes with no exclusionary criteria (e.g., ADD, history of concussion, learning disability were included) that completed preseason baseline tests using the online version of ImPACT. Athletes were assigned to completing the test in either a group setting (n = 150) of 20 athletes per group at their school or individually at a private neuropsychological clinic (n = 166). They found athletes that were tested in a group setting scored significantly lower on verbal memory (83.4 vs. 86.5), visual memory (71.6 vs. 76.7), motor processing speed (35.6 vs. 38.4), and reaction time (0.61 vs. 0.58). The groups also had a higher number of students that obtained invalid results in a group setting (14 athletes) compared to the individual setting (1 athlete). When researchers excluded athletes with ADD, history of concussion, and learning disability athletes still had similar significant results. Due to differences in the brain during the maturation process they controlled for age in motor processing speeds and reaction time variables, and also had similar results. No significance was seen between groups in the total symptom score.
Having a baseline to compare preconcussion neurocognitive function to post concussion neurocognitive function to assist in return to play decision for medical personnel is important. If this baseline is not a true representation then it can deface the validity of the computed-based program. Baseline neurocognitive tests are routinely administered in group settings within high schools and college/university settings to establish a more cost and time efficient process. However, distractions in group like settings can affect the neurocognitive results. This may be why there was a significant difference between group and individual baseline scores within visual memory, verbal memory, motor processing speed, and reaction time, and not in reporting their total symptom score. This was significant even after eliminating the athletes with previous history of concussion, ADD, and those with a learning disability, which can be more easily distracted. This study demonstrates the need for more research on how this test is administered. Other variables like administrators instructions, lighting, seating arrangement, and removing extraneous sounds could all lead to a more controlled environment that a significant difference may not be identified between group versus individual testing. If you use a computer based neurocognitive exam how do you control for a good baseline test? How many athletes do you test per time and do you remain at the test to oversee and answer questions?
Written by: Jane McDevitt MS, ATC, CSCS
Reviewed by: Stephen Thomas
Related Posts:
Reliability of the Online Version of ImPACT in High School Athletes
Reliability of Self-Reported Concussion History
Moser RS, Schatz P, Neidzwski K, & Ott SD (2011). Group versus individual administration affects baseline neurocognitive test performance. The American Journal of Sports Medicine, 39 (11), 2325-30 PMID: 21828367
This is a well-needed research study. Every school I have worked administers baseline testing in a group setting and have often wondered about the impact of distractions on the test results. I know even my own test-taking instructions vary between testing sessions, so it is easy to imagine that between administers instructions could vary greatly. The results of this study are alarming because if we are doing the majority of baseline testing in a group setting and getting lower scores, are we getting accurate information to make RTP decisions after concussion? (Neurocognitive testing is only a piece of RTP decisions, but an important one)
More studies need to be done to solidify these results, but if these trends continue to be reported I think concussion management policies may need to be revised in order to ensure we are making appropriate RTP decisions based on neurocognitive testing baseline. Great article.
This is a very interesting article. Each and every neurocognitive test I have ever administered at baseline was in a group setting. Obviously for the athletes safety it appears to be in the best interest to do individualized testing but how feasible would this be for the majority of the athletic settings? I would be very interested in future research looking into methods of improving baseline scores in a group setting when compared to those done individually. Is it possible that sound dampening headphones and visual barriers keeping auditory and visual distractions to a minimum would allow for group testing while recording improved baseline scores? We could essentially mimic the standardized procedures for GRE and BOC exams during neurocognitive testing as it has been seen to improve individual scores in other forms of computerized testing while allowing for "group" assessment. Great article.
I currently work with D-I collegiate athletics. It is mandatory that our athletes take an annual electronic neurocognitive baseline reassessment before they begin athletic participation. When it comes to controlling the testing environment, we use 6 computers that are isolated in 6 different offices that have a direct connection to our Athletic Training facilities. While taking the test, the athlete is alone in a quiet environment with limited distractions. I believe that this cuts down on the number of invalid test modules and allows our department to collect more accurate baseline measurements. In the event that the athlete has a question or the computer malfunctions, a staff member is immediately accessible to assist them.
Before this study was completed I would have made the hypothesis that group test settings would provide lower scores. I think that this is true for most types of tests. Would a similar assessment of SAT or GRE scores come up with the same result? In order to evaluate the peak of cognitive abilities, as many outside distractions should be controlled for as possible. This proves difficult in most settings, especially when you may have 500 athletes that all need tested in a short period of time. Even if you were able to isolate test-takers, would furniture or wall color affect scores too? What is the MOST ideal environment?
I agree with all your posts. If we do not have valid neurocognitive data then return to play decisions will become more difficult. Even though the neurocognitive assessment is only one part in the return to play chain it is still an important one we rely on. The authors of the the study gaver several suggestions to control outside variables "(1) determine that
athletes understand the purpose and nature of baseline
testing; (2) ensure that athletes understand the test
instructions; (3) encourage good effort on the part of the
athlete and monitor test data for invalid results; and (4)
reduce and control for distractions in the test environment.
Specific ways to control the testing environment and experience may include (1) seating that is comfortably spaced,
where athletes should not be directly next to or across
from each other; (2) removal of extraneous sounds and
interruptions, and use of sound proofing and/or white noise
machines; (3) lighting that does not produce a glare on the
computer screen; (3) clean and functional computer mice
that are able to move with ease; (4) a test administrator
who is present at all times; (5) clear group instructions
and rules provided before testing regarding communication during the testing (raising hands if there is a computer
glitch), with an opportunity for ‘‘Q and A’’ and bathroom
breaks before starting; (6) identifying and removing any
athletes who are overly talkative, not taking the test seriously, and/or are engaging in horseplay, before or during
a group testing, and then testing them later individually;
(7) testing when fatigue is not an issue, such as early in
the day and not immediately following exercise, practice,
or a game."
I have worked in settings that use both a group environment and an isolated environment for computerized neurocognitive testing. I definitely feel that taking the test solo is ideal, however, like many of you have said, it is just not feasible in some settings. Proper education, supervision, and testing instruction is key to getting good results. These tests can be a valuable tool, but we need to remember that the computer is not the only responsible party when it comes to getting a reliable/valid result. As clinicians, we play an important role in ensuring that a group testing environment has a few distractions as possible.
Good post!
In my experience with neurocognitive testing nearly all baseline testing is done in a group setting while follow up post concussion is done individually. In the study design I would be curious to see what kinds of instructions were given to the participants. Were any of the suggestions presented by the authors utilized in the study or were very basic instructions given prior to the testing in both group and individual test takers? The next step in this research would be a follow up with similar groups assessing the authors suggestions to see if the baseline scores of group test takers and individuals would become much closer. However if we are unable to reproduce the same baseline scores in a group setting regardless of instruction are there other ways to address the lower baseline when post concussive measures are taken?
Hailey, This was a retrospective data collection so there was no common script for the 15 schools where the athletes were group tested. There were no rigorous controls (e.g., lighting, computer arrangement) because they wanted to depict what goes on in most high school settings. The study wanted to illuminate that if we are using neurocognitive computer assessment they should administered in the most effective manner. Those medical professionals that utilize this program need to be aware of these variables and try to control for them for valid baselines. Future research should account for these specific variables (e.g., instructions, spacing between computers, noise). If there are proper controls for these variables the baseline scores may get closer to individual testing.