Sports Medicine Research: In the Lab & In the Field: The Data We Never See (Sports Med Res)
Friday, June 22, 2012

The Data We Never See

Publication of Sports Medicine-Related Randomized Controlled Trials Registered in

Chahal J, Tomescu SS, Ravi B, Bach BR Jr, Ogilvie-Harris D, Mohamed NN, Gandhi R. Am J Sports Med. 2012 Jun 7. [Epub ahead of print]

Each month SMR filters through over 2,000 abstracts to identify approximately 20 to 30 articles to share as post or as links on social media. The hope is that these articles will spark discussions and have an influence on clinical care. Unfortunately, there is one thing we have little control over: findings that never get published and details that may be omitted from articles. To improve transparency, the International Committee of Medical Journal Editors decided that all clinical trials must be listed in a public registry (e.g., as a prerequisite for publication. Chahal and colleagues sought to review to determine the publication rates of randomized controlled trials in sports medicine and to compare the registration summaries of randomized trials with final published manuscripts. The authors included clinical trials that assessed operative and nonoperative treatments for joint disorders and common soft tissue injuries. Furthermore, studies were only included if they were reported as completed at least 32 months prior to the search (to provide time for the study to be published). Thirty-four clinical trials (published 2005 to 2010) from North America, Europe, Asia, and Australia were included in these analyses. Overall, 20 out of the 34 clinical trials (59%) were published. Among the 20 published trials, 16 (80%) had at least one discrepancy (e.g., different inclusion/exclusion criteria, different outcome measures, sample size) between the summary and the published article. Seven of these articles had a discrepancy in reporting their primary outcome measure (e.g., reporting the primary outcome as a secondary measure, not stating it was the primary outcome, introducing a new primary outcome). Finally, only 9 out of the 20 published articles included their ID number.

This study highlights the potential discrepancy between clinical trials and the final publication of the clinical trials. One thing to keep in mind is that this may not always a bad thing. For example, it’s a positive thing if it prevents faulty findings from being presented but it becomes concerning when the bias reduces the chances of a study being published because it was not significant. Imagine if I flip a coin a hundred times and only reported heads and never told you about tails. By the end you would think I’m pretty amazing for having flipped the coin 50 times and every time I got heads. In this analysis, the authors report that 59% of clinical trials in sports medicine progressed to publication which is actually better than 43% chance of registered orthopaedic trauma trials being published (Gandhi R et al. 2011) or the 25% publication rate of abstracts at the Orthopaedic and Sports Physical Therapy sections at CSM (2000 to 2004; note these were not just clinical trials; Smith HD et al. 2011). We need to keep this bias in mind and ask questions about what might not be in the literature and why (e.g., null results or bad design). But even if every clinical trial was published the authors note that 80% of the publications had discrepancies between the registration summaries and published paper. Sometimes this may include different sample sizes, inclusion/exclusion criteria, or even what was the primary outcome of the study. This is concerning because we don’t know why there is a discrepancy. To properly assess the quality and outcomes of a study it’s important for us to know what was done, who the population was, and why things might have been changed. Empirically, at SMR we’ve noticed discrepancies even within manuscripts where results differ between the abstract and manuscript or decimal points are in the wrong place. This is not intended to question the validity of research but rather raise awareness that as we read articles and assess their clinical relevance we need to approach them with a critical mind. Research and clinical care thrive when clinicians and researchers have discussions and ask questions. What kind of questions do you have as you read research articles?

Written by: Jeffrey Driban
Reviewed by: Stephen Thomas

Chahal J, Tomescu SS, Ravi B, Bach BR Jr, Ogilvie-Harris D, Mohamed NN, & Gandhi R (2012). Publication of Sports Medicine-Related Randomized Controlled Trials Registered in The American Journal of Sports Medicine PMID: 22679295


Post a Comment

When you submit a comment please click 'Subscribe by Email" (just below the comments) or "Subscribe to: Post Comments (Atom)" (at the bottom of this page) if you would like to receive a notification when another comment has been submitted to this post.

Please note that if you are using Safari and have problems submitting comments you may need to go to your preferences (privacy tab) and stop blocking third party cookies. Sorry for any inconvenience this may pose.