International Journal of Technology Assessment in Health Care

General Essays

Inadequate reporting of trials compromises the applicability of systematic reviews

Gerald Gartlehnera1, Patricia Thiedaa2, Richard A. Hansena2, Laura C. Morgana2, Janelle A. Shumatea3 and Daniel B. Nissmana4

a1 Danube University, Krems

a2 University of North Carolina at Chapel Hill

a3 University of North Carolina Hospitals

a4 Medical University of South Carolina

Abstract

Background: Uncertainty about the applicability of controlled trial findings is an increasing concern for clinicians and policy decision makers. This study aimed to determine whether information reported in studies included in systematic reviews was adequate enough to assess their applicability.

Methods: We used the databases of four recently conducted systematic reviews on the comparative efficacy and safety of second-generation antidepressants, inhaled corticosteroids, Alzheimer's drugs, and targeted immune modulators. We developed and pilot-tested a questionnaire to assess the adequacy of reporting with respect to seven previously validated criteria of study design that distinguish explanatory from pragmatic studies. For each of the 137 included studies, two reviewers independently assessed the adequacy of reporting.

Results: Overall, only 12 percent of the included studies provided sufficient information to reliably distinguish explanatory from pragmatic studies. The areas with the greatest lack of reporting were the setting of the study, methods of adverse event assessment, and sample size considerations to determine a minimally important difference from a patient perspective.

Conclusions: Substantial shortcomings in reporting exist in aspects of study design important to determine whether a study is applicable to specific populations of interest.

Footnotes

The authors declare that they have no competing interests.