International Journal of Technology Assessment in Health Care

Special Section: Alternative Methods for Assessing Technology, Part 1

Selection and Evaluation of Empirical Research in Technology Assessment

Thomas C. Chalmersa1, Peg Hewetta2, Dinah Reitmana3 and Henry S. Sacksa3

a1 Harvard School of Public Health; Boston Veterans Administration Medical Center; Mount Sinai School of Medicine

a2 Harvard School of Public Health

a3 Mount Sinai School of Medicine

Technology assessment involves application of the scientific method to the practice of medicine. Finding all of the assessment reports in a given field is not an easy task. Proper evaluation of those assessments requires the conduct of a prospective experiment in which the sources and results are blinded when the choice is made of papers to exclude and to include, and the process should be carried in duplicate. There are several available data bases for carrying out the search, but because of problems they should be supplemented by reference to the bibliographies of pertinent published articles. Clinical trials included in meta-analyses should be graded by quality and thus facilitate sensitivity analyses. Attention must be paid to the possibility of publication bias. Finally, the advent of meta-analysis makes it desirable to begin randomized controlled trials in areas of uncertainty, even when there is no possibility that individual investigators will encounter enough patients to draw valid conclusions.