International Journal of Technology Assessment in Health Care

METHODS

CAN WE RELIABLY BENCHMARK HEALTH TECHNOLOGY ASSESSMENT ORGANIZATIONS?

Michael Drummonda1, Peter Neumanna2, Bengt Jönssona3, Bryan Lucea4, J. Sanford Schwartza5, Uwe Sieberta6 and Sean D. Sullivana7 for the International Group for HTA Advancement

a1 University of York email: mike.drummond@york.ac.uk

a2 Tufts Medical Center

a3 Stockholm School of Economics

a4 United BioSource Corporation

a5 University of Pennsylvania

a6 University for Health Sciences, Medical Informatics and Technology

a7 University of Washington

Abstract

Objectives: In recent years, there has been growth in the use of health technology assessment (HTA) for making decisions about the reimbursement, coverage, or guidance on the use of health technologies. Given this greater emphasis on the use of HTA, it is important to develop standards of good practice and to benchmark the various HTA organizations against these standards.

Methods: This study discusses the conceptual and methodological challenges associated with benchmarking HTA organizations and proposes a series of audit questions based on a previously published set of principles of good practice.

Results and Conclusions: It is concluded that a benchmarking exercise would be feasible and useful, although the question of who should do the benchmarking requires further discussion. Key issues for further research are the alternative methods for weighting the various principles and for generating an overall score, or summary statement of adherence to the principles. Any weighting system, if developed, would need to be explored in different jurisdictions to assess the extent to which the relative importance of the principles is perceived to vary. Finally, the development and precise wording of the audit questions requires further study, with a view to making the questions as unambiguous as possible, and the reproducibility of the assessments as high as possible.

Footnotes

The International Working Group for HTA Advancement was established in July 2007 with unrestricted funding from the Schering Plough Corporation. The mission of the Working Group is to provide scientifically based leadership to facilitate significant continuous improvement in the development and implementation of practical, rigorous methods into formal health technology assessment (HTA) systems and processes, by facilitating development and adoption of high quality, scientifically driven, objective, and trusted HTA to improve patient outcomes, the health of the public and overall healthcare quality and efficiency. Following the end of the unrestricted grant in December 2010, the Group has sought funding for particular activities. The production of this study was not funded by any external party.

A preliminary version of this paper was presented at the HTAi Conference in Dublin, June 6-9, 2010.