International Journal of Technology Assessment in Health Care


Ian D. Graham a1 1 , Lisa A. Calder a2, Paul C. Hébert a3, Anne O. Carter a4 and Jacqueline M. Tetroe a5
a1 Ottawa Hospital – Civic Campus
a2 University of Western Ontario
a3 University of Ottawa
a4 United Arab Emirates University
a5 Ottawa Hospital – Civic Campus


Objective: To identify and compare clinical practice guideline appraisal instruments.

Methods: Appraisal instruments, defined as instruments intended to be used for guideline evaluation, were identified by searching MEDLINE (1966–99) using the Medical Subject Heading (MeSH) practice guidelines, reviewing bibliographies of the retrieved articles, and contacting authors of guideline appraisal instruments. Two reviewers independently examined the questions/statements from all the instruments and thematically grouped them. The 44 groupings were collapsed into 10 guideline attributes. Using the items, two reviewers independently undertook a content analysis of the instruments.

Results: Fifteen instruments were identified, and two were excluded because they were not focused on evaluation. All instruments were developed after 1992 and contained 8 to 142 questions/statements. Of the 44 items used for the content analysis, the number of items covered by each instrument ranged from 6 to 34. Only the instrument by Cluzeau and colleagues included at least one item for each of the 10 attributes, and it addressed 28 of the 44 items. This instrument and that of Shaneyfelt et al. are the only instruments that have so far been validated.

Conclusions: A comprehensive, concise, and valid instrument could help users systematically judge the quality and utility of clinical practice guidelines. The current instruments vary widely in length and comprehensiveness. There is insufficient evidence to support the exclusive use of any one instrument, although the Cluzeau instrument has received the greatest evaluation. More research is required on the reliability and validity of existing guideline appraisal instruments before any one instrument can become widely adopted.

Key Words: Practice guidelines; Knowledge; Attitudes; Practice; Quality of health care.


1 Ian Graham and Paul Hébert are Career Scientists with the Ontario Ministry of Health. IG was a Medical Research Council of Canada Postdoctoral Health Research Fellow when the study was initiated. We would like to thank Laura McAuley and Christine Niles for their assistance in preparing this manuscript.