Journal of Wine Economics

Shorter Papers

How Expert are “Expert” Wine Judges?*

Robert T. Hodgsona1

a1 Professor emeritus, Humboldt State University, Arcata, California. Owner/winemaker, Fieldbrook Winery. Email: bob@fieldbrookwinery.com.

Abstract

Recent papers by Hodgson (2008) and Gawel and Godden (2008) have questioned the consistency of expert wine judges in a wine competition setting. In the latter paper, a methodology introduced in psychometric research to measure judge reliability corrected for chance was used to quantify judge consistency (Cohen, 1968). This paper extends that notion, suggesting a value of 0.7 for Cohen's weighted kappa might be used to define an expert wine judge. With that criterion, less than 30% of judges who participated in either of the two studies would be considered “expert.” (JEL Classification: C1, L15)

Footnotes

* I would like to thank Karl Storchmann (the editor) and an anonymous referee for their critical suggestions on this manuscript.

Metrics