The Knowledge Engineering Review


Recent research advances in Reinforcement Learning in Spoken Dialogue Systems

Matthew Framptona1 and Oliver Lemona2

a1 Center for the Study of Language and Information, Stanford University, Stanford, CA 94305-4101, USA; e-mail:

a2 School of Mathematical and Computer Sciences, Heriot Watt University, Edinburgh EH14 4AS, UK; e-mail:


This paper will summarize and analyze the work of the different research groups who have recently made significant contributions in using Reinforcement Learning techniques to learn dialogue strategies for Spoken Dialogue Systems (SDSs). This use of stochastic planning and learning has become an important research area in the past 10 years, since it promises automatic data-driven optimization of the behavior of SDSs that were previously hand-coded by expert developers. We survey the most important developments in the field, compare and contrast the different approaches, and describe current open problems.