Natural Language Engineering

Papers

Follow-up question handling in the IMIX and Ritel systems: A comparative study

B. W. VAN SCHOOTENa1, R. OP DEN AKKERa1, S. ROSSETa2, O. GALIBERTa2, A. MAXa2 and G. ILLOUZa2

a1 Human Media Interaction, University of Twente, Netherlands e-mail: schooten@ewi.utwente.nl, infrieks@ewi.utwente.nl

a2 Spoken Language Processing Group (TLP), CNRS-LIMSI, France e-mail: sophie.rosset@limsi.fr, olivier.galibert@limsi.fr, aurelien.max@limsi.fr, gabriel.illouz@limsi.fr

Abstract

One of the basic topics of question answering (QA) dialogue systems is how follow-up questions should be interpreted by a QA system. In this paper, we shall discuss our experience with the IMIX and Ritel systems, for both of which a follow-up question handling scheme has been developed, and corpora have been collected. These two systems are each other's opposites in many respects: IMIX is multimodal, non-factoid, black-box QA, while Ritel is speech, factoid, keyword-based QA. Nevertheless, we will show that they are quite comparable, and that it is fruitful to examine the similarities and differences. We shall look at how the systems are composed, and how real, non-expert, users interact with the systems. We shall also provide comparisons with systems from the literature where possible, and indicate where open issues lie and in what areas existing systems may be improved. We conclude that most systems have a common architecture with a set of common subtasks, in particular detecting follow-up questions and finding referents for them. We characterise these tasks using the typical techniques used for performing them, and data from our corpora. We also identify a special type of follow-up question, the discourse question, which is asked when the user is trying to understand an answer, and propose some basic methods for handling it.

(Received March 01 2007)

(Revised September 20 2007)

(Online publication October 24 2008)