Hostname: page-component-8448b6f56d-tj2md Total loading time: 0 Render date: 2024-04-25T02:57:51.768Z Has data issue: false hasContentIssue false
  • English
  • Français

Assessing the impact of frame semantics on textual entailment

Published online by Cambridge University Press:  16 September 2009

ALJOSCHA BURCHARDT ,
MARCO PENNACCHIOTTI ,
STEFAN THATER  and
MANFRED PINKAL
ALJOSCHA BURCHARDT
Affiliation:
Department of Computational Linguistics, Saarland University, Saarbrücken, Germany e-mail: albu@coli.uni-sb.de, pennacchiotti@coli.uni-sb.de, stth@coli.uni-sb.de, pinkal@coli.uni-sb.de
MARCO PENNACCHIOTTI
Affiliation:
Department of Computational Linguistics, Saarland University, Saarbrücken, Germany e-mail: albu@coli.uni-sb.de, pennacchiotti@coli.uni-sb.de, stth@coli.uni-sb.de, pinkal@coli.uni-sb.de
STEFAN THATER
Affiliation:
Department of Computational Linguistics, Saarland University, Saarbrücken, Germany e-mail: albu@coli.uni-sb.de, pennacchiotti@coli.uni-sb.de, stth@coli.uni-sb.de, pinkal@coli.uni-sb.de
MANFRED PINKAL
Affiliation:
Department of Computational Linguistics, Saarland University, Saarbrücken, Germany e-mail: albu@coli.uni-sb.de, pennacchiotti@coli.uni-sb.de, stth@coli.uni-sb.de, pinkal@coli.uni-sb.de
Get access Rights & Permissions [Opens in a new window]

Abstract

In this article, we underpin the intuition that frame semantic information is a useful resource for modelling textual entailment. To this end, we provide a manual frame semantic annotation for the test set used in the second recognizing textual entailment (RTE) challenge – the FrameNet-annotated textual entailment (FATE) corpus – and discuss experiments we conducted on this basis. In particular, our experiments show that the frame semantic lexicon provided by the Berkeley FrameNet project provides surprisingly good coverage for the task at hand. We identify issues of automatic semantic analysis components, as well as insufficient modelling of the information provided by frame semantic analysis as reasons for ambivalent results of current systems based on frame semantics.

Type
Papers
Information
Natural Language Engineering , Volume 15 , Special Issue 4: Textual Entailment , October 2009 , pp. 527 - 550
Copyright
Copyright © Cambridge University Press 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adams, R., Nicolae, G., Nicolae, C., and Harabagiu, S. 2007. Textual entailment through extended lexical overlap and lexico-semantic matching. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pp. 119–24. Association for Computational Linguistics. Prague, Czech Republic.CrossRefGoogle Scholar
Baker, C. F., Fillmore, C. J., and Lowe, J. B. 1998. The Berkeley FrameNet project. In Proceedings of COLING-ACL, Montreal, Canada.Google Scholar
Bar-Haim, R., Dagan, I., Dolan, B., Ferro, L., Giampiccolo, D., Magnini, B., and Szpektor, I. (eds.) 2006. Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, Venice, Italy.Google Scholar
Bar-Haim, R., Szpektor, I., and Glickman, O. 2005. Definition and analysis of intermediate entailment levels. In Proceedings of ACL 2005 Workshop on Empirical Modeling of Semantic Equivalence and Entailment, pp. 55–60, Ann Arbor, Michigan, Association for Computational Linguistics.Google Scholar
Boas, H. C. 2005. Semantic frames as interlingual representations for multilingual lexical databases. International Journal of Lexicography 18 (4): 445–78.CrossRefGoogle Scholar
Bos, J., and Markert, K. 2006. When logical inference helps determining textual entailment (and when it doesn't). In Proceedings of PASCAL RTE2 Workshop, Venice, Italy.Google Scholar
Brockett, C. 2007. Aligning the RTE 2006 corpus. Technical Report MSR-TR-2007-77. Microsoft Research.Google Scholar
Burchardt, A., Erk, K., and Frank, A. 2005a. A WordNet detour to FrameNet. In Fisseni, B., Schmitz, H.-C., Schröder, B., and Wagner, P. (eds.), Sprachtechnologie, mobile Kommunikation und linguistische Resourcen, vol. 8. Computer Studies in Language and Speech. Frankfurt: Peter Lang.Google Scholar
Burchardt, A., Erk, K., Frank, A., Kowalski, A., Pado, S., and Pinkal, M. 2006. The SALSA corpus: a German corpus resource for lexical semantics. In Proceedings of LREC 2006, Genoa, Italy.Google Scholar
Burchardt, A., and Frank, A. 2006. Approximating textual entailment with LFG and FrameNet frames. In Proceedings of PASCAL RTE2 Workshop, Venice, Italy.Google Scholar
Burchardt, A., Frank, A., and Pinkal, M. 2005b. Building text meaning representations from contextually related frames – a case study. In Proceedings of the Sixth International Workshop on Computational Semantics, IWCS-6, p. 12, Tilburg, The Netherlands.Google Scholar
Burchardt, A., Reiter, N., Thater, S., and Frank, A. 2007. A semantic approach to textual entailment: system evaluation and task analysis. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, Prague, Czech Republic.Google Scholar
Carreras, X., and Marquez, L. (eds.) 2005. Proceedings of the CoNLL shared task: Semantic role labelling, Association for Computational Linguistics, Boston, MA.Google Scholar
Clark, P., Harrison, P., Thompson, J., Murray, W., Hobbs, J., and Fellbaum, C. 2007. On the role of lexical and world knowledge in RTE3. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pp. 54–9. Association for Computational Linguistics. Prague, Czech Republic.Google Scholar
Collins, M. 1999. Head-Driven Statistical Models for Natural Language Parsing. PhD thesis, Philadelphia, PA: University of Pennsylvania.Google Scholar
Dagan, I., Glickman, O., and Magnini, B. 2006. The PASCAL recognising textual entailment challenge. In Quiñonero-Candela, J., Dagan, I., Magnini, B., and D'Alché-Buc, F. (eds.), Evaluating Predictive Uncertainty, Visual Object Categorization and Textual Entailment, vol. 3944, pp. 1–27. Lecture Notes in Computer Science. Heidelberg, Germany: Springer.Google Scholar
De Cao, D., Croce, D., Pennacchiotti, M., and Basili, R. 2008. Combining word sense and usage for modeling frame semantics. In Proceedings of STEP 2008, Venice, Italy.Google Scholar
Erk, K., and Pado, S. 2006. Shalmaneser – a flexible toolbox for semantic role assignment. In Proceedings of LREC 2006, Genoa, Italy.Google Scholar
Fillmore, C. J. 1985. Frames and the semantics of understanding. Quaderni di Semantica IV (2): 222–54.Google Scholar
Fliedner, G. 2007. Linguistically Informed Question Answering. PhD thesis, Saarbrücken, Germany: Saarland University.Google Scholar
Frank, A., Krieger, H.-U., Xu, F., Uszkoreit, H., Crysmann, B., Jörg, B., and Schäfer, U. 2006. Question answering from structured knowledge sources. Journal of Applied Logic, Special Issue on Questions and Answers: Theoretical and Applied Perspectives 5 (1): 20–48.Google Scholar
Garoufi, K. 2007. Towards a Better Understanding of Applied Textual Entailment: Annotation and Evaluation of the RTE-2 Dataset. M.Sc. thesis, Saarbrücken, Germany: Saarland University.Google Scholar
Gildea, D., and Jurafsky, D. 2002. Automatic labeling of semantic roles. Computational Linguistics 28 (3): 245–88.Google Scholar
Hickl, A., and Bensley, J. 2007. A discourse commitment-based framework for recognizing textual entailment. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pp. 171–176. Association for Computational Linguistics. Prague, Czech Republic.Google Scholar
Kouylekov, M., and Magnini, B. 2006. Tree edit distance for recognizing textual entailment: estimating the cost of insertion. In Magnini, B., and Dagan, I. (eds.), Proceedings of the Second PASCAL Recognizing Textual Entailment Challenge. Venice, Italy: Springer.Google Scholar
Lin, D., and Pantel, P. 2001. DIRT-discovery of inference rules from text. In Proceedings of the ACM Conference on Knowledge Discovery and Data Mining (KDD-01), San Francisco, CA.Google Scholar
Litkowski, K. 2006. Componential analysis for recognizing textual entailment. In Proceedings of PASCAL RTE2 Workshop, Venice, Italy.Google Scholar
MacCartney, B., and Manning, C. 2008. Modeling semantic containment and exclusion in natural language inference. In Proceedings of COLING 2008, Manchester, UK.Google Scholar
Manning, C. 2006. Local textual inference: it's hard to circumscribe, but you know it when you see it - and NLP needs it. Unpublished manuscript, Stanford University.Google Scholar
Mihalcea, R., and Edmonds, P. (eds.) 2004. Proceedings of Senseval-3: The Third International Workshop on the Evaluati on of Systems for the Semantic Analysis of Text, Barcelona, Spain.Google Scholar
Monz, C., and de Rijke, M. 2001. Light-weight subsumption checking for computational semantics. In Proceedings of the 3rd Workshop on Inference in Computational Semantics (ICoS-3), Siena, Italy.Google Scholar
Nairn, R., Condoravdi, C., and Karttunen, L. 2006. Computing relative polarity for textual inference. In Proceedings of ICoS-5, Buxton, UK.Google Scholar
Ohara, K., Fujii, S., Ohori, T., Suzuki, R., Saito, H., and Ishizaki, S. 2004. The Japanese FrameNet project: an introduction. In Proceedings of the Workshop on Building Lexical Resources from Semantically Annotated Corpora at LREC 2004, Lisbon, Portugal.Google Scholar
Padó, S. 2007. Cross-Lingual Annotation Projection Models for Role-Semantic Information. PhD thesis, Saarbrücken, Germany: Saarland University.Google Scholar
Padó, S., Pennacchiotti, M., and Sporleder, C. 2008. Semantic role assignment for event nominalisations by leveraging verbal data. In Proceedings of COLING 2008, Manchester, UK.Google Scholar
Palmer, M., Gildea, D., and Kingsbury, P. 2005. The proposition bank: an annotated corpus of semantic roles. Computational Linguistics 31 (1): 71–106.Google Scholar
Pennacchiotti, M., De Cao, D., Marocco, P., and Basili, R. 2008. Towards a vector space model for FrameNet-like resources. In Proceedings of LREC 2006, Marrakech, Morocco.Google Scholar
Raina, R., Haghighi, A., Cox, C., Finkel, J., Michels, J., Toutanova, K., Bill, M., Marie-Catherine, D. M., Christopher, D. M., and Andrew, Y. N. 2005. Robust textual inference using diverse knowledge sources. In Pascal, Proceedings of the First Challenge Workshop, Recognizing Textual Entailment, pp. 65–68. Southampton, UK.Google Scholar
Sekine, S., Inui, K., Dagan, I., Dolan, B., Giampiccolo, D., and Magnini, B. (eds.) (2007). Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing. Association for Computational Linguistics, Prague, Czech Republic.Google Scholar
Snow, R., Vanderwende, L., and Menezes, A. 2006. Effectively using syntax for recognizing false entailment. In Proceedings of HLT/NAACL, New York.Google Scholar
Subirats, C., and Petruck, M. 2003. Surprise! Spanish FrameNet! In Proceedings of the Workshop on Frame Semantics at the XVII International Congress of Linguists, Prague, Czech Republic.Google Scholar
Szpektor, I., Tanev, H., Dagan, I., and Coppola, B. 2004. Scaling web-based acquisition of entailment relations. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, Barcellona, Spain.Google Scholar
Vanderwende, L., and Dolan, W. B. 2005. What syntax can contribute in the entailment task. In Machine Learning Challenges, pp. 205–16. Lecture Notes in Computer Science. Heidelberg, Germany: Springer.Google Scholar
Zanzotto, F. M., and Moschitti, A. 2006. Automatic learning of textual entailments with cross-pair similarities. In Proceedings of the 21st Coling and 44th ACL, pp. 401–8, Sydney, Australia.Google Scholar