Hostname: page-component-8448b6f56d-c47g7 Total loading time: 0 Render date: 2024-04-19T04:34:58.187Z Has data issue: false hasContentIssue false
  • English
  • Français

Effects of perceptual training on second language vowel perception and production

Published online by Cambridge University Press:  25 November 2015

SOLÈNE INCEOGLU
SOLÈNE INCEOGLU*
Affiliation:
Rochester Institute of Technology
*
ADDRESS FOR CORRESPONDENCE Solène Inceoglu, Department of Modern Languages and Cultures, Rochester Institute of Technology, 92 Lomb Memorial Drive, Rochester, NY 14618. E-mail: scigsl@rit.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

This study investigates whether audiovisual training leads to greater improvement in perception and production than auditory training. The participants (n = 60) were American English native speakers enrolled in intermediate French courses. They received audiovisual training, audio-only training, or no training, and were tested at pretest and posttest on their perception and production of French nasal vowels. The results suggest that both training groups improved significantly from the pretest to the posttest, but that the differences between the audiovisual and audio-only groups were not statistically significant. However, the production of the audiovisual training group improved significantly more than the production of the audio-only training group did, suggesting that seeing facial gestures leads to greater improvement in pronunciation.

Type
Articles
Information
Applied Psycholinguistics , Volume 37 , Issue 5 , September 2016 , pp. 1175 - 1199
Copyright
Copyright © Cambridge University Press 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Alm, M., Behne, D. M., Wang, Y., & Eg, R. (2009). Audio-visual identification of place of articulation and voicing in white and babble noise. Journal of the Acoustical Society of America, 126, 377–387.CrossRefGoogle ScholarPubMed
Arnold, P., & Hill, F. (2001). Bisensory augmentation: A speechreading advantage when speech is clearly audible and intact. British Journal of Psychology, 92, 339–355.CrossRefGoogle ScholarPubMed
Baker, W., & Trofimovich, P. (2006). Perceptual paths to accurate production of L2 vowels: The role of individual differences. International Review of Applied Linguistics in Language Teaching, 44, 231–250.CrossRefGoogle Scholar
Benoît, C., Lallouache, T., Mohamadi, T., Tseva, T., & Abry, C. (1991). Nineteen (±two) French visemes for visual speech synthesis. Paper presented at the ESCA Workshop on Speech Synthesis, Autrans, France. Retrieved from http://www.isca-speech.org/archive_open/archive_papers/ssw1/ssw1_253.pdf Google Scholar
Benoît, C., Mohamadi, T., & Kandel, S. (1994). Effects of phonetic context on audio-visual intelligibility of French. Journal of Speech and Hearing Research, 37, 1195–203.Google Scholar
Best, C. T. (1995). A direct realist view of cross-language speech perception. In Strange, W. (Ed.), Speech perception and linguistic experience: Issues in cross-language research (pp. 171–204). Baltimore, MD: York Press.Google Scholar
Binnie, C. A., Montgomery, A. A., & Jackson, P. L. (1974). Auditory and visual contributions to the perception of consonants. Journal of Speech and Hearing Research, 17, 619–630.CrossRefGoogle Scholar
Bohn, O.-S., & Flege, J. E. (1992). The production of new and similar vowels by adult German learners of English. Studies in Second Language Acquisition, 14, 131–156.Google Scholar
Bohn, O.-S., & Flege, J. E. (1997). Perception and production of a new vowel category by adult second language learners. In James, A. & Leather, J. (Eds.), Second-language speech: Structure and process (pp. 53–74). Berlin: Walter de Gruyter.CrossRefGoogle Scholar
Borden, G., Gerber, A., & Milsark, G. (1983). Production and perception of the /r/-/l/ contrast in Korean adults learning English. Language Learning, 33, 499–526.CrossRefGoogle Scholar
Bradlow, A. R., Akahane-Yamada, R., Pisoni, D. B., & Tohkura, Y. (1999). Training Japanese listeners to identify English /r/ and /l/: Long-term retention of learning in perception and production. Perception and Psychophysics, 61, 977–985.Google Scholar
Bradlow, A. R., Pisoni, D. B., Akahane-Yamada, R., & Tohkura, Y. I. (1997). Training Japanese listeners to identify English /r/ and /l/: IV. Some effects of perceptual learning on speech production. Journal of the Acoustical Society of America, 101, 2299–2310.Google Scholar
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C., McGuire, P., et al. (1997). Activation of auditory cortex during silent lipreading. Science, 276, 593–596.Google Scholar
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Google Scholar
Daly, N., Bench, J., & Chappell, H. (1996). Gender differences in speechreadability. Journal of the Academy of Rehabilitative Audiology, 29, 27–40.Google Scholar
de Gelder, B., Bertelson, P., Vroomen, J., & Chen, H. C. (1995). Interlanguage differences in the McGurk effects for Dutch and Cantonese listeners. Paper presented at the Fourth European Conference on Speech Communication and Technology, EUROSPEECH 1995, Madrid, September 18–21. Retrieved from https://www.tamu.edu/faculty/bortfeld/689/689_Spring%2707/other%20readings/DeGelder_etal_Proceedings1995.pdf Google Scholar
Demorest, M. E., Bernstein, L. E., & DeHaven, G. P. (1996). Generalizability of speech-reading performance on nonsense syllables, words, and sentences: Subjects with normal hearing. Journal of Speech, Language, and Hearing Research, 39, 697–713.CrossRefGoogle Scholar
Dörnyei, Z., & Skehan, P. (2003). Individual differences in second language learning. In Doughty, C. J. & Long, M. H. (Eds.), The handbook of second language acquisition (pp. 589–630). Oxford: Blackwell.Google Scholar
Escudero, P. (2005). Linguistic perception and second language acquisition: Explaining the attainment of optimal phonological categorization. Unpublished doctoral dissertation, Utrecht University.Google Scholar
Field, A. (2009). Discovering statistics using SPSS (3rd ed.). London: Sage.Google Scholar
Flege, J. E. (1987). The production of “new” and “similar” phones in a foreign language: Evidence for the effect of equivalence classification. Journal of Phonetics, 15, 47–65.CrossRefGoogle Scholar
Flege, J. E. (1991). Perception and production: The relevance of phonetic input to L2 phonological learning. In Heubner, T. & Ferguson, C. (Eds.), Crosscurrents in second language acquisition and linguistic theory (pp. 249–289). Philadelphia, PA: John Benjamins.Google Scholar
Flege, J. E. (1995). Second language speech learning: Theory, findings, and problems. In Strange, W. (Ed.), Speech perception and linguistic experience: Issues in cross-language research (pp. 233–277). Timonium, MD: York Press.Google Scholar
Flege, J. E., Takagi, N., & Mann, V. (1995). Japanese adults can learn to produce English /ɹ/ and /l/ accurately. Language and Speech, 38, 25–55.CrossRefGoogle ScholarPubMed
Gass, S. M. (1984). Development of speech perception and speech production abilities in adult second language learners. Applied Psycholinguistics, 5, 51–74.Google Scholar
Goto, H. (1971). Auditory perception by normal Japanese adults of the sounds “l” and “r.” Neuropsychologia, 9, 317–323.CrossRefGoogle ScholarPubMed
Grant, K. W., & Seitz, P. F. (1998). Measures of auditory–visual integration in nonsense syllables and sentences. Journal of the Acoustical Society of America, 104, 2438–2450.Google Scholar
Grassegger, H. (1995). McGurk effect in German and Hungarian listeners. Paper presented at the International Congress of Phonetic Sciences, University of Stockholm, Department of Linguistics.Google Scholar
Hardison, D. M. (1996). Bimodal speech perception by native and nonnative speakers of English: Factors influencing the McGurk effect. Language Learning, 46, 3–73.Google Scholar
Hardison, D. M. (2003). Acquisition of second-language speech: Effects of visual cues, context, and talker variability. Applied Psycholinguistics, 24, 495–522.Google Scholar
Hardison, D. M. (2005). Second-language spoken word identification: Effects of perceptual training, visual cues, and phonetic environment. Applied Psycholinguistics, 26, 579–596.CrossRefGoogle Scholar
Hazan, V., Sennema, A., Faulkner, A., Ortega-Llebaria, M., Iba, M., & Chung, H. (2006). The use of visual cues in the perception of non-native consonant contrasts. Journal of the Acoustical Society of America, 119, 1740–1751.CrossRefGoogle ScholarPubMed
Hazan, V., Sennema, A., Iba, M., & Faulkner, A. (2005). Effect of audiovisual perceptual training on the perception and production of consonants by Japanese learners of English. Speech Communication, 47, 360–378.CrossRefGoogle Scholar
Hirata, Y., & Kelly, S. D. (2010). Effects of lips and hands on auditory learning of second-language speech sounds. Journal of Speech, Language, and Hearing Research, 53, 298–310.Google Scholar
Inceoglu, S. (2011). Audio-visual speech perception of French nasal vowels. Paper presented at the Second Language Research Forum, Iowa State University, Ames, IA, October.Google Scholar
Iverson, P., Pinet, M., & Evans, B. G. (2011). Auditory training for experienced and inexperienced second-language learners: Native French speakers learning English vowels. Applied Psycholinguistics, 33, 1–16.Google Scholar
Jesse, A., & Massaro, D. W. (2010). Seeing a singer helps comprehension of the song's lyrics. Psychonomic Bulletin & Review, 17, 323–328.Google Scholar
Jiang, J., Alwan, A., Keating, P. A., Auer, E. T. Jr., & Bernstein, L. E. (2002). On the correlation between facial movements, tongue movements and speech acoustics. Journal on Applied Signal Processing, 11, 1174–1188.Google Scholar
Johnson, F. M., Hicks, L. H., Goldberg, T., & Myslobodsky, M. S. (1988). Sex differences in lipreading. Bulletin of the Psychonomic Society, 26, 106–108.Google Scholar
Kleber, F., Harrington, J., & Reubold, U. (2011). The relationship between the perception and production of coarticulation during a sound change in progress. Language and Speech, 55, 383–405.CrossRefGoogle Scholar
Kluge, D. C., Reis, M. S., Nobre-Oliveira, D., & Bettoni-Techio, M. (2009). The use of visual cues in the perception of English syllable-final nasals by Brazilian EFL learners. In Watkins, M. A., Rauber, A. S., & Baptista, B. O. (Eds.), Recent research in second language phonetics/phonology: Perception and production. (pp. 141–153). Cambridge: Cambridge Scholars Publishing.Google Scholar
Lambacher, S. G., Martens, W. L., Kakehi, K., Marasinghe, C. A., & Molholt, G. (2005). The effects of identification training on the identification and production of American English vowels by native speakers of Japanese. Applied Psycholinguistics, 26, 227–247.Google Scholar
Levy, E. S., & Strange, W. (2008). Perception of French vowels by American English adults with and without French language experience. Journal of Phonetics, 36, 141–157.CrossRefGoogle Scholar
Liberman, A. M., & Mattingly, I. G. (1985). The motor theory of speech perception revised. Cognition, 21, 1–36.Google Scholar
Lisker, L., & Rossi, M. (1992). Auditory and visual cueing of the [±rounded] feature of vowels. Language and Speech, 35, 391–417.Google Scholar
Lively, S. E., Logan, J. S., & Pisoni, D. B. (1993). Training Japanese listeners to identify English /r/ and /l/: II. The role of phonetic environment and talker variability in learning new perceptual categories. Journal of the Acoustical Society of America, 94, 1242–1255.Google Scholar
Lopez-Soto, T., & Kewley-Port, D. (2009). Relation of perception training to production of codas in English as a second language. Proceedings of Meetings on Acoustics, 6, 1–15.Google Scholar
MacLeod, A., & Summerfield, Q. (1990). A procedure for measuring auditory and audiovisual speech-reception thresholds for sentences in noise: Rationale, evaluation, and recommendations for use. British Journal of Audiology, 24, 29–43.Google Scholar
Massaro, D. W., & Light, J. (2003). Read my tongue movements: Bimodal learning to perceive and produce non-native speech /r/ and /l/. Paper presented at Eurospeech (Interspeech), 8th European Conference on Speech Communication and Technology, ISCA, Geneva. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.5.5676&rep=rep1&type=pdf Google Scholar
McGrath, M., & Summerfield, Q. (1985). Intermodal timing relations and audio-visual speech recognition by normal-hearing adults. Journal of the Acoustical Society of America, 77, 678–685.CrossRefGoogle ScholarPubMed
McGrath, M., Summerfield, Q., & Brooke, N. M. (1984). Roles of lips and teeth in lipreading vowels. Proceedings of the Institute of Acoustics, 6, 401–408.Google Scholar
McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.CrossRefGoogle ScholarPubMed
Ménard, L., Dupont, S., Baum, S. R., & Aubin, J. (2009). Production and perception of French vowels by congenitally blind adults and sighted adults. Journal of the Acoustical Society of America, 126, 1406–1414.CrossRefGoogle ScholarPubMed
Mills, A. E. (1983). Acquisition of speech sounds in the visually-handicapped child. In Mills, A. E. (Ed.), Language acquisition in the blind child: Normal and deficient (pp. 45–56). San Diego, CA: College Hill.Google Scholar
Mills, A. E. (1987). The development of phonology in the blind child. In Dodd, B. & Campbell, R. (Eds.), Hearing by eye: The psychology of lip reading (pp. 145–162). Hillsdale, NJ: Erlbaum.Google Scholar
Navarra, J., & Soto-Faraco, S. (2007). Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds. Psychological Research, 71, 4–12.Google Scholar
Öhrström, N., & Traunmüller, H. (2004). Audiovisual perception of Swedish vowels with and without conflicting cues. Paper presented at FONETIK 2004, Stockholm University, Department of Linguistics. Retrieved from http://www2.ling.su.se/fon/fonetik_2004/ohrstrom_traunmuller_fonetik2004.pdf Google Scholar
Ortega-Llebaria, M., Faulkner, A., & Hazan, V. (2001). Auditory–visual L2 speech perception: Effects of visual cues and acoustic-phonetic context for Spanish learners of English. Paper presented at the AVSP 2001 International Conference on Auditory-Visual Speech Processing, ISCA, Scheelsminde, Denmark. Retrieved from http://www.isca-speech.org/archive_open/archive_papers/avsp01/av01_149.pdf Google Scholar
Owens, E., & Blazek, B. (1985). Visemes observed by hearing-impaired and normal-hearing adult viewers. Journal of Speech and Hearing Research, 28, 381–393.CrossRefGoogle ScholarPubMed
Pereira, Y. I. (2013). Perception of English vowels and use of visual cues by learners of English and English native speakers. Journal of the Acoustical Association of America, 133, 3390.CrossRefGoogle Scholar
Plonsky, L., & Oswald, F. L. (2014). How big is “Big”? Interpreting effect sizes in L2 research. Language Learning, 64, 878–912.Google Scholar
Rauber, A., Rato, A., Kluge, D. C., & Santos, G. (2012). TP, v. 3.1 [Computer software]. Retrieved from http://www.worken.com.br/tp_regfree.php/ Google Scholar
Reisberg, D., McLean, J., & Goldfield, A. (1987). Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. In Dodd, B. & Campbell, R. (Eds.), Hearing by eye: The psychology of lip-reading (pp. 97–113). London: Erlbaum.Google Scholar
Robert-Ribes, J., Schwartz, J.-L., Lallouache, T., & Escudier, P. (1998). Complementarity and synergy in bimodal speech: Auditory, visual, and audio-visual identification of French oral vowels in noise. Journal of the Acoustical Society of America, 103, 3677–3689.CrossRefGoogle ScholarPubMed
Rochet, B. L. (1995). Perception and production of second-language speech sounds by adults. In Strange, W. (Ed.), Speech perception and linguistic experience: Issues in cross-language research (pp. 379–410). Timonium, MD: York Press.Google Scholar
Rosenblum, L. D. (2005). Primacy of multimodal speech perception. In Pisoni, D. B. & Remez, R. E. (Eds.), The handbook of speech perception (pp. 51–79). Malden, MA: Blackwell.CrossRefGoogle Scholar
Rosenblum, L. D., Johnson, J. A., & Saldaña, H. M. (1996). Point-light facial displays enhance comprehension of speech in noise. Journal of Speech and Hearing Research, 39, 1159–1170.CrossRefGoogle ScholarPubMed
Sekiyama, K. (1997). Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects. Perception & Psychophysics, 59, 73–80.Google Scholar
Sekiyama, K., & Tohkura, Y. (1991). McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. Journal of the Acoustical Society of America, 90, 1797–1805.Google Scholar
Sheldon, A., & Strange, W. (1982). The acquisition of /r/ and /l/ by Japanese learners of English: Evidence that speech production can precede speech perception. Applied Psycholinguistics, 3, 243–261.CrossRefGoogle Scholar
Smith, L. C. (2001). L2 acquisition of English liquids: Evidence for production independent from perception. In Bonch-Bruevich, X., Crawford, W., Hellermann, J., Higgins, C., & Nguyen, H. (Eds.), Selected proceedings of the 2000 Second Language Research Forum (pp. 3–22). Somerville, MA: Cascadilla Press.Google Scholar
Sommers, M. S., Spehar, B., & Tye-Murray, N. (2005). The effects of signal-to-noise ratio on auditory–visual integration: Integration and encoding are not independent. Journal of the Acoustical Society of America, 117, 2574.CrossRefGoogle Scholar
Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212–215.Google Scholar
Summerfield, Q. (1991). Visual perception of phonetic gestures. In Mattingly, I. G. & Studdert-Kennedy, M. (Eds.), Modularity and the motor theory of speech perception (pp. 117–137). Hillsdale, NJ: Erlbaum.Google Scholar
Summerfield, Q. (1992). Lipreading and audio-visual speech perception. Philosophical Transactions: Biological Sciences, 335, 71–78.Google Scholar
Summerfield, Q., MacLeod, A., McGrath, M., & Brooke, M. (1989). Lips, teeth, and the benefits of lipreading. In Young, A. W. & Ellis, H. D. (Eds.), Handbook of research on face processing (pp. 223–233). Amsterdam: Elsevier.Google Scholar
Traunmüller, H., & Öhrström, N. (2007). Audiovisual perception of openness and lip rounding in front vowels. Journal of Phonetics, 35, 244–258.Google Scholar
Valkenier, B., Duyne, J. Y., Andringa, T. C., & Başkent, D. (2012). Audiovisual perception of congruent and incongruent Dutch front vowels. Journal of Speech, Language, and Hearing Research, 55, 1788–1802.Google Scholar
Walden, B. E., Prosek, R. A., Montgomery, A. A., Scherr, C. K., & Jones, C. J. (1977). Effects of training on the visual recognition of consonants. Journal of Speech and Hearing Research, 20, 130–145.CrossRefGoogle ScholarPubMed
Wang, Y., Behne, D. M., & Jiang, H. (2008). Linguistic experience and audio-visual perception of non-native fricatives. Journal of the Acoustical Society of America, 124, 1716–1726.Google Scholar
Wang, Y., Behne, D. M., & Jiang, H. (2009). Influence of native language phonetic system on audio-visual speech perception. Journal of Phonetics, 37, 344–356.Google Scholar
Wang, Y., Jongman, A., & Sereno, J. A. (2003). Acoustic and perceptual evaluation of Mandarin tone productions before and after perceptual training. Journal of the Acoustical Society of America, 113, 1033–1043.Google Scholar
Werker, J. F., Frost, P. E., & McGurk, H. (1992). La langue et les lèvres: Cross-language influences on bimodal speech perception. Canadian Journal of Psychology, 46, 551–568.Google Scholar
Yamada, R. A., Strange, W., Magnuson, J. S., Pruitt, J. S., & Clarke III, W. D. (1994). The intelligibility of Japanese speakers’ productions of American English /r/, /l/, and /w/, as evaluated by native speakers of American. Paper presented at the International Conference of Spoken Language Processing, Acoustical Society of Japan, Yokohama.CrossRefGoogle Scholar
Yehia, H., Rubin, P., & Vatikiotis-Bateson, E. (1998). Quantitative association of vocal-tract and facial behavior. Speech Communication, 26, 23–43.Google Scholar
Zampini, M. L., & Green, K. P. (2001). The voicing contrast in English and Spanish: The relationship between perception and production. In Nicol, J. (Ed.), One mind, two languages: Bilingual language processing (pp. 23–48). Malden, MA: Blackwell.Google Scholar
Zerling, J.-P. (1989). The three degrees of labialisation of the French steady-state vowels. Paper presented at the European Conference on Speech Communication and Technology, ISCA, Paris. Retrieved from http://dblp.uni-trier.de/db/conf/interspeech/eurospeech1989.html#Zerling89 Google Scholar
Zerling, J.-P. (1990). Aspects articulatoires de la labialité vocalique en français. Contribution à la modélisation à partir de labio-photographies, labiofilms et films radiologiques. Étude statique, dynamique et contrastive. Unpublished doctoral dissertation, University of Strasbourg II.Google Scholar