Hostname: page-component-8448b6f56d-qsmjn Total loading time: 0 Render date: 2024-04-24T00:19:10.648Z Has data issue: false hasContentIssue false
  • English
  • Français

Multimodal research: Addressing the complexity of multimodal environments and the challenges for CALL

Published online by Cambridge University Press:  07 July 2016

Sabine Tan ,
Kay L. O’Halloran  and
Peter Wignell
Sabine Tan
Affiliation:
Faculty of Humanities, Curtin University, Kent Street, Bentley, Western Australia (email: sabine.tan@curtin.edu.au)
Kay L. O’Halloran
Affiliation:
Professor, School of Education, Curtin University, Kent Street, Bentley, Western Australia (email: kay.ohalloran@curtin.edu.au)
Peter Wignell
Affiliation:
Faculty of Humanities, Curtin University, Kent Street, Bentley, Western Australia (email: peter.wignell@curtin.edu.au)
Get access Rights & Permissions [Opens in a new window]

Abstract

Multimodality, the study of the interaction of language with other semiotic resources such as images and sound resources, has significant implications for computer assisted language learning (CALL) with regards to understanding the impact of digital environments on language teaching and learning. In this paper, we explore recent manifestations of CALL in 3-D virtual worlds, illustrated by the example of Second Life. The multimodal analyses of a conventional face-to-face lesson and three language learning activities in Second Life highlight some of the affordances and challenges presented by 3-D virtual environments. The results suggest that while multimodal resources integrate naturally to facilitate language teaching and learning in an orderly, structured and goal-orientated manner in classroom lessons, the often uncoordinated use (or absence) of avatars’ gaze, facial expression, body posture, gesture, as well as the unclear proxemics and use of space pose problems for effective communication in a 3-D virtual world. In addition, a “technology-oriented” register, alongside traditional instructional and regulative genres and registers, is introduced to help students cope with the demands of learning a language in a 3-D virtual environment. The study raises the issue of the relative effectiveness of 3-D virtual worlds for language teaching and learning. In doing so, a digital approach to multimodal research is proposed in order to address the complexity of multimodal learning environments and the various challenges for CALL.

Keywords

multimodalitymultimodal analysissocial semiotic3-D virtual environmentsSecond Life
Type
Regular papers
Information
ReCALL , Volume 28 , Special Issue 3: Multimodal Environments in CALL , September 2016 , pp. 253 - 273
Check for updates
Copyright
Copyright © European Association for Computer Assisted Language Learning 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ackerley, K. and Coccetta, F. (2007) Enriching language learning through a multimedia corpus. ReCALL, 19(3): 351–370.Google Scholar
Berglund, T. Ö. (2009) Multimodal student interaction online: An ecological perspective. ReCALL, 21(2): 186–205.Google Scholar
Canto, S., Jauregi, K. and van den Bergh, H. (2013) Integrating cross-cultural interaction through video-communication and virtual worlds in foreign language teaching programs: Is there an added value? ReCALL, 25(1): 105–121.Google Scholar
Christie, F. (2005) Classroom discourse analysis: A functional perspective. London and New York: Continuum.Google Scholar
Christie, F. and Martin, J. R. (2005) Genre and institutions: Social processes in the workplace and school. London and New York: Continuum.Google Scholar
Ciekanski, M. and Chanier, T. (2008) Developing online multimodal verbal communication to enhance the writing process in an audio-graphic conferencing environment. ReCALL, 20(2): 162–182.CrossRefGoogle Scholar
Codreanu, T. and Combe Celik, C. (2013) Effects of webcams on multimodal interactive learning. ReCALL, 25(1): 30–47.Google Scholar
Deutschmann, M., Panichi, L. and Molka-Danielsen, J. (2009) Designing oral participation in Second Life: A comparative study of two language proficiency courses. ReCALL, 21(2): 206–226.Google Scholar
Develotte, C., Guichon, N. and Vincent, C. (2010) The use of the webcam for teaching a foreign language in a desktop videoconferencing environment. ReCALL, 22(3): 293–312.CrossRefGoogle Scholar
Eggins, S. (2005) An introduction to systemic functional linguistics, (2nd edn.). London: Pinter Publishers.Google Scholar
Godwin-Jones, R. (2014) Games in language learning: Opportunities and challenges. Language Learning & Technology, 18(2): 9–19.Google Scholar
Halliday, M. A. K. (1978) Language as social semiotic. London: Edward Arnold.Google Scholar
Halliday, M. A. K. (1994) An introduction to functional grammar (2nd edn.). London: Edward Arnold.Google Scholar
Halliday, M. A. K. and Hasan, R. (1985) Language, context and text: Aspects of language in a social-semiotic perspective. Waurn Ponds, Victoria: Deakin University.Google Scholar
Halliday, M. A. K. and Matthiessen, C. M. I. M. (2014) Halliday’s introduction to functional grammar. London and New York: Routledge.Google Scholar
Hampel, R. (2014) Making meaning online: Computer-mediated communication for language learning. In: Peti Stantić, A. and Stanojević, M. M. (eds.), Language as information. Proceedings from the CALS Conference 2012. Frankfurt am Main: Peter Lang, 89–106.Google Scholar
Hampel, R. and Hauck, M. (2006) Computer-mediated language learning: making meaning in multimodal virtual learning spaces. The JALT CALL Journal, 2(2): 3–18.Google Scholar
Hampel, R. and Stickler, U. (2012) The use of videoconferencing to support multimodal interaction in an online language classroom. ReCALL, 24(2): 116–137.Google Scholar
Hyland, K. (2009) Academic discourse: English in a global context. London: Continuum International Publishing.Google Scholar
Jauregi, K., Canto, S., de Graaff, R., Koenraad, T. and Moonen, M. (2011) Verbal interaction in Second Life: Towards a pedagogic framework for task design. Computer Assisted Language Learning, 24(1): 77–101.Google Scholar
Jewitt, C. (2003) Re-thinking assessment: Multimodality, literacy and computer-mediated learning. Assessment in Education: Principles, Policy & Practice, 10(1): 83–102.Google Scholar
Jewitt, C. (2006) Technology, literacy and learning: A multimodal approach. London: Routledge.Google Scholar
Jewitt, C. (2008) Multimodality and literacy in school classrooms. Review of Research in Education, 32(1): 241–267.Google Scholar
Jewitt, C. (ed.) (2014) The Routledge handbook of multimodal analysis. London: Routledge.Google Scholar
Jewitt, C. and Kress, G. (eds.) (2003) Multimodal literacy. New York: Peter Lang.Google Scholar
Kress, G. (2010) Multimodality: A social semiotic approach to contemporary communication. New York: Routledge.Google Scholar
Kress, G. and van Leeuwen, T. (2001) Multimodal discourse: The modes and media of contemporary communication discourse. London: Arnold.Google Scholar
Kress, G. and van Leeuwen, T. (2006) Reading images: The grammar of visual design (2nd edn). London: Routledge.Google Scholar
Levy, M. (1997) CALL: Context and conceptualisation. Oxford: Oxford University Press.Google Scholar
Levy, M. (2009) Technologies in use for second language learning. The Modern Language Journal, 93, Focus Issue: 769–782.Google Scholar
Lim, F. V., O’Halloran, K. L. and Podlasov, A. (2012) Spatial pedagogy: Mapping meanings in the use of classroom space. Cambridge Journal of Education, 42(2): 235–251.Google Scholar
Machin, D. (2013) What is multimodal critical discourse studies? Critical Discourse Studies, 10(4): 347–355.Google Scholar
Martin, J. R. (1992) English text: System and structure. Amsterdam: Benjamins.Google Scholar
Martin, J. R. (2002) Meaning beyond the clause: SFL perspectives. Annual Review of Applied Linguistics, 22: 52–74.Google Scholar
Martin, J. R. and Rose, D. (2007) Working with discourse: Meaning beyond the clause, (2nd edn.). London: Continuum.Google Scholar
Moreno Jaén, M. and Pérez Basanta, C. (2009) Developing conversational competence through language awareness and multimodality: The use of DVDs. ReCALL, 21(3): 283–301.Google Scholar
Neville, D. O. (2015) The story in the mind: The effect of 3D gameplay on the structuring of written L2 narratives. ReCALL, 27(1): 21–37.Google Scholar
Norris, S. (2004) Analyzing multimodal interaction: A methodological framework. London and New York: Routledge.Google Scholar
O’Halloran, K. L., Tan, S. and E, M. K. L (2014) Multimodal pragmatics. In: Schneider, K. and Barron, A. (eds.), Pragmatics of discourse. Berlin: De Gruyter Mouton, 239–268.Google Scholar
O’Halloran, K. L., Tan, S. and E, M. K. L. (2015) Multimodal analysis for critical thinking. Learning, Media and Technology. DOI: 10.1080/17439884.2016.1101003.Google Scholar
O’Halloran, K. L., Tan, S. and Smith, B. A. (2016) Multimodal approaches to English for academic purposes. In: Hyland, K. and Shaw, P. (eds.), The Routledge handbook of English for academic purposes. London and New York: Routledge, 257–270.Google Scholar
O’Halloran, K. L., Tan, S. and Wignell, P. (2016 in press) Inter-semiotic translation as resemiotization: A multimodal perspective. In: Fontanille, J., Sonzogni, M. and Troqe, R. (eds.), Signata. Special Issue on Translating: Signs, Texts, Practices.Google Scholar
O’Toole, M. (2011) The language of displayed art (2nd edn.). London and New York: Routledge.Google Scholar
Peterson, M. (2010) Learner participation patterns and strategy use in Second Life: An exploratory case study. ReCALL, 22(3): 273–292.Google Scholar
Peterson, M. (2012) Learner interaction in a massively multiplayer online role playing game (MMORPG): A sociocultural discourse analysis. ReCALL, 24(3): 361–380.Google Scholar
Rama, P. S., Black, R. W., van Es, E. and Warschauer, M. (2012) Affordances for second language learning in World of Warcraft. ReCALL, 24(3): 322–338.Google Scholar
Satar, H. M. (2013) Multimodal language learner interactions via desktop videoconferencing within a framework of social presence: Gaze. ReCALL, 25(1): 122–142.Google Scholar
Sindoni, M. G. (2013) Spoken and written discourse in online interactions: A multimodal approach. London and New York: Routledge.Google Scholar
Sindoni, M. G. (2015) Teaching English multimodally: The use of new travel websites in EFL environments. Teaching English with Technology, 15(2): 19–38.Google Scholar
Sokolik, M. (2014) What constitutes an effective language MOOC? In: Martín-Monje E. and Bárcena Madera, E. (eds.), Language MOOCs: Providing learning, transcending boundaries. Berlin: De Gruyter Open, 16–32.Google Scholar
Stickler, U., Batstone, C., Duensing, A. and Heins, B. (2007) Distant classmates: Speech and silence in online and telephone language tutorials. European Journal of Open, Distance and e-Learning, 2007(2). http://oro.open.ac.uk/32919/1/Stickler_Batstone_Duensing_Heins.html.Google Scholar
Tan, S., Smith, B. A. and O’Halloran, K. L. (2015) Online leadership discourse in higher education: A digital multimodal discourse perspective. Discourse & Communication, 9(5): 559–584.Google Scholar
Unsworth, L. (2008a) Multiliteracies, e-literature and English teaching. Language and Education, 22(1): 62–75.Google Scholar
Unsworth, L. (ed.) (2008b) Multimodal semiotics: Functional analysis in contexts of education. New York: Continuum.Google Scholar
van Leeuwen, T. (1999) Speech, music, sound. Houndsmills: Macmillan Press.Google Scholar
van Leeuwen, T. (2005) Introducing social semiotics. London: Routledge.Google Scholar
Wigham, C. R. and Chanier, T. (2013) A study of verbal and nonverbal communication in Second Life: The ARCHI21 experience. ReCALL, 25(1): 63–84.Google Scholar
Yim, S. and Warschauer, M. (2016) CALL and electronic media. In: Hyland, K. and Shaw, P. (eds.), The Routledge handbook of English for academic purposes. London and New York: Routledge, 592–607.Google Scholar
Zheng, D., Newgarden, K. and Young, M. F. (2012) Multimodal analysis of language learning in World of Warcraft play: Languaging as values-realizing. ReCALL, 24(3): 339–360.Google Scholar